JP7742466B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

People who lack the confidence to express their true selves due to trauma from interpersonal relationships or a lack of education find it difficult to become that ideal person, even if they have a specific ideal image in mind. This is the problem that exists.

The system projects the ideal image of the person set by the user and custom-changes the user's behavior and speech based on that ideal image. It then presents these custom-changed behaviors and speech to the user, and through repeated training, it provides a system that encourages the user to get closer to their ideal image.

1 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 10 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. FIG. 10 is a sequence diagram showing a processing flow of the data processing system in Application Example 1 of Form Example 1. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a third embodiment of the third form example. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used as work memory by the processor.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a symbol is an interface that includes a communication processor, an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be just A, just B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A, a microphone 38B, etc., and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or finger) to receive user input via the indicator. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 is equipped with a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible by the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

"Example 2"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user sets their ideal person in the application. For example, they can set "a compassionate person" as their ideal person.

Step 2: The system custom-changes the user's behavior and comments based on the ideal image of a person. Specifically, it analyzes the user's behavior and comments based on the image of a "compassionate person" and generates appropriate behavior and comments.

Step 3: The system presents the user with the customized actions and statements. For example, it may provide advice to encourage the user to behave more considerately toward others.

Step 4: The user repeatedly practices the suggested actions and statements, allowing them to get closer to their ideal self.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Existing self-improvement systems have difficulty customizing specific actions and speech based on the ideal person the user sets, and lack a means to effectively record the progress of this repetitive training and provide feedback. This poses a challenge in that they do not provide effective support to help users get closer to their ideal self.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image using a generative AI model, means for presenting the custom-changed actions and speech to the user and repeatedly training them, and means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and speech that will bring them closer to their ideal image, and effectively manage their progress.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom-change user behavior and speech.

"Custom Change" refers to changing a user's behavior and words based on an ideal image of a person.

"Repeated training" refers to the user repeatedly practicing the custom-changed actions and utterances that are presented to them.

"Feedback" refers to evaluations and advice provided based on the user's progress through repeated training.

"Recording progress" refers to saving the content and results of the user's repeated training as data.

"Presenting" refers to showing the custom-changed actions and statements generated by the server to the user.

This invention is a system that custom-changes actions and speech based on the ideal person image set by the user, and encourages the user to approach their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: A device operated by the user, such as a smartphone, computer, or tablet.

Server: A server for processing data and running generative AI models.

Software

Application: Software that provides an interface for users to define their ideal person and review their custom-changed behavior and statements.

Generative AI model: An algorithm that custom-changes a user's behavior and speech based on an ideal human image. Specifically, we use generative AI models such as OpenAI's (registered trademark) GPT-4 (registered trademark).

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of a person. For example, they can set "a considerate person" as their ideal image of a person.

The device sends configuration data to the server.

The device sends data about the ideal person set by the user to the server. This data includes information about the ideal person selected by the user.

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects data on the user's past behavior and comments, and custom-changes it based on the ideal image of the person.

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This data includes specific examples of behaviors and speech that the user should practice.

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Users perform repetitive training.

Users practice the proposed custom-changed actions and utterances, practicing repeatedly. The application records the user's progress and provides feedback as needed.

Specific examples

For example, if a user sets "a compassionate person" as their ideal person, the following prompt sentence would be input into the generative AI model:

Example prompt

"Please provide advice to help users be more considerate when speaking with friends."

Based on this prompt, the generative AI model generates specific example utterances that the user can use when conversing with friends. For example, it provides advice such as, "When a friend is in trouble, say, 'Let me know if there's anything I can help you with.'"

In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they click the "Set Ideal Person" button on the application interface, select "Compassionate Person" from the drop-down menu, and press the "Set" button.

Input: User's chosen ideal person (e.g., "a caring person")

Output: Ideal human image configuration data

Step 2:

The device sends the configuration data to the server.

The device sends data about the user's ideal person profile to a server. This transmission is secure using the HTTPS protocol.

Input: Ideal human image data

Output: Configuration data sent to the server

Step 3:

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and social media posts) and customizes it based on the ideal image of the person.

Input: Configuration data, user past behavior and comment data

Output: Custom-changed behavior and speech data

Step 4:

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

Input: Custom-changed behavior and speech data

Output: Examples of specific actions and statements generated

Step 5:

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This transmission is also done using the HTTPS protocol.

Input: Examples of specific actions or statements to be generated

Output: Data sent to the device

Step 6:

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Input: Data sent to the device

Output: Examples of specific actions and statements presented to the user

Step 7:

Users engage in repetitive training.

The user then practices the custom-changed actions and statements presented to them, repeatedly practicing. The application records the user's progress and provides feedback as needed. For example, a user might enter into the application, "Today, I made a kind statement to a friend."

Input: User practice results

Output: Recorded progress data and provided feedback

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional customer service training systems had the problem of being unable to provide effective training because it was difficult to customize specific actions and comments to help staff develop the ideal customer service attitude. Furthermore, there was a lack of repetitive training to help staff approach the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for generating actions and utterances based on the ideal human image using a generative AI model, and means for presenting the generated actions and utterances to the user as prompt sentences. This enables staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repeated training.

A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a person.

The "ideal person" is the specific character and attitude that the user should strive for, and serves as the standard by which the system will custom-change their behavior and speech.

"Projection means" is a function that displays the user's ideal image of a person visually or as text on the system.

"Means for custom change" is a function that allows users to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" is a function for displaying custom-changed actions and comments to users.

"Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and statements.

A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

A "prompt" is a text-based instruction that presents the user with actions and statements generated by the generative AI model based on an ideal human image.

The system for implementing this invention custom-changes behavior and speech based on the ideal person image set by the user, and presents this to the user for repetitive training. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A device such as a smartphone or tablet that allows the user to define their ideal person and check their custom-changed behavior and statements.

2. Server: Processes user input data and uses a generative AI model to generate behaviors and statements based on the ideal human image.

3. Generative AI models: Using generative AI models such as OpenAI's GPT-3 (registered trademark), actions and statements are generated based on the ideal human image set by the user.

Program processing

The server receives the ideal human image and current behavior and speech data sent from the user's device and inputs this as prompt text into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

Hardware and software used

Hardware: Smartphones, tablets, servers

Software: Python, OpenAI API, user interface application

Data processing and calculation

The server sends the user's ideal image of a person and their current actions and statements as prompts to the generative AI model. The generative AI model generates the ideal actions and statements based on the prompts and returns the results to the server. The server then sends the generated actions and statements to the user's device and presents them to the user.

Specific examples

For example, if the user defines their ideal person as "attentive customer service" and their current behavior is "treating customers in an unfriendly manner," the prompt would be as follows:

Example prompt:

Ideal person: Attentive customer service

User behavior: Treating customers rudely

Acting based on the ideal human image:

Based on this prompt, the generative AI model generates actions and statements appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and statements presented, users can get closer to their ideal customer service attitude.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user sets their ideal person image using their device. The user launches the application and inputs their ideal person image (e.g., "thorough customer service"). The input data is sent from the device to the server.

Input: Ideal human image

Output: Ideal human image data sent to the server

Step 2:

The server receives the ideal human image data sent by the user. The server stores this data and uses it for subsequent processing.

Input: Ideal human image data

Output: Saved ideal human image data

Step 3:

The user uses the device to input their current actions and statements. The user inputs their current actions (e.g., "being rude to customers") and sends it from the device to the server.

Input: Current action or statement

Output: Current behavior and speech data sent to the server

Step 4:

The server receives the current activity and comment data sent by the user. The server stores this data and uses it for further processing.

Input: Current activity and speech data

Output: Saved current behavior and speech data

Step 5:

The server generates prompts by combining the stored ideal human image data with current behavior and speech data. The generated prompts are then input into the generative AI model.

Input: Ideal person data, current behavior and speech data

Output: Generated prompt text

Step 6:

The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

Input: Generated prompt text

Output: Ideal actions and statements generated by the generative AI model

Step 7:

The server receives the ideal actions and statements returned by the generative AI model. The server then sends this data to the user's device.

Input: Ideal actions and statements generated by a generative AI model

Output: Ideal behavior and speech data sent to the user's device

Step 8:

The user's device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

Input: Ideal behavior and speech data

Output: Ideal actions and statements presented to the user

Through these processing steps, users learn how to behave and speak in a way that is based on the ideal image of a person, and through repeated training, they can move closer to their ideal attitude.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires sustained effort. Furthermore, there is a lack of specific guidelines for actions and statements for self-improvement, making effective training difficult. Furthermore, there are no appropriate means for users to customize their actions and statements based on their ideal image, making it difficult for users to become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and comments based on the ideal human image, means for presenting the custom-changed actions and comments to the user, means for the user to repeatedly train on the presented actions and comments, means for collecting the user's actions and comments, and means for analyzing the collected actions and comments and customizing them using a generative AI model. This allows the user to obtain specific guidelines for actions and comments, and enables them to get closer to their ideal image through continuous training.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

"Projection means" refers to the function that displays the user's ideal image of the person in visual or text form.

"Means for custom change" refers to the function that allows a user to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" refers to the function for notifying or displaying custom-changed actions or statements to the user.

"Means for repeated training" refers to a function that allows users to train themselves to become closer to their ideal image of a person by repeatedly performing suggested actions and statements.

"Means of collection" refers to the function for recording user actions and comments and saving them as data.

"Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on the ideal image of a person.

A "generative AI model" refers to an artificial intelligence model that converts a user's actions and statements based on an ideal image of a human being.

This invention is a system that custom-changes actions and speech based on the ideal person image set by the user, and presents this to the user for repetitive training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The device used by the user, such as a smartphone, tablet, or computer.

Server: A remote server for processing and storing data.

Software

Application: Software that allows users to define their ideal person and check their custom-changed behavior and statements.

Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user behavior and speech based on an ideal human image.

System Operation

Establishing the ideal human image

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Collecting user behavior and comments

The device records the user's everyday actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

Customize your actions and speech

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), it custom-changes the user's behavior and speech based on the ideal person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to, "I'm busy today, but please let me know if there's anything I can do to help."

Displaying custom actions and statements

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Repetitive training

Users repeatedly execute the custom-changed actions and statements that are presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, users train to become closer to their ideal image of the person they want to be.

Specific examples

Establishing the ideal human image

The user enters "compassionate person" in the application's settings screen and clicks the save button.

Collecting user behavior and comments

The device records the user's voice when they are talking to friends and converts it into text. It also collects text data that the user posts on social media.

Customize your actions and speech

The server inputs the collected text data into a generative AI model, which uses prompts to convert the user's speech based on an ideal human image.

Displaying custom actions and statements

The server sends the custom-modified utterances output by the generative AI model to the device. The device then displays a notification to the user, suggesting something like, "I'm busy today, but please let me know if there's anything I can do to help you."

Repetitive training

Users use the suggestions from the server in their everyday conversations and social media posts. For example, in a conversation with a friend, they might say, "I'm busy today, but please let me know if there's anything I can do to help." By repeating this, users will naturally begin to act and speak in a way that befits a "considerate person."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

Current user statement: "I'm busy today, so I can't help."

Ideal person: "A compassionate person"

Prompt for generative AI model: "Transform the user's utterances into those of a considerate person."

By inputting this prompt into a generative AI model, the user's speech is custom-changed based on the ideal human image.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Input: The ideal person image entered by the user on the settings screen.

Output: The ideal image of a person is saved in the application.

Step 2:

The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media.

Input: User actions and statements.

Output: Collected behavioral and speech data is stored on the device and sent to the server in real time.

Step 3:

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person.

Input: Collected behavioral and speech data, ideal human image.

Output: Custom-changed actions and statements.

Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompts.

Step 4:

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Input: Custom-changed actions and statements.

Output: The custom-changed actions and statements presented to the user.

Specific operation: The device displays the custom-changed message received from the server as a notification.

Step 5:

The user repeatedly executes the custom-changed actions and statements presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts.

Input: The proposed custom-changed actions and statements.

Output: Custom-changed actions and statements performed by the user.

Specific operation: The user uses statements suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to their ideal image of the person they want to be.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In traditional customer service work, there were limited training methods for staff to acquire ideal customer service attitudes, making effective training difficult. Furthermore, there was a lack of specific guidelines or feedback for staff to put ideal customer service attitudes into practice, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient way to provide repeated training to help staff become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for training the ideal customer service attitude using smart glasses worn by staff performing customer service duties, and means for generating custom-changed utterances based on the ideal human image using a generative AI model. This enables staff to efficiently train in and practice the ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to a device or software that visually or audibly displays the ideal human image set by the user.

"Means for custom change" refers to a device or software that changes a user's behavior and speech based on an ideal image of a person.

"Presentation means" refers to a device or software that displays custom-changed actions and statements to the user.

"Means for repeated training" refers to a device or software that allows the user to repeatedly practice custom-changed actions or utterances.

"Customer service" refers to the work of providing products and services to customers.

"Smart glasses" are glasses-type devices that have the function of displaying information.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text and behavior.

"Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

The system for implementing this invention projects the ideal image of a person set by the user, custom-changes the user's actions and speech, and repeatedly trains them. Specifically, it can train ideal customer service attitudes using smart glasses worn by staff performing customer service duties.

System Configuration

The system consists of the following main components:

1. User device: The device on which the user sets their ideal image of the person. This includes smartphones and tablets.

2. Smart glasses: A device worn by staff performing customer service duties. It has the function of visually displaying information.

3. Server: Runs the generative AI model and processes data to custom change user behavior and speech.

4. Generative AI model: An algorithm that generates custom-modified statements based on the user's ideal image of the person.

Program processing

The server receives the user's ideal image of the person and custom-changes the user's behavior and speech based on that image. Specifically, the process is carried out as follows:

1. Setting the ideal person: Using the user device, the user sets the ideal person. For example, they can set "kind and polite customer service" as their ideal image.

2. Enter user utterances: Enter actual utterances made by the user. For example, "Welcome, are you looking for something?"

3. Execution of the generative AI model: The server uses the generative AI model to generate custom-modified utterances based on the ideal human image.

4. Present customized speech: Present customized speech to the user through smart glasses. For example, "Welcome, we'll help you find the product you're looking for."

5. Repetitive training: Users repeatedly practice the custom-modified utterances presented to them.

Hardware and software used

Hardware: Smart glasses (e.g., Google® Glass®), user devices (smartphones, tablets)

Software: Generative AI models (e.g., OpenAI API), data processing servers

Specific examples

For example, let's say the ideal customer service staff member is "friendly and courteous service," and use the following prompt:

Example prompt:

Ideal personality: Kind and courteous customer service

User says: Welcome, are you looking for something?

Custom-changed statements based on the ideal person:

By inputting this prompt into the generative AI model, a custom-modified utterance based on the ideal customer service attitude is generated. For example, a utterance such as, "Welcome, we'll help you find the product you're looking for" is generated.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user sets their ideal image of the person.

Input: The user uses a device (smartphone or tablet) to input their ideal image of a person. For example, "kind and polite customer service."

Data processing: The device sends the input image of the ideal person to the server.

Output: The ideal image of a person is saved on the server.

Step 2:

The user enters the actual statement.

Input: The user uses the device to type what they actually say. For example, "Welcome, is there anything I can help you with?"

Data processing: The device sends the entered comments to the server.

Output: User comments are saved on the server.

Step 3:

The server runs the generative AI model.

Input: The server retrieves the stored ideal person image and user statements.

Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt text is input into the generative AI model to obtain the custom-changed utterances.

Output: The generated custom-changed comments are saved on the server.

Step 4:

Present the custom-changed statement to the user.

Input: The server receives the generated custom-changed statement.

Data processing: The server sends the custom-modified speech to the smart glasses.

Output: The custom-changed speech is displayed on the smart glasses.

Step 5:

Users perform repetitive training.

Input: The user sees the custom-changed speech displayed on the smart glasses.

Specific behavior: The user repeatedly practices the displayed utterance. For example, they practice saying, "Welcome, we'd love to help you find the product you're looking for."

Output: The user's customer service attitude becomes closer to the ideal person.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

"Example 2"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user starts the system and sets their ideal self. This ideal self specifically describes how the user will behave and what they will say.

Step 2: The emotion engine analyzes the user's emotional state in real time. Specifically, it infers the user's emotional state from their facial expressions, tone of voice, choice of words, etc.

Step 3: The emotion engine tracks changes in the user's emotional state. This allows the content of the repetitive training to help the user approach their ideal self to be tailored to the user's emotional state.

Step 4: The user undergoes repeated training to move closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that shares that joy.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Previous systems had the problem that it was difficult to customise behaviour and speech based on the ideal image of the person set by the user, and the training content was not adjusted according to the user's emotional state, resulting in a lack of effective support to help users get closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and means for adjusting the training content based on data obtained from the emotion engine. This enables effective support for the user to get closer to their ideal image.

"User" refers to an individual who uses the system to aim to become closer to their ideal image of a person.

An "ideal person" is a specific set of personality and behavioral characteristics that a user aspires to.

"Projection means" refers to the function that displays the user's ideal image of the person on the system in visual or text format.

"Means for customizing" refers to the ability to change and adjust a user's behavior and comments based on their ideal image of a person.

"Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

"Means for repeated training" refers to a function that allows users to repeatedly practice based on the custom changes presented to them.

The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to the function that changes and adjusts training content according to the user's emotional state based on data obtained from the emotion engine.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The smartphone, tablet, or PC used by the user.

Server: Cloud server for data processing and storage.

Camera and microphone: Devices that capture your facial expressions and tone of voice in real time.

Software

Application: Provides an interface that allows users to define their ideal person and review their custom-changed behavior and statements.

Emotion engine: A software module for analyzing a user's emotional state in real time.

Generative AI model: An AI model that custom-changes a user's behavior and speech based on an ideal human image.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of the person. For example, they can select "a compassionate person."

The server collects user behavior data.

The server collects input data, voice data, and text messages from users' smartphones and PCs in real time, including what users chat with friends and what they say to their voice assistants.

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. For example, if a user says "thank you" infrequently, the server will suggest that they say "thank you" more frequently.

The server presents custom-changed actions and comments to the user.

The server then uses the application's notification function to present the custom-changed actions and comments to the user. For example, it could send a notification such as, "Try saying 'Thank you' the next time you talk to a friend."

Users perform repetitive training.

Based on suggestions from the server, users are made conscious of saying "thank you" more often in their daily lives. By repeating this, users become closer to becoming "considerate people."

The emotion engine analyzes the user's emotional state.

The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

The server adjusts the training content.

The server uses data obtained from the emotion engine to adjust the training content according to the user's emotional state. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

Specific examples

Defining the ideal person

The user opens the application and sets "a compassionate person" as their ideal person.

Custom Change Example

The server suggests that users say "thank you" more frequently when conversing with friends.

Emotion Engine Analysis

If a user smiles during a conversation, the emotion engine will determine that the user is feeling happy.

Adjusting training content

If the user is feeling stressed, the server will provide deep breathing and meditation training.

Example prompt

"If the user sets their ideal person as 'a considerate person,' please suggest what actions and statements they would like to customize."

The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they select their ideal person from options such as "a considerate person" on the application's settings screen and save the settings. The input is the ideal person selected by the user, and the output is data for the set ideal person.

Step 2:

The server collects user behavior data.

The server collects input data, voice data, and text messages from the user's smartphone or PC in real time. Specifically, it collects what the user chats with friends and what they say to their voice assistant. The input is user behavioral data, and the output is a database of the collected behavioral data.

Step 3:

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and speech based on the ideal person. For example, if the user says "thank you" infrequently, the server will suggest that they say "thank you" more often. The input is the collected behavioral data, and the output is suggested custom-changed behaviors and speech.

Step 4:

The server presents custom-changed actions and comments to the user.

The server presents the custom-changed actions and utterances to the user using the application's notification function. Specifically, it sends a notification such as, "Try saying 'thank you' the next time you talk to a friend." The input is a suggestion for a custom-changed action or utterance, and the output is a notification to the user.

Step 5:

Users engage in repetitive training.

Based on suggestions from the server, users are made conscious to say "thank you" more frequently in their daily lives. Specifically, they increase the number of opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

Step 6:

Emotion engine analyzes the user's emotional state

The emotion engine uses the camera and microphone on the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is data on the analyzed emotional state.

Step 7:

The server adjusts the training content.

The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. Specifically, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and training content was not adjusted to reflect the emotional state of staff, making it difficult to provide effective training.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, means for adjusting the training content according to the user's emotional state, and means for providing feedback to the user using a smart device. This enables staff to effectively acquire the ideal customer service attitude.

A "user" is an individual who uses the system to train themselves to behave and speak in a way that reflects their ideal image of a person.

The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

"Projection means" refers to a method or device for visually or audibly presenting the ideal human image set by the user.

"Means for custom change" refers to methods or devices for changing a user's behavior and speech based on an ideal image of a person.

"Means for repeated training" refers to methods or devices that allow users to repeatedly practice custom-changed actions and speech in order to become closer to their ideal image of the person.

"Means for analyzing emotional states in real time" refers to methods or devices for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to a method or device for changing the training content according to the user's emotional state.

A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

"Means for providing feedback" refers to methods or devices for providing users with advice or instructions regarding ideal behavior or speech.

As an embodiment of this invention, we provide a training system that helps staff working in brick-and-mortar stores acquire ideal customer service attitudes. The system includes means for projecting an ideal image of a person set by the user, customizing the user's behavior and speech, and repeatedly training them. It also includes means for analyzing the user's emotional state in real time and adjusting the training content.

Hardware and software used

Hardware

Smart glasses: Equipped with a camera, microphone, and speaker, they capture the user's actions and speech and provide feedback.

Server: A central processing unit for data processing and analysis.

Software

OpenCV: A library for capturing and processing camera images.

EmotionRecognizer: A library that estimates emotions from facial expressions.

SpeechRecognition: Speech recognition library.

TextToSpeech: A library that converts text to speech.

IdealBehaviorModel: A model that generates ideal behaviors and statements.

Data processing and calculations

Camera footage capture

The camera in the smart glasses is used to capture the user's facial expressions and behavior in real time. OpenCV is used to acquire video data, and EmotionRecognizer is used to analyze the emotional state.

Voice recognition

Use the microphone in the smart glasses to capture what the user is saying. Use the SpeechRecognition library to convert the speech data into text.

Feedback generation

Using IdealBehaviorModel, ideal behaviors and utterances are generated based on the user's emotional state and utterances. The generated feedback is converted into speech using the TextToSpeech library and provided to the user through the smart glasses' speakers.

Specific examples

Scenario

Staff working in physical stores will receive training aimed at providing "considerate customer service."

situation

When customers ask questions about products, staff will respond appropriately.

Feedback

If staff are feeling stressed, the system will provide advice to encourage relaxation.

Example prompt

If the user aims to provide "caring customer service," tell them how to respond to customer questions. If the user is feeling stressed, provide them with advice on how to promote relaxation.

In this way, customer service training can be carried out effectively in physical stores.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The user puts on the smart glasses and starts the system. The smart glasses' camera captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is camera video and audio data, and the output is the captured video and audio data.

Step 2:

The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes their emotional state using EmotionRecognizer. The input is the captured video data, and the output is the user's emotional state.

Step 3:

The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data, and the output is text data.

Step 4:

The server uses IdealBehaviorModel to generate ideal behaviors and statements based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal behavior or statement.

Step 5:

The server uses the TextToSpeech library to convert the generated text of ideal actions and statements into speech. The input is the text of the ideal actions and statements, and the output is audio data.

Step 6:

Audio data sent from the server is provided to the user through the smart glasses' speakers. The input is audio data, and the output is audio feedback heard by the user.

Step 7:

The user modifies their actions or statements based on the provided feedback, and new data is captured again through the smart glasses' camera and microphone. This repeats the process from step 1 to step 6. The input is video and audio data of the modified actions or statements, and the output is new feedback.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Previous systems had issues with being unable to provide effective training because it was difficult for users to customize the specific actions and comments that would bring them closer to their ideal image of the person they wanted, and the training content was not adjusted according to the user's emotional state. Furthermore, the lack of a function to analyze the user's emotional state in real time and adjust the training content based on that analysis made it difficult to bring users closer to their ideal image while reducing the psychological burden.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, and means for adjusting the training content in accordance with the emotional state. This allows the user to receive effective training to approach their ideal image.

"Ideal person" refers to the type of person or personality traits that the user should aim to become.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to devices or software used to notify users of custom-changed actions or statements.

"Means for repeated training" refers to a program or system that allows users to repeatedly practice custom-changed actions and utterances.

"Means for analyzing emotional state in real time" refers to devices or software that estimate a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to algorithms or programs that change the content or method of training according to the user's emotional state.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

System Configuration

Users define their ideal image of a person.

Using an application on their device, users can set the ideal person they aspire to be. For example, they can select "a compassionate person." This setting takes into account the user's interpersonal trauma, education, etc.

The server collects user behavior and comments.

The server collects user actions and statements in real time, including text and voice data entered by the user. For example, it collects what users say to friends in chat apps and what they say to voice assistants.

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses natural language processing (NLP) technology to understand what the user is saying. It uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

The server customizes your actions and speech based on your ideal image of a person.

Based on the analysis results, the server custom-changes the user's actions and comments to match their ideal image of a person. For example, "I'm busy today, so I can't help you" could be changed to "I'm busy today, but please let me know if there's anything I can do to help."

The server presents the custom changes to the user.

The server presents the custom-changed actions and comments to the user, who can review them and make any necessary corrections.

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, and choice of words. It uses Microsoft (registered trademark) Azure (registered trademark) Emotion API and Affectiva's SDK.

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it will provide training to promote relaxation, and if the user is feeling joy, it will provide training to encourage behavior that encourages sharing that joy.

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to get closer to their ideal self.

Specific examples

Establishing the ideal human image

The user opens the application, selects "Set your ideal person," and chooses "A compassionate person."

Collecting actions and statements

The application collects chat content between users and friends in the background.

Data Analysis

The chat content collected by the server is analyzed using the Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what the user is saying and generate appropriate responses.

Custom Change Example

If the user says, "I'm busy today and can't help," the server changes this to "I'm busy today, but please let me know if there's anything I can do to help."

Emotion Engine Analysis

When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

Adjusting training content

When a user is feeling stressed, the server provides relaxation training such as "Take a deep breath and relax."

Running the training

The user performs relaxation training presented by the server.

Example prompt

"If a user sets their ideal person as 'a considerate person,' how would they customize their behavior and speech?"

This system allows users to receive effective training to bring them closer to their ideal self.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

Users define their ideal image of a person.

The user launches the application on their device and sets their ideal person. Specifically, they select "Set ideal person" from the application menu and choose their ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is that setting information.

Step 2:

The server collects user behavior and comments.

The server collects user behavior and speech in real time. Specifically, it collects the content of user conversations with friends on chat apps and what users say to voice assistants. The input is the user's text data and voice data, and the output is the collected behavior and speech data.

Step 3:

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

Step 4:

The server custom-changes your behavior and speech based on your ideal person.

Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help." The input is the analysis results, and the output is the custom-changed actions and comments.

Step 5:

The server presents the custom changes to the user.

The server presents the custom-changed actions and statements to the user, who can review them and make corrections as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

Step 6:

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

Step 7:

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it provides training to promote relaxation, and if the user is feeling happy, it provides training to encourage behavior that shares that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

Step 8:

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to approach their ideal self. The input is the adjusted training content, and the output is the user's execution results.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In traditional customer service work, it was difficult for staff to maintain an ideal customer service attitude, and customer satisfaction declined, particularly when their emotional state was unstable. Furthermore, there was a lack of effective training methods to help staff approach their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This enables staff to receive appropriate training according to their emotional state while maintaining an ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to a display device or notification system that notifies users of custom-changed actions and statements.

"Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to become closer to their ideal image of the person they want to be.

An "emotion engine" is software or hardware that analyzes a user's emotional state in real time.

"Means for adjusting training content" is a function for changing the content of training provided to users based on the analysis results of the emotion engine.

"Smart glasses" are wearable devices that can visually display information when worn by the user.

The system for implementing this invention includes a series of means for projecting the ideal human image set by the user, customizing the user's behavior and speech, and repeatedly training them. It also has the ability to use an emotion engine to analyze the user's emotional state in real time and adjust the training content.

Hardware and software used

Hardware: Smart glasses, camera

Software: OpenCV (face detection), EmotionRecognizer (emotion recognition), BehaviorSuggestion (behavior suggestions)

System Configuration

1. A way to project the user's ideal image of a person:

Users can set their ideal image of a person through the application. For example, if their ideal is "kind and polite customer service," the system will use this information to customize their behavior and speech.

2. How to customise user behaviour and comments:

The system runs algorithms to change the user's current behavior and speech based on the ideal person. For example, if the user is feeling stressed, it will make suggestions to encourage relaxation.

3. A method for presenting custom-changed actions and statements to users and repeatedly training them:

Custom-changed actions and speech are displayed on the smart glasses' display. Users can view these and repeatedly practice their ideal actions and speech.

4. Emotion Engine:

The camera captures the user's facial expressions and EmotionRecognizer analyzes their emotional state in real time. This allows the system to provide training content tailored to the user's emotional state.

5. Ways to adjust training content:

Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior that shares that joy.

6. Display method on smart glasses:

Suggested actions and comments are displayed in real time on the smart glasses' display, allowing users to maintain ideal behavior while serving customers.

Specific examples

Scenario: Store staff wear smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

Emotional state: If staff are feeling stressed, offer relaxation suggestions (e.g., "Take a deep breath and relax").

Action suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

Example prompt

Develop an application that custom changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function that uses an emotion engine to analyze the user's emotional state and provide appropriate training.

The flow of the specific processing in Application Example 2 is explained using Figure 18.

Step 1:

The user launches the application and sets their ideal image of the person. The user inputs an ideal image such as "kind and polite customer service." This input data is sent to the server and saved.

Step 2:

The server runs an algorithm that custom-changes behavior and speech based on the user's ideal image of the person. It receives the ideal image of the person and the user's current behavioral data as input, and generates custom-changed behavior and speech as output.

Step 3:

The server sends the custom-changed actions and statements to the smart glasses, which receive them and display them on the screen. The user then views these and practices their ideal actions and statements.

Step 4:

The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data, and generates emotional state data as output.

Step 5:

The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine receives the emotional state data and the ideal human image as input and suggests appropriate actions and statements. The adjusted training content is generated as output.

Step 6:

The server sends the adjusted training content to the smart glasses, which receive it and display it on the screen. The user then views it and practices training that suits their emotional state.

Step 7:

As the user practices the training, the server tracks changes in their emotional state and readjusts the training content as needed. This allows the user to receive optimal training to bring them closer to their ideal image of the person they want to be.

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

"Example 2"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user sets their ideal person in the application. For example, they can set "a compassionate person" as their ideal person.

Step 2: The system custom-changes the user's behavior and comments based on the ideal image of a person. Specifically, it analyzes the user's behavior and comments based on the image of a "compassionate person" and generates appropriate behavior and comments.

Step 3: The system presents the user with the customized actions and statements. For example, it may provide advice to encourage the user to behave more considerately toward others.

Step 4: The user repeatedly practices the suggested actions and statements, allowing them to get closer to their ideal self.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Existing self-improvement systems have difficulty customizing specific actions and speech based on the ideal person the user sets, and lack a means to effectively record the progress of this repetitive training and provide feedback. This poses a challenge in that they do not provide effective support to help users get closer to their ideal self.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image using a generative AI model, means for presenting the custom-changed actions and speech to the user and repeatedly training them, and means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and speech that will bring them closer to their ideal image, and effectively manage their progress.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom-change user behavior and speech.

"Custom Change" refers to changing a user's behavior and words based on an ideal image of a person.

"Repeated training" refers to the user repeatedly practicing the custom-changed actions and utterances that are presented to them.

"Feedback" refers to evaluations and advice provided based on the user's progress through repeated training.

"Recording progress" refers to saving the content and results of the user's repeated training as data.

"Presenting" refers to showing the custom-changed actions and statements generated by the server to the user.

This invention is a system that custom-changes actions and speech based on the ideal person image set by the user, and encourages the user to approach their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: A device operated by the user, such as a smartphone, computer, or tablet.

Server: A server for processing data and running generative AI models.

Software

Application: Software that provides an interface for users to define their ideal person and review their custom-changed behavior and statements.

Generative AI model: An algorithm that custom-changes a user's behavior and speech based on an ideal human image. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of a person. For example, they can set "a considerate person" as their ideal image of a person.

The device sends configuration data to the server.

The device sends data about the ideal person set by the user to the server. This data includes information about the ideal person selected by the user.

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects data on the user's past behavior and comments and customizes it based on the ideal image of the person.

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This data includes specific examples of behaviors and speech that the user should practice.

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Users perform repetitive training.

Users practice the proposed custom-changed actions and utterances, practicing repeatedly. The application records the user's progress and provides feedback as needed.

Specific examples

For example, if a user sets "a compassionate person" as their ideal person, the following prompt sentence would be input into the generative AI model:

Example prompt

"Please provide advice to help users be more considerate when speaking with friends."

Based on this prompt, the generative AI model generates specific example utterances that the user can use when conversing with friends. For example, it provides advice such as, "When a friend is in trouble, say, 'Let me know if there's anything I can help you with.'"

In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they click the "Set Ideal Person" button on the application interface, select "Compassionate Person" from the drop-down menu, and press the "Set" button.

Input: User's chosen ideal person (e.g., "a caring person")

Output: Ideal human image configuration data

Step 2:

The device sends the configuration data to the server.

The device sends data about the user's ideal person profile to a server. This transmission is secure using the HTTPS protocol.

Input: Ideal human image data

Output: Configuration data sent to the server

Step 3:

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and social media posts) and customizes it based on the ideal image of the person.

Input: Configuration data, user past behavior and comment data

Output: Custom-changed behavior and speech data

Step 4:

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

Input: Custom-changed behavior and speech data

Output: Examples of specific actions and statements generated

Step 5:

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This transmission is also done using the HTTPS protocol.

Input: Examples of specific actions or statements to be generated

Output: Data sent to the device

Step 6:

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Input: Data sent to the device

Output: Examples of specific actions and statements presented to the user

Step 7:

Users engage in repetitive training.

The user then practices the custom-changed actions and statements presented to them, repeatedly practicing. The application records the user's progress and provides feedback as needed. For example, a user might enter into the application, "Today, I made a kind statement to a friend."

Input: User practice results

Output: Recorded progress data and provided feedback

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional customer service training systems had the problem of being unable to provide effective training because it was difficult to customize specific actions and comments to help staff develop the ideal customer service attitude. Furthermore, there was a lack of repetitive training to help staff approach the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for generating actions and utterances based on the ideal human image using a generative AI model, and means for presenting the generated actions and utterances to the user as prompt sentences. This enables staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repeated training.

A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a person.

The "ideal person" is the specific character and attitude that the user should strive for, and serves as the standard by which the system will custom-change their behavior and speech.

"Projection means" is a function that displays the user's ideal image of a person visually or as text on the system.

"Means for custom change" is a function that allows users to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" is a function for displaying custom-changed actions and comments to users.

"Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and statements.

A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

A "prompt" is a text-based instruction that presents the user with actions and statements generated by the generative AI model based on an ideal human image.

The system for implementing this invention custom-changes behavior and speech based on the ideal person image set by the user, and presents this to the user for repetitive training. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A device such as a smartphone or tablet that allows the user to define their ideal person and check their custom-changed behavior and statements.

2. Server: Processes user input data and uses a generative AI model to generate behaviors and statements based on the ideal human image.

3. Generative AI models: Using generative AI models such as OpenAI's GPT-3, actions and statements are generated based on the ideal human image set by the user.

Program processing

The server receives the ideal human image and current behavior and speech data sent from the user's device and inputs this as prompt text into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

Hardware and software used

Hardware: Smartphones, tablets, servers

Software: Python, OpenAI API, user interface application

Data processing and calculation

The server sends the user's ideal image of a person and their current actions and statements as prompts to the generative AI model. The generative AI model generates the ideal actions and statements based on the prompts and returns the results to the server. The server then sends the generated actions and statements to the user's device and presents them to the user.

Specific examples

For example, if the user defines their ideal person as "attentive customer service" and their current behavior is "treating customers in an unfriendly manner," the prompt would be as follows:

Example prompt:

Ideal person: Attentive customer service

User behavior: Treating customers rudely

Acting based on the ideal human image:

Based on this prompt, the generative AI model generates actions and statements appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and statements presented, users can get closer to their ideal customer service attitude.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user sets their ideal person image using their device. The user launches the application and inputs their ideal person image (e.g., "thorough customer service"). The input data is sent from the device to the server.

Input: Ideal human image

Output: Ideal human image data sent to the server

Step 2:

The server receives the ideal human image data sent by the user. The server stores this data and uses it for subsequent processing.

Input: Ideal human image data

Output: Saved ideal human image data

Step 3:

The user uses the device to input their current actions and statements. The user inputs their current actions (e.g., "being rude to customers") and sends it from the device to the server.

Input: Current action or statement

Output: Current behavior and speech data sent to the server

Step 4:

The server receives the current activity and comment data sent by the user. The server stores this data and uses it for further processing.

Input: Current activity and speech data

Output: Saved current behavior and speech data

Step 5:

The server generates prompts by combining the stored ideal human image data with current behavior and speech data. The generated prompts are then input into the generative AI model.

Input: Ideal person data, current behavior and speech data

Output: Generated prompt text

Step 6:

The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

Input: Generated prompt text

Output: Ideal actions and statements generated by the generative AI model

Step 7:

The server receives the ideal actions and statements returned by the generative AI model. The server then sends this data to the user's device.

Input: Ideal actions and statements generated by a generative AI model

Output: Ideal behavior and speech data sent to the user's device

Step 8:

The user's device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

Input: Ideal behavior and speech data

Output: Ideal actions and statements presented to the user

Through these processing steps, users learn how to behave and speak in a way that is based on the ideal image of a person, and through repeated training, they can move closer to their ideal attitude.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires sustained effort. Furthermore, there is a lack of specific guidelines for actions and statements for self-improvement, making effective training difficult. Furthermore, there are no appropriate means for users to customize their actions and statements based on their ideal image, making it difficult for users to become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and comments based on the ideal human image, means for presenting the custom-changed actions and comments to the user, means for the user to repeatedly train on the presented actions and comments, means for collecting the user's actions and comments, and means for analyzing the collected actions and comments and customizing them using a generative AI model. This allows the user to obtain specific guidelines for actions and comments, and enables them to get closer to their ideal image through continuous training.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

"Projection means" refers to the function that displays the user's ideal image of the person in visual or text form.

"Means for custom change" refers to the function that allows a user to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" refers to the function for notifying or displaying custom-changed actions or statements to the user.

"Means for repeated training" refers to a function that allows users to train themselves to become closer to their ideal image of a person by repeatedly performing suggested actions and statements.

"Means of collection" refers to the function for recording user actions and comments and saving them as data.

"Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on the ideal image of a person.

A "generative AI model" refers to an artificial intelligence model that converts a user's actions and statements based on an ideal image of a human being.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, presenting this to the user for repetitive training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The device used by the user, such as a smartphone, tablet, or computer.

Server: A remote server for processing and storing data.

Software

Application: Software that allows users to define their ideal person and check their custom-changed behavior and statements.

Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user behavior and speech based on an ideal human image.

System Operation

Establishing the ideal human image

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Collecting user behavior and comments

The device records the user's everyday actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

Customize your actions and speech

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), it custom-changes the user's behavior and speech based on the ideal person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to, "I'm busy today, but please let me know if there's anything I can do to help."

Displaying custom actions and statements

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Repetitive training

Users repeatedly execute the custom-changed actions and statements that are presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, users train to become closer to their ideal image of the person they want to be.

Specific examples

Establishing the ideal human image

The user enters "compassionate person" in the application's settings screen and clicks the save button.

Collecting user behavior and comments

The device records the user's voice when they are talking to friends and converts it into text. It also collects text data that the user posts on social media.

Customize your actions and speech

The server inputs the collected text data into a generative AI model, which uses prompts to convert the user's speech based on an ideal human image.

Displaying custom actions and statements

The server sends the custom-modified utterances output by the generative AI model to the device. The device then displays a notification to the user, suggesting something like, "I'm busy today, but please let me know if there's anything I can do to help you."

Repetitive training

Users use the suggestions from the server in their everyday conversations and social media posts. For example, in a conversation with a friend, they might say, "I'm busy today, but please let me know if there's anything I can do to help." By repeating this, users will naturally begin to act and speak in a way that befits a "considerate person."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

Current user statement: "I'm busy today, so I can't help."

Ideal person: "A compassionate person"

Prompt for generative AI model: "Transform the user's utterances into those of a considerate person."

By inputting this prompt into a generative AI model, the user's speech is custom-changed based on the ideal human image.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Input: The ideal person image entered by the user on the settings screen.

Output: The ideal image of a person is saved in the application.

Step 2:

The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media.

Input: User actions and statements.

Output: Collected behavioral and speech data is stored on the device and sent to the server in real time.

Step 3:

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person.

Input: Collected behavioral and speech data, ideal human image.

Output: Custom-changed actions and statements.

Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompts.

Step 4:

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Input: Custom-changed actions and statements.

Output: The custom-changed actions and statements presented to the user.

Specific operation: The device displays the custom-changed message received from the server as a notification.

Step 5:

The user repeatedly executes the custom-changed actions and statements presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts.

Input: The proposed custom-changed actions and statements.

Output: Custom-changed actions and statements performed by the user.

Specific operation: The user uses statements suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to their ideal image of the person they want to be.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In traditional customer service work, there were limited training methods for staff to acquire ideal customer service attitudes, making effective training difficult. Furthermore, there was a lack of specific guidelines or feedback for staff to put ideal customer service attitudes into practice, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient way to provide repeated training to help staff become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for training the ideal customer service attitude using smart glasses worn by staff performing customer service duties, and means for generating custom-changed utterances based on the ideal human image using a generative AI model. This enables staff to efficiently train in and practice the ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to a device or software that visually or audibly displays the ideal human image set by the user.

"Means for custom change" refers to a device or software that changes a user's behavior and speech based on an ideal image of a person.

"Presentation means" refers to a device or software that displays custom-changed actions and statements to the user.

"Means for repeated training" refers to a device or software that allows the user to repeatedly practice custom-changed actions or utterances.

"Customer service" refers to the work of providing products and services to customers.

"Smart glasses" are glasses-type devices that have the function of displaying information.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text and behavior.

"Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

The system for implementing this invention projects the ideal image of a person set by the user, custom-changes the user's actions and speech, and repeatedly trains them. Specifically, it can train ideal customer service attitudes using smart glasses worn by staff performing customer service duties.

System Configuration

The system consists of the following main components:

1. User device: The device on which the user sets their ideal image of the person. This includes smartphones and tablets.

2. Smart glasses: A device worn by staff performing customer service duties. It has the function of visually displaying information.

3. Server: Runs the generative AI model and processes data to custom change user behavior and speech.

4. Generative AI model: An algorithm that generates custom-modified statements based on the user's ideal image of the person.

Program processing

The server receives the user's ideal image of the person and custom-changes the user's behavior and speech based on that image. Specifically, the process is carried out as follows:

1. Setting the ideal person: Using the user device, the user sets the ideal person. For example, they can set "kind and polite customer service" as their ideal image.

2. Enter user utterances: Enter actual utterances made by the user. For example, "Welcome, are you looking for something?"

3. Execution of the generative AI model: The server uses the generative AI model to generate custom-modified utterances based on the ideal human image.

4. Present customized utterances: Present customized utterances to the user through the smart glasses. For example, "Welcome, we'll help you find the product you're looking for."

5. Repetitive training: Users repeatedly practice the custom-modified utterances presented to them.

Hardware and software used

Hardware: Smart glasses (e.g., Google Glass), user devices (smartphones, tablets)

Software: Generative AI models (e.g., OpenAI API), data processing servers

Specific examples

For example, let's say the ideal customer service staff member is "friendly and courteous service," and use the following prompt:

Example prompt:

Ideal personality: Kind and courteous customer service

User says: Welcome, are you looking for something?

Custom-changed statements based on the ideal person:

By inputting this prompt into the generative AI model, a custom-modified utterance based on the ideal customer service attitude is generated. For example, a utterance such as, "Welcome, we'll help you find the product you're looking for" is generated.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user sets their ideal image of the person.

Input: The user uses a device (smartphone or tablet) to input their ideal image of a person. For example, "kind and polite customer service."

Data processing: The device sends the input image of the ideal person to the server.

Output: The ideal image of a person is saved on the server.

Step 2:

The user enters the actual statement.

Input: The user uses the device to type what they actually say. For example, "Welcome, is there anything I can help you with?"

Data processing: The device sends the entered comments to the server.

Output: User comments are saved on the server.

Step 3:

The server runs the generative AI model.

Input: The server retrieves the stored ideal person image and user statements.

Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt text is input into the generative AI model to obtain the custom-changed utterances.

Output: The generated custom-changed comments are saved on the server.

Step 4:

Present the custom-changed statement to the user.

Input: The server receives the generated custom-changed statement.

Data processing: The server sends the custom-modified speech to the smart glasses.

Output: The custom-changed speech is displayed on the smart glasses.

Step 5:

Users perform repetitive training.

Input: The user sees the custom-changed speech displayed on the smart glasses.

Specific behavior: The user repeatedly practices the displayed utterance. For example, they practice saying, "Welcome, we'd love to help you find the product you're looking for."

Output: The user's customer service attitude becomes closer to the ideal person.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

"Example 2"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user starts the system and sets their ideal self. This ideal self specifically describes how the user will behave and what they will say.

Step 2: The emotion engine analyzes the user's emotional state in real time. Specifically, it infers the user's emotional state from their facial expressions, tone of voice, choice of words, etc.

Step 3: The emotion engine tracks changes in the user's emotional state. This allows the content of the repetitive training to help the user approach their ideal self to be tailored to the user's emotional state.

Step 4: The user undergoes repeated training to move closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that shares that joy.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Previous systems had the problem that it was difficult to customise behaviour and speech based on the ideal image of the person set by the user, and the training content was not adjusted according to the user's emotional state, resulting in a lack of effective support to help users get closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and means for adjusting the training content based on data obtained from the emotion engine. This enables effective support for the user to get closer to their ideal image.

"User" refers to an individual who uses the system to aim to become closer to their ideal image of a person.

An "ideal person" is a specific set of personality and behavioral characteristics that a user aspires to.

"Projection means" refers to the function that displays the user's ideal image of the person on the system in visual or text format.

"Means for customizing" refers to the ability to change and adjust a user's behavior and comments based on their ideal image of a person.

"Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

"Means for repeated training" refers to a function that allows users to repeatedly practice based on the custom changes presented to them.

The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to the function that changes and adjusts training content according to the user's emotional state based on data obtained from the emotion engine.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The smartphone, tablet, or PC used by the user.

Server: Cloud server for data processing and storage.

Camera and microphone: Devices that capture your facial expressions and tone of voice in real time.

Software

Application: Provides an interface that allows users to define their ideal person and review their custom-changed behavior and statements.

Emotion engine: A software module for analyzing a user's emotional state in real time.

Generative AI model: An AI model that custom-changes a user's behavior and speech based on an ideal human image.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of the person. For example, they can select "a compassionate person."

The server collects user behavior data.

The server collects input data, voice data, and text messages from users' smartphones and PCs in real time, including what users chat with friends and what they say to their voice assistants.

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. For example, if a user says "thank you" infrequently, the server will suggest that they say "thank you" more frequently.

The server presents custom-changed actions and comments to the user.

The server then uses the application's notification function to present the custom-changed actions and comments to the user. For example, it could send a notification such as, "Try saying 'Thank you' the next time you talk to a friend."

Users perform repetitive training.

Based on suggestions from the server, users are made conscious of saying "thank you" more often in their daily lives. By repeating this, users become closer to becoming "considerate people."

The emotion engine analyzes the user's emotional state.

The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

The server adjusts the training content.

The server uses data obtained from the emotion engine to adjust the training content according to the user's emotional state. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

Specific examples

Defining the ideal person

The user opens the application and sets "a compassionate person" as their ideal person.

Custom Change Example

The server suggests that users say "thank you" more frequently when conversing with friends.

Emotion Engine Analysis

If a user smiles during a conversation, the emotion engine will determine that the user is feeling happy.

Adjusting training content

If the user is feeling stressed, the server will provide deep breathing and meditation training.

Example prompt

"If the user sets their ideal person as 'a considerate person,' please suggest what actions and statements they would like to customize."

The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they select their ideal person from options such as "a considerate person" on the application's settings screen and save the settings. The input is the ideal person selected by the user, and the output is data for the set ideal person.

Step 2:

The server collects user behavior data.

The server collects input data, voice data, and text messages from the user's smartphone or PC in real time. Specifically, it collects what the user chats with friends and what they say to their voice assistant. The input is user behavioral data, and the output is a database of the collected behavioral data.

Step 3:

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and speech based on the ideal person. For example, if the user says "thank you" infrequently, the server will suggest that they say "thank you" more often. The input is the collected behavioral data, and the output is suggested custom-changed behaviors and speech.

Step 4:

The server presents custom-changed actions and comments to the user.

The server presents the custom-changed actions and utterances to the user using the application's notification function. Specifically, it sends a notification such as, "Try saying 'thank you' the next time you talk to a friend." The input is a suggestion for a custom-changed action or utterance, and the output is a notification to the user.

Step 5:

Users engage in repetitive training.

Based on suggestions from the server, users are made conscious to say "thank you" more frequently in their daily lives. Specifically, they increase the number of opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

Step 6:

Emotion engine analyzes the user's emotional state

The emotion engine uses the camera and microphone on the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is data on the analyzed emotional state.

Step 7:

The server adjusts the training content.

The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. Specifically, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and training content was not adjusted to reflect the emotional state of staff, making it difficult to provide effective training.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, means for adjusting the training content according to the user's emotional state, and means for providing feedback to the user using a smart device. This enables staff to effectively acquire the ideal customer service attitude.

A "user" is an individual who uses the system to train themselves to behave and speak in a way that reflects their ideal image of a person.

The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

"Projection means" refers to a method or device for visually or audibly presenting the ideal human image set by the user.

"Means for custom change" refers to methods or devices for changing a user's behavior and speech based on an ideal image of a person.

"Means for repeated training" refers to methods or devices that allow users to repeatedly practice custom-changed actions and speech in order to become closer to their ideal image of the person.

"Means for analyzing emotional states in real time" refers to methods or devices for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to a method or device for changing the training content according to the user's emotional state.

A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

"Means for providing feedback" refers to methods or devices for providing users with advice or instructions regarding ideal behavior or speech.

As an embodiment of this invention, we provide a training system that helps staff working in brick-and-mortar stores acquire ideal customer service attitudes. The system includes means for projecting an ideal image of a person set by the user, customizing the user's behavior and speech, and repeatedly training them. It also includes means for analyzing the user's emotional state in real time and adjusting the training content.

Hardware and software used

Hardware

Smart glasses: Equipped with a camera, microphone, and speaker, they capture the user's actions and speech and provide feedback.

Server: A central processing unit for data processing and analysis.

Software

OpenCV: A library for capturing and processing camera images.

EmotionRecognizer: A library that estimates emotions from facial expressions.

SpeechRecognition: Speech recognition library.

TextToSpeech: A library for converting text to speech.

IdealBehaviorModel: A model that generates ideal behaviors and statements.

Data processing and calculations

Camera footage capture

The camera in the smart glasses is used to capture the user's facial expressions and behavior in real time. OpenCV is used to acquire video data, and EmotionRecognizer is used to analyze the emotional state.

Voice recognition

Use the microphone in the smart glasses to capture what the user is saying. Use the SpeechRecognition library to convert the speech data into text.

Feedback generation

Using IdealBehaviorModel, ideal behaviors and utterances are generated based on the user's emotional state and utterances. The generated feedback is converted into speech using the TextToSpeech library and provided to the user through the smart glasses' speakers.

Specific examples

Scenario

Staff working in physical stores will receive training aimed at providing "considerate customer service."

situation

When customers ask questions about products, staff will respond appropriately.

Feedback

If staff are feeling stressed, the system will provide advice to encourage relaxation.

Example prompt

If the user aims to provide "caring customer service," tell them how to respond to customer questions. If the user is feeling stressed, provide them with advice on how to promote relaxation.

In this way, customer service training can be carried out effectively in physical stores.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The user puts on the smart glasses and starts the system. The smart glasses' camera captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is camera video and audio data, and the output is the captured video and audio data.

Step 2:

The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes their emotional state using EmotionRecognizer. The input is the captured video data, and the output is the user's emotional state.

Step 3:

The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data, and the output is text data.

Step 4:

The server uses IdealBehaviorModel to generate ideal behaviors and statements based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal behavior or statement.

Step 5:

The server uses the TextToSpeech library to convert the generated text of ideal actions and statements into speech. The input is the text of the ideal actions and statements, and the output is audio data.

Step 6:

Audio data sent from the server is provided to the user through the smart glasses' speakers. The input is audio data, and the output is audio feedback heard by the user.

Step 7:

The user modifies their actions or statements based on the provided feedback, and new data is captured again through the smart glasses' camera and microphone. This repeats the process from step 1 to step 6. The input is video and audio data of the modified actions or statements, and the output is new feedback.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Previous systems had issues with being unable to provide effective training because it was difficult for users to customize the specific actions and comments that would bring them closer to their ideal image of the person they wanted, and the training content was not adjusted according to the user's emotional state. Furthermore, the lack of a function to analyze the user's emotional state in real time and adjust the training content based on that analysis made it difficult to bring users closer to their ideal image while reducing the psychological burden.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, and means for adjusting the training content in accordance with the emotional state. This allows the user to receive effective training to approach their ideal image.

"Ideal person" refers to the type of person or personality traits that the user should aim to become.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to devices or software used to notify users of custom-changed actions or statements.

"Means for repeated training" refers to a program or system that allows users to repeatedly practice custom-changed actions and utterances.

"Means for analyzing emotional state in real time" refers to devices or software that estimate a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to algorithms or programs that change the content or method of training according to the user's emotional state.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

System Configuration

Users define their ideal image of a person.

Using an application on their device, users can set the ideal person they aspire to be. For example, they can select "a compassionate person." This setting takes into account the user's interpersonal trauma, education, etc.

The server collects user behavior and comments.

The server collects user actions and statements in real time, including text and voice data entered by the user. For example, it collects what users say to friends in chat apps and what they say to voice assistants.

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses natural language processing (NLP) technology to understand what the user is saying. It uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

The server customizes your actions and speech based on your ideal image of a person.

Based on the analysis results, the server custom-changes the user's actions and comments to match their ideal image of a person. For example, "I'm busy today, so I can't help you" could be changed to "I'm busy today, but please let me know if there's anything I can do to help."

The server presents the custom changes to the user.

The server presents the custom-changed actions and comments to the user, who can review them and make any necessary corrections.

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, choice of words, etc. It uses Microsoft Azure's Emotion API and Affectiva's SDK.

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it will provide training to promote relaxation, and if the user is feeling joy, it will provide training to encourage behavior that encourages sharing that joy.

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to get closer to their ideal self.

Specific examples

Establishing the ideal human image

The user opens the application, selects "Set your ideal person," and chooses "A compassionate person."

Collecting actions and statements

The application collects chat content between users and friends in the background.

Data Analysis

The chat content collected by the server is analyzed using the Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what the user is saying and generate appropriate responses.

Custom Change Example

If the user says, "I'm busy today and can't help," the server will change this to "I'm busy today, but please let me know if there's anything I can do to help."

Emotion Engine Analysis

When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

Adjusting training content

When a user is feeling stressed, the server provides relaxation training such as "Take a deep breath and relax."

Running the training

The user performs relaxation training presented by the server.

Example prompt

"If a user sets their ideal person as 'a considerate person,' how would they customize their behavior and speech?"

This system allows users to receive effective training to bring them closer to their ideal self.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

Users define their ideal image of a person.

The user launches the application on their device and sets their ideal person. Specifically, they select "Set ideal person" from the application menu and choose their ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is that setting information.

Step 2:

The server collects user behavior and comments.

The server collects user behavior and speech in real time. Specifically, it collects the content of user conversations with friends on chat apps and what users say to voice assistants. The input is the user's text data and voice data, and the output is the collected behavior and speech data.

Step 3:

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

Step 4:

The server custom-changes your behavior and speech based on your ideal person.

Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help." The input is the analysis results, and the output is the custom-changed actions and comments.

Step 5:

The server presents the custom changes to the user.

The server presents the custom-changed actions and statements to the user, who can review them and make corrections as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

Step 6:

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

Step 7:

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it provides training to promote relaxation, and if the user is feeling happy, it provides training to encourage behavior that shares that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

Step 8:

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to approach their ideal self. The input is the adjusted training content, and the output is the user's execution results.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In traditional customer service work, it was difficult for staff to maintain an ideal customer service attitude, and customer satisfaction declined, especially when their emotional state was unstable. Furthermore, there was a lack of effective training methods to help staff approach their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This enables staff to receive appropriate training according to their emotional state while maintaining an ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to a display device or notification system that notifies users of custom-changed actions and statements.

"Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to become closer to their ideal image of the person they want to be.

An "emotion engine" is software or hardware that analyzes a user's emotional state in real time.

"Means for adjusting training content" is a function for changing the content of training provided to users based on the analysis results of the emotion engine.

"Smart glasses" are wearable devices that can visually display information when worn by the user.

The system for implementing this invention includes a series of means for projecting the ideal human image set by the user, customizing the user's behavior and speech, and repeatedly training them. It also has the ability to use an emotion engine to analyze the user's emotional state in real time and adjust the training content.

Hardware and software used

Hardware: Smart glasses, camera

Software: OpenCV (face detection), EmotionRecognizer (emotion recognition), BehaviorSuggestion (behavior suggestions)

System Configuration

1. A way to project the user's ideal image of a person:

Users can set their ideal image of a person through the application. For example, if their ideal is "kind and polite customer service," the system will use this information to customize their behavior and speech.

2. How to customise user behaviour and comments:

The system runs algorithms to change the user's current behavior and speech based on the ideal person. For example, if the user is feeling stressed, it will make suggestions to encourage relaxation.

3. A method for presenting custom-changed actions and statements to users and repeatedly training them:

Custom-changed actions and speech are displayed on the smart glasses' display. Users can view these and repeatedly practice their ideal actions and speech.

4. Emotion Engine:

The camera captures the user's facial expressions, and EmotionRecognizer analyzes their emotional state in real time. This allows the system to provide training content tailored to the user's emotional state.

5. Ways to adjust training content:

Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior that shares that joy.

6. Display method on smart glasses:

Suggested actions and comments are displayed in real time on the smart glasses' display, allowing users to maintain ideal behavior while serving customers.

Specific examples

Scenario: Store staff wear smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

Emotional state: If staff are feeling stressed, offer relaxation suggestions (e.g., "Take a deep breath and relax").

Action suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

Example prompt

Develop an application that custom changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function that uses an emotion engine to analyze the user's emotional state and provide appropriate training.

The flow of the specific processing in Application Example 2 will be explained using Figure 18.

Step 1:

The user launches the application and sets their ideal image of the person. The user inputs an ideal image such as "kind and polite customer service." This input data is sent to the server and saved.

Step 2:

The server runs an algorithm that custom-changes behavior and speech based on the user's ideal image of the person. It receives the ideal image of the person and the user's current behavioral data as input, and generates custom-changed behavior and speech as output.

Step 3:

The server sends the custom-changed actions and statements to the smart glasses, which receive them and display them on the screen. The user then views these and practices their ideal actions and statements.

Step 4:

The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data, and generates emotional state data as output.

Step 5:

The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine receives the emotional state data and the ideal human image as input and suggests appropriate actions and statements. The adjusted training content is generated as output.

Step 6:

The server sends the adjusted training content to the smart glasses, which receive it and display it on the screen. The user then views it and practices training that corresponds to their emotional state.

Step 7:

As the user practices the training, the server tracks changes in their emotional state and readjusts the training content as needed. This allows the user to receive optimal training to bring them closer to their ideal image of the person they want to be.

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

"Example 2"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user sets their ideal person in the application. For example, they can set "a compassionate person" as their ideal person.

Step 2: The system custom-changes the user's behavior and comments based on the ideal image of a person. Specifically, it analyzes the user's behavior and comments based on the image of a "compassionate person" and generates appropriate behavior and comments.

Step 3: The system presents the user with the customized actions and statements. For example, it may provide advice to encourage the user to behave more considerately toward others.

Step 4: The user repeatedly practices the suggested actions and statements, allowing them to get closer to their ideal self.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Existing self-improvement systems have difficulty customizing specific actions and speech based on the ideal person the user sets, and lack a means to effectively record the progress of this repetitive training and provide feedback. This poses a challenge in that they do not provide effective support to help users get closer to their ideal self.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image using a generative AI model, means for presenting the custom-changed actions and speech to the user and repeatedly training them, and means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and speech that will bring them closer to their ideal image, and effectively manage their progress.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom-change user behavior and speech.

"Custom Change" refers to changing a user's behavior and words based on an ideal image of a person.

"Repeated training" refers to the user repeatedly practicing the custom-changed actions and utterances that are presented to them.

"Feedback" refers to evaluations and advice provided based on the user's progress through repeated training.

"Recording progress" refers to saving the content and results of the user's repeated training as data.

"Presenting" refers to showing the custom-changed actions and statements generated by the server to the user.

This invention is a system that custom-changes actions and speech based on the ideal person image set by the user, and encourages the user to approach their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: A device operated by the user, such as a smartphone, computer, or tablet.

Server: A server for processing data and running generative AI models.

Software

Application: Software that provides an interface for users to define their ideal person and review their custom-changed behavior and statements.

Generative AI model: An algorithm that custom-changes a user's behavior and speech based on an ideal human image. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of a person. For example, they can set "a compassionate person" as their ideal image of a person.

The device sends configuration data to the server.

The device sends data about the ideal person set by the user to the server. This data includes information about the ideal person selected by the user.

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects data on the user's past behavior and comments and customizes it based on the ideal image of the person.

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This data includes specific examples of behaviors and speech that the user should practice.

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Users perform repetitive training.

Users practice the proposed custom-changed actions and utterances, practicing repeatedly. The application records the user's progress and provides feedback as needed.

Specific examples

For example, if a user sets "a compassionate person" as their ideal person, the following prompt sentence would be input into the generative AI model:

Example prompt

"Please provide advice to help users be more considerate when speaking with friends."

Based on this prompt, the generative AI model generates specific example utterances that the user can use when conversing with friends. For example, it provides advice such as, "When a friend is in trouble, say, 'Let me know if there's anything I can help you with.'"

In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they click the "Set Ideal Person" button on the application interface, select "Compassionate Person" from the drop-down menu, and press the "Set" button.

Input: User's chosen ideal person (e.g., "a caring person")

Output: Ideal human image configuration data

Step 2:

The device sends the configuration data to the server.

The device sends data about the user's ideal person profile to a server. This transmission is secure using the HTTPS protocol.

Input: Ideal human image data

Output: Configuration data sent to the server

Step 3:

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and social media posts) and customizes it based on the ideal image of the person.

Input: Configuration data, user past behavior and comment data

Output: Custom-changed behavior and speech data

Step 4:

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

Input: Custom-changed behavior and speech data

Output: Examples of specific actions and statements generated

Step 5:

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This transmission is also done using the HTTPS protocol.

Input: Examples of specific actions or statements to be generated

Output: Data sent to the device

Step 6:

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Input: Data sent to the device

Output: Examples of specific actions and statements presented to the user

Step 7:

Users engage in repetitive training.

The user then practices the custom-changed actions and statements presented to them, repeatedly practicing. The application records the user's progress and provides feedback as needed. For example, a user might enter into the application, "Today, I made a kind statement to a friend."

Input: User practice results

Output: Recorded progress data and provided feedback

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional customer service training systems had the problem of being unable to provide effective training because it was difficult to customize specific actions and comments to help staff develop the ideal customer service attitude. Furthermore, there was a lack of repetitive training to help staff approach the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for generating actions and utterances based on the ideal human image using a generative AI model, and means for presenting the generated actions and utterances to the user as prompt sentences. This enables staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repeated training.

A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a person.

The "ideal person" is the specific character and attitude that the user should strive for, and serves as the standard by which the system will custom-change their behavior and speech.

"Projection means" is a function that displays the user's ideal image of a person visually or as text on the system.

"Means for custom change" is a function that allows users to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" is a function for displaying custom-changed actions and comments to users.

"Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and statements.

A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

A "prompt" is a text-based instruction that presents the user with actions and statements generated by the generative AI model based on an ideal human image.

The system for implementing this invention custom-changes behavior and speech based on the ideal person image set by the user, and presents this to the user for repetitive training. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A device such as a smartphone or tablet that allows the user to define their ideal person and check their custom-changed behavior and statements.

2. Server: Processes user input data and uses a generative AI model to generate behaviors and statements based on the ideal human image.

3. Generative AI models: Using generative AI models such as OpenAI's GPT-3, actions and statements are generated based on the ideal human image set by the user.

Program processing

The server receives the ideal human image and current behavior and speech data sent from the user's device and inputs this as prompt text into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

Hardware and software used

Hardware: Smartphones, tablets, servers

Software: Python, OpenAI API, user interface application

Data processing and calculation

The server sends the user's ideal image of a person and their current actions and statements as prompts to the generative AI model. The generative AI model generates the ideal actions and statements based on the prompts and returns the results to the server. The server then sends the generated actions and statements to the user's device and presents them to the user.

Specific examples

For example, if the user defines their ideal person as "attentive customer service" and their current behavior is "treating customers in a rude manner," the prompt would be as follows:

Example prompt:

Ideal person: Attentive customer service

User behavior: Treating customers rudely

Acting based on the ideal human image:

Based on this prompt, the generative AI model generates actions and statements appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and statements presented, users can get closer to their ideal customer service attitude.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user sets their ideal image of a person using their device. The user launches the application and inputs their ideal image of a person (e.g., "thoughtful customer service"). The input data is sent from the device to the server.

Input: Ideal human image

Output: Ideal human image data sent to the server

Step 2:

The server receives the ideal human image data sent by the user. The server stores this data and uses it for subsequent processing.

Input: Ideal human image data

Output: Saved ideal human image data

Step 3:

The user uses the device to input their current actions and statements. The user inputs their current actions (e.g., "being rude to customers") and sends it from the device to the server.

Input: Current action or statement

Output: Current behavior and speech data sent to the server

Step 4:

The server receives the current activity and comment data sent by the user. The server stores this data and uses it for further processing.

Input: Current activity and speech data

Output: Saved current behavior and speech data

Step 5:

The server generates prompts by combining the stored ideal human image data with current behavior and speech data. The generated prompts are then input into the generative AI model.

Input: Ideal person data, current behavior and speech data

Output: Generated prompt text

Step 6:

The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

Input: Generated prompt text

Output: Ideal actions and statements generated by the generative AI model

Step 7:

The server receives the ideal actions and statements returned by the generative AI model. The server then sends this data to the user's device.

Input: Ideal actions and statements generated by a generative AI model

Output: Ideal behavior and speech data sent to the user's device

Step 8:

The user's device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

Input: Ideal behavior and speech data

Output: Ideal actions and statements presented to the user

Through these processing steps, users learn how to behave and speak in a way that is based on the ideal image of a person, and through repeated training, they can move closer to their ideal attitude.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires sustained effort. Furthermore, there is a lack of specific guidelines for actions and statements for self-improvement, making effective training difficult. Furthermore, there are no appropriate means for users to customize their actions and statements based on their ideal image, making it difficult for users to become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and comments based on the ideal human image, means for presenting the custom-changed actions and comments to the user, means for the user to repeatedly train on the presented actions and comments, means for collecting the user's actions and comments, and means for analyzing the collected actions and comments and customizing them using a generative AI model. This allows the user to obtain specific guidelines for actions and comments, and enables them to get closer to their ideal image through continuous training.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

"Projection means" refers to the function that displays the user's ideal image of the person in visual or text form.

"Means for custom change" refers to the function that allows a user to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" refers to the function for notifying or displaying custom-changed actions or statements to the user.

"Means for repeated training" refers to a function that allows users to train themselves to become closer to their ideal image of a person by repeatedly performing suggested actions and statements.

"Means of collection" refers to the function for recording user actions and comments and saving them as data.

"Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on the ideal image of a person.

A "generative AI model" refers to an artificial intelligence model that converts a user's actions and statements based on an ideal image of a human being.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, presenting this to the user for repetitive training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The device used by the user, such as a smartphone, tablet, or computer.

Server: A remote server for processing and storing data.

Software

Application: Software that allows users to define their ideal person and check their custom-changed behavior and statements.

Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user behavior and speech based on an ideal human image.

System Operation

Establishing the ideal human image

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Collecting user behavior and comments

The device records the user's everyday actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

Customize your actions and speech

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), it custom-changes the user's behavior and speech based on the ideal person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to, "I'm busy today, but please let me know if there's anything I can do to help."

Displaying custom actions and statements

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Repetitive training

Users repeatedly execute the custom-changed actions and statements that are presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, users train to become closer to their ideal image of the person they want to be.

Specific examples

Establishing the ideal human image

The user enters "compassionate person" in the application's settings screen and clicks the save button.

Collecting user behavior and comments

The device records the user's voice when they are talking to friends and converts it into text. It also collects text data that the user posts on social media.

Customize your actions and speech

The server inputs the collected text data into a generative AI model, which uses prompts to convert the user's speech based on an ideal human image.

Displaying custom actions and statements

The server sends the custom-modified utterances output by the generative AI model to the device. The device then displays a notification to the user, suggesting something like, "I'm busy today, but please let me know if there's anything I can do to help you."

Repetitive training

Users use the comments suggested by the server in their everyday conversations and social media posts. For example, in a conversation with a friend, they might say, "I'm busy today, but please let me know if there's anything I can do to help." By repeating this, users will naturally begin to act and speak in a way that befits a "considerate person."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

Current user statement: "I'm busy today, so I can't help."

Ideal person: "A compassionate person"

Prompt for generative AI model: "Transform the user's utterances into those of a considerate person."

By inputting this prompt into a generative AI model, the user's speech is custom-changed based on the ideal human image.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Input: The ideal person image entered by the user on the settings screen.

Output: The ideal image of a person is saved in the application.

Step 2:

The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media.

Input: User actions and statements.

Output: Collected behavioral and speech data is stored on the device and sent to the server in real time.

Step 3:

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person.

Input: Collected behavioral and speech data, ideal human image.

Output: Custom-changed actions and statements.

Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompt sentences.

Step 4:

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Input: Custom-changed actions and statements.

Output: The custom-changed actions and statements presented to the user.

Specific operation: The device displays the custom-changed message received from the server as a notification.

Step 5:

The user repeatedly executes the custom-changed actions and statements presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts.

Input: The proposed custom-changed actions and statements.

Output: Custom-changed actions and statements performed by the user.

Specific operation: The user uses statements suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to their ideal image of the person they want to be.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In traditional customer service work, there were limited training methods for staff to acquire ideal customer service attitudes, making effective training difficult. Furthermore, there was a lack of specific guidelines or feedback for staff to put ideal customer service attitudes into practice, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient way to provide repeated training to help staff become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for training the ideal customer service attitude using smart glasses worn by staff performing customer service duties, and means for generating custom-changed utterances based on the ideal human image using a generative AI model. This enables staff to efficiently train in and practice the ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to a device or software that visually or audibly displays the ideal human image set by the user.

"Means for custom change" refers to a device or software that changes a user's behavior and speech based on an ideal image of a person.

"Presentation means" refers to a device or software that displays custom-changed actions and statements to the user.

"Means for repeated training" refers to a device or software that allows the user to repeatedly practice custom-changed actions or utterances.

"Customer service" refers to the work of providing products and services to customers.

"Smart glasses" are glasses-type devices that have the function of displaying information.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text and behavior.

"Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

The system for implementing this invention projects the ideal image of a person set by the user, custom-changes the user's actions and speech, and repeatedly trains them. Specifically, it can train ideal customer service attitudes using smart glasses worn by staff performing customer service duties.

System Configuration

The system consists of the following main components:

1. User device: The device on which the user sets their ideal image of the person. This includes smartphones and tablets.

2. Smart glasses: A device worn by staff performing customer service duties. It has the function of visually displaying information.

3. Server: Runs the generative AI model and processes data to custom change user behavior and speech.

4. Generative AI model: An algorithm that generates custom-modified statements based on the user's ideal image of the person.

Program processing

The server receives the user's ideal image of the person and custom-changes the user's behavior and speech based on that image. Specifically, the process is carried out as follows:

1. Setting the ideal person: Using the user device, the user sets the ideal person. For example, they can set "kind and polite customer service" as their ideal image.

2. Enter user utterances: Enter actual utterances made by the user. For example, "Welcome, are you looking for something?"

3. Execution of the generative AI model: The server uses the generative AI model to generate custom-modified utterances based on the ideal human image.

4. Present customized speech: Present customized speech to the user through smart glasses. For example, "Welcome, we'll help you find the product you're looking for."

5. Repetitive training: Users repeatedly practice the custom-modified utterances presented to them.

Hardware and software used

Hardware: Smart glasses (e.g., Google Glass), user devices (smartphones, tablets)

Software: Generative AI models (e.g., OpenAI API), data processing servers

Specific examples

For example, let's say the ideal customer service staff member is "friendly and courteous service," and use the following prompt:

Example prompt:

Ideal personality: Kind and courteous customer service

User says: Welcome, are you looking for something?

Custom-changed statements based on the ideal person:

By inputting this prompt into the generative AI model, a custom-modified utterance based on the ideal customer service attitude is generated. For example, a utterance such as, "Welcome, we'll help you find the product you're looking for" is generated.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user sets their ideal image of the person.

Input: The user uses a device (smartphone or tablet) to input their ideal image of a person. For example, "kind and polite customer service."

Data processing: The device sends the input image of the ideal person to the server.

Output: The ideal image of a person is saved on the server.

Step 2:

The user enters the actual statement.

Input: The user uses the device to type what they actually say. For example, "Welcome, is there anything I can help you with?"

Data processing: The device sends the entered comments to the server.

Output: User comments are saved on the server.

Step 3:

The server runs the generative AI model.

Input: The server retrieves the stored ideal person image and user statements.

Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt text is input into the generative AI model to obtain the custom-changed utterances.

Output: The generated custom-changed comments are saved on the server.

Step 4:

Present the custom-changed statement to the user.

Input: The server receives the generated custom-changed statement.

Data processing: The server sends the custom-modified speech to the smart glasses.

Output: The custom-changed speech is displayed on the smart glasses.

Step 5:

Users perform repetitive training.

Input: The user sees the custom-changed speech displayed on the smart glasses.

Specific behavior: The user repeatedly practices the displayed utterance. For example, they practice saying, "Welcome, we'd love to help you find the product you're looking for."

Output: The user's customer service attitude becomes closer to the ideal person.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

"Example 2"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user starts the system and sets their ideal self. This ideal self specifically describes how the user will behave and what they will say.

Step 2: The emotion engine analyzes the user's emotional state in real time. Specifically, it infers the user's emotional state from their facial expressions, tone of voice, choice of words, etc.

Step 3: The emotion engine tracks changes in the user's emotional state. This allows the content of the repetitive training to help the user approach their ideal self to be tailored to the user's emotional state.

Step 4: The user undergoes repeated training to move closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that shares that joy.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Previous systems had the problem that it was difficult to customise behaviour and speech based on the ideal image of the person set by the user, and the training content was not adjusted according to the user's emotional state, resulting in a lack of effective support to help users get closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and means for adjusting the training content based on data obtained from the emotion engine. This enables effective support for the user to get closer to their ideal image.

"User" refers to an individual who uses the system to aim to become closer to their ideal image of a person.

An "ideal person" is a specific set of personality and behavioral characteristics that a user aspires to.

"Projection means" refers to the function that displays the user's ideal image of a person in visual or text form on the system.

"Means for customizing" refers to the ability to change and adjust a user's behavior and comments based on their ideal image of a person.

"Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

"Means for repeated training" refers to a function that allows users to repeatedly practice based on the custom changes presented to them.

The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to the function that changes and adjusts training content according to the user's emotional state based on data obtained from the emotion engine.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The smartphone, tablet, or PC used by the user.

Server: Cloud server for data processing and storage.

Camera and microphone: Devices that capture your facial expressions and tone of voice in real time.

Software

Application: Provides an interface that allows users to define their ideal person and review their custom-changed behavior and statements.

Emotion engine: A software module for analyzing a user's emotional state in real time.

Generative AI model: An AI model that custom-changes a user's behavior and speech based on an ideal human image.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of the person. For example, they can select "a compassionate person."

The server collects user behavior data.

The server collects input data, voice data, and text messages from users' smartphones and PCs in real time, including what users chat with friends and what they say to their voice assistants.

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. For example, if a user says "thank you" infrequently, the server will suggest that they say "thank you" more frequently.

The server presents custom-changed actions and comments to the user.

The server then uses the application's notification function to present the custom-changed actions and comments to the user. For example, it could send a notification such as, "Try saying 'Thank you' the next time you talk to a friend."

Users perform repetitive training.

Based on suggestions from the server, users are made conscious of saying "thank you" more often in their daily lives. By repeating this, users become closer to becoming "considerate people."

The emotion engine analyzes the user's emotional state.

The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

The server adjusts the training content.

The server uses data obtained from the emotion engine to adjust the training content according to the user's emotional state. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

Specific examples

Defining the ideal person

The user opens the application and sets "a compassionate person" as their ideal person.

Custom Change Example

The server suggests that users say "thank you" more frequently when conversing with friends.

Emotion Engine Analysis

If a user smiles during a conversation, the emotion engine will determine that the user is feeling happy.

Adjusting training content

If the user is feeling stressed, the server will provide deep breathing and meditation training.

Example prompt

"If the user sets their ideal person as 'a considerate person,' please suggest what actions and statements they would like to customize."

The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they select their ideal person from options such as "a considerate person" on the application's settings screen and save the settings. The input is the ideal person selected by the user, and the output is data for the set ideal person.

Step 2:

The server collects user behavior data.

The server collects input data, voice data, and text messages from the user's smartphone or PC in real time. Specifically, it collects what the user chats with friends and what they say to their voice assistant. The input is user behavioral data, and the output is a database of the collected behavioral data.

Step 3:

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and speech based on the ideal person. For example, if the user says "thank you" infrequently, the server will suggest that they say "thank you" more often. The input is the collected behavioral data, and the output is suggested custom-changed behaviors and speech.

Step 4:

The server presents custom-changed actions and comments to the user.

The server presents the custom-changed actions and utterances to the user using the application's notification function. Specifically, it sends a notification such as, "Try saying 'thank you' the next time you talk to a friend." The input is a suggestion for a custom-changed action or utterance, and the output is a notification to the user.

Step 5:

Users engage in repetitive training.

Based on suggestions from the server, users are made conscious to say "thank you" more frequently in their daily lives. Specifically, they increase the number of opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

Step 6:

Emotion engine analyzes the user's emotional state

The emotion engine uses the camera and microphone on the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is data on the analyzed emotional state.

Step 7:

The server adjusts the training content.

The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. Specifically, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and training content was not adjusted to reflect the emotional state of staff, making it difficult to provide effective training.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, means for adjusting the training content according to the user's emotional state, and means for providing feedback to the user using a smart device. This enables staff to effectively acquire the ideal customer service attitude.

A "user" is an individual who uses the system to train themselves to behave and speak in a way that reflects their ideal image of a person.

The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

"Projection means" refers to a method or device for visually or audibly presenting the ideal human image set by the user.

"Means for custom change" refers to methods or devices for changing a user's behavior and speech based on an ideal image of a person.

"Means for repeated training" refers to methods or devices that allow users to repeatedly practice custom-changed actions and speech in order to become closer to their ideal image of the person.

"Means for analyzing emotional states in real time" refers to methods or devices for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to a method or device for changing the training content according to the user's emotional state.

A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

"Means for providing feedback" refers to methods or devices for providing users with advice or instructions regarding ideal behavior or speech.

As an embodiment of this invention, we provide a training system that helps staff working in brick-and-mortar stores acquire ideal customer service attitudes. The system includes means for projecting an ideal image of a person set by the user, customizing the user's behavior and speech, and repeatedly training them. It also includes means for analyzing the user's emotional state in real time and adjusting the training content.

Hardware and software used

Hardware

Smart glasses: Equipped with a camera, microphone, and speaker, they capture the user's actions and speech and provide feedback.

Server: A central processing unit for data processing and analysis.

Software

OpenCV: A library for capturing and processing camera images.

EmotionRecognizer: A library that estimates emotions from facial expressions.

SpeechRecognition: Speech recognition library.

TextToSpeech: A library for converting text to speech.

IdealBehaviorModel: A model that generates ideal behaviors and statements.

Data processing and calculations

Camera footage capture

The camera in the smart glasses is used to capture the user's facial expressions and behavior in real time. OpenCV is used to acquire video data, and EmotionRecognizer is used to analyze the emotional state.

Voice recognition

Use the microphone in the smart glasses to capture what the user is saying. Use the SpeechRecognition library to convert the speech data into text.

Feedback generation

Using IdealBehaviorModel, ideal behaviors and utterances are generated based on the user's emotional state and utterances. The generated feedback is converted into speech using the TextToSpeech library and provided to the user through the smart glasses' speakers.

Specific examples

Scenario

Staff working in physical stores will receive training aimed at providing "considerate customer service."

situation

When customers ask questions about products, staff will respond appropriately.

Feedback

If staff are feeling stressed, the system will provide advice to encourage relaxation.

Example prompt

If the user aims to provide "caring customer service," tell them how to respond to customer questions. If the user is feeling stressed, provide them with advice on how to promote relaxation.

In this way, customer service training can be carried out effectively in physical stores.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The user puts on the smart glasses and starts the system. The smart glasses' camera captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is camera video and audio data, and the output is the captured video and audio data.

Step 2:

The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes their emotional state using EmotionRecognizer. The input is the captured video data, and the output is the user's emotional state.

Step 3:

The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data, and the output is text data.

Step 4:

The server uses IdealBehaviorModel to generate ideal behaviors and statements based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal behavior or statement.

Step 5:

The server uses the TextToSpeech library to convert the generated text of ideal actions and statements into speech. The input is the text of the ideal actions and statements, and the output is audio data.

Step 6:

Audio data sent from the server is provided to the user through the smart glasses' speakers. The input is audio data, and the output is audio feedback heard by the user.

Step 7:

The user modifies their actions or statements based on the provided feedback, and new data is captured again through the smart glasses' camera and microphone. This repeats the process from step 1 to step 6. The input is video and audio data of the modified actions or statements, and the output is new feedback.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Previous systems had issues with being unable to provide effective training because it was difficult for users to customize the specific actions and comments that would bring them closer to their ideal image of the person they wanted, and the training content was not adjusted according to the user's emotional state. Furthermore, the lack of a function to analyze the user's emotional state in real time and adjust the training content based on that analysis made it difficult to bring users closer to their ideal image while reducing the psychological burden.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, and means for adjusting the training content in accordance with the emotional state. This allows the user to receive effective training to approach their ideal image.

"Ideal person" refers to the type of person or personality traits that the user should aim to become.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to devices or software used to notify users of custom-changed actions or statements.

"Means for repeated training" refers to a program or system that allows users to repeatedly practice custom-changed actions and utterances.

"Means for analyzing emotional state in real time" refers to devices or software that estimate a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to algorithms or programs that change the content or method of training according to the user's emotional state.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

System Configuration

Users define their ideal image of a person.

Using an application on their device, users can set the ideal person they aspire to be. For example, they can select "a compassionate person." This setting takes into account the user's interpersonal trauma, education, etc.

The server collects user behavior and comments.

The server collects user actions and statements in real time, including text and voice data entered by the user. For example, it collects what users say to friends in chat apps and what they say to voice assistants.

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses natural language processing (NLP) technology to understand what the user is saying. It uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

The server customizes your actions and speech based on your ideal image of a person.

Based on the analysis results, the server custom-changes the user's actions and comments to match their ideal image of a person. For example, "I'm busy today, so I can't help you" could be changed to "I'm busy today, but please let me know if there's anything I can do to help."

The server presents the custom changes to the user.

The server presents the custom-changed actions and comments to the user, who can review them and make any necessary corrections.

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, choice of words, etc. It uses Microsoft Azure's Emotion API and Affectiva's SDK.

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it will provide training to promote relaxation, and if the user is feeling joy, it will provide training to encourage behavior that encourages sharing that joy.

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to get closer to their ideal self.

Specific examples

Establishing the ideal human image

The user opens the application, selects "Set your ideal person," and chooses "A compassionate person."

Collecting actions and statements

The application collects chat content between users and friends in the background.

Data Analysis

The chat content collected by the server is analyzed using the Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what the user is saying and generate appropriate responses.

Custom Change Example

If the user says, "I'm busy today and can't help," the server changes this to "I'm busy today, but please let me know if there's anything I can do to help."

Emotion Engine Analysis

When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

Adjusting training content

When a user is feeling stressed, the server provides relaxation training such as "Take a deep breath and relax."

Running the training

The user performs relaxation training presented by the server.

Example prompt

"If a user sets their ideal person as 'a considerate person,' how would they customize their actions and words?"

This system allows users to receive effective training to bring them closer to their ideal self.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

Users define their ideal image of a person.

The user launches the application on their device and sets their ideal person. Specifically, they select "Set ideal person" from the application menu and choose their ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is that setting information.

Step 2:

The server collects user behavior and comments.

The server collects user behavior and speech in real time. Specifically, it collects the content of user conversations with friends on chat apps and what users say to voice assistants. The input is the user's text data and voice data, and the output is the collected behavior and speech data.

Step 3:

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

Step 4:

The server custom-changes your behavior and speech based on your ideal person.

Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help." The input is the analysis results, and the output is the custom-changed actions and comments.

Step 5:

The server presents the custom changes to the user.

The server presents the custom-changed actions and statements to the user, who can review them and make corrections as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

Step 6:

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

Step 7:

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it provides training to promote relaxation, and if the user is feeling happy, it provides training to encourage behavior that shares that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

Step 8:

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to approach their ideal self. The input is the adjusted training content, and the output is the user's execution results.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In traditional customer service work, it was difficult for staff to maintain an ideal customer service attitude, and customer satisfaction declined, particularly when their emotional state was unstable. Furthermore, there was a lack of effective training methods to help staff approach their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This enables staff to receive appropriate training according to their emotional state while maintaining an ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to a display device or notification system that notifies users of custom-changed actions and statements.

"Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to become closer to their ideal image of the person they want to be.

An "emotion engine" is software or hardware that analyzes a user's emotional state in real time.

"Means for adjusting training content" is a function for changing the content of training provided to users based on the analysis results of the emotion engine.

"Smart glasses" are wearable devices that can visually display information when worn by the user.

The system for implementing this invention includes a series of means for projecting the ideal human image set by the user, customizing the user's behavior and speech, and repeatedly training them. It also has the ability to use an emotion engine to analyze the user's emotional state in real time and adjust the training content.

Hardware and software used

Hardware: Smart glasses, camera

Software: OpenCV (face detection), EmotionRecognizer (emotion recognition), BehaviorSuggestion (behavior suggestions)

System Configuration

1. A way to project the user's ideal image of a person:

Users can set their ideal image of a person through the application. For example, if their ideal is "kind and polite customer service," the system will use this information to customize their behavior and speech.

2. How to customise user behaviour and comments:

The system runs algorithms to change the user's current behavior and speech based on the ideal person. For example, if the user is feeling stressed, it will make suggestions to encourage relaxation.

3. A method for presenting custom-changed actions and statements to users and repeatedly training them:

Custom-changed actions and speech are displayed on the smart glasses' display. Users can view these and repeatedly practice their ideal actions and speech.

4. Emotion Engine:

The camera captures the user's facial expressions and EmotionRecognizer analyzes their emotional state in real time. This allows the system to provide training content tailored to the user's emotional state.

5. Ways to adjust training content:

Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior that shares that joy.

6. Display method on smart glasses:

Suggested actions and comments are displayed in real time on the smart glasses' display, allowing users to maintain ideal behavior while serving customers.

Specific examples

Scenario: Store staff wear smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

Emotional state: If staff are feeling stressed, offer relaxation suggestions (e.g., "Take a deep breath and relax").

Action suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

Example prompt

Develop an application that custom changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function that uses an emotion engine to analyze the user's emotional state and provide appropriate training.

The flow of the specific processing in Application Example 2 is explained using Figure 18.

Step 1:

The user launches the application and sets their ideal image of the person. The user inputs an ideal image such as "kind and polite customer service." This input data is sent to the server and saved.

Step 2:

The server runs an algorithm that custom-changes behavior and speech based on the user's ideal image of the person. It receives the ideal image of the person and the user's current behavioral data as input, and generates custom-changed behavior and speech as output.

Step 3:

The server sends the custom-changed actions and statements to the smart glasses, which receive them and display them on the screen. The user then views these and practices their ideal actions and statements.

Step 4:

The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data, and generates emotional state data as output.

Step 5:

The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine receives the emotional state data and the ideal human image as input and suggests appropriate actions and statements. The adjusted training content is generated as output.

Step 6:

The server sends the adjusted training content to the smart glasses, which receive it and display it on the screen. The user then views it and practices training that corresponds to their emotional state.

Step 7:

As the user practices the training, the server tracks changes in their emotional state and readjusts the training content as needed. This allows the user to receive optimal training to bring them closer to their ideal image of the person they want to be.

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

"Example 2"

The system of the present invention includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them. Specifically, the user sets an ideal human image on the application. For example, if the user sets a "compassionate person" as their ideal human image, the system custom-changes the user's actions and speech based on the "compassionate person." The system then presents the custom-changed actions and speech to the user and repeatedly trains them, encouraging the user to approach their ideal image.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user sets their ideal person in the application. For example, they can set "a compassionate person" as their ideal person.

Step 2: The system custom-changes the user's behavior and comments based on the ideal image of a person. Specifically, it analyzes the user's behavior and comments based on the image of a "compassionate person" and generates appropriate behavior and comments.

Step 3: The system presents the user with the customized actions and statements. For example, it may provide advice to encourage the user to behave more considerately toward others.

Step 4: The user repeatedly practices the suggested actions and statements, allowing them to get closer to their ideal self.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Existing self-improvement systems have difficulty customizing specific actions and speech based on the ideal person the user sets, and lack a means to effectively record the progress of this repetitive training and provide feedback. This poses a challenge in that they do not provide effective support to help users get closer to their ideal self.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image using a generative AI model, means for presenting the custom-changed actions and speech to the user and repeatedly training them, and means for recording the progress of the repeated training and providing feedback. This enables the user to learn and practice specific actions and speech that will bring them closer to their ideal image, and effectively manage their progress.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate data and custom-change user behavior and speech.

"Custom Change" refers to changing a user's behavior and words based on an ideal image of a person.

"Repeated training" refers to the user repeatedly practicing the custom-changed actions and utterances that are presented to them.

"Feedback" refers to evaluations and advice provided based on the user's progress through repeated training.

"Recording progress" refers to saving the content and results of the user's repeated training as data.

"Presenting" refers to showing the custom-changed actions and statements generated by the server to the user.

This invention is a system that custom-changes actions and speech based on the ideal person image set by the user, and encourages the user to approach their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: A device operated by the user, such as a smartphone, computer, or tablet.

Server: A server for processing data and running generative AI models.

Software

Application: Software that provides an interface for users to define their ideal person and review their custom-changed behavior and statements.

Generative AI model: An algorithm that custom-changes a user's behavior and speech based on an ideal human image. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of a person. For example, they can set "a considerate person" as their ideal image of a person.

The device sends configuration data to the server.

The device sends data about the ideal person set by the user to the server. This data includes information about the ideal person selected by the user.

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received data to analyze the user's behavior and comments. Specifically, the server collects data on the user's past behavior and comments and customizes it based on the ideal image of the person.

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This data includes specific examples of behaviors and speech that the user should practice.

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Users perform repetitive training.

Users practice the proposed custom-changed actions and utterances, practicing repeatedly. The application records the user's progress and provides feedback as needed.

Specific examples

For example, if a user sets "a compassionate person" as their ideal person, the following prompt sentence would be input into the generative AI model:

Example prompt

"Please provide advice to help users be more considerate when speaking with friends."

Based on this prompt, the generative AI model generates specific example utterances that the user can use when conversing with friends. For example, it provides advice such as, "When a friend is in trouble, say, 'Let me know if there's anything I can help you with.'"

In this way, users can learn and practice behaviors and speech based on their ideal image of a person.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they click the "Set Ideal Person" button on the application interface, select "Compassionate Person" from the drop-down menu, and press the "Set" button.

Input: User's chosen ideal person (e.g., "a caring person")

Output: Ideal human image configuration data

Step 2:

The device sends the configuration data to the server.

The device sends data about the user's ideal person profile to a server. This transmission is secure using the HTTPS protocol.

Input: Ideal human image data

Output: Configuration data sent to the server

Step 3:

The server analyzes the data using a generative AI model.

The server uses a generative AI model based on the received configuration data to analyze the user's behavior and comments. Specifically, the server collects the user's past behavior and comment data (e.g., chat logs and social media posts) and customizes it based on the ideal image of the person.

Input: Configuration data, user past behavior and comment data

Output: Custom-changed behavior and speech data

Step 4:

The server generates custom-modified actions and statements.

The server uses a generative AI model to generate actions and statements based on the user's ideal image of a person. For example, it generates specific examples of actions and statements that would be expected of a "compassionate person."

Input: Custom-changed behavior and speech data

Output: Examples of specific actions and statements generated

Step 5:

The server sends the generated data to the device.

The server then sends the generated custom-changed behavior and speech data to the device. This transmission is also done using the HTTPS protocol.

Input: Examples of specific actions or statements to be generated

Output: Data sent to the device

Step 6:

The device presents the data to the user.

The device then presents the received data to the user, who can then view specific examples of the custom-changed actions and statements through the application interface.

Input: Data sent to the device

Output: Examples of specific actions and statements presented to the user

Step 7:

Users engage in repetitive training.

The user then practices the custom-changed actions and statements presented to them, repeatedly practicing. The application records the user's progress and provides feedback as needed. For example, a user might enter into the application, "Today, I made a kind statement to a friend."

Input: User practice results

Output: Recorded progress data and provided feedback

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional customer service training systems had the problem of being unable to provide effective training because it was difficult to customize specific actions and comments to help staff develop the ideal customer service attitude. Furthermore, there was a lack of repetitive training to help staff approach the ideal customer service attitude, making it difficult for them to maintain the ideal attitude when actually serving customers.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for generating actions and utterances based on the ideal human image using a generative AI model, and means for presenting the generated actions and utterances to the user as prompt sentences. This enables staff to concretely understand the ideal customer service attitude and maintain the ideal attitude in actual customer service through repeated training.

A "user" is an individual who uses the system to learn to behave and speak in a way that is consistent with the ideal image of a person.

The "ideal person" is the specific character and attitude that the user should strive for, and serves as the standard by which the system will custom-change their behavior and speech.

"Projection means" is a function that displays the user's ideal image of a person visually or as text on the system.

"Means for custom change" is a function that allows users to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" is a function for displaying custom-changed actions and comments to users.

"Means for repeated training" is a function that allows users to repeatedly practice custom-changed actions and statements.

A "generative AI model" is an artificial intelligence model that generates actions and statements based on the ideal human image set by the user.

A "prompt" is a text-based instruction that presents the user with actions and statements generated by the generative AI model based on an ideal human image.

The system for implementing this invention custom-changes behavior and speech based on the ideal person image set by the user, and presents this to the user for repetitive training. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A device such as a smartphone or tablet that allows the user to define their ideal person and check their custom-changed behavior and statements.

2. Server: Processes user input data and uses a generative AI model to generate behaviors and statements based on the ideal human image.

3. Generative AI models: Using generative AI models such as OpenAI's GPT-3, actions and statements are generated based on the ideal human image set by the user.

Program processing

The server receives the ideal human image and current behavior and speech data sent from the user's device and inputs this as prompt text into the generative AI model. The generative AI model generates behavior and speech based on the ideal human image and returns the results to the server. The server then sends the generated behavior and speech to the user's device and presents them to the user.

Hardware and software used

Hardware: Smartphones, tablets, servers

Software: Python, OpenAI API, user interface application

Data processing and calculation

The server sends the user's ideal image of a person and their current actions and statements as prompts to the generative AI model. The generative AI model generates the ideal actions and statements based on the prompts and returns the results to the server. The server then sends the generated actions and statements to the user's device and presents them to the user.

Specific examples

For example, if the user defines their ideal person as "attentive customer service" and their current behavior is "treating customers in an unfriendly manner," the prompt would be as follows:

Example prompt:

Ideal person: Attentive customer service

User behavior: Treating customers rudely

Acting based on the ideal human image:

Based on this prompt, the generative AI model generates actions and statements appropriate for "considerate customer service" and presents the results to the user. By repeatedly practicing the actions and statements presented, users can get closer to their ideal customer service attitude.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user sets their ideal person image using their device. The user launches the application and inputs their ideal person image (e.g., "thorough customer service"). The input data is sent from the device to the server.

Input: Ideal human image

Output: Ideal human image data sent to the server

Step 2:

The server receives the ideal human image data sent by the user. The server stores this data and uses it for subsequent processing.

Input: Ideal human image data

Output: Saved ideal human image data

Step 3:

The user uses the device to input their current actions and statements. The user inputs their current actions (e.g., "being rude to customers") and sends it from the device to the server.

Input: Current action or statement

Output: Current behavior and speech data sent to the server

Step 4:

The server receives the current activity and comment data sent by the user. The server stores this data and uses it for further processing.

Input: Current activity and speech data

Output: Saved current behavior and speech data

Step 5:

The server generates prompts by combining the stored ideal human image data with current behavior and speech data. The generated prompts are then input into the generative AI model.

Input: Ideal person data, current behavior and speech data

Output: Generated prompt text

Step 6:

The server sends the generated prompt text to the generative AI model, which then generates the ideal behavior or speech based on the prompt text.

Input: Generated prompt text

Output: Ideal actions and statements generated by the generative AI model

Step 7:

The server receives the ideal actions and statements returned by the generative AI model. The server then sends this data to the user's device.

Input: Ideal actions and statements generated by a generative AI model

Output: Ideal behavior and speech data sent to the user's device

Step 8:

The user's device receives the ideal behavior and speech data sent from the server and presents it to the user. The user confirms the presented behavior and speech and performs repeated training.

Input: Ideal behavior and speech data

Output: Ideal actions and statements presented to the user

Through these processing steps, users learn how to behave and speak in a way that is based on the ideal image of a person, and through repeated training, they can move closer to their ideal attitude.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In modern society, many people aim to improve themselves and become closer to their ideal image, but the process is difficult and requires sustained effort. Furthermore, there is a lack of specific guidelines for actions and statements for self-improvement, making effective training difficult. Furthermore, there are no appropriate means for users to customize their actions and statements based on their ideal image, making it difficult for users to become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and comments based on the ideal human image, means for presenting the custom-changed actions and comments to the user, means for the user to repeatedly train on the presented actions and comments, means for collecting the user's actions and comments, and means for analyzing the collected actions and comments and customizing them using a generative AI model. This allows the user to obtain specific guidelines for actions and comments, and enables them to get closer to their ideal image through continuous training.

"Ideal person" refers to the specific person or personality traits that a user should aim for.

"Projection means" refers to the function that displays the user's ideal image of the person in visual or text form.

"Means for custom change" refers to the function that allows a user to change their current behavior and comments based on their ideal image of the person.

"Means of presentation" refers to the function for notifying or displaying custom-changed actions or statements to the user.

"Means for repeated training" refers to a function that allows users to train themselves to become closer to their ideal image of a person by repeatedly performing suggested actions and statements.

"Means of collection" refers to the function for recording user actions and comments and saving them as data.

"Means of analysis" refers to the function of analyzing collected user behavior and comment data and extracting information for custom changes based on the ideal image of a person.

A "generative AI model" refers to an artificial intelligence model that converts a user's actions and statements based on an ideal image of a human being.

This invention is a system that custom-changes actions and speech based on the ideal person image set by the user, and presents this to the user for repetitive training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The device used by the user, such as a smartphone, tablet, or computer.

Server: A remote server for processing and storing data.

Software

Application: Software that allows users to define their ideal person and check their custom-changed behavior and statements.

Generative AI model: An artificial intelligence model (e.g., GPT-4) that transforms user behavior and speech based on an ideal human image.

System Operation

Establishing the ideal human image

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Collecting user behavior and comments

The device records the user's everyday actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media. The collected data is sent to a server in real time.

Customize your actions and speech

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), it custom-changes the user's behavior and speech based on the ideal person. For example, if a user says, "I'm busy today, so I can't help you," the generative AI model converts this to, "I'm busy today, but please let me know if there's anything I can do to help."

Displaying custom actions and statements

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Repetitive training

Users repeatedly execute the custom-changed actions and statements that are presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts. In this way, users train to become closer to their ideal image of the person they want to be.

Specific examples

Establishing the ideal human image

The user enters "compassionate person" in the application's settings screen and clicks the save button.

Collecting user behavior and comments

The device records the user's voice when they are talking to friends and converts it into text. It also collects text data that the user posts on social media.

Customize your actions and speech

The server inputs the collected text data into a generative AI model, which uses prompts to convert the user's speech based on an ideal human image.

Displaying custom actions and statements

The server sends the custom-modified utterances output by the generative AI model to the device. The device then displays a notification to the user, suggesting something like, "I'm busy today, but please let me know if there's anything I can do to help you."

Repetitive training

Users use the suggestions from the server in their everyday conversations and social media posts. For example, in a conversation with a friend, they might say, "I'm busy today, but please let me know if there's anything I can do to help." By repeating this, users will naturally begin to act and speak in a way that befits a "considerate person."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

Current user statement: "I'm busy today, so I can't help."

Ideal person: "A compassionate person"

Prompt for generative AI model: "Transform the user's utterances into those of a considerate person."

By inputting this prompt into a generative AI model, the user's speech is custom-changed based on the ideal human image.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user launches the application and goes to the settings screen. There, they enter their ideal image of a person. For example, they might enter "a considerate person."

Input: The ideal person image entered by the user on the settings screen.

Output: The ideal image of a person is saved in the application.

Step 2:

The device records the user's daily actions and comments. For example, it collects the content of the user's conversations with friends and the content posted on social media.

Input: User actions and statements.

Output: Collected behavioral and speech data is stored on the device and sent to the server in real time.

Step 3:

The server analyzes the received user behavior and speech data. Using a generative AI model (e.g., GPT-4), the user's behavior and speech are custom-changed based on the ideal image of a person.

Input: Collected behavioral and speech data, ideal human image.

Output: Custom-changed actions and statements.

Specific operation: The server inputs the collected text data into the generative AI model and converts the user's utterances using prompts.

Step 4:

The server sends the custom-changed actions and statements to the device. The device notifies the user. The user confirms the notification and actually uses the custom-changed actions and statements.

Input: Custom-changed actions and statements.

Output: The custom-changed actions and statements presented to the user.

Specific operation: The device displays the custom-changed message received from the server as a notification.

Step 5:

The user repeatedly executes the custom-changed actions and statements presented to them. For example, they use statements suggested by the generative AI model in their daily conversations and social media posts.

Input: The proposed custom-changed actions and statements.

Output: Custom-changed actions and statements performed by the user.

Specific operation: The user uses statements suggested by the server in conversations with friends and in social media posts. By repeating this process, the user trains to become closer to their ideal image of the person they want to be.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In traditional customer service work, there were limited training methods for staff to acquire ideal customer service attitudes, making effective training difficult. Furthermore, there was a lack of specific guidelines or feedback for staff to put ideal customer service attitudes into practice, making it difficult to improve customer service quality. Furthermore, there was a need for an efficient way to provide repeated training to help staff become closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and utterances based on the ideal human image, means for presenting the custom-changed actions and utterances to the user and repeatedly training them, means for training the ideal customer service attitude using smart glasses worn by staff performing customer service duties, and means for generating custom-changed utterances based on the ideal human image using a generative AI model. This enables staff to efficiently train in and practice the ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to a device or software that visually or audibly displays the ideal human image set by the user.

"Means for custom change" refers to a device or software that changes a user's behavior and speech based on an ideal image of a person.

"Presentation means" refers to a device or software that displays custom-changed actions and statements to the user.

"Means for repeated training" refers to a device or software that allows the user to repeatedly practice custom-changed actions or utterances.

"Customer service" refers to the work of providing products and services to customers.

"Smart glasses" are glasses-type devices that have the function of displaying information.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text and behavior.

"Custom-changed statements" are statements made by users that have been altered based on their ideal image of a person.

The system for implementing this invention projects the ideal image of a person set by the user, custom-changes the user's actions and speech, and repeatedly trains them. Specifically, it can train ideal customer service attitudes using smart glasses worn by staff performing customer service duties.

System Configuration

The system consists of the following main components:

1. User device: The device on which the user sets their ideal image of the person. This includes smartphones and tablets.

2. Smart glasses: A device worn by staff performing customer service duties. It has the function of visually displaying information.

3. Server: Runs the generative AI model and processes data to custom change user behavior and speech.

4. Generative AI model: An algorithm that generates custom-modified statements based on the user's ideal image of the person.

Program processing

The server receives the user's ideal image of the person and custom-changes the user's behavior and speech based on that image. Specifically, the process is carried out as follows:

1. Setting the ideal person: Using the user device, the user sets the ideal person. For example, they can set "kind and polite customer service" as their ideal image.

2. Enter user utterances: Enter actual utterances made by the user. For example, "Welcome, are you looking for something?"

3. Execution of the generative AI model: The server uses the generative AI model to generate custom-modified utterances based on the ideal human image.

4. Present customized speech: Present customized speech to the user through smart glasses. For example, "Welcome, we'll help you find the product you're looking for."

5. Repetitive training: Users repeatedly practice the custom-modified utterances presented to them.

Hardware and software used

Hardware: Smart glasses (e.g., Google Glass), user devices (smartphones, tablets)

Software: Generative AI models (e.g., OpenAI API), data processing servers

Specific examples

For example, let's say the ideal customer service staff member is "friendly and courteous service," and use the following prompt:

Example prompt:

Ideal personality: Kind and courteous customer service

User says: Welcome, are you looking for something?

Custom-changed statements based on the ideal person:

By inputting this prompt into the generative AI model, a custom-modified utterance based on the ideal customer service attitude is generated. For example, a utterance such as, "Welcome, we'll help you find the product you're looking for" is generated.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user sets their ideal image of the person.

Input: The user uses a device (smartphone or tablet) to input their ideal image of a person. For example, "kind and polite customer service."

Data processing: The device sends the input image of the ideal person to the server.

Output: The ideal image of a person is saved on the server.

Step 2:

The user enters the actual statement.

Input: The user uses the device to type what they actually say. For example, "Welcome, is there anything I can help you with?"

Data processing: The device sends the entered comments to the server.

Output: User comments are saved on the server.

Step 3:

The server runs the generative AI model.

Input: The server retrieves the stored ideal person image and user statements.

Data calculation: The server uses the generative AI model to generate custom-changed utterances based on the ideal human image. The prompt text is input into the generative AI model to obtain the custom-changed utterances.

Output: The generated custom-changed comments are saved on the server.

Step 4:

Present the custom-changed statement to the user.

Input: The server receives the generated custom-changed statement.

Data processing: The server sends the custom-modified speech to the smart glasses.

Output: The custom-changed speech is displayed on the smart glasses.

Step 5:

Users perform repetitive training.

Input: The user sees the custom-changed speech displayed on the smart glasses.

Specific behavior: The user repeatedly practices the displayed utterance. For example, they practice saying, "Welcome, we'd love to help you find the product you're looking for."

Output: The user's customer service attitude becomes closer to the ideal person.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

"Example 2"

One embodiment of the present invention incorporates an emotion engine. This emotion engine analyzes the user's emotional state in real time. Specifically, it infers the emotional state from the user's facial expressions, tone of voice, choice of words, etc. The emotion engine also tracks changes in the user's emotional state. This makes it possible to adjust the content of repetitive training to help the user approach their ideal self in accordance with the user's emotional state. For example, when the user is feeling stressed, training that encourages relaxation is provided. On the other hand, when the user is feeling joy, training that encourages behavior that shares that joy is provided. This allows the user to train to approach their ideal self in a way that is appropriate for their emotional state.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user boots up the system and sets their ideal image. This ideal image is the user's

It specifically shows what actions a user will take and what they will say.

Step 2: The emotion engine analyzes the user's emotional state in real time. Specifically, it infers the user's emotional state from their facial expressions, tone of voice, choice of words, etc.

Step 3: The emotion engine tracks changes in the user's emotional state. This allows the content of the repetitive training to help the user approach their ideal self to be tailored to the user's emotional state.

Step 4: The user undergoes repeated training to move closer to their ideal self. This training is adjusted according to the user's emotional state. For example, when the user is feeling stressed, training is provided to promote relaxation. On the other hand, when the user is feeling joy, training is provided to encourage behavior that shares that joy.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Previous systems had the problem that it was difficult to customise behaviour and speech based on the ideal image of the person set by the user, and the training content was not adjusted according to the user's emotional state, resulting in a lack of effective support to help users get closer to their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, and means for adjusting the training content based on data obtained from the emotion engine. This enables effective support for the user to get closer to their ideal image.

"User" refers to an individual who uses the system to aim to become closer to their ideal image of a person.

An "ideal person" is a specific set of personality and behavioral characteristics that a user aspires to.

"Projection means" refers to the function that displays the user's ideal image of the person on the system in visual or text format.

"Means for customizing" refers to the ability to change and adjust a user's behavior and comments based on their ideal image of a person.

"Means of presentation" refers to the function of notifying or displaying custom-changed actions or comments to the user.

"Means for repeated training" refers to a function that allows users to repeatedly practice based on the custom changes presented to them.

The "emotion engine" refers to a function that analyzes a user's emotional state in real time based on their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to the function that changes and adjusts training content according to the user's emotional state based on data obtained from the emotion engine.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

Hardware and software used

Hardware

Device: The smartphone, tablet, or PC used by the user.

Server: Cloud server for data processing and storage.

Camera and microphone: Devices that capture your facial expressions and tone of voice in real time.

Software

Application: Provides an interface that allows users to define their ideal person and review their custom-changed behavior and statements.

Emotion engine: A software module for analyzing a user's emotional state in real time.

Generative AI model: An AI model that custom-changes a user's behavior and speech based on an ideal human image.

Data processing and calculation

Users define their ideal image of a person.

The user launches the application and sets their ideal image of the person. For example, they can select "a compassionate person."

The server collects user behavior data.

The server collects input data, voice data, and text messages from users' smartphones and PCs in real time, including what users chat with friends and what they say to their voice assistants.

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. For example, if a user says "thank you" infrequently, the server will suggest that they say "thank you" more frequently.

The server presents custom-changed actions and comments to the user.

The server then uses the application's notification function to present the custom-changed actions and comments to the user. For example, it could send a notification such as, "Try saying 'Thank you' the next time you talk to a friend."

Users perform repetitive training.

Based on suggestions from the server, users are made conscious of saying "thank you" more often in their daily lives. By repeating this, users become closer to becoming "considerate people."

The emotion engine analyzes the user's emotional state.

The emotion engine uses the user's smartphone camera and microphone to analyze facial expressions and tone of voice in real time. For example, if a user is smiling while speaking, the emotion engine will determine that the user is feeling happy.

The server adjusts the training content.

The server uses data obtained from the emotion engine to adjust the training content according to the user's emotional state. For example, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax."

Specific examples

Defining the ideal person

The user opens the application and sets "a compassionate person" as their ideal person.

Custom Change Example

The server suggests that users say "thank you" more frequently when conversing with friends.

Emotion Engine Analysis

If a user smiles during a conversation, the emotion engine will determine that the user is feeling happy.

Adjusting training content

If the user is feeling stressed, the server will provide deep breathing and meditation training.

Example prompt

"If the user sets their ideal person as 'a considerate person,' please suggest what actions and statements they would like to customize."

The system supports users' growth by providing them with specific actions and statements to help them become closer to their ideal self, and by providing training tailored to their emotional state.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1:

Users define their ideal image of a person.

The user launches the application and sets their ideal person. Specifically, they select their ideal person from options such as "a considerate person" on the application's settings screen and save the settings. The input is the ideal person selected by the user, and the output is data for the set ideal person.

Step 2:

The server collects user behavior data.

The server collects input data, voice data, and text messages from the user's smartphone or PC in real time. Specifically, it collects what the user chats with friends and what they say to their voice assistant. The input is user behavioral data, and the output is a database of the collected behavioral data.

Step 3:

The server analyzes the data and performs custom changes.

The server analyzes the collected data using natural language processing (NLP) technology. Specifically, it uses a generative AI model to change the user's behavior and speech based on the ideal person. For example, if the user says "thank you" infrequently, the server will suggest that they say "thank you" more often. The input is the collected behavioral data, and the output is suggested custom-changed behaviors and speech.

Step 4:

The server presents custom-changed actions and comments to the user.

The server presents the custom-changed actions and utterances to the user using the application's notification function. Specifically, it sends a notification such as, "Try saying 'thank you' the next time you talk to a friend." The input is a suggestion for a custom-changed action or utterance, and the output is a notification to the user.

Step 5:

Users engage in repetitive training.

Based on suggestions from the server, users are made conscious to say "thank you" more frequently in their daily lives. Specifically, they increase the number of opportunities to say "thank you" in conversations with friends and family. The input is the suggestions from the server, and the output is the user's actual behavior.

Step 6:

Emotion engine analyzes the user's emotional state

The emotion engine uses the camera and microphone on the user's smartphone to analyze facial expressions and tone of voice in real time. Specifically, if the user is smiling and talking, the emotion engine determines that the user is feeling happy. The input is the user's facial expressions and tone of voice, and the output is data on the analyzed emotional state.

Step 7:

The server adjusts the training content.

The server adjusts the training content according to the user's emotional state based on the data obtained from the emotion engine. Specifically, if the user is feeling stressed, the server will provide training such as "Take a deep breath and relax." The input is the analyzed emotional state data, and the output is the adjusted training content.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional customer service training systems lacked specific feedback to help staff develop ideal customer service attitudes, and training content was not adjusted to reflect the emotional state of staff, making it difficult to provide effective training.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, means for adjusting the training content according to the user's emotional state, and means for providing feedback to the user using a smart device. This enables staff to effectively acquire the ideal customer service attitude.

A "user" is an individual who uses the system to train themselves to behave and speak in a way that reflects their ideal image of a person.

The "ideal person" is a model of specific behaviors and attitudes that users should aim for.

"Projection means" refers to a method or device for visually or audibly presenting the ideal human image set by the user.

"Means for custom change" refers to methods or devices for changing a user's behavior and speech based on an ideal image of a person.

"Means for repeated training" refers to methods or devices that allow users to repeatedly practice custom-changed actions and speech in order to become closer to their ideal image of the person.

"Means for analyzing emotional state in real time" refers to methods or devices for inferring emotions from a user's facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to a method or device for changing the training content according to the user's emotional state.

A "smart device" is a device that has a built-in camera, microphone, speaker, etc. and is used to provide feedback to the user.

"Means for providing feedback" refers to methods or devices for providing users with advice or instructions regarding ideal behavior or speech.

As an embodiment of this invention, we provide a training system that helps staff working in brick-and-mortar stores acquire ideal customer service attitudes. The system includes means for projecting an ideal image of a person set by the user, customizing the user's behavior and speech, and repeatedly training them. It also includes means for analyzing the user's emotional state in real time and adjusting the training content.

Hardware and software used

Hardware

Smart glasses: Equipped with a camera, microphone, and speaker, they capture the user's actions and speech and provide feedback.

Server: A central processing unit for data processing and analysis.

Software

OpenCV: A library for capturing and processing camera images.

EmotionRecognizer: A library that estimates emotions from facial expressions.

SpeechRecognition: Speech recognition library.

TextToSpeech: A library for converting text to speech.

IdealBehaviorModel: A model that generates ideal behaviors and statements.

Data processing and calculations

Camera footage capture

The camera in the smart glasses is used to capture the user's facial expressions and behavior in real time. OpenCV is used to acquire video data, and EmotionRecognizer is used to analyze the emotional state.

Voice recognition

Use the microphone in the smart glasses to capture what the user is saying. Use the SpeechRecognition library to convert the speech data into text.

Feedback generation

Using IdealBehaviorModel, ideal behaviors and utterances are generated based on the user's emotional state and utterances. The generated feedback is converted into speech using the TextToSpeech library and provided to the user through the smart glasses' speakers.

Specific examples

Scenario

Staff working in physical stores will receive training aimed at providing "considerate customer service."

situation

When customers ask questions about products, staff will respond appropriately.

Feedback

If staff are feeling stressed, the system will provide advice to encourage relaxation.

Example prompt

If the user aims to provide "caring customer service," tell them how to respond to customer questions. If the user is feeling stressed, provide them with advice on how to promote relaxation.

In this way, customer service training can be carried out effectively in physical stores.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The user puts on the smart glasses and starts the system. The smart glasses' camera captures the user's facial expressions and actions in real time, and the microphone captures the user's speech. The input is camera video and audio data, and the output is the captured video and audio data.

Step 2:

The server uses OpenCV to process the video data sent from the smart glasses. Specifically, it extracts the user's facial expressions from the video data and analyzes their emotional state using EmotionRecognizer. The input is the captured video data, and the output is the user's emotional state.

Step 3:

The server uses the SpeechRecognition library to convert the voice data sent from the smart glasses into text. The input is the captured voice data, and the output is text data.

Step 4:

The server uses IdealBehaviorModel to generate ideal behaviors and statements based on the user's emotional state and text data. The input is the user's emotional state and text data, and the output is the text of the ideal behavior or statement.

Step 5:

The server uses the TextToSpeech library to convert the generated text of ideal actions and statements into speech. The input is the text of the ideal actions and statements, and the output is audio data.

Step 6:

Audio data sent from the server is provided to the user through the smart glasses' speakers. The input is audio data, and the output is audio feedback heard by the user.

Step 7:

The user modifies their actions or statements based on the provided feedback, and new data is captured again through the smart glasses' camera and microphone. This repeats the process from step 1 to step 6. The input is video and audio data of the modified actions or statements, and the output is new feedback.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Previous systems had issues with being unable to provide effective training because it was difficult for users to customize the specific actions and comments that would bring them closer to their ideal image of the person they wanted, and the training content was not adjusted according to the user's emotional state. Furthermore, the lack of a function to analyze the user's emotional state in real time and adjust the training content based on that analysis made it difficult to bring users closer to their ideal image while reducing the psychological burden.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for projecting an ideal human image set by the user, means for customizing the user's actions and speech based on the ideal human image, means for presenting the custom-changed actions and speech to the user and repeatedly training them, means for analyzing the user's emotional state in real time, and means for adjusting the training content in accordance with the emotional state. This allows the user to receive effective training to approach their ideal image.

"Ideal person" refers to the type of person or personality traits that the user should aim to become.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to devices or software used to notify users of custom-changed actions or statements.

"Means for repeated training" refers to a program or system that allows users to repeatedly practice custom-changed actions and utterances.

"Means for analyzing emotional state in real time" refers to devices or software that estimate a user's emotional state by analyzing their facial expressions, tone of voice, choice of words, etc.

"Means for adjusting training content" refers to algorithms or programs that change the content or method of training according to the user's emotional state.

This invention is a system that custom-changes behavior and speech based on the ideal person image set by the user, and helps the user get closer to their ideal image through repeated training. Specific embodiments of this system are described below.

System Configuration

Users define their ideal image of a person.

Using an application on their device, users can set the ideal person they aspire to be. For example, they can select "a compassionate person." This setting takes into account the user's interpersonal trauma, education, etc.

The server collects user behavior and comments.

The server collects user actions and statements in real time, including text and voice data entered by the user. For example, it collects what users say to friends in chat apps and what they say to voice assistants.

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses natural language processing (NLP) technology to understand what the user is saying. It uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4.

The server customizes your actions and speech based on your ideal image of a person.

Based on the analysis results, the server custom-changes the user's actions and comments to match their ideal image of a person. For example, "I'm busy today, so I can't help you" could be changed to "I'm busy today, but please let me know if there's anything I can do to help."

The server presents the custom changes to the user.

The server presents the custom-changed actions and comments to the user, who can review them and make any necessary corrections.

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it analyzes the user's facial expressions, tone of voice, choice of words, etc. It uses Microsoft Azure's Emotion API and Affectiva's SDK.

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it will provide training to promote relaxation, and if the user is feeling joy, it will provide training to encourage behavior that encourages sharing that joy.

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to get closer to their ideal self.

Specific examples

Establishing the ideal human image

The user opens the application, selects "Set your ideal person," and chooses "A compassionate person."

Collecting actions and statements

The application collects chat content between users and friends in the background.

Data Analysis

The chat content collected by the server is analyzed using the Google Cloud Natural Language API to perform sentiment analysis. It also uses OpenAI's GPT-4 to understand what the user is saying and generate appropriate responses.

Custom Change Example

If the user says, "I'm busy today and can't help," the server changes this to "I'm busy today, but please let me know if there's anything I can do to help."

Emotion Engine Analysis

When a user speaks into the camera, the server uses Microsoft Azure's Emotion API to determine whether the user is feeling stressed based on their facial expressions and tone of voice.

Adjusting training content

When a user is feeling stressed, the server provides relaxation training such as "Take a deep breath and relax."

Running the training

The user performs relaxation training presented by the server.

Example prompt

"If a user sets their ideal person as 'a considerate person,' how would they customize their behavior and speech?"

This system allows users to receive effective training to bring them closer to their ideal self.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

Users define their ideal image of a person.

The user launches the application on their device and sets their ideal person. Specifically, they select "Set ideal person" from the application menu and choose their ideal person from options such as "a considerate person." The input is the ideal person selected by the user, and the output is that setting information.

Step 2:

The server collects user behavior and comments.

The server collects user behavior and speech in real time. Specifically, it collects the content of user conversations with friends on chat apps and what users say to voice assistants. The input is the user's text data and voice data, and the output is the collected behavior and speech data.

Step 3:

Analyze the data collected by the server

The server analyzes the collected data. Specifically, it uses generative AI models such as Google Cloud Natural Language API and OpenAI's GPT-4 to understand what users say and perform sentiment analysis and key phrase extraction. The input is the collected behavioral and utterance data, and the output is the analysis results.

Step 4:

The server custom-changes your behavior and speech based on your ideal person.

Based on the analysis results, the server custom-changes the user's actions and comments to match the ideal person. For example, "I'm busy today, so I can't help you" is changed to "I'm busy today, but please let me know if there's anything I can do to help." The input is the analysis results, and the output is the custom-changed actions and comments.

Step 5:

The server presents the custom changes to the user.

The server presents the custom-changed actions and statements to the user, who can review them and make corrections as necessary. The input is the custom-changed actions and statements, and the output is the content presented to the user.

Step 6:

The server uses an emotion engine to analyze the user's emotional state.

The server uses an emotion engine to analyze the user's emotional state in real time. Specifically, it uses Microsoft Azure's Emotion API and Affectiva's SDK to analyze the user's facial expressions, tone of voice, choice of words, etc. The input is the user's facial expressions, tone of voice, and choice of words, and the output is the analysis of the user's emotional state.

Step 7:

The server adjusts the training content based on the user's emotional state.

The server adjusts the training content according to the user's emotional state. For example, if the user is feeling stressed, it provides training to promote relaxation, and if the user is feeling happy, it provides training to encourage behavior that shares that joy. The input is the analysis result of the emotional state, and the output is the adjusted training content.

Step 8:

User performs training

The user performs the training content presented by the server. This allows the user to undergo repeated training to approach their ideal self. The input is the adjusted training content, and the output is the user's execution results.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In traditional customer service work, it was difficult for staff to maintain an ideal customer service attitude, and customer satisfaction declined, particularly when their emotional state was unstable. Furthermore, there was a lack of effective training methods to help staff approach their ideal image.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes a means for projecting an ideal human image set by the user, a means for customizing the user's actions and speech based on the ideal human image, a means for presenting the custom-changed actions and speech to the user and repeatedly training them, an emotion engine for analyzing the user's emotional state in real time, a means for adjusting the training content based on the emotion engine, and a means for displaying on the smart glasses. This enables staff to receive appropriate training according to their emotional state while maintaining an ideal customer service attitude.

"User" means an individual or organization that uses the system.

The "ideal person" is a model of behavior and attitude that users should aim for.

"Projection means" refers to devices or software that visually or audibly display the ideal human image set by the user.

"Means for custom change" refers to algorithms or programs that change a user's behavior and speech based on an ideal image of a person.

"Means of presentation" refers to a display device or notification system that notifies users of custom-changed actions and statements.

"Means for repeated training" is a function that allows users to repeatedly practice their custom-changed actions and statements in order to become closer to their ideal image of the person they want to be.

An "emotion engine" is software or hardware that analyzes a user's emotional state in real time.

"Means for adjusting training content" is a function for changing the content of training provided to users based on the analysis results of the emotion engine.

"Smart glasses" are wearable devices that can visually display information when worn by the user.

The system for implementing this invention includes a series of means for projecting the ideal human image set by the user, customizing the user's behavior and speech, and repeatedly training them. It also has the ability to use an emotion engine to analyze the user's emotional state in real time and adjust the training content.

Hardware and software used

Hardware: Smart glasses, camera

Software: OpenCV (face detection), EmotionRecognizer (emotion recognition), BehaviorSuggestion (behavior suggestions)

System Configuration

1. A way to project the user's ideal image of a person:

Users can set their ideal image of a person through the application. For example, if their ideal is "kind and polite customer service," the system will use this information to customize their behavior and speech.

2. How to customise user behaviour and comments:

The system runs algorithms to change the user's current behavior and speech based on the ideal person. For example, if the user is feeling stressed, it will make suggestions to encourage relaxation.

3. A method for presenting custom-changed actions and statements to users and repeatedly training them:

Custom-changed actions and speech are displayed on the smart glasses' display. Users can view these and repeatedly practice their ideal actions and speech.

4. Emotion Engine:

The camera captures the user's facial expressions and EmotionRecognizer analyzes their emotional state in real time. This allows the system to provide training content tailored to the user's emotional state.

5. Ways to adjust training content:

Based on the analysis results of the emotion engine, the BehaviorSuggestion engine suggests appropriate actions and statements. For example, if the user is feeling happy, it provides training to encourage behavior that shares that joy.

6. Display method on smart glasses:

Suggested actions and comments are displayed in real time on the smart glasses' display, allowing users to maintain ideal behavior while serving customers.

Specific examples

Scenario: Store staff wear smart glasses to help them maintain a "friendly and courteous" attitude while serving customers.

Emotional state: If staff are feeling stressed, offer relaxation suggestions (e.g., "Take a deep breath and relax").

Action suggestion: Suggest a kind statement to customers such as, "Please let me know if you need any help."

Example prompt

Develop an application that custom changes actions and comments in real time based on the ideal customer service attitude set by the user, and displays them on the smart glasses display. Include a function that uses an emotion engine to analyze the user's emotional state and provide appropriate training.

The flow of the specific processing in Application Example 2 is explained using Figure 18.

Step 1:

The user launches the application and sets their ideal image of the person. The user inputs an ideal image such as "kind and polite customer service." This input data is sent to the server and saved.

Step 2:

The server runs an algorithm that custom-changes behavior and speech based on the user's ideal image of the person. It receives the ideal image of the person and the user's current behavioral data as input, and generates custom-changed behavior and speech as output.

Step 3:

The server sends the custom-changed actions and statements to the smart glasses, which receive them and display them on the screen. The user then views these and practices their ideal actions and statements.

Step 4:

The camera captures the user's facial expressions and transmits them to the server in real time. The server uses EmotionRecognizer to analyze the user's emotional state from the input facial expression data, and generates emotional state data as output.

Step 5:

The server adjusts the training content based on the emotional state data. The BehaviorSuggestion engine receives the emotional state data and the ideal human image as input and suggests appropriate actions and statements. The adjusted training content is generated as output.

Step 6:

The server sends the adjusted training content to the smart glasses, which receive it and display it on the screen. The user then views it and practices training that corresponds to their emotional state.

Step 7:

As the user practices the training, the server tracks changes in their emotional state and readjusts the training content as needed. This allows the user to receive optimal training to bring them closer to their ideal image of the person they want to be.

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged above and below the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the top side of the concentric circles, and the emotion of "discomfort" is arranged on the bottom side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a calm impression.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances such as posture and blood sugar levels, and when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery life, so that when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotion maps may be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and emotional brain physiological signal analysis systems, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions belonging to an area called "reaction," where sensation is dominant. The right half of the emotion map is lined with emotions belonging to an area called "situation," where situational awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may also be performed by multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The following types of processors can be used as hardware resources for executing specific processes. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically for executing specific processes. All processors have built-in or connected memory, and all use the memory to execute specific processes.

The hardware resource that executes the specific processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific processing may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiments.

(Claim 1)
The system includes a means for projecting an ideal human image set by a user, a means for custom-changing the user's actions and speech based on the ideal human image, and a means for presenting the custom-changed actions and speech to the user and repeatedly training them.

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

(Claim 4)
The system according to claim 1 , wherein the means for custom-changing the user's actions and speech based on the ideal human image includes an emotion engine that recognizes the user's emotions.

(Claim 5)
5. The system according to claim 4, wherein the emotion engine analyzes the user's emotional state in real time and custom-changes the user's behavior and speech based on the results of the analysis.

(Claim 6)
5. The system of claim 4, wherein the emotional engine tracks changes in the user's emotional state and adjusts the repetitive training based on the changes.

"Example 1"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image using a generative AI model;
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
means for recording the progress of said repetitive training and providing feedback;
A system including:

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Application Example 1"

(Claim 1)
A means to project the ideal image of a person set by the user,
A way to customize user behavior and speech based on the ideal person,
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
A method for generating actions and statements based on an ideal human image using a generative AI model, and
means for presenting the generated actions or utterances to the user as prompt sentences;
A system including:

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Example 2"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image;
A means for presenting the custom-changed actions and statements to a user;
A means for the user to repeatedly train the suggested actions and utterances;
A means of collecting user actions and comments,
A method to analyze collected actions and comments and use generative AI models to make custom changes.
A system including:

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Application Example 2"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image;
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
A method for training ideal customer service attitudes using smart glasses worn by staff performing customer service duties.
A means for generating custom-modified utterances based on an ideal human profile using a generative AI model; and
A system including:

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Example 1: Combining Emotion Engines"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image;
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
An emotion engine that analyzes the user's emotional state in real time,
a means for adjusting training content based on data obtained from the emotion engine;
A system including:

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Application example 1: Combining emotion engines"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image;
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
A means of analyzing the user's emotional state in real time;
a means for adjusting training content according to the user's emotional state;
a means for providing feedback to the user using a smart device;

A system including

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Example 2: Combining Emotion Engines"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image;
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
A means of analyzing the user's emotional state in real time;
a means for adjusting training content in response to said emotional state;

A system including

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

"Application Example 2: Combining Emotion Engines"

(Claim 1)
A means to project the ideal image of a person set by the user,
A means for customizing the user's actions and comments based on the ideal human image;
A means for presenting the customized behaviors and utterances to the user and repeatedly training them;
An emotion engine that analyzes the user's emotional state in real time,
a means for adjusting training content based on the emotion engine;
a means for displaying on the smart glasses;
A system including:

(Claim 2)
The system according to claim 1 , wherein the ideal person image is set taking into consideration the user's traumatic relationships, education, etc.

(Claim 3)
The system of claim 1 , wherein the repetitive training encourages the user to approach their ideal image.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (3)

means for storing information about an ideal human image set by a user;
A means for collecting at least one of user actions and statements as data;
A means for generating a prompt sentence that generates an instruction to custom change at least one of the user's behavior and speech based on information about the ideal person image set by the user and at least one of the collected data about the user's behavior and speech;
A means for generating a suggested message including at least one of an action and a utterance obtained by custom-changing at least one of the user's action and utterance using the generated prompt sentence and a generation AI model;
and means for presenting the generated suggestion message to the user. means for identifying an emotional state of the user using an emotion engine;
The system of claim 1, wherein the means for generating the prompt sentence generates a prompt sentence that generates instructions to custom change at least one of the user's behavior and utterances based on information about the ideal person image set by the user, collected data about at least one of the user's behavior and utterances, and the identified emotional state of the user. The system of claim 1 , wherein the means for presenting to the user is a display of smart glasses.