JP7825005B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

When using generative AI, the time required to enter prompts is cumbersome, creating a digital divide for people who have difficulty using the service, particularly the elderly.

We provide an accessory device that can be connected to a smartphone. This device includes a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The user authentication means is a fingerprint authentication means, and the operation means is a one-click operation means. This allows prompt input to the generative AI immediately from a smartphone, realizing smarter use of generative AI. It also contributes to bridging the digital divide for people who have difficulty using services, such as the elderly.

1 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 10 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. FIG. 10 is a sequence diagram showing a processing flow of the data processing system in Application Example 1 of Form Example 1. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a third embodiment of the third form example. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used by the processor as work memory.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a symbol is an interface that includes a communication processor, an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be just A, just B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A, a microphone 38B, etc., and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or finger) to receive user input via the indicator. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 is equipped with a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible by the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using a wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the fingerprint information with the registered user's fingerprint. The operation means involves the user pressing a button on the device, which inputs prompts to the generative AI.

"Example 2"

As a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user connects an accessory device that can be connected to a smartphone to the smartphone using a wireless communication method such as Bluetooth.

Step 2: The user places their finger on the fingerprint authentication sensor on the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the registered user's fingerprint information.

Step 3: If authentication is successful, the device's buttons will be enabled.

Step 4: When the user presses a button on the device, an application on the smartphone is launched and prompts are input to the generative AI.

"Example 2"

Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

Step 2: If authentication is successful, the device's buttons will be enabled.

Step 3: When the user presses the button, an application on the smartphone is launched and prompt input is sent to the generative AI.

Step 4: This allows users to easily utilize generative AI without having to operate their smartphone.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In systems that use conventional generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompt text, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience for users when using generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer, and can be used with a variety of functions by installing applications.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

"Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

"Prompt input" is the act of inputting instructions or questions to a generative AI.

"User authentication means" refers to technology or devices used to verify a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and verify the user's identity based on that information.

"Operation means" refers to the interface or device that allows the user to operate the accessory device or smartphone.

A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

An "application" is software that runs on a smartphone and provides specific functions or services.

"Generation results" are output such as text and images generated by the generative AI based on the prompt text.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts into a generative AI model. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment:

Accessory devices are external devices that connect to your smartphone and have the following main functions:

User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

Operation method: Equipped with a one-click button, making it easy for users to operate.

Smartphone:

The smartphone is connected to the accessory device using wireless communication means such as Bluetooth. An application for providing prompt input to the generative AI model is installed on the smartphone.

Generative AI model:

The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

System Operation

User authentication:

The user places their finger on the fingerprint authentication sensor installed on the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

Prompt Input:

After successful authentication, the user presses a button on the accessory device. The smartphone application launches and a prompt input screen appears. The user enters a prompt statement for the generative AI model.

Send to generative AI model:

The smartphone sends the input prompt to the generative AI model. The server operates the generative AI model based on the received prompt and creates the generated results.

Displaying the generated results:

The server sends the generated results to the smartphone. The smartphone displays the received results to the user. The user can check the displayed results and save or share them as needed.

Specific examples

If a user wants to have a generative AI create a new poem, the specific steps are as follows:

1. User authentication:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The accessory device reads the fingerprint and performs authentication.

If authentication is successful, a signal indicating successful authentication will be sent to your smartphone.

2. Prompt input:

The user presses a button on the accessory device.

The smartphone application will launch and a prompt input screen will appear.

The user enters the prompt text "Create a new poem."

3. Send to generative AI model:

The smartphone sends the entered prompt to the generative AI model.

The server runs a generative AI model based on the prompt text to generate the poem.

4. Displaying the generated results:

The server sends the generated poem to your smartphone.

The smartphone will then display the received poem to the user.

Users can review the displayed poems and save or share them as needed.

In this way, users can use the accessory device and smartphone to provide prompt input to the generative AI model and obtain generated results.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Connecting Accessory Devices

Input: User opens their smartphone's Bluetooth settings and selects an accessory device.

How it works: Your smartphone establishes a connection with the accessory device using Bluetooth.

Output: The smartphone notifies you that the connection was successful.

Step 2: User Authentication

Input: The user places their finger on the fingerprint sensor on the accessory device.

How it works: The accessory device reads your fingerprint and compares it with the fingerprint information stored in its internal database.

Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

Step 3: Prepare for prompt input

Input: The smartphone receives a signal indicating successful authentication.

How it works: The smartphone displays a prompt input screen for the generative AI.

Output: The user sees the prompt input screen and begins entering the prompt statement.

Step 4: Enter the prompt

Input: The user enters a prompt for the generative AI through a smartphone application.

How it works: The smartphone receives the entered prompt text and formats the data internally.

Output: The prepared prompt is ready to be sent to the generative AI model.

Step 5: Send to the generative AI model

Input: The smartphone sends the prepared prompt to the generative AI model.

Operation: The server runs the generative AI model based on the received prompt text and creates the generated results.

Output: The generated results are sent from the server to the smartphone.

Step 6: View the generated results

Input: The smartphone receives the generated results from the server.

How it works: The smartphone displays the generated results to the user.

Output: The user checks the displayed generated results and saves or shares them as needed.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient user authentication security, creating the risk of operational errors and unauthorized access. This can potentially reduce factory productivity and safety.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for generating instructions for managing and operating robots in a factory; and means for transmitting the instructions to the robots using wireless communication means. This makes it possible to efficiently and safely manage and operate robots in a factory.

An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

"Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions and questions to the generative AI.

"User authentication means" is a function for verifying and authenticating a user's identity.

"Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

"Operation means" is an interface that allows the user to input prompts to the generative AI.

"One-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

"Means for generating instructions for managing and operating robots within a factory" refers to a function for generating specific work instructions for robots within a factory.

"Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

"Means for transmitting to robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

This invention is a system that efficiently and safely manages and operates robots in a factory using accessory devices that can be connected to a smartphone. A specific embodiment of this system is described below.

System Configuration

The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

Hardware

Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

Accessory device: Connects to a smartphone and includes fingerprint authentication means and operation means.

Fingerprint authentication method: Scans the user's fingerprint for authentication.

Wireless communication method: Data is sent and received between smartphones and factory robots using Bluetooth, Wi-Fi, etc.

Factory robot: Receives instructions and performs actions.

Software

Generative AI model: Generates specific instructions for factory robots based on prompt input from the user.

Smartphone application: Provides the user interface and manages communication with the generative AI model.

Operating Procedure

1. User authentication: The user places their finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

2. Prompt input: An authenticated user uses an operation method (one-click operation method) to input a prompt to the generative AI model.

3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

5. Robot operation: The factory robot operates according to the instructions it receives.

Specific examples

For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions. Below are some example prompts:

"Move to assembly line 3 and begin welding."

"Move to the inspection area to perform quality checks."

"Go get part A and take it to assembly line 2."

This makes it possible to efficiently and safely manage and operate robots within factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user places their finger on the accessory device and is authenticated using the fingerprint authentication means.

Input: User's fingerprint

Data processing: The fingerprint authentication device scans the fingerprint and compares it with registered fingerprint data.

Output: Authentication result (success or failure)

Specific operation: The fingerprint authentication sensor scans the user's fingerprint and compares it with an internal database for authentication.

Step 2:

If authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

Input: Authentication result (success), user prompt

Data processing: If the authentication result is successful, a prompt text is sent to the generative AI model.

Output: Prompt input to the generative AI model

Specific operation: When the user presses the button, the smartphone application sends a prompt statement to the generative AI model.

Step 3:

Based on the prompts, a generative AI model generates specific instructions for the factory robot.

Input: User prompt

Data processing: The generative AI model analyzes the prompt and generates appropriate instructions.

Output: Instructions to factory robots

Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

Step 4:

The generated instructions are sent to the factory robot using wireless communication means.

Input: Instructions from a generative AI model

Data processing: Send instructions to factory robots via wireless communication means.

Output: Send instructions to factory robots

Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

Step 5:

The factory robot will act according to the instructions it receives.

Input: Instructions for factory robots

Data processing: Factory robots analyze instructions and perform corresponding actions.

Output: Robot movement

Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

To use conventional generative AI models, users had to manually operate their smart devices and input prompts. This was time-consuming and inconvenient, especially when their hands were full or when a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. A system that solves these issues and allows users to use generative AI models more easily and safely is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for using the operation means to send a signal to the smart device and launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without having to directly operate the smart device, and the user authentication means also improves security.

"User authentication means" refers to the means used to verify a user's identity and grant access to a system.

"Operation means" refers to the interface or device that allows the user to operate the system.

A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

"Means for transmitting signals" refers to means for transmitting specific instructions or data from the operating means to the smart device.

A "specific application" is a software program designed to provide a specific function or service.

A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

A "prompt" is text input used to give instructions or ask questions to a generative AI model.

"Means for generating a response" refers to the means by which a generative AI model creates an appropriate response based on a prompt.

"Means for sending a response" refers to means for sending the generated response to a smart device.

Form for implementing the invention

This invention is a system that allows users to use generative AI models without directly operating a smart device. Specific embodiments of this system are described below.

Hardware and software used

Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

Software: Fingerprint authentication system, smart device application, generative AI model

System Configuration

1. User authentication method:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The fingerprint authentication system captures fingerprint data and authenticates the user.

2. Operating means:

If authentication is successful, the buttons on the accessory device will be enabled.

The user presses the enabled button.

3. Signal transmission method:

When the button is pressed, the accessory device sends a signal to the smart device.

The smart device receives the signal and launches a specific application.

4. Prompt sending method:

The launched application sends a prompt to the generative AI model.

Prompts are pre-defined and contain the instructions or questions the user wants to ask the generated AI model.

5. Response generation method:

The server uses a generative AI model to generate a response based on the received prompt.

6. Response sending method:

The server generates a response and sends it to the smart device.

The smart device receives the response and displays it to the user.

Specific examples

When a user places their finger on the fingerprint authentication sensor on the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device, which launches an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

Example prompt

"Tell me what the weather is like today."

"Check the schedule for the next meeting."

"Tell me the latest news."

This system allows users to easily use generated AI models without having to directly operate a smart device, and also improves security through user authentication measures.

The flow of the identification process in Example 2 will be explained using Figure 13.

Program processing flow

Step 1:

The user places their finger on the fingerprint authentication sensor on the accessory device.

Input: User fingerprint

Data processing: The fingerprint sensor captures fingerprint data.

Output: Captured fingerprint data

Step 2:

The device captures the fingerprint data and performs authentication.

Input: Captured fingerprint data

Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

Output: Authentication result (success or failure)

Step 3:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Data processing: If authentication is successful, enable the device button.

Output: Enabled button

Step 4:

The user presses the button.

Input: Enabled button

Data processing: The device detects that a button has been pressed.

Output: Button press signal

Step 5:

The terminal sends a signal to the smart device to launch the application.

Input: Button press signal

Data processing: The terminal sends a signal to the smart device to launch a specific application.

Output: Launched application

Step 6:

The application sends a prompt to the generative AI model.

Input: Launched application

Data processing: The application sends pre-defined prompts to the generative AI model.

Output: Prompt sent to the generative AI model

Step 7:

The server generates a response using a generative AI model.

Input: Prompt sent to the generative AI model

Data calculation: The generative AI model analyzes the prompt and generates an appropriate response.

Output: Generated response

Step 8:

The server sends the response to the smart device.

Input: Generated response

Data processing: The server sends the generated response to the smart device.

Output: Response sent to smart device

Step 9:

Your smart device will display the response.

Input: Response sent to smart device

Data processing: The smart device displays the response received to the user.

Output: Response displayed to the user

In this way, users can use generative AI models without directly operating their smart devices.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional food delivery systems require users to use their smartphones to place orders, making them cumbersome to operate. In addition, insufficient user authentication poses security risks. Furthermore, voice recognition-based ordering systems are not widespread, resulting in low user convenience. There is a need to solve these problems and provide a system that allows users to use food delivery services easily and safely.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

An accessory device connectable to a smart device includes a user authentication means that authenticates a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means that recognizes the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the results of the order using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

"Generative AI" is an artificial intelligence technology that generates text and other content based on user instructions and prompts.

"Instruction input" refers to commands or requests that a user inputs to a generative AI.

"User authentication means" refers to technology or devices used to verify and authenticate a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and authenticate the user based on that information.

"Operation means" refers to the interface or buttons that users use to operate accessory devices or smart devices.

"Voice recognition means" refers to technology or devices that analyze the user's voice and convert it into text data.

"Voice data" refers to data that has been digitally recorded from the user's voice.

"Transmission means" refers to the technology or equipment used to transmit data to other devices or systems.

This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

2. Accessory device: Additional hardware that is connected to a smart device and used, including fingerprint authentication means, operation means, and voice recognition means.

3. Generative AI: Artificial intelligence technology that generates text and other content based on user instructions and prompts.

Program processing

Hardware

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

Smart device: Functions as the application execution environment.

Microphone: Captures the user's voice as a means of voice recognition.

Software

Fingerprint authentication library: A software module for performing fingerprint authentication.

Speech recognition library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

Food delivery API: Place orders using the food delivery service's API.

Processing flow

1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device. If authentication is successful, the device proceeds to the next step.

2. Voice input: The user presses the operation means (one-click button) of the attached device and verbally states the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

3. Prompt generation: Generate a prompt sentence for the generative AI based on the acquired text data.

4. AI response generation: The generative AI model generates the optimal response based on the prompt.

5. Order execution: Based on the generated response, the order is completed using the food delivery API.

Specific examples

When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the generative AI model suggests the most suitable pizza restaurant and menu item, and completes the order using a food delivery API.

Example prompt

"User wants to order: Pizza"

In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphone.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint data.

Processing: The fingerprint authentication library analyzes the fingerprint data and compares it with registered fingerprint data.

Output: Authentication result (success or failure).

Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

Step 2:

The user presses the operating means (one-click button) of the accessory device.

Input: User operation (button press).

Processing: Detects that the operating means has been pressed and activates voice recognition mode.

Output: Activate voice recognition mode.

Specific operation: The sensor detects that a button has been pressed, and the voice recognition library begins preparing for voice input.

Step 3:

The user verbally communicates the order details.

Input: User's voice data.

Processing: The microphone captures the audio data, and the speech recognition library converts it into text data.

Output: Text data (order details).

Specific operation: The microphone captures the user's voice, and the speech recognition library analyzes the voice and converts it into text.

Step 4:

The device generates a prompt based on the acquired text data.

Input: Text data (order details).

Processing: Convert the text data into a prompt in the format "The user would like to order: [Order details]".

Output: Prompt statement.

Specific operation: Converts text data into prompt text according to the format.

Step 5:

The device sends a prompt to the generative AI model.

Input: Prompt text.

Processing: The generative AI model analyzes the prompt and generates the optimal response.

Output: AI response (optimal order content).

Specific operation: The prompt sentence is sent to the generative AI model, and the AI generates a response.

Step 6:

The device uses the food delivery API based on the generated response to complete the order.

Input: AI response (optimal order content).

Processing: Fulfill orders using the food delivery API.

Output: Order confirmation (data indicating the order was completed successfully).

Specific operation: Sends the AI response to the food delivery API and confirms the order.

The server may further include means for recognizing the user's emotions using an emotion engine, which will be described later, and may obtain the output of the generative AI using the voice data acquired by the voice recognition means, a prompt statement including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt statement such as "User wants to order: pizza, user's emotion: sadness" may be generated, and the generative AI model may suggest the optimal pizza restaurant and menu item taking the user's emotion into consideration, and the order may be completed using a food delivery API.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, it recognizes emotions from the user's tone of voice and facial expression, and feeds this information back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

"Example 2"

Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user operates an accessory device that can be connected to a smartphone.

Step 2: The device's user authentication means authenticates the user. One possible authentication method is fingerprint authentication.

Step 3: The authenticated user provides prompt input to the generative AI.

Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

Step 5: The emotion engine feeds that information back to the generative AI.

Step 6: The generative AI generates an appropriate response based on the user's emotions.

"Example 2"

Step 1: The user speaks to the accessory device.

Step 2: The device's microphone captures the user's voice and sends the audio data to the emotion engine.

Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

Step 4: The emotion engine feeds the results back to the generative AI.

Step 5: The generative AI generates an appropriate response based on the user's emotions.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Existing smartphone accessory devices allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses based on the user's intentions and emotions, resulting in a lack of improvement in the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

A "generative AI model" is an artificial intelligence system that generates output such as text or voice based on input prompts.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

"User authentication means" refers to technical means for verifying a user's identity.

"Fingerprint authentication means" refers to a technical means that reads a user's fingerprint and performs authentication based on that information.

"Operation means" refers to the interface that allows the user to operate the accessory device.

A "one-click operation method" is an interface that allows users to complete an operation with a single click.

"Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

"Emotion information" is data regarding the user's emotional state obtained by emotion recognition means.

"Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment

The accessory device connects to your smartphone using wireless communication methods such as Bluetooth. The device includes the following main components:

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates them.

Action button: An interface that allows the user to start prompt input.

Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

Smartphone

The smartphone runs a dedicated application that works in conjunction with the accessory device. This application provides the following functions:

Prompt input interface: A screen where users can enter instructions or questions for the generative AI model.

Data transmission function: Sends prompts and emotion information to the server.

Server

The server provides backend services that handle data communication between smartphones and generative AI models. The server has the following functions:

Data relay function: Sends prompts and emotional information received from a smartphone to the generative AI model.

Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

Specific examples

Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

1. User authentication

The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

2. Prompt Input

The user opens a dedicated application on their smartphone and enters a prompt. For example, they might enter, "What's the weather like today?" The device then prepares to send this prompt to the generative AI model.

3. Emotion recognition

The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The device prepares this emotion data to send to the generative AI model.

4. Sending prompts to generative AI models

The device sends the prompt and emotion data entered by the user to the server. The server then sends this data to the generative AI model. The generative AI model generates an appropriate response based on the prompt and emotion data.

5. Receiving responses from generative AI models

The server receives the response from the generative AI model and forwards it to the device. The device then prepares to display this response to the user.

6. Displaying the response

The device displays the response received from the generative AI model to the user. For example, it might say, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

Example prompt

"What's the weather like today?"

"Please let me know when the next meeting is scheduled."

"Please tell me the latest news."

In this way, users can naturally interact with generative AI models through accessory devices.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1: User Authentication

The user places their finger on the fingerprint authentication sensor on the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

Step 2: Connect your accessory device to your smartphone

The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

Step 3: Enter the prompt

The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

Step 4: Emotion Recognition

The accessory device uses an emotion engine that senses the user's vocal tone and facial expressions to analyze the user's emotional state. The input is the user's vocal tone and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

Step 5: Send prompts to the generative AI model

The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

Step 6: Receive a response from the generative AI model

The server receives the response from the generative AI model and forwards it to the device. The input is the response data from the generative AI model, and the output is the result of forwarding to the device. The device prepares to display this response to the user. Once the response is received, the device displays "Response received successfully."

Step 7: View the response

The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts if necessary.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

An "accessory device connectable to a smartphone" is an external device that connects to a smartphone using wireless communication means and provides specific functions.

"Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions and questions to the generative AI.

"User authentication means" refers to technology used to verify a user's identity, ensuring that only specific users can operate the system.

"Fingerprint authentication" is a technology that scans a user's fingerprint and authenticates the user based on that information.

"Operation means" refers to the physical or software interface that allows the user to operate the system.

A "one-click operation means" is an interface that allows users to perform a specific function with a single click.

"Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

"Means for feeding back emotional information to the generative AI" is a function that sends recognized emotional information to the generative AI, allowing the generative AI to generate responses and suggestions based on that information.

This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

System Configuration

The system consists of the following main components:

1. Accessory devices that can be connected to smartphones

Wireless communication method: Connects to a smartphone using Bluetooth or other means.

User authentication method: User authentication is performed using a fingerprint sensor.

Operation method: The user uses a one-click operation method to input prompts to the generative AI.

Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

2. Generative AI

Prompt Input: Receives instructions or questions from the user and generates responses or suggestions.

Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

Program processing

The server performs the following process:

1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

3. Emotion recognition: Analyze the user's tone of voice and facial expressions to recognize emotions.

4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

Hardware and software used

Hardware: Bluetooth module, fingerprint sensor, camera (for facial recognition), microphone (for voice recognition)

Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

Specific examples

If the generative AI recognizes that the user is tired, it will suggest "food delivery options recommended for when you're tired." For example, it might generate a prompt like this:

The user is feeling tired. Please suggest some food delivery options.

In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The server connects the smartphone and accessory device using a Bluetooth module. The input is pairing information between the smartphone and accessory device, and the output is the establishment of the connection. Specifically, it uses the Bluetooth connection library to establish communication between the smartphone and accessory device.

Step 2:

The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

Step 3:

The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotional information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotions.

Step 4:

The server generates a prompt for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the emotional information is converted into text format and a prompt suitable for the generative AI is created.

Step 5:

The server uses a generative AI model to generate responses and suggestions in response to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI generates the response or suggestion.

Step 6:

The device presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional systems using generative AI models required users to operate their smartphones, making them difficult to use. They also were unable to generate appropriate responses based on the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing their emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to obtain an appropriate response according to their emotions.

"User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

"Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

"Emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from its tone and content.

A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

This invention provides a system that includes a user authentication means, an operation means, an emotion recognition means, and a feedback means. Specific embodiments of this system are described below.

First, the user places their finger on the accessory device's fingerprint authentication sensor. The fingerprint authentication sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which compares the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Please tell me about today's weather" is input to the generative AI model.

Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotions. The analysis results are sent to the generative AI model via the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," it generates the response "Let's take a break today and refresh ourselves."

This system allows users to easily use generative AI models without having to operate a smartphone. It also improves the user experience by enabling appropriate responses based on the user's emotions.

For example, when a user places their finger on the fingerprint authentication sensor on an accessory device and presses a button after successful authentication, an application on the smartphone launches and the following prompt is input into the generative AI model:

"Can you tell me about today's weather?"

As another example, when a user asks an accessory device, "How are you feeling today?", the emotion recognition means recognizes the emotion from the user's tone of voice and inputs the following prompt sentence into the generative AI model:

"The user seems a little tired. Please generate an encouraging message."

This allows users to easily utilize generative AI without having to operate their smartphone.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint

Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

Output: Captured fingerprint data

Step 2:

The device captures fingerprint data and sends it to the server.

Input: Captured fingerprint data

Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and, when sending that data to the server, displays "Authentication in progress..." on the device screen.

Output: Fingerprint data sent to the server

Step 3:

The server authenticates the fingerprint data.

Input: Fingerprint data sent to the server

Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

Output: Authentication result (success or failure)

Step 4:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Specific operation: If the server successfully authenticates, it sends a signal to the device to activate the button. The button on the device will light up green.

Output: Enabled button

Step 5:

The user presses the button.

Input: Enabled button

Specific Action: The user presses the enabled button, which triggers the next step.

Output: Button press signal

Step 6:

The device launches the application on the smartphone.

Input: Button press signal

Specific operation: The device launches a dedicated application on the smartphone. The application logo will appear on the smartphone screen.

Output: Launched application

Step 7:

The application prompts input to the generative AI model.

Input: Launched application

Specific operation: The application inputs a prompt to the generative AI model. The screen displays "Sending prompt...". For example, the prompt input is "Tell me about the weather today."

Output: Prompt sent to the generative AI model

Step 8:

The user speaks to the accessory device.

Input: User's voice

Specific operation: The user speaks to the accessory device. For example, "How are you feeling today?" The device's microphone lights up red, indicating that it is capturing audio data.

Output: Captured audio data

Step 9:

The device captures voice data and sends it to the emotion engine.

Input: Captured audio data

Specific operation: The device uses the built-in microphone to capture audio data and sends that data to the emotion engine.

Output: Audio data sent to the emotion engine

Step 10:

The emotion engine analyzes the user's emotions from the tone of the voice data.

Input: Audio data sent to the emotion engine

Specific operation: The emotion engine analyzes the tone of the voice data and determines the user's emotions. The device screen will display "Analyzing emotions..."

Output: Analyzed emotional information

Step 11:

The emotion engine feeds the analysis results back into the generative AI model.

Input: Analyzed emotional information

Specific operation: The emotion engine feeds the analysis results back to the generative AI model. For example, it sends information such as "The user seems a little tired."

Output: Emotional information sent to the generative AI model

Step 12:

The generative AI model generates responses based on the user's emotions.

Input: Emotional information sent to the generative AI model

Specific operation: The generative AI model generates an appropriate response based on the emotional information fed back. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today to refresh yourself." The response is displayed on the smartphone screen.

Output: Generated response

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

With conventional food delivery services, when users order using their smartphones, the process is cumbersome and they don't offer appropriate suggestions based on the user's mood. In particular, there are issues with the lack of relaxing meal suggestions for tired or stressed users, which hinders the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for enabling prompt input to the generative AI, user authentication means for operating the accessory device, operation means for a user authenticated by the user authentication means to input prompts to the generative AI, voice capture means for capturing the user's voice and recognizing their emotions, emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotions, and means for the generative AI to generate an appropriate response based on the emotions recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

"Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

"Prompt input" refers to the act of inputting instructions or questions to generative AI.

An "accessory device" is an external device that can be connected to a smartphone to provide specific functions.

"User authentication means" refers to a means for verifying a user's identity and ensuring that only specific users can operate the system.

"Fingerprint authentication means" refers to a means of authenticating a user by reading their fingerprint.

"Operation means" refers to the interface, buttons, and other means that users use to operate the system.

A "one-click operation method" is a method that allows the user to complete an operation with a single click.

"Audio capture means" means for recording or capturing a user's voice.

"Emotion recognition means" refers to a means for analyzing captured voice data and recognizing the user's emotions.

"Response" refers to the response or suggestion that generative AI generates in response to user input.

A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method. An AI generation model (e.g., AIGenerator) is used as the generative AI.

For example, if a user says, "I'm tired today," the voice capture means captures the voice and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

Examples of prompts to input to a generative AI model include:

User Emotion: Fatigue

Prompt for generative AI: "Please recommend a relaxing meal for a tired user."

This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate their smartphone.

The flow of the specific processing in Application Example 2 is explained using Figure 18.

Step 1:

The user places their finger on the fingerprint authentication sensor of an accessory device connected to their smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

Step 2:

If authentication is successful, the accessory device's operation means is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and begins prompt input to the generative AI.

Step 3:

The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and voice recognition software converts the data into digital form.

Step 4:

The captured voice data is sent to the emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

Step 5:

The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt sentence to the generative AI. The generative AI creates a prompt sentence to generate an appropriate response based on the emotional data.

Step 6:

The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

Step 7:

The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user via the terminal's display or audio output device.

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is

These results are obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using a wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the fingerprint information with the registered user's fingerprint. The operation means involves the user pressing a button on the device, which inputs prompts to the generative AI.

"Example 2"

As a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user connects an accessory device that can be connected to a smartphone to the smartphone using a wireless communication method such as Bluetooth.

Step 2: The user places their finger on the fingerprint authentication sensor on the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the registered user's fingerprint information.

Step 3: If authentication is successful, the device's buttons will be enabled.

Step 4: When the user presses a button on the device, an application on the smartphone is launched and prompts are input to the generative AI.

"Example 2"

Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

Step 2: If authentication is successful, the device's buttons will be enabled.

Step 3: When the user presses the button, an application on the smartphone is launched and prompt input is sent to the generative AI.

Step 4: This allows users to easily utilize generative AI without having to operate their smartphone.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In systems that use conventional generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompt text, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience for users when using generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

"Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

"Prompt input" is the act of inputting instructions or questions to a generative AI.

"User authentication means" refers to technology or devices used to verify a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and verify the user's identity based on that information.

"Operation means" refers to the interface or device that allows the user to operate the accessory device or smartphone.

A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

An "application" is software that runs on a smartphone and provides specific functions or services.

"Generation results" are output such as text and images generated by the generative AI based on the prompt text.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts into a generative AI model. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment:

Accessory devices are external devices that connect to your smartphone and have the following main functions:

User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

Operation method: Equipped with a one-click button, making it easy for users to operate.

Smartphone:

The smartphone is connected to the accessory device using wireless communication means such as Bluetooth. An application for providing prompt input to the generative AI model is installed on the smartphone.

Generative AI model:

The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

System Operation

User authentication:

The user places their finger on the fingerprint authentication sensor installed on the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

Prompt Input:

After successful authentication, the user presses a button on the accessory device. The smartphone application launches and a prompt input screen appears. The user enters a prompt statement for the generative AI model.

Send to generative AI model:

The smartphone sends the input prompt to the generative AI model. The server operates the generative AI model based on the received prompt and creates the generated results.

Display generated results:

The server sends the generated results to the smartphone. The smartphone displays the received results to the user. The user can check the displayed results and save or share them as needed.

Specific examples

If a user wants to have a generative AI create a new poem, the specific steps are as follows:

1. User authentication:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The accessory device reads the fingerprint and performs authentication.

If authentication is successful, a signal indicating successful authentication will be sent to your smartphone.

2. Prompt input:

The user presses a button on the accessory device.

The smartphone application will launch and a prompt input screen will appear.

The user enters the prompt text "Create a new poem."

3. Send to generative AI model:

The smartphone sends the entered prompt to the generative AI model.

The server runs a generative AI model based on the prompt text to generate the poem.

4. Displaying the generated results:

The server sends the generated poem to your smartphone.

The smartphone will then display the received poem to the user.

Users can review the displayed poems and save or share them as needed.

In this way, users can use the accessory device and smartphone to provide prompt input to the generative AI model and obtain generated results.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Connecting Accessory Devices

Input: User opens their smartphone's Bluetooth settings and selects an accessory device.

How it works: Your smartphone establishes a connection with the accessory device using Bluetooth.

Output: The smartphone notifies you that the connection was successful.

Step 2: User Authentication

Input: The user places their finger on the fingerprint sensor on the accessory device.

How it works: The accessory device reads your fingerprint and compares it with the fingerprint information stored in its internal database.

Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

Step 3: Prepare for prompt input

Input: The smartphone receives a signal indicating successful authentication.

How it works: The smartphone displays a prompt input screen for the generative AI.

Output: The user sees the prompt input screen and begins entering the prompt statement.

Step 4: Enter the prompt

Input: The user enters a prompt for the generative AI through a smartphone application.

How it works: The smartphone receives the entered prompt text and formats the data internally.

Output: The prepared prompt is ready to be sent to the generative AI model.

Step 5: Send to the generative AI model

Input: The smartphone sends the prepared prompt to the generative AI model.

Operation: The server runs the generative AI model based on the received prompt text and creates the generated results.

Output: The generated results are sent from the server to the smartphone.

Step 6: View the generated results

Input: The smartphone receives the generated results from the server.

How it works: The smartphone displays the generated results to the user.

Output: The user checks the displayed generated results and saves or shares them as needed.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient user authentication security, creating the risk of operational errors and unauthorized access. This can potentially reduce factory productivity and safety.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for generating instructions for managing and operating robots in a factory; and means for transmitting the instructions to the robots using wireless communication means. This makes it possible to efficiently and safely manage and operate robots in a factory.

An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

"Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

"User authentication means" is a function for verifying and authenticating a user's identity.

"Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

"Operation means" is an interface that allows the user to input prompts to the generative AI.

"One-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

"Means for generating instructions for managing and operating robots within a factory" refers to a function for generating specific work instructions for robots within a factory.

"Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

"Means for transmitting to robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

This invention is a system that efficiently and safely manages and operates robots in a factory using accessory devices that can be connected to a smartphone. A specific embodiment of this system is described below.

System Configuration

The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

Hardware

Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

Accessory device: Connects to a smartphone and includes fingerprint authentication means and operation means.

Fingerprint authentication method: Scans the user's fingerprint for authentication.

Wireless communication method: Data is sent and received between smartphones and factory robots using Bluetooth, Wi-Fi, etc.

Factory robot: Receives instructions and performs actions.

Software

Generative AI model: Generates specific instructions for factory robots based on prompt input from the user.

Smartphone application: Provides the user interface and manages communication with the generative AI model.

Operating Procedure

1. User authentication: The user places their finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

2. Prompt input: An authenticated user uses an operation method (one-click operation method) to input a prompt to the generative AI model.

3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

5. Robot operation: The factory robot operates according to the instructions it receives.

Specific examples

For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions. Below are some example prompts:

"Move to assembly line 3 and begin welding."

"Move to the inspection area to perform quality checks."

"Go get part A and take it to assembly line 2."

This makes it possible to efficiently and safely manage and operate robots within factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user places their finger on the accessory device and is authenticated using the fingerprint authentication means.

Input: User's fingerprint

Data processing: The fingerprint authentication device scans the fingerprint and compares it with registered fingerprint data.

Output: Authentication result (success or failure)

Specific operation: The fingerprint authentication sensor scans the user's fingerprint and compares it with an internal database for authentication.

Step 2:

If authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

Input: Authentication result (success), user prompt

Data processing: If the authentication result is successful, a prompt text is sent to the generative AI model.

Output: Prompt input to the generative AI model

Specific operation: When the user presses the button, the smartphone application sends a prompt statement to the generative AI model.

Step 3:

Based on the prompts, a generative AI model generates specific instructions for the factory robot.

Input: User prompt

Data processing: The generative AI model analyzes the prompt and generates appropriate instructions.

Output: Instructions to factory robots

Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

Step 4:

The generated instructions are sent to the factory robot using wireless communication means.

Input: Instructions from a generative AI model

Data processing: Send instructions to factory robots via wireless communication means.

Output: Send instructions to factory robots

Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

Step 5:

The factory robot will act according to the instructions it receives.

Input: Instructions for factory robots

Data processing: Factory robots analyze instructions and perform corresponding actions.

Output: Robot movement

Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

To use conventional generative AI models, users had to manually operate their smart devices and input prompts. This was time-consuming and inconvenient, especially when their hands were full or when a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. A system that solves these issues and allows users to use generative AI models more easily and safely is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for using the operation means to send a signal to the smart device and launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without having to directly operate the smart device, and the user authentication means also improves security.

"User authentication means" refers to the means used to verify a user's identity and grant access to a system.

"Operation means" refers to the interface or device that allows the user to operate the system.

A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

"Means for transmitting signals" refers to means for transmitting specific instructions or data from the operating means to the smart device.

A "specific application" is a software program designed to provide a specific function or service.

A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

A "prompt" is text input used to give instructions or ask questions to a generative AI model.

"Means for generating a response" refers to the means by which a generative AI model creates an appropriate response based on a prompt.

"Means for sending a response" refers to means for sending the generated response to a smart device.

Form for implementing the invention

This invention is a system that allows users to use generative AI models without directly operating a smart device. Specific embodiments of this system are described below.

Hardware and software used

Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

Software: Fingerprint authentication system, smart device application, generative AI model

System Configuration

1. User authentication method:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The fingerprint authentication system captures fingerprint data and authenticates the user.

2. Operating means:

If authentication is successful, the buttons on the accessory device will be enabled.

The user presses the enabled button.

3. Signal transmission method:

When the button is pressed, the accessory device sends a signal to the smart device.

The smart device receives the signal and launches a specific application.

4. Prompt sending method:

The launched application sends a prompt to the generative AI model.

Prompts are pre-defined and contain the instructions or questions the user wants to ask the generated AI model.

5. Response generation method:

The server uses a generative AI model to generate a response based on the received prompt.

6. Response sending method:

The server generates a response and sends it to the smart device.

The smart device receives the response and displays it to the user.

Specific examples

When a user places their finger on the fingerprint authentication sensor on the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device, which launches an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

Example prompt

"Tell me what the weather is like today."

"Check the schedule for the next meeting."

"Tell me the latest news."

This system allows users to easily use generated AI models without having to directly operate a smart device, and also improves security through user authentication measures.

The flow of the identification process in Example 2 will be explained using Figure 13.

Program processing flow

Step 1:

The user places their finger on the fingerprint authentication sensor on the accessory device.

Input: User fingerprint

Data processing: The fingerprint sensor captures fingerprint data.

Output: Captured fingerprint data

Step 2:

The device captures the fingerprint data and performs authentication.

Input: Captured fingerprint data

Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

Output: Authentication result (success or failure)

Step 3:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Data processing: If authentication is successful, enable the device button.

Output: Enabled button

Step 4:

The user presses the button.

Input: Enabled button

Data processing: The device detects that a button has been pressed.

Output: Button press signal

Step 5:

The terminal sends a signal to the smart device to launch the application.

Input: Button press signal

Data processing: The terminal sends a signal to the smart device to launch a specific application.

Output: Launched application

Step 6:

The application sends a prompt to the generative AI model.

Input: Launched application

Data processing: The application sends pre-defined prompts to the generative AI model.

Output: Prompt sent to the generative AI model

Step 7:

The server generates a response using a generative AI model.

Input: Prompt sent to the generative AI model

Data calculation: The generative AI model analyzes the prompt and generates an appropriate response.

Output: Generated response

Step 8:

The server sends the response to the smart device.

Input: Generated response

Data processing: The server sends the generated response to the smart device.

Output: Response sent to smart device

Step 9:

Your smart device will display the response.

Input: Response sent to smart device

Data processing: The smart device displays the response received to the user.

Output: Response displayed to the user

In this way, users can use generative AI models without directly operating their smart devices.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional food delivery systems require users to use their smartphones to place orders, making them cumbersome to operate. In addition, insufficient user authentication poses security risks. Furthermore, voice recognition-based ordering systems are not widespread, resulting in low user convenience. There is a need to solve these problems and provide a system that allows users to use food delivery services easily and safely.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

An accessory device connectable to a smart device includes a user authentication means that authenticates a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means that recognizes the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the results of the order using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

"Generative AI" is an artificial intelligence technology that generates text and other content based on user instructions and prompts.

"Instruction input" refers to commands or requests that a user inputs to a generative AI.

"User authentication means" refers to technology or devices used to verify and authenticate a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and authenticate the user based on that information.

"Operation means" refers to the interface or buttons that users use to operate accessory devices or smart devices.

"Voice recognition means" refers to technology or devices that analyze the user's voice and convert it into text data.

"Voice data" refers to data that has been digitally recorded from the user's voice.

"Transmission means" refers to the technology or equipment used to transmit data to other devices or systems.

This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

2. Accessory device: Additional hardware that is connected to a smart device and used, including fingerprint authentication means, operation means, and voice recognition means.

3. Generative AI: Artificial intelligence technology that generates text and other content based on user instructions and prompts.

Program processing

Hardware

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

Smart device: Functions as the application execution environment.

Microphone: Captures the user's voice as a means of voice recognition.

Software

Fingerprint authentication library: A software module for performing fingerprint authentication.

Speech recognition library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

Food delivery API: Place orders using the food delivery service's API.

Processing flow

1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device. If authentication is successful, the device proceeds to the next step.

2. Voice input: The user presses the operation means (one-click button) of the attached device and verbally states the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

3. Prompt generation: Generate a prompt sentence for the generative AI based on the acquired text data.

4. AI response generation: The generative AI model generates the optimal response based on the prompt.

5. Order execution: Based on the generated response, the order is completed using the food delivery API.

Specific examples

When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the generative AI model suggests the most suitable pizza restaurant and menu item, and completes the order using a food delivery API.

Example prompt

"User wants to order: Pizza"

In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphone.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint data.

Processing: The fingerprint authentication library analyzes the fingerprint data and compares it with registered fingerprint data.

Output: Authentication result (success or failure).

Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

Step 2:

The user presses the operating means (one-click button) of the accessory device.

Input: User operation (button press).

Processing: Detects that the operating means has been pressed and activates voice recognition mode.

Output: Activate voice recognition mode.

Specific operation: The sensor detects that a button has been pressed, and the voice recognition library begins preparing for voice input.

Step 3:

The user verbally communicates the order details.

Input: User's voice data.

Processing: The microphone captures the audio data, and the speech recognition library converts it into text data.

Output: Text data (order details).

Specific operation: The microphone captures the user's voice, and the speech recognition library analyzes the voice and converts it into text.

Step 4:

The device generates a prompt based on the acquired text data.

Input: Text data (order details).

Processing: Convert the text data into a prompt in the format "The user would like to order: [Order details]".

Output: Prompt statement.

Specific operation: Converts text data into prompt text according to the format.

Step 5:

The device sends a prompt to the generative AI model.

Input: Prompt text.

Processing: The generative AI model analyzes the prompt and generates the optimal response.

Output: AI response (optimal order content).

Specific operation: The prompt sentence is sent to the generative AI model, and the AI generates a response.

Step 6:

The device uses the food delivery API based on the generated response to complete the order.

Input: AI response (optimal order content).

Processing: Fulfill orders using the food delivery API.

Output: Order confirmation (data indicating the order was completed successfully).

Specific operation: Sends the AI response to the food delivery API and confirms the order.

The server may further include means for recognizing the user's emotions using an emotion engine, which will be described later, and may obtain the output of the generative AI using the voice data acquired by the voice recognition means, a prompt statement including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt statement such as "User wants to order: pizza, user's emotion: sadness" may be generated, and the generative AI model may suggest the optimal pizza restaurant and menu item taking the user's emotion into consideration, and the order may be completed using a food delivery API.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, it recognizes emotions from the user's tone of voice and facial expression, and feeds this information back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

"Example 2"

Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user operates an accessory device that can be connected to a smartphone.

Step 2: The device's user authentication means authenticates the user. One possible authentication method is fingerprint authentication.

Step 3: The authenticated user provides prompt input to the generative AI.

Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

Step 5: The emotion engine feeds that information back to the generative AI.

Step 6: The generative AI generates an appropriate response based on the user's emotions.

"Example 2"

Step 1: The user speaks to the accessory device.

Step 2: The device's microphone captures the user's voice and sends the audio data to the emotion engine.

Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

Step 4: The emotion engine feeds the results back to the generative AI.

Step 5: The generative AI generates an appropriate response based on the user's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Existing smartphone accessory devices allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses based on the user's intentions and emotions, resulting in a lack of improvement in the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

A "generative AI model" is an artificial intelligence system that generates output such as text or voice based on input prompts.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

"User authentication means" refers to technical means for verifying a user's identity.

"Fingerprint authentication means" refers to a technical means that reads a user's fingerprint and performs authentication based on that information.

"Operation means" refers to the interface that allows the user to operate the accessory device.

A "one-click operation method" is an interface that allows users to complete an operation with a single click.

"Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

"Emotion information" is data regarding the user's emotional state obtained by emotion recognition means.

"Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment

The accessory device connects to your smartphone using wireless communication methods such as Bluetooth. The device includes the following main components:

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates them.

Action button: An interface that allows the user to start prompt input.

Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

Smartphone

The smartphone runs a dedicated application that works in conjunction with the accessory device. This application provides the following functions:

Prompt input interface: A screen where users can enter instructions or questions for the generative AI model.

Data transmission function: Sends prompts and emotion information to the server.

Server

The server provides backend services that handle data communication between smartphones and generative AI models. The server has the following functions:

Data relay function: Sends prompts and emotional information received from a smartphone to the generative AI model.

Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

Specific examples

Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

1. User authentication

The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

2. Prompt Input

The user opens a dedicated application on their smartphone and enters a prompt. For example, they might enter, "What's the weather like today?" The device then prepares to send this prompt to the generative AI model.

3. Emotion recognition

The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The device prepares this emotion data to send to the generative AI model.

4. Sending prompts to generative AI models

The device sends the prompt and emotion data entered by the user to the server. The server then sends this data to the generative AI model. The generative AI model generates an appropriate response based on the prompt and emotion data.

5. Receiving responses from generative AI models

The server receives the response from the generative AI model and forwards it to the device. The device then prepares to display this response to the user.

6. Displaying the response

The device displays the response received from the generative AI model to the user. For example, it might say, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

Example prompt

"What's the weather like today?"

"Please let me know when the next meeting is scheduled."

"Please tell me the latest news."

In this way, users can naturally interact with generative AI models through accessory devices.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1: User Authentication

The user places their finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

Step 2: Connect your accessory device to your smartphone

The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

Step 3: Enter the prompt

The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

Step 4: Emotion Recognition

The accessory device uses an emotion engine that senses the user's vocal tone and facial expressions to analyze the user's emotional state. The input is the user's vocal tone and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

Step 5: Send prompts to the generative AI model

The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

Step 6: Receive a response from the generative AI model

The server receives the response from the generative AI model and forwards it to the device. The input is the response data from the generative AI model, and the output is the result of forwarding to the device. The device prepares to display this response to the user. Once the response is received, the device displays "Response received successfully."

Step 7: View the response

The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts if necessary.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

An "accessory device connectable to a smartphone" is an external device that connects to a smartphone using wireless communication means and provides specific functions.

"Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

"User authentication means" refers to technology used to verify a user's identity and ensure that only specific users can operate the system.

"Fingerprint authentication" is a technology that scans a user's fingerprint and authenticates the user based on that information.

"Operation means" refers to the physical or software interface that allows the user to operate the system.

A "one-click operation means" is an interface that allows users to perform a specific function with a single click.

"Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

"Means for feeding back emotional information to the generative AI" is a function that sends recognized emotional information to the generative AI, allowing the generative AI to generate responses and suggestions based on that information.

This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

System Configuration

The system consists of the following main components:

1. Accessory devices that can be connected to smartphones

Wireless communication method: Connects to a smartphone using Bluetooth or other means.

User authentication method: User authentication is performed using a fingerprint sensor.

Operation method: The user uses a one-click operation method to input prompts to the generative AI.

Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

2. Generative AI

Prompt Input: Receives instructions or questions from the user and generates responses or suggestions.

Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

Program processing

The server performs the following process:

1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

3. Emotion recognition: Analyze the user's tone of voice and facial expressions to recognize emotions.

4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

Hardware and software used

Hardware: Bluetooth module, fingerprint sensor, camera (for facial recognition), microphone (for voice recognition)

Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

Specific examples

If the generative AI recognizes that the user is tired, it will suggest "food delivery options recommended for when you're tired." For example, it might generate a prompt like this:

The user is feeling tired. Please suggest some food delivery options.

In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The server connects the smartphone and accessory device using a Bluetooth module. The input is pairing information between the smartphone and accessory device, and the output is the establishment of the connection. Specifically, it uses the Bluetooth connection library to establish communication between the smartphone and accessory device.

Step 2:

The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

Step 3:

The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotional information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotions.

Step 4:

The server generates a prompt for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the emotional information is converted into text format and a prompt suitable for the generative AI is created.

Step 5:

The server uses a generative AI model to generate responses and suggestions in response to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI generates the response or suggestion.

Step 6:

The device presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional systems using generative AI models required users to operate their smartphones, making them difficult to use. They also were unable to generate appropriate responses based on the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing their emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to obtain an appropriate response according to their emotions.

"User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

"Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

"Emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from its tone and content.

A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

This invention provides a system that includes a user authentication means, an operation means, an emotion recognition means, and a feedback means. Specific embodiments of this system are described below.

First, the user places their finger on the accessory device's fingerprint authentication sensor. The fingerprint authentication sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which compares the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Please tell me about today's weather" is input to the generative AI model.

Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotions. The analysis results are sent to the generative AI model via the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," it generates the response "Let's take a break today and refresh ourselves."

This system allows users to easily use generative AI models without having to operate a smartphone. It also improves the user experience by enabling appropriate responses based on the user's emotions.

For example, when a user places their finger on the fingerprint authentication sensor on an accessory device and presses a button after successful authentication, an application on the smartphone launches and the following prompt is input into the generative AI model:

"Can you tell me about today's weather?"

As another example, when a user asks an accessory device, "How are you feeling today?", the emotion recognition means recognizes the emotion from the user's tone of voice and inputs the following prompt sentence into the generative AI model:

"The user seems a little tired. Please generate an encouraging message."

This allows users to easily utilize generative AI without having to operate their smartphone.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint

Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

Output: Captured fingerprint data

Step 2:

The device captures fingerprint data and sends it to the server.

Input: Captured fingerprint data

Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and, when sending that data to the server, displays "Authentication in progress..." on the device screen.

Output: Fingerprint data sent to the server

Step 3:

The server authenticates the fingerprint data.

Input: Fingerprint data sent to the server

Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

Output: Authentication result (success or failure)

Step 4:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Specific operation: If the server successfully authenticates, it sends a signal to the device to activate the button. The button on the device will light up green.

Output: Enabled button

Step 5:

The user presses the button.

Input: Enabled button

Specific Action: The user presses the enabled button, which triggers the next step.

Output: Button press signal

Step 6:

The device launches the application on the smartphone.

Input: Button press signal

Specific operation: The device launches a dedicated application on the smartphone. The application logo will appear on the smartphone screen.

Output: Launched application

Step 7:

The application prompts input to the generative AI model.

Input: Launched application

Specific operation: The application inputs a prompt to the generative AI model. The screen displays "Sending prompt...". For example, the prompt input is "Tell me about the weather today."

Output: Prompt sent to the generative AI model

Step 8:

The user speaks to the accessory device.

Input: User's voice

Specific operation: The user speaks to the accessory device. For example, "How are you feeling today?" The device's microphone lights up red, indicating that it is capturing audio data.

Output: Captured audio data

Step 9:

The device captures voice data and sends it to the emotion engine.

Input: Captured audio data

Specific operation: The device uses the built-in microphone to capture audio data and sends that data to the emotion engine.

Output: Audio data sent to the emotion engine

Step 10:

The emotion engine analyzes the user's emotions from the tone of the voice data.

Input: Audio data sent to the emotion engine

Specific operation: The emotion engine analyzes the tone of the voice data and determines the user's emotions. The device screen will display "Analyzing emotions..."

Output: Analyzed emotional information

Step 11:

The emotion engine feeds the analysis results back into the generative AI model.

Input: Analyzed emotional information

Specific operation: The emotion engine feeds the analysis results back to the generative AI model. For example, it sends information such as "The user seems a little tired."

Output: Emotional information sent to the generative AI model

Step 12:

The generative AI model generates responses based on the user's emotions.

Input: Emotional information sent to the generative AI model

Specific operation: The generative AI model generates an appropriate response based on the emotional information fed back. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today to refresh yourself." The response is displayed on the smartphone screen.

Output: Generated response

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

With conventional food delivery services, when users order using their smartphones, the process is cumbersome and they don't offer appropriate suggestions based on the user's mood. In particular, there are issues with the lack of relaxing meal suggestions for tired or stressed users, which hinders the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for enabling prompt input to the generative AI, user authentication means for operating the accessory device, operation means for a user authenticated by the user authentication means to input prompts to the generative AI, voice capture means for capturing the user's voice and recognizing their emotions, emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotions, and means for the generative AI to generate an appropriate response based on the emotions recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

"Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

"Prompt input" refers to the act of inputting instructions or questions to generative AI.

An "accessory device" is an external device that can be connected to a smartphone to provide specific functions.

"User authentication means" refers to a means for verifying a user's identity and ensuring that only specific users can operate the system.

"Fingerprint authentication means" refers to a means of authenticating a user by reading their fingerprint.

"Operation means" refers to the interface, buttons, and other means that users use to operate the system.

A "one-click operation method" is a method that allows the user to complete an operation with a single click.

"Audio capture means" means for recording or capturing a user's voice.

"Emotion recognition means" refers to a means for analyzing captured voice data and recognizing the user's emotions.

"Response" refers to the response or suggestion that generative AI generates in response to user input.

A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method. An AI generation model (e.g., AIGenerator) is used as the generative AI.

For example, if a user says, "I'm tired today," the voice capture means captures the voice and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

Examples of prompts to input to a generative AI model include:

User Emotion: Fatigue

Prompt for generative AI: "Please recommend a relaxing meal for a tired user."

This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate their smartphone.

The flow of the specific processing in Application Example 2 will be explained using Figure 18.

Step 1:

The user places their finger on the fingerprint authentication sensor of an accessory device connected to their smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

Step 2:

If authentication is successful, the accessory device's operation means is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and begins prompt input to the generative AI.

Step 3:

The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and voice recognition software converts the data into digital form.

Step 4:

The captured voice data is sent to the emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

Step 5:

The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt sentence to the generative AI. The generative AI creates a prompt sentence to generate an appropriate response based on the emotional data.

Step 6:

The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

Step 7:

The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user via the terminal's display or audio output device.

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using a wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the fingerprint information with the registered user's fingerprint. The operation means involves the user pressing a button on the device, which inputs prompts to the generative AI.

"Example 2"

As a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user connects an accessory device that can be connected to a smartphone to the smartphone using a wireless communication method such as Bluetooth.

Step 2: The user places their finger on the fingerprint authentication sensor on the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the registered user's fingerprint information.

Step 3: If authentication is successful, the device's buttons will be enabled.

Step 4: When the user presses a button on the device, an application on the smartphone is launched and prompts are input to the generative AI.

"Example 2"

Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

Step 2: If authentication is successful, the device's buttons will be enabled.

Step 3: When the user presses the button, an application on the smartphone is launched and prompt input is sent to the generative AI.

Step 4: This allows users to easily utilize generative AI without having to operate their smartphone.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In systems that use conventional generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompt text, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience for users when using generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

"Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

"Prompt input" is the act of inputting instructions or questions to a generative AI.

"User authentication means" refers to technology or devices used to verify a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and verify the user's identity based on that information.

"Operation means" refers to the interface or device that allows the user to operate the accessory device or smartphone.

A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

An "application" is software that runs on a smartphone and provides specific functions or services.

"Generation results" are output such as text and images generated by the generative AI based on the prompt text.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts into a generative AI model. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment:

Accessory devices are external devices that connect to your smartphone and have the following main functions:

User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

Operation method: Equipped with a one-click button, making it easy for users to operate.

Smartphone:

The smartphone is connected to the accessory device using wireless communication means such as Bluetooth. An application for providing prompt input to the generative AI model is installed on the smartphone.

Generative AI model:

The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

System Operation

User authentication:

The user places their finger on the fingerprint authentication sensor installed on the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

Prompt Input:

After successful authentication, the user presses a button on the accessory device. The smartphone application launches and a prompt input screen appears. The user enters a prompt statement for the generative AI model.

Send to generative AI model:

The smartphone sends the input prompt to the generative AI model. The server operates the generative AI model based on the received prompt and creates the generated results.

Display generated results:

The server sends the generated results to the smartphone. The smartphone displays the received results to the user. The user can check the displayed results and save or share them as needed.

Specific examples

If a user wants to have a generative AI create a new poem, the specific steps are as follows:

1. User authentication:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The accessory device reads the fingerprint and performs authentication.

If authentication is successful, a signal indicating successful authentication will be sent to your smartphone.

2. Prompt input:

The user presses a button on the accessory device.

The smartphone application will launch and a prompt input screen will appear.

The user enters the prompt text "Create a new poem."

3. Send to generative AI model:

The smartphone sends the entered prompt to the generative AI model.

The server runs a generative AI model based on the prompt text to generate the poem.

4. Displaying the generated results:

The server sends the generated poem to your smartphone.

The smartphone will then display the received poem to the user.

Users can review the displayed poems and save or share them as needed.

In this way, users can use the accessory device and smartphone to provide prompt input to the generative AI model and obtain generated results.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Connecting Accessory Devices

Input: User opens their smartphone's Bluetooth settings and selects an accessory device.

How it works: Your smartphone establishes a connection with the accessory device using Bluetooth.

Output: The smartphone notifies you that the connection was successful.

Step 2: User Authentication

Input: The user places their finger on the fingerprint sensor on the accessory device.

How it works: The accessory device reads your fingerprint and compares it with the fingerprint information stored in its internal database.

Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

Step 3: Prepare for prompt input

Input: The smartphone receives a signal indicating successful authentication.

How it works: The smartphone displays a prompt input screen for the generative AI.

Output: The user sees the prompt input screen and begins entering the prompt statement.

Step 4: Enter the prompt

Input: The user enters a prompt for the generative AI through a smartphone application.

How it works: The smartphone receives the entered prompt text and formats the data internally.

Output: The prepared prompt is ready to be sent to the generative AI model.

Step 5: Send to the generative AI model

Input: The smartphone sends the prepared prompt to the generative AI model.

Operation: The server runs the generative AI model based on the received prompt text and creates the generated results.

Output: The generated results are sent from the server to the smartphone.

Step 6: View the generated results

Input: The smartphone receives the generated results from the server.

How it works: The smartphone displays the generated results to the user.

Output: The user checks the displayed generated results and saves or shares them as needed.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient user authentication security, creating the risk of operational errors and unauthorized access. This can potentially reduce factory productivity and safety.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for generating instructions for managing and operating robots in a factory; and means for transmitting the instructions to the robots using wireless communication means. This makes it possible to efficiently and safely manage and operate robots in a factory.

An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

"Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions and questions to the generative AI.

"User authentication means" is a function for verifying and authenticating a user's identity.

"Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

"Operation means" is an interface that allows the user to input prompts to the generative AI.

"One-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

"Means for generating instructions for managing and operating robots within a factory" refers to a function for generating specific work instructions for robots within a factory.

"Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

"Means for transmitting to robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

This invention is a system that efficiently and safely manages and operates robots in a factory using accessory devices that can be connected to a smartphone. A specific embodiment of this system is described below.

System Configuration

The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

Hardware

Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

Accessory device: Connects to a smartphone and includes fingerprint authentication means and operation means.

Fingerprint authentication method: Scans the user's fingerprint for authentication.

Wireless communication method: Data is sent and received between smartphones and factory robots using Bluetooth, Wi-Fi, etc.

Factory robot: Receives instructions and performs actions.

Software

Generative AI model: Generates specific instructions for factory robots based on prompt input from the user.

Smartphone application: Provides the user interface and manages communication with the generative AI model.

Operating Procedure

1. User authentication: The user places their finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

2. Prompt input: An authenticated user uses an operation method (one-click operation method) to input a prompt to the generative AI model.

3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

5. Robot operation: The factory robot operates according to the instructions it receives.

Specific examples

For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions. Below are some example prompts:

"Move to assembly line 3 and begin welding."

"Move to the inspection area to perform quality checks."

"Go get part A and take it to assembly line 2."

This makes it possible to efficiently and safely manage and operate robots within factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user places their finger on the accessory device and is authenticated using the fingerprint authentication means.

Input: User's fingerprint

Data processing: The fingerprint authentication device scans the fingerprint and compares it with registered fingerprint data.

Output: Authentication result (success or failure)

Specific operation: The fingerprint authentication sensor scans the user's fingerprint and compares it with an internal database for authentication.

Step 2:

If authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

Input: Authentication result (success), user prompt

Data processing: If the authentication result is successful, a prompt text is sent to the generative AI model.

Output: Prompt input to the generative AI model

Specific operation: When the user presses the button, the smartphone application sends a prompt statement to the generative AI model.

Step 3:

Based on the prompts, a generative AI model generates specific instructions for the factory robot.

Input: User prompt

Data processing: The generative AI model analyzes the prompt and generates appropriate instructions.

Output: Instructions to factory robots

Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

Step 4:

The generated instructions are sent to the factory robot using wireless communication means.

Input: Instructions from a generative AI model

Data processing: Send instructions to factory robots via wireless communication means.

Output: Send instructions to factory robots

Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

Step 5:

The factory robot will act according to the instructions it receives.

Input: Instructions for factory robots

Data processing: Factory robots analyze instructions and perform corresponding actions.

Output: Robot movement

Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

To use conventional generative AI models, users had to manually operate their smart devices and input prompts. This was time-consuming and inconvenient, especially when their hands were full or when a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. A system that solves these issues and allows users to use generative AI models more easily and safely is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for using the operation means to send a signal to the smart device and launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without having to directly operate the smart device, and the user authentication means also improves security.

"User authentication means" refers to the means used to verify a user's identity and grant access to a system.

"Operation means" refers to the interface or device that allows the user to operate the system.

A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

"Means for transmitting signals" refers to means for transmitting specific instructions or data from the operating means to the smart device.

A "specific application" is a software program designed to provide a specific function or service.

A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

A "prompt" is text input used to give instructions or ask questions to a generative AI model.

"Means for generating a response" refers to the means by which a generative AI model creates an appropriate response based on a prompt.

"Means for sending a response" refers to means for sending the generated response to a smart device.

Form for implementing the invention

This invention is a system that allows users to use generative AI models without directly operating a smart device. Specific embodiments of this system are described below.

Hardware and software used

Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

Software: Fingerprint authentication system, smart device application, generative AI model

System Configuration

1. User authentication method:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The fingerprint authentication system captures fingerprint data and authenticates the user.

2. Operating means:

If authentication is successful, the buttons on the accessory device will be enabled.

The user presses the enabled button.

3. Signal transmission method:

When the button is pressed, the accessory device sends a signal to the smart device.

The smart device receives the signal and launches a specific application.

4. Prompt sending method:

The launched application sends a prompt to the generative AI model.

Prompts are pre-defined and contain the instructions or questions the user wants to ask the generated AI model.

5. Response generation method:

The server uses a generative AI model to generate a response based on the received prompt.

6. Response sending method:

The server generates a response and sends it to the smart device.

The smart device receives the response and displays it to the user.

Specific examples

When a user places their finger on the fingerprint authentication sensor on the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device, which launches an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

Example prompt

"Tell me what the weather is like today."

"Check the schedule for the next meeting."

"Tell me the latest news."

This system allows users to easily use generative AI models without having to directly operate a smart device, and also improves security through user authentication measures.

The flow of the identification process in Example 2 will be explained using Figure 13.

Program processing flow

Step 1:

The user places their finger on the fingerprint authentication sensor on the accessory device.

Input: User fingerprint

Data processing: The fingerprint sensor captures fingerprint data.

Output: Captured fingerprint data

Step 2:

The device captures the fingerprint data and performs authentication.

Input: Captured fingerprint data

Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

Output: Authentication result (success or failure)

Step 3:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Data processing: If authentication is successful, enable the device button.

Output: Enabled button

Step 4:

The user presses the button.

Input: Enabled button

Data processing: The device detects that a button has been pressed.

Output: Button press signal

Step 5:

The terminal sends a signal to the smart device to launch the application.

Input: Button press signal

Data processing: The terminal sends a signal to the smart device to launch a specific application.

Output: Launched application

Step 6:

The application sends a prompt to the generative AI model.

Input: Launched application

Data processing: The application sends pre-defined prompts to the generative AI model.

Output: Prompt sent to the generative AI model

Step 7:

The server generates a response using a generative AI model.

Input: Prompt sent to the generative AI model

Data calculation: The generative AI model analyzes the prompt and generates an appropriate response.

Output: Generated response

Step 8:

The server sends the response to the smart device.

Input: Generated response

Data processing: The server sends the generated response to the smart device.

Output: Response sent to smart device

Step 9:

Your smart device will display the response.

Input: Response sent to smart device

Data processing: The smart device displays the response received to the user.

Output: Response displayed to the user

In this way, users can use generative AI models without directly operating their smart devices.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional food delivery systems require users to use their smartphones to place orders, making them cumbersome to operate. In addition, insufficient user authentication poses security risks. Furthermore, voice recognition-based ordering systems are not widespread, resulting in low user convenience. There is a need to solve these problems and provide a system that allows users to use food delivery services easily and safely.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

An accessory device connectable to a smart device includes a user authentication means that authenticates a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means that recognizes the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the results of the order using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

"Generative AI" is an artificial intelligence technology that generates text and other content based on user instructions and prompts.

"Instruction input" refers to commands or requests that a user inputs to a generative AI.

"User authentication means" refers to technology or devices used to verify and authenticate a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and authenticate the user based on that information.

"Operation means" refers to the interface or buttons that users use to operate accessory devices or smart devices.

"Voice recognition means" refers to technology or devices that analyze the user's voice and convert it into text data.

"Voice data" refers to data that has been digitally recorded from the user's voice.

"Transmission means" refers to the technology or equipment used to transmit data to other devices or systems.

This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

2. Accessory device: Additional hardware that is connected to a smart device and used, including fingerprint authentication means, operation means, and voice recognition means.

3. Generative AI: Artificial intelligence technology that generates text and other content based on user instructions and prompts.

Program processing

Hardware

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

Smart device: Functions as the application execution environment.

Microphone: Captures the user's voice as a means of voice recognition.

Software

Fingerprint authentication library: A software module for performing fingerprint authentication.

Speech recognition library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

Food delivery API: Place orders using the food delivery service's API.

Processing flow

1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device. If authentication is successful, the device proceeds to the next step.

2. Voice input: The user presses the operation means (one-click button) of the attached device and verbally states the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

3. Prompt generation: Generate a prompt sentence for the generative AI based on the acquired text data.

4. AI response generation: The generative AI model generates the optimal response based on the prompt.

5. Order execution: Based on the generated response, the order is completed using the food delivery API.

Specific examples

When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the generative AI model suggests the most suitable pizza restaurant and menu item, and completes the order using a food delivery API.

Example prompt

"User wants to order: Pizza"

In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphone.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint data.

Processing: The fingerprint authentication library analyzes the fingerprint data and compares it with registered fingerprint data.

Output: Authentication result (success or failure).

Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

Step 2:

The user presses the operating means (one-click button) of the accessory device.

Input: User operation (button press).

Processing: Detects that the operating means has been pressed and activates voice recognition mode.

Output: Activate voice recognition mode.

Specific operation: The sensor detects that a button has been pressed, and the voice recognition library begins preparing for voice input.

Step 3:

The user verbally communicates the order details.

Input: User's voice data.

Processing: The microphone captures the audio data, and the speech recognition library converts it into text data.

Output: Text data (order details).

Specific operation: The microphone captures the user's voice, and the speech recognition library analyzes the voice and converts it into text.

Step 4:

The device generates a prompt based on the acquired text data.

Input: Text data (order details).

Processing: Convert the text data into a prompt in the format "The user would like to order: [Order details]".

Output: Prompt statement.

Specific operation: Converts text data into prompt text according to the format.

Step 5:

The device sends a prompt to the generative AI model.

Input: Prompt text.

Processing: The generative AI model analyzes the prompt and generates the optimal response.

Output: AI response (optimal order content).

Specific operation: The prompt sentence is sent to the generative AI model, and the AI generates a response.

Step 6:

The device uses the food delivery API based on the generated response to complete the order.

Input: AI response (optimal order content).

Processing: Fulfill orders using the food delivery API.

Output: Order confirmation (data indicating the order was completed successfully).

Specific operation: Sends the AI response to the food delivery API and confirms the order.

The server may further include means for recognizing the user's emotions using an emotion engine, which will be described later, and may obtain the output of the generative AI using the voice data acquired by the voice recognition means, a prompt statement including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt statement such as "User wants to order: pizza, user's emotion: sadness" may be generated, and the generative AI model may suggest the optimal pizza restaurant and menu item taking the user's emotion into consideration, and the order may be completed using a food delivery API.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, it recognizes emotions from the user's tone of voice and facial expression, and feeds this information back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

"Example 2"

Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user operates an accessory device that can be connected to a smartphone.

Step 2: The device's user authentication means authenticates the user. One possible authentication method is fingerprint authentication.

Step 3: The authenticated user provides prompt input to the generative AI.

Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

Step 5: The emotion engine feeds that information back to the generative AI.

Step 6: The generative AI generates an appropriate response based on the user's emotions.

"Example 2"

Step 1: The user speaks to the accessory device.

Step 2: The device's microphone captures the user's voice and sends the audio data to the emotion engine.

Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

Step 4: The emotion engine feeds the results back to the generative AI.

Step 5: The generative AI generates an appropriate response based on the user's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Existing smartphone accessory devices allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses based on the user's intentions and emotions, resulting in a lack of improvement in the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

A "generative AI model" is an artificial intelligence system that generates output such as text or voice based on input prompts.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

"User authentication means" refers to technical means for verifying a user's identity.

"Fingerprint authentication means" refers to a technical means that reads a user's fingerprint and performs authentication based on that information.

"Operation means" refers to the interface that allows the user to operate the accessory device.

A "one-click operation method" is an interface that allows users to complete an operation with a single click.

"Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

"Emotion information" is data regarding the user's emotional state obtained by emotion recognition means.

"Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment

The accessory device connects to your smartphone using wireless communication methods such as Bluetooth. The device includes the following main components:

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates them.

Action button: An interface that allows the user to start prompt input.

Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

Smartphone

The smartphone runs a dedicated application that works in conjunction with the accessory device. This application provides the following functions:

Prompt input interface: A screen where users can enter instructions or questions for the generative AI model.

Data transmission function: Sends prompts and emotion information to the server.

Server

The server provides backend services that handle data communication between smartphones and generative AI models. The server has the following functions:

Data relay function: Sends prompts and emotional information received from a smartphone to the generative AI model.

Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

Specific examples

Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

1. User authentication

The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

2. Prompt Input

The user opens a dedicated application on their smartphone and enters a prompt. For example, they might enter, "What's the weather like today?" The device then prepares to send this prompt to the generative AI model.

3. Emotion recognition

The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The device prepares this emotion data to send to the generative AI model.

4. Sending prompts to generative AI models

The device sends the prompt and emotion data entered by the user to the server. The server then sends this data to the generative AI model. The generative AI model generates an appropriate response based on the prompt and emotion data.

5. Receiving responses from generative AI models

The server receives the response from the generative AI model and forwards it to the device. The device then prepares to display this response to the user.

6. Displaying the response

The device displays the response received from the generative AI model to the user. For example, it might say, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

Example prompt

"What's the weather like today?"

"Please let me know when the next meeting is scheduled."

"Please tell me the latest news."

In this way, users can naturally interact with generative AI models through accessory devices.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1: User Authentication

The user places their finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

Step 2: Connect your accessory device to your smartphone

The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

Step 3: Enter the prompt

The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

Step 4: Emotion Recognition

The accessory device uses an emotion engine that senses the user's vocal tone and facial expressions to analyze the user's emotional state. The input is the user's vocal tone and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

Step 5: Send prompts to the generative AI model

The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

Step 6: Receive a response from the generative AI model

The server receives the response from the generative AI model and forwards it to the device. The input is the response data from the generative AI model, and the output is the result of forwarding to the device. The device prepares to display this response to the user. Once the response is received, the device displays "Response received successfully."

Step 7: View the response

The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts if necessary.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

An "accessory device connectable to a smartphone" is an external device that connects to a smartphone using wireless communication means and provides specific functions.

"Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

"User authentication means" refers to technology used to verify a user's identity and ensure that only specific users can operate the system.

"Fingerprint authentication" is a technology that scans a user's fingerprint and authenticates the user based on that information.

"Operation means" refers to the physical or software interface that allows the user to operate the system.

A "one-click operation means" is an interface that allows users to perform a specific function with a single click.

"Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

"Means for feeding back emotional information to the generative AI" is a function that sends recognized emotional information to the generative AI, allowing the generative AI to generate responses and suggestions based on that information.

This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

System Configuration

The system consists of the following main components:

1. Accessory devices that can be connected to smartphones

Wireless communication method: Connects to a smartphone using Bluetooth or other means.

User authentication method: User authentication is performed using a fingerprint sensor.

Operation method: The user uses a one-click operation method to input prompts to the generative AI.

Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

2. Generative AI

Prompt Input: Receives instructions or questions from the user and generates responses or suggestions.

Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

Program processing

The server performs the following process:

1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

3. Emotion recognition: Analyze the user's tone of voice and facial expressions to recognize emotions.

4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

Hardware and software used

Hardware: Bluetooth module, fingerprint sensor, camera (for facial recognition), microphone (for voice recognition)

Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

Specific examples

If the generative AI recognizes that the user is tired, it will suggest "food delivery options recommended for when you're tired." For example, it might generate a prompt like this:

The user is feeling tired. Please suggest some food delivery options.

In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The server connects the smartphone and accessory device using a Bluetooth module. The input is pairing information between the smartphone and accessory device, and the output is the establishment of the connection. Specifically, it uses the Bluetooth connection library to establish communication between the smartphone and accessory device.

Step 2:

The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

Step 3:

The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotional information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotions.

Step 4:

The server generates a prompt for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the emotional information is converted into text format and a prompt suitable for the generative AI is created.

Step 5:

The server uses a generative AI model to generate responses and suggestions in response to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI generates the response or suggestion.

Step 6:

The device presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional systems using generative AI models required users to operate their smartphones, making them difficult to use. They also were unable to generate appropriate responses based on the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing their emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to obtain an appropriate response according to their emotions.

"User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

"Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

"Emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from its tone and content.

A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

This invention provides a system that includes a user authentication means, an operation means, an emotion recognition means, and a feedback means. Specific embodiments of this system are described below.

First, the user places their finger on the accessory device's fingerprint authentication sensor. The fingerprint authentication sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which compares the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Please tell me about today's weather" is input to the generative AI model.

Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotions. The analysis results are sent to the generative AI model via the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," it generates the response "Let's take a break today and refresh ourselves."

This system allows users to easily use generative AI models without having to operate a smartphone. It also improves the user experience by enabling appropriate responses based on the user's emotions.

For example, when a user places their finger on the fingerprint authentication sensor on an accessory device and presses a button after successful authentication, an application on the smartphone launches and the following prompt is input into the generative AI model:

"Can you tell me about today's weather?"

As another example, when a user asks an accessory device, "How are you feeling today?", the emotion recognition means recognizes the emotion from the user's tone of voice and inputs the following prompt sentence into the generative AI model:

"The user seems a little tired. Please generate an encouraging message."

This allows users to easily utilize generative AI without having to operate their smartphone.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint

Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

Output: Captured fingerprint data

Step 2:

The device captures fingerprint data and sends it to the server.

Input: Captured fingerprint data

Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and, when sending that data to the server, displays "Authentication in progress..." on the device screen.

Output: Fingerprint data sent to the server

Step 3:

The server authenticates the fingerprint data.

Input: Fingerprint data sent to the server

Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

Output: Authentication result (success or failure)

Step 4:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Specific operation: If the server successfully authenticates, it sends a signal to the device to activate the button. The button on the device will light up green.

Output: Enabled button

Step 5:

The user presses the button.

Input: Enabled button

Specific Action: The user presses the enabled button, which triggers the next step.

Output: Button press signal

Step 6:

The device launches the application on the smartphone.

Input: Button press signal

Specific operation: The device launches a dedicated application on the smartphone. The application logo will appear on the smartphone screen.

Output: Launched application

Step 7:

The application prompts input to the generative AI model.

Input: Launched application

Specific operation: The application inputs a prompt to the generative AI model. The screen displays "Sending prompt...". For example, the prompt input is "Tell me about the weather today."

Output: Prompt sent to the generative AI model

Step 8:

The user speaks to the accessory device.

Input: User's voice

Specific operation: The user speaks to the accessory device. For example, "How are you feeling today?" The device's microphone lights up red, indicating that it is capturing audio data.

Output: Captured audio data

Step 9:

The device captures voice data and sends it to the emotion engine.

Input: Captured audio data

Specific operation: The device uses the built-in microphone to capture audio data and sends that data to the emotion engine.

Output: Audio data sent to the emotion engine

Step 10:

The emotion engine analyzes the user's emotions from the tone of the voice data.

Input: Audio data sent to the emotion engine

Specific operation: The emotion engine analyzes the tone of the voice data and determines the user's emotions. The device screen will display "Analyzing emotions..."

Output: Analyzed emotional information

Step 11:

The emotion engine feeds the analysis results back into the generative AI model.

Input: Analyzed emotional information

Specific operation: The emotion engine feeds the analysis results back to the generative AI model. For example, it sends information such as "The user seems a little tired."

Output: Emotional information sent to the generative AI model

Step 12:

The generative AI model generates responses based on the user's emotions.

Input: Emotional information sent to the generative AI model

Specific operation: The generative AI model generates an appropriate response based on the emotional information fed back. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today to refresh yourself." The response is displayed on the smartphone screen.

Output: Generated response

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

With conventional food delivery services, when users order using their smartphones, the process is cumbersome and they don't offer appropriate suggestions based on the user's mood. In particular, there are issues with the lack of relaxing meal suggestions for tired or stressed users, which hinders the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for enabling prompt input to the generative AI, user authentication means for operating the accessory device, operation means for a user authenticated by the user authentication means to input prompts to the generative AI, voice capture means for capturing the user's voice and recognizing their emotions, emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotions, and means for the generative AI to generate an appropriate response based on the emotions recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

"Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

"Prompt input" refers to the act of inputting instructions or questions to generative AI.

An "accessory device" is an external device that can be connected to a smartphone to provide specific functions.

"User authentication means" refers to a means for verifying a user's identity and ensuring that only specific users can operate the system.

"Fingerprint authentication means" refers to a means of authenticating a user by reading their fingerprint.

"Operation means" refers to the interface, buttons, and other means that users use to operate the system.

A "one-click operation method" is a method that allows the user to complete an operation with a single click.

"Audio capture means" means for recording or capturing a user's voice.

"Emotion recognition means" refers to a means for analyzing captured voice data and recognizing the user's emotions.

"Response" refers to the response or suggestion that generative AI generates in response to user input.

A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method. An AI generation model (e.g., AIGenerator) is used as the generative AI.

For example, if a user says, "I'm tired today," the voice capture means captures the voice and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

Examples of prompts to input to a generative AI model include:

User Emotion: Fatigue

Prompt for generative AI: "Please recommend a relaxing meal for a tired user."

This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate their smartphone.

The flow of the specific processing in Application Example 2 is explained using Figure 18.

Step 1:

The user places their finger on the fingerprint authentication sensor of an accessory device connected to their smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

Step 2:

If authentication is successful, the accessory device's operation means is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and begins prompt input to the generative AI.

Step 3:

The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and the voice recognition software converts the data into a digital format.

Step 4:

The captured voice data is sent to the emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

Step 5:

The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt sentence to the generative AI. The generative AI creates a prompt sentence to generate an appropriate response based on the emotional data.

Step 6:

The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

Step 7:

The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user via the terminal's display or audio output device.

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is

These results are obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. Specifically, the accessory device is connected to the smartphone using a wireless communication means such as Bluetooth, and prompt input to the generative AI is performed through an application running on the smartphone. The user authentication means includes a fingerprint authentication sensor built into the device, which performs authentication by comparing the fingerprint information with the registered user's fingerprint. The operation means involves the user pressing a button on the device, which inputs prompts to the generative AI.

"Example 2"

As a specific example of use, a user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device, and if authentication is successful, the button on the device is enabled. When the user presses the button, an application on the smartphone is launched and a prompt input is sent to the generative AI. This allows the user to easily utilize generative AI without having to operate the smartphone.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user connects an accessory device that can be connected to a smartphone to the smartphone using a wireless communication method such as Bluetooth.

Step 2: The user places their finger on the fingerprint authentication sensor on the accessory device, and the fingerprint authentication sensor authenticates the user by matching it with the registered user's fingerprint information.

Step 3: If authentication is successful, the device's buttons will be enabled.

Step 4: When the user presses a button on the device, an application on the smartphone is launched and prompts are input to the generative AI.

"Example 2"

Step 1: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device.

Step 2: If authentication is successful, the device's buttons will be enabled.

Step 3: When the user presses the button, an application on the smartphone is launched and prompt input is sent to the generative AI.

Step 4: This allows users to easily utilize generative AI without having to operate their smartphone.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In systems that use conventional generative AI models, users are required to perform complex operations and undergo multi-step authentication when entering prompt text, resulting in a poor user experience. Furthermore, there is a lack of efficient means for quickly and accurately obtaining generated results. This reduces the convenience for users when using generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for running an application on the smartphone for inputting prompts to the generative AI; and means for sending prompts to the generative AI model and receiving generation results. This allows the user to input prompts to the generative AI model with simple operations and quickly and accurately obtain generation results.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer, and can be used to install applications and utilize a variety of functions.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

"Generative AI" is an artificial intelligence model that generates text, images, etc. based on input prompts.

"Prompt input" is the act of inputting instructions or questions to a generative AI.

"User authentication means" refers to technology or devices used to verify a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and verify the user's identity based on that information.

"Operation means" refers to the interface or device that allows the user to operate the accessory device or smartphone.

A "one-click operation means" is an interface or device that allows a user to perform a specific function with a single click.

An "application" is software that runs on a smartphone and provides specific functions or services.

"Generation results" are output such as text and images generated by the generative AI based on the prompt text.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to easily input prompts into a generative AI model. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment:

Accessory devices are external devices that connect to your smartphone and have the following main functions:

User authentication method: Equipped with a fingerprint authentication sensor that reads the user's fingerprint information.

Operation method: Equipped with a one-click button, making it easy for users to operate.

Smartphone:

The smartphone is connected to the accessory device using wireless communication means such as Bluetooth. An application for providing prompt input to the generative AI model is installed on the smartphone.

Generative AI model:

The generative AI model runs on a server and generates text and images based on prompts sent from the smartphone.

System Operation

User authentication:

The user places their finger on the fingerprint authentication sensor installed on the accessory device. The accessory device reads the fingerprint information and compares it with the fingerprint information stored in its internal database. If authentication is successful, the accessory device sends a signal indicating successful authentication to the smartphone.

Prompt Input:

After successful authentication, the user presses a button on the accessory device. The smartphone application launches and a prompt input screen appears. The user enters a prompt statement for the generative AI model.

Send to generative AI model:

The smartphone sends the input prompt to the generative AI model. The server operates the generative AI model based on the received prompt and creates the generated results.

Display generated results:

The server sends the generated results to the smartphone. The smartphone displays the received results to the user. The user can check the displayed results and save or share them as needed.

Specific examples

If a user wants to have a generative AI create a new poem, the specific steps are as follows:

1. User authentication:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The accessory device reads the fingerprint and performs authentication.

If authentication is successful, a signal indicating successful authentication will be sent to your smartphone.

2. Prompt input:

The user presses a button on the accessory device.

The smartphone application will launch and a prompt input screen will appear.

The user enters the prompt text "Create a new poem."

3. Send to generative AI model:

The smartphone sends the entered prompt to the generative AI model.

The server runs a generative AI model based on the prompt text to generate the poem.

4. Displaying the generated results:

The server sends the generated poem to your smartphone.

The smartphone will then display the received poem to the user.

Users can review the displayed poems and save or share them as needed.

In this way, users can use the accessory device and smartphone to provide prompt input to the generative AI model and obtain generated results.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Connecting Accessory Devices

Input: User opens their smartphone's Bluetooth settings and selects an accessory device.

How it works: Your smartphone establishes a connection with the accessory device using Bluetooth.

Output: The smartphone notifies you that the connection was successful.

Step 2: User Authentication

Input: The user places their finger on the fingerprint sensor on the accessory device.

How it works: The accessory device reads your fingerprint and compares it with the fingerprint information stored in its internal database.

Output: If authentication is successful, the accessory device sends a signal to the smartphone indicating successful authentication.

Step 3: Prepare for prompt input

Input: The smartphone receives a signal indicating successful authentication.

How it works: The smartphone displays a prompt input screen for the generative AI.

Output: The user sees the prompt input screen and begins entering the prompt statement.

Step 4: Enter the prompt

Input: The user enters a prompt for the generative AI through a smartphone application.

How it works: The smartphone receives the entered prompt text and formats the data internally.

Output: The prepared prompt is ready to be sent to the generative AI model.

Step 5: Send to the generative AI model

Input: The smartphone sends the prepared prompt to the generative AI model.

Operation: The server runs the generative AI model based on the received prompt text and creates the generated results.

Output: The generated results are sent from the server to the smartphone.

Step 6: View the generated results

Input: The smartphone receives the generated results from the server.

How it works: The smartphone displays the generated results to the user.

Output: The user checks the displayed generated results and saves or shares them as needed.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

To efficiently and safely manage and operate robots in factories, it is necessary to transmit instructions to the robots quickly and accurately. However, conventional methods involve complex operations and lack sufficient user authentication security, creating the risk of operational errors and unauthorized access. This can potentially reduce factory productivity and safety.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; means for generating instructions for managing and operating robots in a factory; and means for transmitting the instructions to the robots using wireless communication means. This makes it possible to efficiently and safely manage and operate robots in a factory.

An "accessory device connectable to a smartphone" is a device that connects to a smartphone using wireless communication means and provides additional functions.

"Generative AI" is an artificial intelligence system that generates instructions and information based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

"User authentication means" is a function for verifying and authenticating a user's identity.

"Fingerprint authentication means" is a function that scans a user's fingerprint and uses that information to authenticate the user.

"Operation means" is an interface that allows the user to input prompts to the generative AI.

"One-click operation means" is a function that allows the user to input prompts to the generative AI with a single click.

"Means for generating instructions for managing and operating robots within a factory" refers to a function for generating specific work instructions for robots within a factory.

"Wireless communication means" refers to a function for sending and receiving data between devices using wireless communication technologies such as Bluetooth and Wi-Fi.

"Means for transmitting to robot" is a function for transmitting the generated instructions to the robot using wireless communication means.

This invention is a system that efficiently and safely manages and operates robots in a factory using accessory devices that can be connected to a smartphone. A specific embodiment of this system is described below.

System Configuration

The system consists of a smartphone, accessory devices, a generative AI model, wireless communication means, fingerprint authentication means, operation means, and a factory robot.

Hardware

Smartphone: Functions as a user interface and connects to accessory devices via wireless communication means.

Accessory device: Connects to a smartphone and includes fingerprint authentication means and operation means.

Fingerprint authentication method: Scans the user's fingerprint for authentication.

Wireless communication method: Data is sent and received between smartphones and factory robots using Bluetooth, Wi-Fi, etc.

Factory robot: Receives instructions and performs actions.

Software

Generative AI model: Generates specific instructions for factory robots based on prompt input from the user.

Smartphone application: Provides the user interface and manages communication with the generative AI model.

Operating Procedure

1. User authentication: The user places their finger on the accessory device and authenticates using the fingerprint authentication method. If authentication is successful, the user can enter prompts.

2. Prompt input: An authenticated user uses an operation method (one-click operation method) to input a prompt to the generative AI model.

3. Instruction generation: Based on the prompts, the generative AI model generates specific instructions for the factory robot.

4. Instruction transmission: The generated instructions are transmitted to the factory robot using wireless communication means.

5. Robot operation: The factory robot operates according to the instructions it receives.

Specific examples

For example, when a robot needs to perform a specific task in a factory, a manager can use a smartphone and accessory devices to give the robot instructions. Below are some example prompts:

"Move to assembly line 3 and begin welding."

"Move to the inspection area to perform quality checks."

"Go get part A and take it to assembly line 2."

This makes it possible to efficiently and safely manage and operate robots within factories. The system utilizes user authentication and generative AI to reduce the risk of incorrect operation and unauthorized access, improving factory productivity and safety.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user places their finger on the accessory device and is authenticated using the fingerprint authentication means.

Input: User's fingerprint

Data processing: The fingerprint authentication device scans the fingerprint and compares it with registered fingerprint data.

Output: Authentication result (success or failure)

Specific operation: The fingerprint authentication sensor scans the user's fingerprint and compares it with an internal database for authentication.

Step 2:

If authentication is successful, the user uses the operation means (one-click operation means) to input a prompt to the generative AI model.

Input: Authentication result (success), user prompt

Data processing: If the authentication result is successful, a prompt text is sent to the generative AI model.

Output: Prompt input to the generative AI model

Specific operation: When the user presses the button, the smartphone application sends a prompt statement to the generative AI model.

Step 3:

Based on the prompts, a generative AI model generates specific instructions for the factory robot.

Input: User prompt

Data processing: The generative AI model analyzes the prompt and generates appropriate instructions.

Output: Instructions to factory robots

Specific actions: The generative AI model analyzes the prompt and generates specific instructions, such as "Move to assembly line 3 and start welding."

Step 4:

The generated instructions are sent to the factory robot using wireless communication means.

Input: Instructions from a generative AI model

Data processing: Send instructions to factory robots via wireless communication means.

Output: Send instructions to factory robots

Specific operation: Instructions are sent from a smartphone to a factory robot using Bluetooth or Wi-Fi.

Step 5:

The factory robot will act according to the instructions it receives.

Input: Instructions for factory robots

Data processing: Factory robots analyze instructions and perform corresponding actions.

Output: Robot movement

Specific operation: The factory robot operates according to instructions such as "move to assembly line 3 and start welding."

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

To use conventional generative AI models, users had to manually operate their smart devices and input prompts. This was time-consuming and inconvenient, especially when their hands were full or when a quick response was required. Furthermore, from a security perspective, user authentication was sometimes insufficient, creating a risk of unauthorized use by third parties. A system that solves these issues and allows users to use generative AI models more easily and safely is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for operation by an authenticated user, a means for using the operation means to send a signal to the smart device and launch a specific application, a means for the launched application to send a prompt sentence to the generative AI model, and a means for the generative AI model to generate a response based on the prompt sentence and send the response to the smart device. This allows the user to easily use the generative AI model without having to directly operate the smart device, and the user authentication means also improves security.

"User authentication means" refers to the means used to verify a user's identity and grant access to a system.

"Operation means" refers to the interface or device that allows the user to operate the system.

A "smart device" is an electronic device that can connect to the Internet and has the ability to run applications.

"Means for transmitting signals" refers to means for transmitting specific instructions or data from the operating means to the smart device.

A "specific application" is a software program designed to provide a specific function or service.

A "generative AI model" is an artificial intelligence model that generates a response based on an input prompt.

A "prompt" is text input used to give instructions or ask questions to a generative AI model.

"Means for generating a response" refers to the means by which a generative AI model creates an appropriate response based on a prompt.

"Means for sending a response" refers to means for sending the generated response to a smart device.

Form for implementing the invention

This invention is a system that allows users to use generative AI models without directly operating a smart device. Specific embodiments of this system are described below.

Hardware and software used

Hardware: Accessory devices (with fingerprint authentication sensors), smart devices (smartphones, etc.)

Software: Fingerprint authentication system, smart device application, generative AI model

System Configuration

1. User authentication method:

The user places their finger on the fingerprint authentication sensor on the accessory device.

The fingerprint authentication system captures fingerprint data and authenticates the user.

2. Operating means:

If authentication is successful, the buttons on the accessory device will be enabled.

The user presses the enabled button.

3. Signal transmission method:

When the button is pressed, the accessory device sends a signal to the smart device.

The smart device receives the signal and launches a specific application.

4. Prompt sending method:

The launched application sends a prompt to the generative AI model.

Prompts are pre-defined and contain the instructions or questions the user wants to ask the generated AI model.

5. Response generation method:

The server uses a generative AI model to generate a response based on the received prompt.

6. Response sending method:

The server generates a response and sends it to the smart device.

The smart device receives the response and displays it to the user.

Specific examples

When a user places their finger on the fingerprint authentication sensor on the accessory device, the terminal captures the fingerprint data and performs authentication. If authentication is successful, the button on the terminal is enabled. When the user presses the button, the terminal sends a signal to the smart device, which launches an application. The application launches and sends a prompt sentence, "Tell me what the weather is today," to the generative AI model. The server uses the generative AI model to generate a response, "Today's weather is sunny," and sends it to the smart device. The smart device displays the response, and the user can see the information, "Today's weather is sunny," on the screen.

Example prompt

"Tell me what the weather is like today."

"Check the schedule for the next meeting."

"Tell me the latest news."

This system allows users to easily use generative AI models without having to directly operate a smart device, and also improves security through user authentication measures.

The flow of the identification process in Example 2 will be explained using Figure 13.

Program processing flow

Step 1:

The user places their finger on the fingerprint authentication sensor on the accessory device.

Input: User fingerprint

Data processing: The fingerprint sensor captures fingerprint data.

Output: Captured fingerprint data

Step 2:

The device captures the fingerprint data and performs authentication.

Input: Captured fingerprint data

Data calculation: The fingerprint authentication system analyzes the fingerprint data and authenticates the user.

Output: Authentication result (success or failure)

Step 3:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Data processing: If authentication is successful, enable the device button.

Output: Enabled button

Step 4:

The user presses the button.

Input: Enabled button

Data processing: The device detects that a button has been pressed.

Output: Button press signal

Step 5:

The terminal sends a signal to the smart device to launch the application.

Input: Button press signal

Data processing: The terminal sends a signal to the smart device to launch a specific application.

Output: Launched application

Step 6:

The application sends a prompt to the generative AI model.

Input: Launched application

Data processing: The application sends pre-defined prompts to the generative AI model.

Output: Prompt sent to the generative AI model

Step 7:

The server generates a response using a generative AI model.

Input: Prompt sent to the generative AI model

Data calculation: The generative AI model analyzes the prompt and generates an appropriate response.

Output: Generated response

Step 8:

The server sends the response to the smart device.

Input: Generated response

Data processing: The server sends the generated response to the smart device.

Output: Response sent to smart device

Step 9:

Your smart device will display the response.

Input: Response sent to smart device

Data processing: The smart device displays the response received to the user.

Output: Response displayed to the user

In this way, users can use generative AI models without directly operating their smart devices.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional food delivery systems require users to use their smartphones to place orders, making them cumbersome to operate. In addition, insufficient user authentication poses security risks. Furthermore, voice recognition-based ordering systems are not widespread, resulting in low user convenience. There is a need to solve these problems and provide a system that allows users to use food delivery services easily and safely.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

An accessory device connectable to a smart device includes a user authentication means that authenticates a user operating the accessory device through fingerprint authentication, a button provided on the accessory device that a user authenticated by the user authentication means presses to input instructions to the generative AI, and a voice recognition means that recognizes the user's voice in response to the button being pressed. The server includes a means for acquiring the output of the generative AI using a prompt statement including voice data acquired by the voice recognition means and the generative AI. The prompt statement instructs the user to place an order through a food delivery service, and the acquiring means acquires the results of the order using a food delivery API. This allows users to easily and safely use food delivery services using only their voice, without having to operate their smartphone.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that has internet connectivity and can run applications.

An "accessory device" is additional hardware that is connected to a smart device to provide specific functionality.

"Generative AI" is an artificial intelligence technology that generates text and other content based on user instructions and prompts.

"Instruction input" refers to commands or requests that a user inputs to a generative AI.

"User authentication means" refers to technology or devices used to verify and authenticate a user's identity.

"Fingerprint authentication means" refers to technology or devices that read a user's fingerprint and authenticate the user based on that information.

"Operation means" refers to the interface or buttons that users use to operate accessory devices or smart devices.

"Voice recognition means" refers to technology or devices that analyze the user's voice and convert it into text data.

"Voice data" refers to data that has been digitally recorded from the user's voice.

"Transmission means" refers to the technology or equipment used to transmit data to other devices or systems.

This invention provides a system that allows users to easily and safely use generative AI using an accessory device that can be connected to a smart device. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. Smart device: A portable electronic device such as a smartphone or tablet that has internet connectivity and can run applications.

2. Accessory device: Additional hardware that is connected to a smart device and used, including fingerprint authentication means, operation means, and voice recognition means.

3. Generative AI: Artificial intelligence technology that generates text and other content based on user instructions and prompts.

Program processing

Hardware

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates the user based on that information.

Smart device: Functions as the application execution environment.

Microphone: Captures the user's voice as a means of voice recognition.

Software

Fingerprint authentication library: A software module for performing fingerprint authentication.

Speech recognition library: A software module for converting speech into text data (e.g., Google® Speech-to-Text).

Generative AI model: An AI model that generates responses based on user prompts (e.g., GPT-3 (registered trademark)).

Food delivery API: Place orders using the food delivery service's API.

Processing flow

1. User authentication: The user authenticates by placing their finger on the fingerprint authentication sensor on the accessory device. If authentication is successful, the device proceeds to the next step.

2. Voice input: The user presses the operation means (one-click button) of the attached device and verbally states the order details. The microphone picks up the voice, and the voice recognition library converts it into text data.

3. Prompt generation: Generate a prompt sentence for the generative AI based on the acquired text data.

4. AI response generation: The generative AI model generates the optimal response based on the prompt.

5. Order execution: Based on the generated response, the order is completed using the food delivery API.

Specific examples

When a user places their finger on the fingerprint sensor for authentication and says "I want to order pizza," the speech recognition library converts this into text data. When the generative AI model receives the prompt "The user wants to order: pizza," the generative AI model suggests the most suitable pizza restaurant and menu item, and completes the order using a food delivery API.

Example prompt

"User wants to order: Pizza"

In this way, users can easily and safely use food delivery services using only their voice, without having to operate their smartphone.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint data.

Processing: The fingerprint authentication library analyzes the fingerprint data and compares it with registered fingerprint data.

Output: Authentication result (success or failure).

Specific operation: The fingerprint authentication sensor reads the user's fingerprint, and the fingerprint authentication library analyzes the data to perform authentication.

Step 2:

The user presses the operating means (one-click button) of the accessory device.

Input: User operation (button press).

Processing: Detects that the operating means has been pressed and activates voice recognition mode.

Output: Activate voice recognition mode.

Specific operation: The sensor detects that a button has been pressed, and the voice recognition library begins preparing for voice input.

Step 3:

The user verbally communicates the order details.

Input: User's voice data.

Processing: The microphone captures the audio data, and the speech recognition library converts it into text data.

Output: Text data (order details).

Specific operation: The microphone captures the user's voice, and the speech recognition library analyzes the voice and converts it into text.

Step 4:

The device generates a prompt based on the acquired text data.

Input: Text data (order details).

Processing: Convert the text data into a prompt in the format "The user would like to order: [Order details]".

Output: Prompt statement.

Specific operation: Converts text data into prompt text according to the format.

Step 5:

The device sends a prompt to the generative AI model.

Input: Prompt text.

Processing: The generative AI model analyzes the prompt and generates the optimal response.

Output: AI response (optimal order content).

Specific operation: The prompt sentence is sent to the generative AI model, and the AI generates a response.

Step 6:

The device uses the food delivery API based on the generated response to complete the order.

Input: AI response (optimal order content).

Processing: Fulfill orders using the food delivery API.

Output: Order confirmation (data indicating the order was completed successfully).

Specific operation: Sends the AI response to the food delivery API and confirms the order.

The server may further include means for recognizing the user's emotions using an emotion engine, which will be described later, and may obtain the output of the generative AI using the voice data acquired by the voice recognition means, a prompt statement including the user's emotion, and the generative AI. For example, if the user's emotion is recognized as "sadness," a prompt statement such as "User wants to order: pizza, user's emotion: sadness" may be generated, and the generative AI model may suggest the optimal pizza restaurant and menu item taking the user's emotion into consideration, and the order may be completed using a food delivery API.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides an accessory device connectable to a smartphone. This device includes a means for enabling prompt input to the generative AI, a user authentication means, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI. The device also includes an emotion engine that recognizes the user's emotions. Specifically, it recognizes emotions from the user's tone of voice and facial expression, and feeds this information back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotions.

"Example 2"

Another embodiment of the present invention provides a system in which an emotion engine recognizes emotions from the tone of a user's voice. Specifically, when a user speaks to an accessory device, a microphone within the device captures the user's voice and transmits the voice data to the emotion engine. The emotion engine analyzes the user's emotion from the tone of the voice data and feeds the results back to the generative AI. This enables the generative AI to generate an appropriate response according to the user's emotion.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user operates an accessory device that can be connected to a smartphone.

Step 2: The device's user authentication means authenticates the user. One possible authentication method is fingerprint authentication.

Step 3: The authenticated user provides prompt input to the generative AI.

Step 4: The device's emotion engine recognizes emotions from the user's tone of voice and facial expressions.

Step 5: The emotion engine feeds that information back to the generative AI.

Step 6: The generative AI generates an appropriate response based on the user's emotions.

"Example 2"

Step 1: The user speaks to the accessory device.

Step 2: The device's microphone captures the user's voice and sends the audio data to the emotion engine.

Step 3: The emotion engine analyzes the user's emotions from the tone of the voice data.

Step 4: The emotion engine feeds the results back to the generative AI.

Step 5: The generative AI generates an appropriate response based on the user's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Existing smartphone accessory devices allow prompt input to generative AI models, but they have the problem of being unable to generate responses that take the user's emotional state into account. This makes it difficult to obtain appropriate responses based on the user's intentions and emotions, resulting in a lack of improvement in the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone, and includes: means for enabling prompt input to the generative AI model; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI model; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI model. This makes it possible to obtain an appropriate response from the generative AI model that takes into account the user's emotional state.

A "smartphone" is a mobile information terminal that combines the functions of a mobile phone and a computer.

An "accessory device" is an external device that connects to a smartphone to provide additional functionality.

A "generative AI model" is an artificial intelligence system that generates output such as text or voice based on input prompts.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

"User authentication means" refers to technical means for verifying a user's identity.

"Fingerprint authentication means" refers to a technical means that reads a user's fingerprint and performs authentication based on that information.

"Operation means" refers to the interface that allows the user to operate the accessory device.

A "one-click operation method" is an interface that allows users to complete an operation with a single click.

"Emotion recognition means" refers to a technical means for recognizing emotions by analyzing the user's tone of voice and facial expressions.

"Emotion information" is data regarding the user's emotional state obtained by emotion recognition means.

"Feedback means" refers to the technical means for transmitting acquired emotional information to the generative AI model.

Form for implementing the invention

This invention is a system that uses an accessory device connectable to a smartphone to provide prompt input to a generative AI model and obtain a response that takes into account the user's emotional state. A specific embodiment of this system is described below.

Hardware and software configuration

Accessory equipment

The accessory device connects to your smartphone using wireless communication methods such as Bluetooth. The device includes the following main components:

Fingerprint authentication sensor: Reads the user's fingerprint and authenticates them.

Action button: An interface that allows the user to start prompt input.

Emotion recognition engine: Analyzes the user's tone of voice and facial expressions to obtain emotional information.

Smartphone

The smartphone runs a dedicated application that works in conjunction with the accessory device. This application provides the following functions:

Prompt input interface: A screen where users can enter instructions or questions for the generative AI model.

Data transmission function: Sends prompts and emotion information to the server.

Server

The server provides backend services that handle data communication between smartphones and generative AI models. The server has the following functions:

Data relay function: Sends prompts and emotional information received from a smartphone to the generative AI model.

Response reception function: Receives responses from the generative AI model and forwards them to the smartphone.

Specific examples

Consider a scenario in which a user uses an accessory device connected to a smartphone to ask a generative AI model questions.

1. User authentication

The user places their finger on the accessory device's fingerprint authentication sensor to authenticate. The terminal (smartphone) receives the fingerprint data from the accessory device and compares it with registered fingerprint data. If authentication is successful, the user can proceed to the next step.

2. Prompt Input

The user opens a dedicated application on their smartphone and enters a prompt. For example, they might enter, "What's the weather like today?" The device then prepares to send this prompt to the generative AI model.

3. Emotion recognition

The accessory device analyzes the user's emotional state using an emotion engine that senses the user's tone of voice and facial expressions. The device prepares this emotion data to send to the generative AI model.

4. Sending prompts to generative AI models

The device sends the prompt and emotion data entered by the user to the server. The server then sends this data to the generative AI model. The generative AI model generates an appropriate response based on the prompt and emotion data.

5. Receiving responses from generative AI models

The server receives the response from the generative AI model and forwards it to the device. The device then prepares to display this response to the user.

6. Displaying the response

The device displays the response received from the generative AI model to the user. For example, it might say, "It's sunny today. Have a nice day." The user can review this response and enter further prompts if necessary.

Example prompt

"What's the weather like today?"

"Please let me know when the next meeting is scheduled."

"Please tell me the latest news."

In this way, users can naturally interact with generative AI models through accessory devices.

The flow of the identification process in Example 1 will be explained using Figure 15.

Step 1: User Authentication

The user places their finger on the fingerprint authentication sensor of the accessory device. The terminal receives the fingerprint data from the accessory device and compares it with registered fingerprint data. The input is the user's fingerprint data, and the output is the authentication result. The terminal analyzes the fingerprint data and checks whether it matches the registered data. If authentication is successful, the terminal displays "Authentication successful."

Step 2: Connect your accessory device to your smartphone

The terminal establishes a connection with an accessory device via Bluetooth communication. The input is a Bluetooth connection request and the output is the connection status. The terminal receives a connection request from the accessory device and establishes the connection. The server monitors this connection and relays data as necessary. If the connection is successful, the terminal displays "Connection successful."

Step 3: Enter the prompt

The user opens a dedicated application on their smartphone and inputs a prompt. For example, they might input "What's the weather like today?" The input is the user's prompt, and the output is the prompt data. The device prepares to send this prompt to the generative AI model. When the user has completed their input, the device displays "Prompt input completed."

Step 4: Emotion Recognition

The accessory device uses an emotion engine that senses the user's vocal tone and facial expressions to analyze the user's emotional state. The input is the user's vocal tone and facial expression data, and the output is emotion data. The device prepares this emotion data to send to the generative AI model. Once the emotion data is acquired, the device displays "Emotion data acquisition successful."

Step 5: Send prompts to the generative AI model

The device sends the prompt and emotion data entered by the user to the server. The input is the prompt data and emotion data, and the output is the result of transmission to the server. The server then sends this data to the generative AI model. If the data transmission is successful, the device displays "Data transmission successful."

Step 6: Receive a response from the generative AI model

The server receives the response from the generative AI model and forwards it to the device. The input is the response data from the generative AI model, and the output is the result of forwarding to the device. The device prepares to display this response to the user. Once the response is received, the device displays "Response received successfully."

Step 7: View the response

The device displays the response received from the generative AI model to the user. For example, it displays "It's sunny today. Have a nice day." The input is the response data from the generative AI model, and the output is the display result to the user. The user can review this response and enter further prompts if necessary.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional food delivery services struggled to provide customized recommendations based on users' emotions and circumstances. They also lacked a means for users to use generative AI to simplify the ordering process. This could result in a poor user experience and reduced frequency of service use.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server is an accessory device connectable to a smartphone and includes: means for enabling prompt input to the generative AI; user authentication means for operating the accessory device; operation means for a user authenticated by the user authentication means to input prompts to the generative AI; emotion recognition means for recognizing the user's emotions; and means for feeding back emotion information recognized by the emotion recognition means to the generative AI. This makes it possible to propose customized food delivery according to the user's emotions.

An "accessory device connectable to a smartphone" is an external device that connects to a smartphone using wireless communication means and provides specific functions.

"Generative AI" is an artificial intelligence system that generates responses and suggestions based on prompt input from the user.

"Means to enable prompt input" is a function that provides an interface for users to input instructions or questions to the generative AI.

"User authentication means" refers to technology used to verify a user's identity and ensure that only specific users can operate the system.

"Fingerprint authentication" is a technology that scans a user's fingerprint and authenticates the user based on that information.

"Operation means" refers to the physical or software interface that allows the user to operate the system.

A "one-click operation means" is an interface that allows users to perform a specific function with a single click.

"Emotion recognition means" is a technology that analyzes the user's tone of voice and facial expressions to recognize their emotions.

"Means for feeding back emotional information to the generative AI" is a function that sends recognized emotional information to the generative AI, allowing the generative AI to generate responses and suggestions based on that information.

This invention is a system that uses an accessory device connectable to a smartphone to simplify the ordering process when a user uses a food delivery service through generative AI and makes customized suggestions based on the user's emotions.

System Configuration

The system consists of the following main components:

1. Accessory devices that can be connected to smartphones

Wireless communication method: Connects to a smartphone using Bluetooth or other means.

User authentication method: User authentication is performed using a fingerprint sensor.

Operation method: The user uses a one-click operation method to input prompts to the generative AI.

Emotion recognition: Analyzes the user's tone of voice and facial expressions to recognize emotions.

2. Generative AI

Prompt Input: Receives instructions or questions from the user and generates responses or suggestions.

Emotional feedback: Generate customized responses and suggestions based on recognized emotional information.

Program processing

The server performs the following process:

1. Bluetooth connection: Connect your smartphone and accessory device using Bluetooth.

2. User authentication: The fingerprint sensor scans the user's fingerprint for authentication.

3. Emotion recognition: Analyze the user's tone of voice and facial expressions to recognize emotions.

4. Prompt generation: Generate prompts for the generative AI based on the recognized emotions.

5. AI Response Generation: Use generative AI to generate responses and suggestions to prompts.

Hardware and software used

Hardware: Bluetooth module, fingerprint sensor, camera (for facial recognition), microphone (for voice recognition)

Software: Bluetooth connection library, fingerprint authentication library, emotion recognition engine, generative AI model

Specific examples

If the generative AI recognizes that the user is tired, it will suggest "food delivery options recommended for when you're tired." For example, it might generate a prompt like this:

The user is feeling tired. Please suggest some food delivery options.

In this way, it becomes possible to offer customized food delivery suggestions based on the user's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 16.

Step 1:

The server connects the smartphone and accessory device using a Bluetooth module. The input is pairing information between the smartphone and accessory device, and the output is the establishment of the connection. Specifically, it uses the Bluetooth connection library to establish communication between the smartphone and accessory device.

Step 2:

The device uses a fingerprint authentication sensor to scan the user's fingerprint and perform user authentication. The input is the user's fingerprint data, and the output is the authentication result. Specifically, the fingerprint authentication library is used to compare the scanned fingerprint data with registered data to determine whether authentication was successful.

Step 3:

The device uses a microphone and camera to capture the user's tone of voice and facial expressions to perform emotion recognition. The input is voice data and image data, and the output is recognized emotional information. Specifically, an emotion recognition engine is used to analyze the voice data and image data to identify the user's emotions.

Step 4:

The server generates a prompt for the generative AI based on the recognized emotional information. The input is emotional information, and the output is a prompt. Specifically, the emotional information is converted into text format and a prompt suitable for the generative AI is created.

Step 5:

The server uses a generative AI model to generate responses and suggestions in response to prompts. The input is the prompt, and the output is the generated response or suggestion. Specifically, the prompt is input into the generative AI model, and the AI generates the response or suggestion.

Step 6:

The device presents the generated response or suggestion to the user. The input is the generated response or suggestion, and the output is what is displayed to the user. Specifically, the response or suggestion is displayed on the smartphone display so that the user can check it.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional systems using generative AI models required users to operate their smartphones, making them difficult to use. They also were unable to generate appropriate responses based on the user's emotions, limiting the user experience. This made it difficult for users to effectively utilize generative AI models.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes a user authentication means, an operation means for a user authenticated by the user authentication means to input prompts to the generative AI model, an emotion recognition means for capturing the user's voice data and analyzing their emotions, and a means for feeding back the emotion information analyzed by the emotion recognition means to the generative AI model. This allows the user to easily use the generative AI model without operating a smartphone, and further enables the user to obtain an appropriate response according to their emotions.

"User authentication means" refers to a means for verifying a user's identity, and includes biometric authentication technologies such as fingerprint authentication and facial recognition.

"Operation means" refers to the means by which a user inputs prompts to the generated AI model, and includes interfaces such as buttons and touchscreens.

"Emotion recognition means" is a means for capturing a user's voice data and analyzing the user's emotions from its tone and content.

A "generative AI model" is an artificial intelligence model that generates appropriate responses based on prompts from the user.

"Prompt input" is the act of a user inputting instructions or questions into a generative AI model.

The "feedback means" is a means for transmitting the emotional information analyzed by the emotion recognition means to the generative AI model.

This invention provides a system that includes a user authentication means, an operation means, an emotion recognition means, and a feedback means. Specific embodiments of this system are described below.

First, the user places their finger on the accessory device's fingerprint authentication sensor. The fingerprint authentication sensor captures the user's fingerprint data and sends it to the device. The device then sends this data to the server, which compares the fingerprint data with an existing database to authenticate the user. If authentication is successful, the device's buttons are enabled.

Next, when the user presses the enabled button, the device launches a dedicated application on the smartphone. This application has the function of inputting prompt sentences to the generative AI model. For example, the prompt sentence "Please tell me about today's weather" is input to the generative AI model.

Furthermore, when the user speaks to the accessory device, the microphone in the terminal captures voice data. This voice data is sent to the emotion recognition means, which analyzes the tone of the voice data to analyze the user's emotions. The analysis results are sent to the generative AI model via the feedback means. The generative AI model generates an appropriate response based on this emotional information. For example, in response to the prompt "The user seems a little tired. Please generate an encouraging message," it generates the response "Let's take a break today and refresh ourselves."

This system allows users to easily use generative AI models without having to operate a smartphone. It also improves the user experience by enabling appropriate responses based on the user's emotions.

For example, when a user places their finger on the fingerprint authentication sensor on an accessory device and presses a button after successful authentication, an application on the smartphone launches and the following prompt is input into the generative AI model:

"Can you tell me about today's weather?"

As another example, when a user asks an accessory device, "How are you feeling today?", the emotion recognition means recognizes the emotion from the user's tone of voice and inputs the following prompt sentence into the generative AI model:

"The user seems a little tired. Please generate an encouraging message."

This allows users to easily utilize generative AI without having to operate their smartphone.

The flow of the identification process in Example 2 will be explained using Figure 17.

Step 1:

The user places their finger on the fingerprint sensor.

Input: User's fingerprint

Specific operation: When the user places their finger on the fingerprint authentication sensor on the accessory device, the sensor will glow blue.

Output: Captured fingerprint data

Step 2:

The device captures fingerprint data and sends it to the server.

Input: Captured fingerprint data

Specific operation: The device captures fingerprint data from the fingerprint authentication sensor and, when sending that data to the server, the device screen displays "Authentication in progress..."

Output: Fingerprint data sent to the server

Step 3:

The server authenticates the fingerprint data.

Input: Fingerprint data sent to the server

Specific operation: The server compares the received fingerprint data with an existing database and authenticates the user.

Output: Authentication result (success or failure)

Step 4:

If authentication is successful, the device's buttons will be enabled.

Input: Authentication result (success)

Specific operation: If the server successfully authenticates, it sends a signal to the device to activate the button. The button on the device will light up green.

Output: Enabled button

Step 5:

The user presses the button.

Input: Enabled button

Specific Action: The user presses the enabled button, which triggers the next step.

Output: Button press signal

Step 6:

The device launches the application on the smartphone.

Input: Button press signal

Specific operation: The device launches a dedicated application on the smartphone. The application logo will appear on the smartphone screen.

Output: Launched application

Step 7:

The application prompts input to the generative AI model.

Input: Launched application

Specific operation: The application inputs a prompt to the generative AI model. The screen displays "Sending prompt...". For example, the prompt input is "Tell me about the weather today."

Output: Prompt sent to the generative AI model

Step 8:

The user speaks to the accessory device.

Input: User's voice

Specific operation: The user speaks to the accessory device. For example, "How are you feeling today?" The device's microphone lights up red, indicating that it is capturing audio data.

Output: Captured audio data

Step 9:

The device captures voice data and sends it to the emotion engine.

Input: Captured audio data

Specific operation: The device uses the built-in microphone to capture audio data and sends that data to the emotion engine.

Output: Audio data sent to the emotion engine

Step 10:

The emotion engine analyzes the user's emotions from the tone of the voice data.

Input: Audio data sent to the emotion engine

Specific operation: The emotion engine analyzes the tone of the voice data and determines the user's emotions. The device screen will display "Analyzing emotions..."

Output: Analyzed emotional information

Step 11:

The emotion engine feeds the analysis results back into the generative AI model.

Input: Analyzed emotional information

Specific operation: The emotion engine feeds the analysis results back to the generative AI model. For example, it sends information such as "The user seems a little tired."

Output: Emotional information sent to the generative AI model

Step 12:

The generative AI model generates responses based on the user's emotions.

Input: Emotional information sent to the generative AI model

Specific operation: The generative AI model generates an appropriate response based on the emotional information fed back. For example, in response to the prompt "Generate an encouraging message," it generates the response "Take a break today to refresh yourself." The response is displayed on the smartphone screen.

Output: Generated response

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

With conventional food delivery services, when users order using their smartphones, the process is cumbersome and they don't offer appropriate suggestions based on the user's mood. In particular, there are issues with the lack of relaxing meal suggestions for tired or stressed users, which hinders the user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for enabling prompt input to the generative AI, user authentication means for operating the accessory device, operation means for a user authenticated by the user authentication means to input prompts to the generative AI, voice capture means for capturing the user's voice and recognizing their emotions, emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the user's emotions, and means for the generative AI to generate an appropriate response based on the emotions recognized by the emotion recognition means. This enables the user to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate a smartphone.

"Generative AI" is an artificial intelligence system that generates appropriate responses and suggestions based on user input.

"Prompt input" refers to the act of inputting instructions or questions to generative AI.

An "accessory device" is an external device that can be connected to a smartphone to provide specific functions.

"User authentication means" refers to a means for verifying a user's identity and ensuring that only specific users can operate the system.

"Fingerprint authentication means" refers to a means of authenticating a user by reading their fingerprint.

"Operation means" refers to the interface, buttons, and other means that users use to operate the system.

A "one-click operation method" is a method that allows the user to complete an operation with a single click.

"Audio capture means" means for recording or capturing a user's voice.

"Emotion recognition means" refers to a means for analyzing captured voice data and recognizing the user's emotions.

"Response" refers to the response or suggestion that generative AI generates in response to user input.

A system for implementing this invention is configured as follows: First, a user uses an accessory device connected to a smartphone. This accessory device is equipped with a fingerprint authentication means, and authentication is performed by the user placing their finger on it. If authentication is successful, the operation means of the accessory device is enabled, and the user can use the one-click operation means to input prompts to the generative AI.

Next, the voice capture means captures the user's voice. This voice data is sent to the emotion recognition means, which analyzes the user's emotions. The emotion recognition means recognizes the user's emotions from the tone and content of the voice data and feeds the results back to the generative AI. The generative AI generates an appropriate response based on the recognized emotions.

Specifically, the following hardware and software are used: A fingerprint sensor is used as the fingerprint authentication method. A microphone and voice recognition software (e.g., speech_recognition library) are used as the voice capture method. An emotion engine (e.g., EmotionEngine) is used as the emotion recognition method. An AI generation model (e.g., AIGenerator) is used as the generative AI.

For example, if a user says, "I'm tired today," the voice capture means captures the voice and the emotion recognition means detects "fatigue." Based on this emotion, the generative AI generates a response suggesting a "relaxing meal." In this way, appropriate food delivery suggestions are made according to the user's emotions.

Examples of prompts to input to a generative AI model include:

User Emotion: Fatigue

Prompt for generative AI: "Please recommend a relaxing meal for a tired user."

This system allows users to receive appropriate food delivery suggestions through voice and emotion recognition without having to operate their smartphone.

The flow of the specific processing in Application Example 2 is explained using Figure 18.

Step 1:

The user places their finger on the fingerprint authentication sensor of an accessory device connected to their smartphone. The input is the user's fingerprint data, and the output is the authentication result. The accessory device reads the fingerprint data and compares it with its internal database for authentication. If authentication is successful, the device proceeds to the next step.

Step 2:

If authentication is successful, the accessory device's operation means is enabled. The user uses the one-click operation means to input a prompt to the generative AI. The input is the user's click operation, and the output is the trigger for the prompt input. The accessory device detects the click operation and begins prompt input to the generative AI.

Step 3:

The device's voice capture means captures the user's voice. The input is the user's voice data and the output is the captured voice data. The device's microphone records the user's voice and the voice recognition software converts the data into a digital format.

Step 4:

The captured voice data is sent to the emotion recognition means. The input is the captured voice data and the output is the recognized emotion data. The emotion engine analyzes the voice data and recognizes the user's emotion from the tone and content.

Step 5:

The recognized emotional data is fed back to the generative AI. The input is the recognized emotional data, and the output is a prompt sentence to the generative AI. The generative AI creates a prompt sentence to generate an appropriate response based on the emotional data.

Step 6:

The generative AI generates an appropriate response based on the prompt. The input is the prompt and the output is the generated response. The generative AI analyzes the prompt and generates food delivery suggestions based on the user's emotions.

Step 7:

The generated response is provided to the user. The input is the generated response, and the output is a suggestion to the user. Appropriate food delivery suggestions are displayed or spoken to the user via the terminal's display or audio output device.

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged above and below the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the top side of the concentric circles, and the emotion of "discomfort" is arranged on the bottom side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a sense of calm.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Robots, cars, motorcycles, etc. can also create emotions based on various balances, such as posture and battery level, so that when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotion maps are, for example, based on Dr. Mitsuyoshi's Emotion Map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https://ci.n

ii.ac.jp/naid/500000375379). The left half of the emotion map lists emotions that belong to an area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions that belong to an area called "situation," where situation awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may also be performed by multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The hardware resources that execute specific processes can include the following types of processors. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource that executes specific processes by executing software, i.e., a program. Processors also include FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), and ASICs (Application Specific Integrated Circuits), which are specially designed to execute specific processes.

Specialized electrical circuits are processors with specialized circuit configurations. Each processor has built-in or connected memory, and each processor uses memory to perform specific processes.

The hardware resource that executes a specific process consists of one of these various processors.

It may also be configured as a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resource that executes a specific process may be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiments.

(Claim 1)

A system comprising an accessory device connectable to a smartphone, a means for enabling prompt input to generative AI, a user authentication means for operating the accessory device, and an operation means for a user authenticated by the user authentication means to input prompts to the generative AI.

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

(Claim 4)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means, and further includes an emotion engine that recognizes the user's emotions.

(Claim 5)

The system of claim 4, wherein the emotion engine recognizes emotions from the user's tone of voice.

(Claim 6)

The system of claim 4, wherein the emotion engine recognizes emotions from the user's facial expressions.

.

"Example 1"

(Claim 1)

An accessory device connectable to a smartphone,
A means to enable prompt input to the generative AI; and
a user authentication means for operating the accessory device;
an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI;
A means for running an application on a smartphone for inputting prompts to the generative AI;
means for sending a prompt sentence to the generative AI model and receiving a generated result;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Application Example 1"

(Claim 1)

An accessory device connectable to a smartphone, the accessory device enabling prompt input to the generative AI;
a user authentication means for operating the accessory device;
an operation means for a user authenticated by the user authentication means to input a prompt to the generation system AI;
means for generating instructions for managing and operating robots in a factory;
means for transmitting the instruction to the robot using wireless communication means;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Example 2"

(Claim 1)

A user authentication means;
an operation means for an authenticated user to operate;
A means for transmitting a signal to the smart device using the operation means to launch a specific application;
A means for the launched application to send a prompt to the generative AI model;
a means for the generative AI model to generate a response based on the prompt sentence and transmit the response to the smart device;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Application Example 2"

(Claim 1)

An accessory device connectable to the smart device, which allows instructions to be input to the generative AI;
a user authentication means for operating the accessory device;
an operation means for a user authenticated by the user authentication means to input instructions to the generation system AI;
a speech recognition means;
A means for transmitting the voice data acquired by the voice recognition means to a generation system AI;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Example 1: Combining Emotion Engines"

(Claim 1)

an accessory device connectable to a smartphone, the accessory device enabling prompt input to the generative AI model;
a user authentication means for operating the accessory device;
an operation means for a user authenticated by the user authentication means to input a prompt to the generative AI model;
an emotion recognition means for recognizing an emotion of a user;
a means for feeding back the emotion information recognized by the emotion recognition means to a generative AI model;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Application example 1: Combining emotion engines"

(Claim 1)

An accessory device connectable to a smartphone, the accessory device enabling prompt input to the generative AI;
a user authentication means for operating the accessory device;
an operation means for a user authenticated by the user authentication means to input a prompt to the generation system AI;
an emotion recognition means for recognizing an emotion of a user;
a means for feeding back emotion information recognized by the emotion recognition means to a generative AI;

A system including

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Example 2: Combining Emotion Engines"

(Claim 1)

A user authentication means;
An operation means for a user authenticated by the user authentication means to input a prompt to the generated AI model;
an emotion recognition means for capturing voice data of a user and analyzing the user's emotions;
a means for feeding back the emotion information analyzed by the emotion recognition means to a generative AI model;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

"Application Example 2: Combining Emotion Engines"

(Claim 1)

means for enabling prompt input to the generative AI;
a user authentication means for operating the accessory device;
an operation means for a user authenticated by the user authentication means to input a prompt to the generation system AI;
a voice capture means for capturing a user's voice and recognizing emotions;
emotion recognition means for analyzing the voice data captured by the voice capture means and recognizing the emotion of the user;
means for generating an appropriate response by a generative AI based on the emotion recognized by the emotion recognition means;
A system including:

(Claim 2)

The system of claim 1, wherein the user authentication means is a fingerprint authentication means.

(Claim 3)

The system of claim 1, wherein the operation means is a one-click operation means.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (1)

1. A system including an accessory device connectable to a smart device,
a user authentication means for authenticating a user who operates the accessory device by fingerprint authentication;
an operation means for enabling an operation means of the accessory device when authentication by the user authentication means is successful;
a button provided on the accessory device, the button being pressed to input an instruction to the generative AI after the operation means is enabled ;
a voice recognition means for recognizing a user's voice in response to pressing of the button;
a means for acquiring an output of the generative AI using a prompt sentence including the voice data acquired by the voice recognition means and the generative AI;
means for analyzing the voice data acquired by the voice recognition means and recognizing the user's emotion using an emotion engine that recognizes the user's emotion from the tone and content of the voice data ;
The acquiring means includes:
generating a prompt sentence including the voice data acquired by the voice recognition means and the user's emotion recognized by the emotion engine ;
the prompt instructs the user to order from a food delivery service;
The acquiring means acquires the order result using the generative AI and the food delivery API.
system.