JP7849429B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent Document 1 discloses a persona chatbot control method performed by at least one processor, comprising the steps of: receiving a user utterance; adding the user utterance to a prompt containing instructions related to a description of the chatbot's character; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Japanese Patent Publication No. 2022-180282

In restaurant operations, significant time and effort are required for menu creation, product description writing, and translation. Furthermore, customers often experience stressful situations such as waiting in line with waitstaff and cashiers, and having to ask questions when choosing from the menu. In particular, services for foreign tourists are often inadequate due to language barriers.

This invention provides a means for creating menus using a smartphone, an AI means for generating product descriptions from smartphone photos, and a means for AI to generate translations. This reduces the operational burden on restaurants and improves the customer experience. Furthermore, the AI explains questions that arise during menu selection and provides recommendations based on preferences, resulting in a more personalized service. In addition, the multilingual translation AI enhances services for foreign tourists.

This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. This shows an emotion map where multiple emotions are mapped. This shows an emotion map where multiple emotions are mapped. This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. This is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 of Example 2. This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3. This is a sequence diagram showing the processing flow of the data processing system in Example 1 of Embodiment 1, which is an example of a configuration in which an emotion engine is combined. This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. This is a sequence diagram showing the processing flow of the data processing system in Example 2 of the Form 2 when an emotion engine is combined. This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3, which combines an emotion engine. This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined.

An example of an embodiment of the system relating to the technology of this disclosure will be described below with reference to the attached drawings.

First, let's explain the terminology used in the following explanation.

In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (TENSOR PROCESSING UNIT®).

In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

In the following embodiments, the coded communication interface (I/F) is an interface including a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi®, or Bluetooth®.

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." That is, "A and/or B" means that it could be A alone, B alone, or a combination of A and B. Furthermore, the same concept applies when expressing three or more things linked by "and/or" in this specification.

[First Embodiment]

Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

As shown in Figure 1, the data processing system 10 comprises a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to the bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting it in a form perceptible to the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication interface 44 is connected to the network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. The storage 32 stores a specific processing program 56. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception and output processing. The storage 50 stores the reception and output program 60. The reception and output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception and output program 60 from the storage 50 and executes the read reception and output program 60 on the RAM 48. The reception and output processing is realized by the processor 46 operating as a control unit 46A according to the reception and output program 60 executed on the RAM 48.

Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be explained.

"Example of Morphology 1"

In this embodiment of the present invention, a restaurant operator creates a menu using a smartphone. Specifically, the operator uploads photos of products taken with the smartphone's camera, and AI processes those photos.

It automatically recognizes the characteristics and features of the product. Based on this recognition, it generates a product description. This product description is then displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Furthermore, in this embodiment of the present invention, the AI generates the translation work. Specifically, the generated product description is translated into multiple languages. This makes the product description easier for foreign tourists to understand. For example, it is possible to translate into major tourist languages such as English, Chinese, and Korean.

"Example of Morphology 3"

Furthermore, in this embodiment of the present invention, the customer experience is also improved. Specifically, when a customer is considering a menu, the AI explains their questions and recommends products according to their preferences. For example, if a customer inputs information such as "I like spicy food," the AI will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI will provide an answer to that question.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: The restaurant operator takes photos of the products using their smartphone camera.

Step 2: Upload the photos you took to the system.

Step 3: The system's AI automatically recognizes the product's characteristics and features from the photograph.

Step 4: The AI generates a product description based on the recognition results.

Step 5: The generated product description is displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Step 1: Obtain the product description generated by the AI.

Step 2: The AI translates the product description into multiple languages.

Step 3: The translated product description is displayed to foreign tourists on the restaurant's website or application.

"Example of Morphology 3"

Step 1: When customers are considering the menu, they input their questions into the AI.

Step 2: The AI generates the answer to that question.

Step 3: The AI recommends products based on the customer's preferences.

Step 4: The AI-generated answers and recommendations are displayed to the customer.

(Example 1)

Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

When restaurant operators create menus, the process of taking photos of products and then recognizing their characteristics and features to generate product descriptions is time-consuming. Furthermore, there is a lack of means to provide multilingual support for foreign tourists and to enhance the customer experience. Therefore, there is a need for increased efficiency in menu creation and improved customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a means for creating menus using a smartphone; an artificial intelligence (AI) means for generating product descriptions from smartphone photos; a means for AI to generate translations; a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a means for AI to explain questions during menu consideration; a means for recommending items according to preferences; a means for uploading product photos taken with a smartphone camera, for AI to automatically recognize the characteristics and features of the products from those photos; and a means for generating product descriptions based on the recognition results. This enables increased efficiency in menu creation and improved customer satisfaction.

A "smartphone" is a multi-functional mobile device that, in addition to the functions of a regular mobile phone, also offers internet connectivity and the ability to use applications.

"Menu creation methods" refer to the methods and tools that restaurant operators use to create menus, particularly those using smartphones.

"Artificial intelligence tools" refer to technologies that use machine learning and data analysis to automatically perform specific tasks.

"Means of generating translation work using artificial intelligence" refers to methods and technologies that use artificial intelligence to translate text into multiple languages.

"Methods for improving the customer experience" refer to methods and tools for improving the convenience and satisfaction customers experience when using a service.

"Methods for using artificial intelligence to explain questions during menu consideration" refers to methods and technologies in which artificial intelligence automatically provides answers to questions that customers may have when considering menu options.

"Methods of recommending based on preferences" refer to methods and technologies that recommend appropriate products and services based on customer preferences and past choices.

"Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

"Methods for generating product descriptions" refers to methods and technologies for creating product descriptions in natural language based on recognized product characteristics and features.

This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, users upload photos of their products taken with their smartphone cameras, and artificial intelligence (AI) automatically recognizes the characteristics and features of the products from these photos. Based on this recognition, the system generates product descriptions. These product descriptions are then displayed to customers on the restaurant's website or application.

Hardware and software to be used

Smartphone: A device used for taking and uploading photos.

Server: Performs data processing and runs AI models.

Artificial intelligence models: Image recognition models and natural language generation models based on TENSORFLOW® and PyTorch (e.g., GPT-3®, BERT).

Data processing and data calculation

1. The user takes a photo of the product with their smartphone.

The user launches their smartphone's camera app and takes a picture of the product. For example, they might take a picture of a new dessert, "Chocolate Cake."

2. The user uploads photos to the system.

The user opens a dedicated application, selects the photos they've taken, and presses the upload button. The photos are then sent to the server via the internet.

3. The server receives the photos and inputs them into the AI model.

The server receives the uploaded photos and inputs them into an AI model for image processing. The AI model used here is based on TensorFlow or PyTorch.

4. The server uses an AI model to recognize the characteristics and features of the product.

The server uses an AI model to recognize the characteristics and features of a product from a photograph. For example, it extracts information such as the type of dessert, main ingredients, and visual characteristics.

5. The server generates a product description based on the recognition results.

The server generates product descriptions based on the AI's recognition results. These product descriptions are created using natural language generation technology. The software used includes GPT-3 and BERT.

6. The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Customers can then view them.

Specific examples

A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description, such as "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing," and displays it on the website.

Example of a prompt message

Please upload a photo of your new dessert. AI will recognize the product's characteristics and features from the photo and generate a product description.

The flow of the specific processing in Example 1 will be explained using Figure 11.

Step 1:

The user takes a photo of the product with their smartphone.

Input: Product (e.g., Chocolate cake)

Output: Photos of the photographed product

Specific action: The user launches the camera app on their smartphone and takes a picture of the product. For example, they might take a picture of the new dessert, "Chocolate Cake."

Step 2:

The user uploads a photo to the system.

Input: Photographs of the product

Output: Photo data sent to the server

Specific operation: The user opens a dedicated application, selects the photo they have taken, and presses the upload button. The photo is sent to the server via the internet.

Step 3:

The server receives the photos and inputs them into the AI model.

Input: Photo data sent to the server

Output: Photo data input to the AI model

Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

Step 4:

The server uses an AI model to recognize the characteristics and features of the product.

Input: Photo data entered into the AI model

Output: Characteristics and features of the recognized product (e.g., chocolate cake, cream topping, berry decoration)

Specific operation: The server runs an AI model to extract product characteristics and features from a photograph. For example, it recognizes the type of dessert, main ingredients, and visual features.

Step 5:

The server generates a product description based on the recognition results.

Input: Characteristics and features of the recognized product

Output: Generated product description (Example: "This chocolate cake features rich chocolate and creamy toppings. The berry decorations make it visually appealing.")

Specific operation: The server uses GPT-3 or BERT to generate product descriptions based on the recognition results.

Step 6:

The server displays the product description on the website or application.

Input: Generated product description

Output: Product description displayed on the website or application

Specific operation: The server saves the product description to the website's database, sends the data to the front-end for display, and the customer can view it.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

Traditional restaurant menu creation was often done manually, which was time-consuming and labor-intensive, and made it difficult to adequately convey the appeal of the products. Furthermore, the lack of multilingual support for foreign tourists and insufficient recommendation features tailored to customer preferences were also problems. In addition, there were limited means to improve the in-store customer experience, such as having to wait for waitstaff or in line at the register, highlighting the need for increased customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an AI method for generating product descriptions from smartphone photos; a method for AI-generated translations; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI-generated explanations of questions during menu consideration; a method for making recommendations based on preferences; a method for recognizing product characteristics and features using an image recognition model; and a method for generating product descriptions based on recognized features using a natural language generation model. This enables efficient menu creation and automatic generation of attractive product descriptions, facilitating multilingual support, improved customer experience, and personalized recommendations.

"A method for creating menus using a smartphone" refers to a method for efficiently creating restaurant menus using a smartphone's camera and applications.

"AI methods for generating product descriptions from smartphone photos" refers to artificial intelligence methods that analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"A method for generating translations using AI" refers to a method of translating product descriptions and menu contents into multiple languages using artificial intelligence.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"An AI-powered solution for questions during menu consideration" refers to a method where artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending based on preferences" refers to a method of recommending appropriate products or menu items based on a customer's past choices and preferences.

"A means of recognizing the characteristics and features of a product using an image recognition model" refers to a method of automatically recognizing the characteristics and features of a product from a photograph using image recognition technology.

"A means of generating product descriptions based on features recognized using a natural language generation model" refers to a method of automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

The system for implementing this invention includes a means for creating menus using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for AI to generate translations, a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register, a means for AI to explain questions when considering menus, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on recognized features using a natural language generation model.

System Program

The server implements the system using the following hardware and software.

Hardware:

Smartphone (with camera)

Server (for hosting AI models)

Software:

TensorFlow: A library for image recognition

OpenAI® GPT-3: API for natural language generation

Explanation of the process

Image recognition:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of the food from the image. For example, features such as "chicken curry, spicy, tomato-based" might be recognized.

Natural language generation:

The server converts the recognized features into a string and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is: "Generate an appealing product description for a dish with the following features: Chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it might generate a description such as: "This spicy chicken curry features juicy chicken simmered in a tomato-based sauce. Aromatic spices will whet your appetite."

Translation work:

The generated product descriptions are translated into multiple languages as needed. The server uses AI to perform translations into multiple languages, such as English and Chinese.

Improving the customer experience:

Customers can use their smartphones to select menu items and place orders in-store. This reduces waiting times at the counter and cashier, enabling smoother service.

Explanation of the question and recommendations:

The AI automatically provides answers to questions customers may have when considering menu options. It also recommends appropriate products and menu items based on the customer's past choices and preferences.

In this way, menu creation becomes more efficient, attractive product descriptions are automatically generated, and multilingual support, improved customer experience, and personalized recommendations become possible.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The input is the photo of the food, and the output is the image data sent to the server.

Step 2:

The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of dishes from uploaded image data. The input is image data, and the output is a list of the recognized characteristics and features. Specifically, the image data is preprocessed, input into the model, and prediction results are obtained.

Step 3:

The server converts the recognized characteristics and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of characteristics and features, and the output is a prompt statement. Specifically, it formats the characteristics and features and generates the prompt statement.

Step 4:

The server receives a response from GPT-3 and generates an attractive product description. The input is a prompt, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and retrieves the product description.

Step 5:

The server translates the generated product description into multiple languages as needed. The input is the product description, and the output is the translated product description. Specifically, it calls a translation API to translate into multiple languages.

Step 6:

Users use their smartphones to select menu items and place orders in the store. Input is generated product descriptions, and output is the user's order information. Specifically, the application displays product descriptions, and the user selects their order.

Step 7:

The server automatically provides answers to questions that arise when users consider menu options, using AI. The input is the user's question, and the output is the AI's answer. Specifically, it analyzes the question and generates an appropriate answer.

Step 8:

The server recommends appropriate products and menu items based on the user's past choices and preferences. The input is the user's past selection data, and the output is the recommended products and menu items. Specifically, it analyzes past selection data and uses a recommendation algorithm to suggest products.

(Example 2)

Next, we will describe Example 2 of the 2nd embodiment. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

Traditional systems often required manual translation of product descriptions into multiple languages, resulting in significant time and effort. Furthermore, services for foreign tourists were inadequate, leading to difficulties in understanding product descriptions. Additionally, insufficient efforts to improve the customer experience and address questions during menu selection could potentially lower customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for creating menus using a smart device, means for artificial intelligence (AI) to generate product descriptions from images on the smart device, means for AI to generate translation tasks, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations based on preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into the generation AI model and obtaining translation results, means for saving translation results to a database, means for sending translation results to a terminal, and means for the terminal to display the translation results. This automates the multilingual translation of product descriptions, improves services for foreign tourists, enhances the customer experience, and resolves questions during menu consideration.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that possesses advanced computing power and communication capabilities.

"Menu creation methods" refer to functions and applications used to create menus for restaurants and retail stores using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

"Means of improving the customer experience" refer to features and methods that enhance the convenience and satisfaction customers experience when using products or services.

"A means for AI to explain questions during menu consideration" refers to a function where artificial intelligence automatically provides answers to questions and concerns that arise when customers are considering menu options.

"Preference-based recommendations" refers to a function that recommends appropriate products and services based on a customer's past choices and preferences.

"Methods for translating product descriptions into multiple languages" refers to functions for translating product descriptions into several languages.

A "generative AI model" is an artificial intelligence model trained to perform tasks such as text generation and translation.

A "prompt statement" is an instruction given to a generative AI model to perform a specific task.

"Means for saving translation results to a database" refers to the function of saving translation results generated by a generative AI model to a database.

"Means for sending translation results to the terminal" refers to the function that sends translation results stored in the database to the user's terminal.

"Means for the terminal to display translation results" refers to the function that displays the received translation results on the user's terminal screen.

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translation tasks, means for improving the customer experience, means for artificial intelligence to explain questions when considering menus, means for recommending according to preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into a generation AI model and obtaining translation results, means for saving translation results in a database, means for sending translation results to a terminal, and means for the terminal to display translation results.

Hardware and software to be used

Hardware:

Smart devices (smartphones, tablets, etc.)

Server (a computer with high-performance computing capabilities)

Software:

Generative AI models (e.g., OpenAI's GPT-4®)

Database Management System

Communication protocol (e.g., HTTP/HTTPS)

Data processing and data calculation

Product description generation:

The user takes a picture of the product using a smart device and sends the image to the server. The server uses artificial intelligence to generate a product description from the image. Specifically, it uses an image analysis algorithm to recognize the product's characteristics and features, and then generates a product description based on that.

Prompt message generation:

The server generates prompt text using a generative AI model. The prompt text specifies the target language for translation (e.g., English, Chinese, Korean).

Execution of translation work:

The server inputs the generated prompt text and product description into the AI model and retrieves the translation result. The AI model then performs multilingual translation based on the input text.

Saving and submitting translation results:

The server saves the acquired translation results to a database. The saved translation results are sent to the user's device, which then displays the translation.

Specific examples

Specific example:

The user enters the Japanese product description: "This product uses high-quality materials."

The server generates the prompt message: "Translate the following product description into English: This product uses high-quality materials."

The server inputs the prompt text and product description into the generated AI model and retrieves the English translation result: "This product uses high-quality materials."

The server saves the translation results to a database and sends them to the terminal.

The device displays the translation results, and the user confirms them.

This system enables multilingual translation of product descriptions through collaboration between the server, terminal, and user.

The flow of the specific processing in Example 2 will be explained using Figure 13.

Step 1:

The user enters the product description.

The user enters the product description in text format using a smart device. The entered product description is then sent from the smart device to the server.

Input: Product description text

Output: Product description text sent to the server

Step 2:

The server receives the product description.

The server receives the product description text sent by the user and temporarily stores it in memory.

Input: Product description text submitted by the user

Output: Product description text saved in memory

Step 3:

The server generates the prompt message.

The server generates prompt messages based on the specified languages for translation. For example, if translation is required for English, Chinese, and Korean, it will generate prompt messages corresponding to each language.

Input: Product description text, target language for translation

Output: Generated prompt message

Step 4:

The server inputs prompt text and product description into the generated AI model.

The server inputs the generated prompt text and product description text into the AI model. The AI model then performs translation based on the input text.

Input: Prompt message, product description text

Output: Data input to the generative AI model

Step 5:

The server retrieves the translation result.

The server retrieves translation results from the generative AI model. The retrieved translation results are separated by language.

Input: Data entered into the generative AI model

Output: Translated text

Step 6:

The server saves the translation results to the database.

The server saves the retrieved translation results to a database. When saving, it associates the original Japanese product description with the translated result.

Input: Translated text, original product description text

Output: Translation results stored in the database

Step 7:

The server sends the translation result to the terminal.

The server sends the saved translation results to the user's device. The transmitted data is provided in a format accessible to the user.

Input: Translation results stored in the database

Output: Translation result sent to the terminal

Step 8:

The device displays the translation result.

The terminal receives the translation results sent from the server and displays them to the user. The user can review the displayed translation results and make corrections or re-translates as needed.

Input: Translation result sent from the server

Output: Translation result displayed on the terminal

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing goods, making it difficult for them to understand product descriptions. Furthermore, visually impaired individuals and those with reading and writing difficulties also have limited means of understanding product descriptions. This can lead to a diminished customer experience and potentially impact sales.

In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, and a means for translating product descriptions into multiple languages and displaying and playing them aloud in the user's set language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired individuals, and people with reading and writing difficulties.

"A method for creating menus using a smartphone" refers to a method for creating menus using a smartphone.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

"Methods for generating translation work using artificial intelligence" refers to methods of translating text into multiple languages using artificial intelligence.

"Methods for improving the customer experience" refer to measures taken to improve the customer's experience when purchasing a product.

"A method for using artificial intelligence to explain questions that arise when customers are considering menu options" refers to a method in which artificial intelligence answers questions that customers may have when considering menu options.

"Methods of recommending based on preferences" refer to methods of recommending products and services based on customer preferences.

"A means of translating product descriptions into multiple languages and displaying and playing them in the user's preferred language" refers to a means of translating product descriptions into multiple languages and displaying and playing them in the user's preferred language.

To implement this invention, it is necessary to construct a system that combines a smartphone, artificial intelligence technology, a translation API, and a voice playback function.

First, smartphones have a camera function that allows users to photograph products. When a user photographs a product, the image data is sent to a server. The server then generates a product description using image recognition technology. This image recognition technology might include, for example, Google® Cloud Vision API.

Next, the generated product description is translated into multiple languages using artificial intelligence on the server. Translation services such as the Google Cloud Translation API are used for this translation. The translated text is then sent to the user's smartphone and displayed based on their language settings.

Furthermore, the translated product descriptions are played back using the smartphone's audio playback function. This makes the product accessible to visually impaired individuals and those with reading and writing difficulties.

As a concrete example, consider a tourist trying to purchase a leather wallet at a physical store in Japan. When the tourist scans the wallet's tag with their smartphone camera, the app automatically recognizes the product description and translates it into the user's chosen language (e.g., English). The translated product description would be displayed as "This is a high-quality leather wallet," and also played aloud.

Examples of prompt messages include the following:

I am developing an application to translate product descriptions into multiple languages. Please translate the following text into the specified languages.

Text: "This is a high-quality leather wallet."

Target language: Japanese

In this way, foreign tourists, visually impaired individuals, and people with reading and writing difficulties will be able to understand product descriptions without encountering language barriers when purchasing items in physical stores.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user takes a picture of the product using their smartphone camera. The input is the product image taken with the smartphone camera. The output is the captured product image data.

Step 2:

The device sends the captured product image data to the server. The input is the product image data. The output is the product image data sent to the server.

Step 3:

The server generates product descriptions using image recognition technology. The input is product image data. The server extracts text information from the images using APIs such as Google Cloud Vision API and generates the product description. The output is the generated product description text.

Step 4:

The server translates the generated product description text into multiple languages. The input is the product description text. The server uses the Google Cloud Translation API to translate it into the specified languages. The output is the translated product description text.

Step 5:

The server sends the translated product description text to the terminal. The input is the translated product description text. The output is the translated product description text sent to the terminal.

Step 6:

The device displays the translated product description text. The input is the translated product description text. The output is the translated product description displayed on the smartphone screen.

Step 7:

The device plays the translated product description text as audio. The input is the translated product description text. The device uses the smartphone's audio playback function to convert the text to audio and play it. The output is the translated product description played as audio.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

Traditional menu creation and customer experience improvement systems fail to adequately answer users' questions when considering menu items and to provide product recommendations tailored to individual preferences. Furthermore, they are insufficient in reducing waiting times and providing multilingual services. This can lead to decreased customer satisfaction and negatively impact store sales.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for the user to input information, means for transmitting the input information to the server, means for the server to analyze the input information using a generated AI model, means for generating recommendations and answers based on the analysis results, means for transmitting the generated recommendations and answers to a terminal, and means for the terminal to display the recommendations and answers to the user. This enables appropriate answers to questions the user may have when considering a menu, and product recommendations tailored to individual preferences. Furthermore, it can reduce waiting times and provide multilingual services, leading to improved customer satisfaction and increased store sales.

A "smart device" is an electronic device used by users to input information and communicate with a server, and includes smartphones and tablets.

"Artificial intelligence tools" refer to algorithms and models that analyze input data and generate appropriate responses or recommendations.

A "generative AI model" refers to a machine learning model that analyzes user input information and generates appropriate responses and recommendations.

"Methods for improving the customer experience" refer to measures that reduce waiting times and allow customers to use services comfortably.

"A means for artificial intelligence to explain questions that arise when considering menu options" refers to a method by which artificial intelligence provides appropriate answers to questions that users may have when considering menu options.

"Methods for recommending products based on preferences" refers to methods for recommending appropriate products based on the user's preferences.

"Means for user information input" refers to the interface through which users input information in formats such as text or voice.

"Means for sending inputted information to the server" refers to the communication means used to send information entered by the user to the server.

"Means for a server to analyze input information using a generated AI model" refers to the means by which a server uses a generated AI model to analyze information received from a user.

"Means for generating recommendations and responses based on analysis results" refers to methods for generating recommendations and responses for users based on the analysis results of a generation AI model.

"Means for sending generated recommendations and responses to the device" refers to the communication methods used by the server to send generated recommendations and responses to the user's device.

"Means by which a device displays recommendations and answers to a user" refers to the means by which a user's device displays recommendations and answers received from a server to the user.

This invention is a system in which a user inputs information using a smart device, a server analyzes that information using a generated AI model, and then generates and provides appropriate recommendations and responses to the user. A specific embodiment of this system is described below.

First, the user inputs information into the system using a smart device such as a smartphone or tablet. For example, the user might input "I like spicy food." This information is entered through the smart device's interface.

Next, the smart device sends the entered information to the server. In this process, the smart device converts the input data into an appropriate format (e.g., JSON format) and sends it to the server via the network.

The server analyzes the received user input using a generation AI model (for example, OpenAI's GPT-4). Specifically, it analyzes information such as "I like spicy food" and searches a database related to spicy food.

The server recommends products tailored to the user's preferences based on the analysis results. For example, it generates a list of spicy dishes and presents it to the user. Furthermore, if the user asks, "What are the ingredients in this dish?", the server generates an answer to that question.

The generated recommendations and responses are sent from the server to the smart device. The server converts the data into the appropriate format and sends it to the smart device over the network.

Smart devices display recommendations and answers received from the server to the user. For example, they might display, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot."

As a concrete example, the following prompt message is shown.

"I like spicy food. Could you recommend some dishes?"

"What are the ingredients in this dish?"

In this way, a system is realized in which the server, smart device, and user work together to improve the customer experience. This system allows users to receive appropriate answers to questions when considering menus and product recommendations tailored to their individual preferences. Furthermore, it enables reduced waiting times and the provision of multilingual services, leading to improved customer satisfaction and increased store sales. The flow of specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters information using a terminal.

The user opens the application on their smartphone or tablet, enters "I like spicy food" into the text box, and presses the submit button. The input data is in text format and represents information about the user's preferences.

Step 2:

The terminal sends the entered information to the server.

The terminal converts the user's input, "I like spicy food," into JSON format and sends it to the server using the HTTPS protocol. The input data is user preference information in text format, and the output data is request data in JSON format.

Step 3:

The server analyzes the input information using a generated AI model.

The server parses the received JSON data and inputs the information "likes spicy food" into a generative AI model. The generative AI model (e.g., GPT-4) searches a database of spicy dishes and generates a list of related dishes. The input data is a request in JSON format, and the output data is the list of dishes resulting from the analysis.

Step 4:

The server generates recommendations and responses based on the analysis results.

The server generates a list of spicy dishes based on the analysis results obtained from the generated AI model. For example, it might list dish names such as "Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is the list of dishes resulting from the analysis, while the output data is a recommendation list presented to the user.

Step 5:

The server sends recommendations and responses generated by the server to the device.

The server converts the generated list of spicy dishes into JSON format and sends it to the terminal using the HTTPS protocol. The input data is a recommendation list, and the output data is a JSON response.

Step 6:

The device displays recommendations and answers to the user.

The terminal parses the received JSON data and displays to the user, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is response data in JSON format, and the output data is text information displayed to the user.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

Traditional food delivery systems have problems such as difficulty for customers to obtain detailed information when choosing a menu and a lack of personalized recommendations. Furthermore, the lack of a means to provide real-time information on ingredients and nutritional content has been a factor in lowering customer satisfaction.

In Application Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting time, a means for artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, a means for recommending dishes based on customer preferences, and a means for providing information on the ingredients of the dishes. As a result, customers can obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

A "menu creation method" is a function for generating a list of dishes and drinks that customers can choose from.

"An artificial intelligence method for generating product descriptions from images" refers to a function that analyzes images taken with a smart device and automatically generates product descriptions based on their content.

"A means of generating translation work using artificial intelligence" refers to a function that automatically translates text between different languages.

"Methods for improving the customer experience by reducing waiting times" refer to features that reduce the waiting time customers experience when using a service.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are choosing from a menu.

"A method of recommending based on preferences" refers to a function that recommends appropriate products and services based on the customer's preferences and past choices.

"A method for recommending dishes based on customer preferences" refers to a function that recommends appropriate dishes based on the preference information entered by the customer.

"Means of providing information on the ingredients of a dish" refers to a function that provides information about the ingredients and components contained in a specific dish.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from images on a smart device, means for generating translations using artificial intelligence, means for improving the customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for making recommendations according to preferences, means for recommending dishes based on customer preferences, and means for providing information on the ingredients of dishes.

System Program

This system's program is implemented using Python and the OpenAI API. The server receives customer input information and uses a generative AI model to provide appropriate recommendations and information.

Explanation of the process

The server receives customer preferences and questions sent from smart devices and generates prompts based on this information. These generated prompts are input into a generation AI model via the OpenAI API, which then generates appropriate answers and recommendations. The generated information is then sent back to the smart device and provided to the customer.

The hardware used will be smart devices such as smartphones and tablets. The software used will be Python and the OpenAI API.

Specific examples

For example, if a customer enters "I like spicy food," the server will generate the following prompt:

"Please recommend some dishes for people who like spicy food."

Entering this prompt into the OpenAI API will cause the generating AI model to recommend spicy dishes such as "Mapo Tofu" and "Kimchi Hot Pot."

Furthermore, when a customer asks, "What are the ingredients in Mapo Tofu?", the server generates the following prompt:

"What are the ingredients in Mapo Tofu?"

When this prompt is entered into the OpenAI API, the generating AI model will provide ingredient information such as "tofu, ground pork, green onions, garlic, ginger, chili bean paste, sweet bean paste, soy sauce, sake, sugar, and chicken broth."

This allows customers to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the application using a smart device and enters their preferences and answers questions. The entered information is sent to the server in text format. Examples of input include "I like spicy food" and "What are the ingredients in Mapo Tofu?".

Step 2:

The server analyzes the input information received from the user and generates an appropriate prompt. For example, in response to the input "I like spicy food," it generates the prompt "Please recommend some dishes for someone who likes spicy food." This prompt is then input into the generation AI model.

Step 3:

The server sends the generated prompt to the OpenAI API and retrieves appropriate answers and recommendations based on the generated AI model. For example, in response to the prompt "What dishes would you recommend for someone who likes spicy food?", answers such as "Mapo Tofu" or "Kimchi Hot Pot" would be generated.

Step 4:

The server analyzes the responses and recommendations obtained from the generative AI model and converts them into a format for user presentation. For example, it formats responses like "Mapo Tofu" and "Kimchi Hot Pot" into a list.

Step 5:

The server sends formatted responses and recommendations to the smart device. The user can then view this information on the smart device's screen.

Step 6:

Users can select menu items based on the information provided or ask further detailed questions. For example, they might re-enter the question, "What are the ingredients in Mapo Tofu?"

Step 7:

The server receives user input again and repeats the same process. Specifically, it generates a prompt again, inputs it into the generation AI model, retrieves the answer, formats it, and provides it to the user.

This allows users to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

"Example of Morphology 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and recommends products accordingly. Specifically, when a user is selecting a product, the system recognizes their emotions from their facial expressions and tone of voice, and recommends products that match those emotions. For example, if a user shows a joyful expression, the system recommends products that are likely to make that user happy.

"Example of Morphology 2"

Furthermore, the emotion engine generates product descriptions based on the user's emotions. Specifically, it recognizes the user's emotions and selects words that match those emotions to generate a product description. For example, if the user is feeling down, it will select words that will cheer them up to generate a product description. (Example 3)

Furthermore, the emotion engine recognizes the user's emotions and provides services accordingly. Specifically, it recognizes the user's emotions and provides services that match those emotions. For example, if a user is showing anger, it might offer a service such as an apology.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system recommends products that match the recognized emotions.

"Example of Morphology 2"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system selects words that match the recognized emotion and generates a product description.

"Example of Morphology 3"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system provides services that match the recognized emotions.

(Example 1)

Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

Traditional restaurant menu creation and customer experience improvement have been time-consuming and labor-intensive. In particular, generating product descriptions from photos and providing personalized recommendations consume significant human resources. Furthermore, multilingual services for foreign tourists are inadequate, highlighting the need for improved customer satisfaction. Additionally, the inability to provide product recommendations based on customer emotions can lead to a decline in the quality of the customer experience. An efficient system is needed to address these challenges.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an artificial intelligence (AI) method for generating product descriptions from smartphone photos; a method for AI to generate translation tasks; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI to explain questions during menu consideration; a method for recommending items according to preferences; and a method for recognizing user emotions and recommending products based on those emotions. This enables more efficient menu creation, product recommendations tailored to customer preferences, enhanced service for foreign tourists through multilingual support, and product recommendations based on customer emotions.

"A method for creating menus using a smartphone" refers to a method for creating restaurant menus using a smartphone's camera and applications.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"Methods for generating translation work using artificial intelligence" refers to methods of translating product descriptions and menu contents into multiple languages using artificial intelligence technology.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"A method for using artificial intelligence to explain questions during menu consideration" refers to a method in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending products based on preferences" refers to a method of recommending appropriate products based on a customer's past choices and preferences.

"A method for recognizing user emotions and recommending products based on those emotions" refers to a method of recognizing emotions by analyzing the user's facial expressions and tone of voice, and then recommending products that match those emotions.

This invention is a system that allows restaurant operators to create menus using smartphones and improve the customer experience. A specific embodiment of this system is described below.

First, the user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they would launch the smartphone's camera app, frame the dessert, and press the shutter button.

Next, the device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in an appropriate format (e.g., JPEG, PNG).

The server analyzes the received photos using image recognition software (e.g., Google Cloud Vision API). The server recognizes the characteristics of the product from the photo (e.g., chocolate cake, fruit parfait). For example, the Google Cloud Vision API might return the label "chocolate cake."

Subsequently, the server generates a product description based on the recognized characteristics. Using a generation AI model (e.g., OpenAI GPT-3), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

The generated product descriptions are displayed on the restaurant's website or application by the server. Specifically, the product descriptions are sent to the front-end in HTML or JSON format and displayed in the user interface.

Furthermore, when a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, the smartphone's camera captures the user's smile, and the microphone records the user's voice tone.

The server analyzes the captured data using an emotion engine (e.g., Microsoft® Azure® Emotion API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait."

As a concrete example, consider a scenario where a user takes a photo of a new dessert with their smartphone and uploads it to the system. The server uses the Google Cloud Vision API to analyze the photo and recognize its characteristic as "chocolate cake." The AI then generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

Furthermore, if a user displays an expression of joy while viewing the menu using their smartphone camera, the Microsoft Azure Emotion API recognizes that expression. The server then recommends a "fruit parfait," which is likely to bring joy to the user.

Example of a prompt message:

"Please upload photos of the product taken with your smartphone. Our AI will automatically recognize the product's characteristics and generate a product description. It will also use your camera and microphone to recommend products based on your emotions."

In this way, a system is realized in which servers, terminals, and users work together to efficiently create restaurant menus and recommend products to customers.

The flow of the specific processing in Example 1 will be explained using Figure 17.

Step 1:

The user takes a photo of the product.

The user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they launch the smartphone's camera app, frame the dessert, and press the shutter button. The input is the photograph taken, and the output is the image data stored on the smartphone.

Step 2:

The device uploads the photos to the server.

The device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. The photo data is sent in an appropriate format (e.g., JPEG, PNG). The input is the image data on the smartphone, and the output is the image data sent to the server.

Step 3:

The server analyzes the photo and recognizes the product's characteristics.

The server analyzes the received photograph using image recognition software (e.g., an image recognition API). The server recognizes the characteristics of the product from the photograph (e.g., chocolate cake, fruit parfait). For example, the image recognition API might return the label "chocolate cake." The input is the image data sent to the server, and the output is the recognized product characteristic information.

Step 4:

The server generates the product description.

The server generates a product description based on the recognized characteristics. Using a generative AI model (e.g., a generative AI model), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture." The input is product characteristic information, and the output is the generated product description.

Step 5:

The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Specifically, it sends the product descriptions in HTML or JSON format to the front-end and displays them in the user interface. The input is the generated product description, and the output is the product description displayed on the website or application.

Step 6:

When a user selects a product, the device captures the user's emotions.

When a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, a smartphone's camera captures the user's smile, and the microphone records the user's voice tone. The input is the user's facial expressions and voice tone, and the output is the captured emotional data.

Step 7:

The server analyzes emotions and recommends appropriate products.

The server analyzes the captured data using an emotion engine (e.g., an emotion analysis API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait." The input is the captured emotion data, and the output is the recommended product information.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

Traditional restaurants faced significant challenges in menu creation and improving the customer experience, requiring considerable time and effort. Specifically, there were multiple obstacles, including menu creation, product description generation, personalized recommendations, and multilingual support. Furthermore, providing service that resonated with customer emotions was difficult, making improving customer satisfaction a challenge.

In Application Example 1, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on the smart device, a means for the AI to generate translations, a means for improving the customer experience, a means for the AI to explain questions during menu consideration, a means for recommending according to preferences, and a means for recognizing customer emotions and recommending products based on those emotions. This enables efficient menu creation, automatic generation of product descriptions, multilingual support, improved customer experience, and emotion-based recommendations.

A "smart device" is a portable electronic device with advanced functions, such as a smartphone or smart glasses.

"Menu creation methods" refer to functions and methods for creating restaurant menus using smart devices.

"AI methods for generating product descriptions" refers to artificial intelligence technology that automatically recognizes the characteristics and features of a product from a photograph taken with a smart device and generates a product description based on that recognition.

"A means of generating translation work using AI" refers to a function that automatically performs multilingual translation using artificial intelligence.

"Methods for improving the customer experience" refer to methods and functions that enhance the customer experience, such as eliminating the need to wait in line with waitstaff or at the cash register.

"An AI-powered solution for questions during menu consideration" refers to a function where artificial intelligence automatically explains questions that arise when customers are considering menu options.

"A method for recommending products based on preferences" refers to a function that recommends appropriate products based on the customer's preferences.

"A means of recognizing customer emotions and recommending products based on those emotions" refers to a function that recognizes customer emotions from facial expressions, tone of voice, etc., and recommends products that match those emotions.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from photos on a smart device using AI, means for generating translations using AI, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations according to preferences, and means for recognizing customer emotions and recommending products based on those emotions.

Program Processing Description

Hardware

The server will be equipped with a high-performance GPU. Smart devices such as smartphones and smart glasses will be used as terminals.

Software

The server uses the following software:

OpenCV: Image processing library

Keras: A deep learning library

Transformers: Generative AI models (such as GPT-3)

Data processing and calculations

1. Menu creation method:

Take a photo of the food using the device's camera and upload it to the server.

The server uses OpenCV to preprocess images and Keras to recognize product characteristics and features from the images.

Based on the recognition results, product descriptions are generated using Transformers.

2. Methods for AI to generate translation work:

The server uses a generative AI model to translate the generated product descriptions into multiple languages.

In particular, we will implement multilingual support to enhance services for foreign tourists.

3. Means of improving the customer experience:

The terminal allows customers to complete ordering and payment on their smart devices, eliminating the need to wait in line with waitstaff or at the cash register.

4. How AI can explain questions that arise when considering menu options:

The terminal sends questions that arise when customers are considering the menu to the server, and the server automatically generates explanations using AI.

5. Methods for recommending based on preferences:

The server recommends appropriate products based on the customer's past order history and preferences.

6. Means for recognizing customer emotions and recommending products based on those emotions:

The server analyzes the customer's facial expressions and voice tone through the device's camera, and uses Keras to recognize their emotions.

Based on recognized emotions, Transformers are used to recommend appropriate products.

Specific examples and prompt messages

Specific example:

When a customer is wearing smart glasses and looking at a menu, the smart glasses analyze the customer's facial expression and display a message saying, "You seem happy, so we recommend dessert!"

Example prompts for a generative AI model:

"This dish looks like delicious pasta. Please generate a detailed product description."

Thus, the embodiment of this invention combines smart devices and AI technology to enable menu creation and improve the customer experience in restaurants.

The flow of the specific processing in Application Example 1 will be explained using Figure 18.

Step 1:

The user takes a photo of the food using a smart device (smartphone or smart glasses). The photo is saved on the device.

Step 2:

The device uploads the captured photos to the server. The server performs image preprocessing using OpenCV. Specifically, it resizes and denoises the images.

Step 3:

The server analyzes pre-processed images using Keras to recognize the characteristics and features of the product. The input is the pre-processed image, and the output is data indicating the product's characteristics and features.

Step 4:

The server generates product descriptions using Transformers based on recognized characteristics and features. The input is data describing the product's characteristics and features, and the output is the generated product description.

Step 5:

The server uses a generative AI model to translate the generated product descriptions into multiple languages. The input is the generated product description, and the output is the translated product description in multiple languages.

Step 6:

The terminal allows customers to complete their orders and payments on their smart devices, eliminating the need to wait in line with waiters or at the cash register. Input is the customer's order information, and output is a notification that the order has been completed.

Step 7:

The terminal sends questions that arise when customers consider the menu to the server. The server automatically generates explanations using AI. The input is the customer's question, and the output is the generated explanation.

Step 8:

The server recommends appropriate products based on the customer's past order history and preferences. The input is the customer's order history and preference data, and the output is the recommended products.

Step 9:

The system analyzes the customer's facial expressions and voice tone through the terminal's camera, and the server uses Keras to recognize their emotions. The input is data on the customer's facial expressions and voice tone, and the output is the recognized emotion.

Step 10:

The server uses Transformers to recommend appropriate products based on the recognized emotions. The input is the recognized emotions, and the output is the recommended products.

(Example 2)

Next, we will describe Example 2 of the 2nd embodiment. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

Modern consumers demand quick and accurate understanding of product descriptions, as well as multilingual information. They also require personalized product descriptions that cater to their emotions. However, traditional systems struggle to meet these demands, making the improvement of services, particularly for foreign tourists, a significant challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for having artificial intelligence explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This enables users to quickly and accurately understand product descriptions and provides information in multiple languages. Furthermore, by providing personalized product descriptions tailored to user emotions, customer satisfaction can be improved.

A "smart device" is a portable electronic device that has internet connectivity and can run applications.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Means of improving the customer experience" refer to features that enhance the convenience and satisfaction customers experience when using a service.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function that allows artificial intelligence to provide appropriate answers to questions that arise when customers are considering menu options.

"Preference-based recommendation mechanisms" refer to features that recommend appropriate products and services based on a customer's past choices and preferences.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions from their input and actions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to a function for translating generated product descriptions into multiple languages.

Modes for carrying out the invention

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for generating translations using artificial intelligence, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

Hardware and software to be used

The server generates product descriptions using a generative AI model (e.g., OpenAI's GPT-4). The generated product descriptions are translated into multiple languages using a translation engine (e.g., Google Translate API). To recognize user emotions, an emotion engine (e.g., IBM Watson®'s Sentiment Analysis API) is used. Smart devices are portable electronic devices with internet connectivity capable of running applications.

Explanation of the program's processing

The server receives text entered by the user through the terminal and generates a product description using a generative AI model. For example, if the user enters "This product is high quality and long-lasting," the server will generate a product description based on this text.

The generated product descriptions are translated into multiple languages using a translation engine. For example, they are translated into major tourist languages such as English, Chinese, and Korean. Specifically, a description like "This product is high quality and long-lasting" is translated.

When a user inputs emotional text or audio data through their device, the server uses an emotion engine to recognize the user's emotions. For example, if a user inputs "I'm feeling down today," the server analyzes this text and recognizes the user's emotion as "down."

The server uses a generative AI model to generate product descriptions that respond to the user's emotions, based on the emotions it recognizes. For example, if the user is feeling down, the server will select uplifting words to generate a product description.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Examples of specific cases and prompt messages

As a concrete example, consider the following scenario.

Scenario 1: Multilingual Translation

If a user enters "This product is high quality and long-lasting" in Japanese, the server will translate this product description into English, Chinese, and Korean.

Scenario 2: Product Description Based on Emotions

If a user enters "I'm feeling down today," the server uses an emotion engine to recognize the user's emotions and generates a product description that will cheer them up.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Example of a prompt message

Examples of prompts for a generative AI model are as follows:

Please translate this Japanese product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'

"Please generate a product description that will cheer up a user who is feeling down. The original description is 'This product is high quality and long-lasting.'"

The above describes the embodiments for carrying out this invention.

The flow of the specific processing in Example 2 will be explained using Figure 19.

Step 1:

The user uses a terminal to input the text that will form the basis of the product description. For example, the user might input, "This product is high quality and long-lasting." The entered text is then sent to the server.

Step 2:

The server uses a generative AI model to generate a product description based on the text entered by the user. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description based on the user's input: 'This product is high quality and long-lasting.'" The generative AI model generates a product description based on this prompt and returns it to the server. The output is the generated product description.

Step 3:

The server sends the generated product description to the translation engine for translation into multiple languages. Specifically, the server sends the following prompt to the translation engine: "Translate this product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'" The translation engine performs the translation based on this prompt and returns it to the server. The output is the product description translated into multiple languages.

Step 4:

The user inputs text or audio data related to their emotions through their device. For example, the user might input, "I'm feeling down today." The input text or audio data is then sent to the server.

Step 5:

The server uses an emotion engine to recognize the user's emotions. Specifically, the server sends the following prompt to the emotion engine: "Analyze the user's input 'I'm feeling down today' and recognize the emotion." Based on this prompt, the emotion engine analyzes the emotion and returns the result to the server. The output is the recognized emotion.

Step 6:

The server uses a generative AI model to generate a product description that responds to the user's emotions, based on the emotions it has perceived. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description that will cheer up the user if they are feeling down. The original description is 'This product is high quality and long-lasting.'" Based on this prompt, the generative AI model generates an emotion-responsive product description and returns it to the server. The output is the emotion-responsive product description.

Step 7:

The server sends a product description to the user's device that corresponds to the generated emotion. The user can then view the uplifting product description through their device. Specifically, if the user enters "I'm feeling down today," the server will provide the user with a product description such as, "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

The above outlines the specific processing flow of this system's program.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

Traditional e-commerce sites often provide product descriptions in a single language, which posed a problem for foreign tourists and multilingual users, making them difficult to understand. Furthermore, the lack of emotionally resonant product descriptions made it difficult to increase user purchase intent. Additionally, users sometimes spent a significant amount of time trying to understand product descriptions, resulting in a less-than-ideal purchasing experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for artificial intelligence to explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This makes it possible to provide product descriptions tailored to user emotions in multiple languages, making them easily understandable to foreign tourists and multilingual users. Furthermore, it can increase user purchasing intent and improve the purchasing experience.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

"Menu creation methods" refer to functions or software that allow users to create menus for products and services using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Customer experience improvement measures" refer to features and software designed to enhance the customer experience when using a product or service.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate explanations for questions that arise when users are considering menu options.

"Preference-based recommendation" refers to a function that recommends appropriate products and services based on the user's preferences and past behavior.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to functions or software for translating generated product descriptions into multiple languages.

This invention will now describe embodiments for carrying out this invention. This system includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

System Configuration

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones, tablets, and smart glasses.

Server: A high-performance computer responsible for data processing and storage.

Software

Artificial intelligence model: Generates product descriptions using the OpenAI API.

Translation software: Multilingual translation is performed using the Google Translate API.

Emotion recognition software: Uses EmotionRecognizer to recognize the user's emotions.

Data processing and data calculation

1. Image Analysis: Images of products taken with a smart device are sent to the server. The server uses an image analysis algorithm to extract product characteristics and features and generate a product description.

2. Emotion Recognition: The smart device's camera and microphone are used to analyze the user's facial expressions and voice tone. EmotionRecognizer software recognizes the user's emotions and sends the results to the server.

3. Product Description Generation: The server generates prompt text based on the user's emotions and uses the OpenAI API to generate a product description appropriate to those emotions.

4. Multilingual Translation: The generated product description is translated into multiple languages using the Google Translate API. The translation results support major languages such as English, Chinese, and Korean.

Specific examples

For example, if a user is feeling down, the smart device's camera captures the user's facial expression, and the EmotionRecognizer recognizes it as "downcast." The server then generates the following prompt:

"Generate a product description that will cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time."

This prompt is sent to the OpenAI API to generate a sentiment-appropriate product description. The generated product description will look like this:

"This product was created to brighten your life, even just a little. It's made from high-quality materials and is built to last."

Next, we will translate this product description into multiple languages using the Google Translate API.

In this way, it becomes possible to provide product descriptions tailored to the user's emotions in multiple languages.

The flow of the specific processing in Application Example 2 will be explained using Figure 20.

Step 1:

The user takes a photo of the product with their smart device.

Input: Product image

Output: Captured image data

Specific operation: The user uses a smart device such as a smartphone or tablet to take a picture of the product they are considering purchasing. This image data is saved on the smart device.

Step 2:

The device sends the captured image to the server.

Input: Captured image data

Output: Image data sent to the server

Specific operation: The smart device transmits the captured image data to the server via the internet. The HTTP protocol and similar protocols are used for transmission.

Step 3:

The server performs image analysis and extracts the product's characteristics and features.

Input: Sent image data

Output: Characteristics and features of the extracted products

Specific operation: The server uses image analysis algorithms to automatically extract product characteristics and features from image data. For example, it recognizes the product's color, shape, brand logo, etc.

Step 4:

The server generates a product description based on the extracted characteristics and features.

Input: Characteristics and features of the extracted products

Output: Generated product description

Specific operation: The server uses a generative AI model to generate product descriptions based on extracted characteristics and features. For example, a description such as "This product is made from high-quality materials and will last a long time" might be generated.

Step 5:

The device recognizes the user's emotions.

Input: User's facial expressions and tone of voice

Output: Recognized user emotions

Specific operation: Using the camera and microphone of a smart device, the EmotionRecognizer software captures the user's facial expressions and voice tone, recognizing the user's emotions. For example, emotions such as "depressed" or "happy" can be recognized.

Step 6:

The server generates prompt messages that respond to the user's emotions.

Input: Recognized user sentiment, generated product description

Output: Prompt message based on emotion

Specific operation: The server generates prompt messages to send to the AI model based on the recognized user's emotions. For example, a prompt message like, "Generate a product description to cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time," might be generated.

Step 7:

The server uses an AI model to generate product descriptions that are appropriate for the user's emotions.

Input: Prompt text that responds to emotion

Output: Product description suitable for emotions

Specific operation: The server uses a generative AI model (e.g., OpenAI's API) to generate emotionally appropriate product descriptions based on the prompt text. For example, a description like, "This product was created to brighten your life a little. It is made from high-quality materials and is durable," might be generated.

Step 8:

The server translates the generated product description into multiple languages.

Input: Generated product description

Output: Product description translated into multiple languages

Specific operation: The server uses the Google Translate API to translate the generated product description into multiple languages, such as English, Chinese, and Korean. For example, a translation might be: "This product is made to brighten your life a little. It is made of high-quality materials and lasts long."

Step 9:

The server sends the product description, translated into multiple languages, to the device.

Input: Product description translated into multiple languages

Output: Translation result sent to the terminal

Specific operation: The server sends product descriptions translated into multiple languages to smart devices via the internet. HTTP protocol and similar protocols are used for transmission.

Step 10:

The device displays product descriptions translated into multiple languages to the user.

Input: Translation result sent to the device

Output: Multilingual product description displayed to the user

Specific operation: The smart device displays the product description, translated into multiple languages, to the user. This allows the user to understand the emotionally relevant product description in multiple languages.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

Traditional customer experience improvement systems often rely on manual processes for menu creation, product descriptions, and translation, resulting in inefficiency. Furthermore, they may fail to adequately address customer questions and provide personalized recommendations, potentially leading to decreased customer satisfaction. Additionally, the lack of features to recognize and respond to customer emotions hinders the improvement of the overall customer experience.

In Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and a means for recognizing user emotions and providing services corresponding to those emotions. This enables quick answers to customer questions when considering menus, product recommendations tailored to preferences, and significantly improves customer satisfaction by providing services that respond to customer emotions.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

"Methods for generating translation work using artificial intelligence" refers to functions that automatically translate text and audio using artificial intelligence.

"Methods for improving customer experience by reducing waiting times" refer to functions that shorten the waiting time customers experience when using a service, thereby providing a smoother experience.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are considering menu options.

"A method for recommending products based on preferences" refers to a function where artificial intelligence recommends appropriate products based on the customer's preferences and past choices.

"A means of recognizing user emotions and providing services that respond to those emotions" refers to a function that recognizes user emotions from their facial expressions and voice, and provides appropriate services that correspond to those emotions.

This invention is a system for improving customer experience and includes means for creating menus using smart devices, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for recommending products according to preferences, and means for recognizing user emotions and providing services that respond to those emotions.

Hardware and software to be used

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones and tablets.

Server: A high-performance server (e.g., a server with an NVIDIA GPU).

Software

Generative AI model: GPT-4 is used as an example.

Emotion recognition engine: Affectiva SDK is used as an example.

Program Processing Description

The server receives information sent from the smart device and performs analysis using a generative AI model. For example, if the user inputs "I like spicy food," the generative AI model generates a list of spicy dishes, and the server sends this list to the smart device. The smart device then displays the received list to the user.

Furthermore, when a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to generate ingredient information and sends it to the smart device. The smart device then displays the received ingredient information to the user.

Furthermore, smart devices use their built-in cameras and microphones to capture the user's facial expressions and voice, and use an emotion recognition engine to recognize the user's emotions. For example, if the user shows an angry expression, the smart device sends that information to a server. The server uses a generative AI model to generate an apology and sends it to the smart device. The smart device then displays the received apology to the user.

Examples of specific cases and prompt messages

Example 1: User enters "I like spicy food."

The user opens the app on their smartphone and enters "I like spicy food."

The server generates a list of spicy dishes using an AI model.

The server generates a list and sends it to the smart device.

A smart device displays a list of spicy dishes to the user.

Example prompt: "Please list dishes you would recommend to a customer who likes spicy food."

Example 2: A user asks, "What are the ingredients in this dish?"

The user opens the tablet app and asks, "What are the ingredients in this dish?"

The server generates ingredient information using an AI model.

The server generates ingredient information and sends it to the smart device.

The smart device displays ingredient information to the user.

Example prompt: "Please tell me the ingredients for this dish."

Example 3: The user shows an angry expression.

The user displays an angry expression while using their smartphone.

Smart devices use cameras to capture the user's facial expressions, which are then analyzed by an emotion recognition engine.

The smart device sends information about the anger it recognizes to the server.

The server generates an apology using an AI model.

The server generates an apology message and sends it to the smart device.

Smart devices display an apology message to the user.

Example prompt: "Generate an apology for when the user is angry."

In this way, the server, smart device, and user work together to operate the system and improve the customer experience. The flow of a specific process in Example 3 will be explained using Figure 21.

Step 1:

The user inputs information through their device. The user uses a smartphone or tablet to input questions and preferences into the system. For example, they might input "I like spicy food." The input data is saved on the device in text format.

Step 2:

The terminal sends the input information to the server. The terminal sends the user-entered information to the server via the internet. The input data is sent to the server in text format.

Step 3:

The server analyzes the input information using a generative AI model. The server passes the received input information to the generative AI model (e.g., GPT-4) for analysis. The generative AI model generates appropriate responses or recommendations based on the input information. For example, given the information "likes spicy food," it generates a list of spicy dishes. The input data is in text format, and the output data is in list format.

Step 4:

The server generates appropriate answers and recommendations based on the analysis results. The server generates answers and recommendations for the user based on the analysis results obtained from the generating AI model. For example, it might generate a list of spicy dishes. The output data is in list format.

Step 5:

The server sends the generated responses and recommendations to the device. The server sends the generated responses and recommendations to the device via the internet. The output data is sent to the device in list format.

Step 6:

The device displays answers and recommendations to the user. The device displays answers and recommendations received from the server to the user. The user can review the displayed information. The output data is displayed in list format.

Step 7:

The device recognizes the user's emotions using an emotion engine. The device captures the user's facial expressions and voice using its built-in camera and microphone, and recognizes the user's emotions using an emotion engine (e.g., Affectiva SDK). Input data is in image or audio format, and output data is in emotion information format.

Step 8:

The device transmits recognized emotion information to the server. The device transmits the recognized emotion information to the server via the internet. Input data is sent to the server in emotion information format, and output data is also sent in emotion information format.

Step 9:

The server generates appropriate services based on emotional information. The server uses a generative AI model to generate appropriate services based on the received emotional information. For example, if the user shows an angry expression, it will generate an apology. Input data is in emotional information format, and output data is in text format.

Step 10:

The server sends the generated service to the terminal. The server sends the generated service to the terminal via the internet. The output data is sent to the terminal in text format.

Step 11:

The terminal provides services to the user. The terminal delivers services received from the server to the user. For example, it displays an apology. The output data is displayed in text format.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

Traditional food delivery services faced challenges such as users spending a significant amount of time choosing from menus and difficulty in providing services tailored to user preferences and emotions. Furthermore, multilingual support for foreign tourists was insufficient, highlighting the need for improved user experience. There is a demand to address these challenges and provide a more comfortable and personalized service.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing the user's emotions and providing services corresponding to those emotions. This allows users to efficiently select menu items and receive personalized services tailored to their individual preferences and emotions. Furthermore, multilingual support enhances services for foreign tourists.

A "smart device" is a portable electronic device with advanced computing capabilities, such as a smartphone or tablet.

A "menu creation method" refers to a method or device for generating a list of products or services that a user can select.

"An artificial intelligence system for generating product descriptions from images" refers to an artificial intelligence system that uses image recognition technology to automatically analyze the characteristics and features of a product and generates a product description based on that analysis.

"Means of generating translation work using artificial intelligence" refers to an artificial intelligence system that automatically translates text and audio between different languages.

"Customer experience improvement measures" refer to methods and devices that enhance the convenience and satisfaction users experience when using a service.

"A system where artificial intelligence explains questions that arise when considering menu options" refers to a system where artificial intelligence provides appropriate answers to questions that users may have when choosing from a menu.

A "method for recommending products based on preferences" is a system that recommends appropriate products and services based on the user's past choices and input information.

An "emotion recognition system" is a system that analyzes a user's emotions from their facial expressions, tone of voice, etc., and responds accordingly.

The system for carrying out this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing user emotions and providing services corresponding to those emotions.

System Program

The program for this system performs the following operations:

Hardware and software

Hardware:

Smart devices (smartphones, tablets, etc.)

Camera (to capture the user's facial expressions)

Computer (a processing unit for executing programs)

Software:

OpenCV (image processing library)

Keras (deep learning library)

Transformers (a library of generative AI models)

Data processing and data calculation

1. Menu creation method:

The server generates a list of products and services that users can select using their smart devices.

2. Artificial intelligence methods for generating product descriptions from images:

The server analyzes images captured by the smart device's camera and automatically recognizes the product's characteristics and features.

Based on the recognized information, generate a product description.

3. Methods for generating translation tasks using artificial intelligence:

The server automatically translates text and audio between different languages.

In particular, we will provide multilingual support to enhance services for foreign tourists.

4. Measures to improve customer experience:

The server reduces waiting times for users when accessing the service, improving convenience.

5. How artificial intelligence can explain questions that arise during menu planning:

The server uses a generative AI model to provide appropriate answers to questions that arise when users select menu items.

6. Methods for recommending products based on preferences:

The server recommends appropriate products and services based on the user's past choices and input information.

7. Emotion recognition means:

The server analyzes the user's facial expressions and voice tone captured by the camera to recognize their emotions.

Respond appropriately to the recognized emotions.

Specific examples

When a user enters "I like spicy food," the server uses a generative AI model to recommend spicy dishes.

When a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to provide information about the ingredients.

If the user displays an angry expression, the server uses emotion recognition to apologize with the message, "I'm sorry. Is there a problem?"

Example of a prompt message

"I like spicy food. What do you recommend?"

"What are the ingredients in this dish?"

"How would you respond if a user is showing signs of anger?"

In this way, users can efficiently select from menus and receive personalized service tailored to their individual preferences and feelings. Furthermore, multilingual support enhances services for foreign tourists.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The user opens the menu using a smart device.

Input: A menu display request initiated by the user.

Data Processing: The server retrieves menu information from the database and sends it to the smart device.

Output: A menu will appear on your smart device.

Step 2:

The user takes a photo of the food using the camera on their smart device.

Input: An image of a dish taken by the user.

Data Processing: The server receives images and analyzes them using an image recognition algorithm (OpenCV).

Output: The analysis results extract the characteristics and features of the dishes.

Step 3:

The server generates a product description from the image.

Input: Data on the characteristics and features of the dish.

Data Processing: The server uses a generative AI model to generate product descriptions based on characteristics and features.

Output: Product description text is generated and sent to the smart device.

Step 4:

Users enter questions when considering menu options.

Input: The question entered by the user (e.g., "What are the ingredients in this dish?").

Data Processing: The server generates answers to questions using a generative AI model.

Output: The answer text is generated and displayed on the smart device.

Step 5:

The server recommends products based on the user's preferences.

Input: User's past selections and input information (e.g., "I like spicy food").

Data Processing: The server uses a generative AI model to recommend products based on user preferences.

Output: A list of recommended products is displayed on your smart device.

Step 6:

The user captures their facial expressions using the camera on their smart device.

Input: User's facial expression image.

Data Processing: The server analyzes emotions using a facial recognition algorithm (Keras).

Output: As an analysis result, user sentiment data is generated.

Step 7:

The server provides services tailored to the user's emotions.

Input: User emotion data (e.g., anger).

Data Processing: The server generates appropriate responses based on sentiment data (e.g., apology messages).

Output: The corresponding message will be displayed on the smart device.

Step 8:

The server performs the translation.

Input: Text or audio data entered by the user.

Data Processing: The server uses a translation algorithm to translate the input data into multiple languages.

Output: Translated text and audio data are displayed on the smart device.

The specific processing unit 290 transmits the result of the specific processing to the smart device 14. On the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input regarding the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. On the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is the generative AI ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives prompts containing instructions, as well as inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions provided by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini® (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example of a configuration in which specific processing is performed by the data processing device 12 was given, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[Second Embodiment]

Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

As shown in Figure 3, the data processing system 210 comprises a data processing device 12 and a smart eye.

It is equipped with a mirror 214. An example of a data processing device 12 is a server.

The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. It captures images of the area surrounding the user 20 (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

The communication interface 44 is connected to the network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via the network 54. The exchange of various types of information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is performed by the processor 46 on the RAM 48.

This is achieved by operating as the control unit 46A according to the force program 60.

Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be explained.

"Example of Morphology 1"

In one embodiment of this invention, a restaurant operator creates a menu using a smartphone. Specifically, the operator uploads photos of products taken with the smartphone's camera, and AI automatically recognizes the characteristics and features of the products from the photos. Based on this recognition, a product description is generated. This product description is then displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Furthermore, in this embodiment of the present invention, the AI generates the translation work. Specifically, the generated product description is translated into multiple languages. This makes the product description easier for foreign tourists to understand. For example, it is possible to translate into major tourist languages such as English, Chinese, and Korean.

"Example of Morphology 3"

Furthermore, in this embodiment of the present invention, the customer experience is also improved. Specifically, when a customer is considering a menu, the AI explains their questions and recommends products according to their preferences. For example, if a customer inputs information such as "I like spicy food," the AI will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI will provide an answer to that question.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: The restaurant operator takes photos of the products using their smartphone camera.

Step 2: Upload the photos you took to the system.

Step 3: The system's AI automatically recognizes the product's characteristics and features from the photograph.

Step 4: The AI generates a product description based on the recognition results.

Step 5: The generated product description is displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Step 1: Obtain the product description generated by the AI.

Step 2: The AI translates the product description into multiple languages.

Step 3: The translated product description is displayed to foreign tourists on the restaurant's website or application.

"Example of Morphology 3"

Step 1: When customers are considering the menu, they input their questions into the AI.

Step 2: The AI generates the answer to that question.

Step 3: The AI recommends products based on the customer's preferences.

Step 4: The AI-generated answers and recommendations are displayed to the customer.

(Example 1)

Next, we will describe Example 1 of Embodiment 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

When restaurant operators create menus, the process of taking photos of products and then recognizing their characteristics and features to generate product descriptions is time-consuming. Furthermore, there is a lack of means to provide multilingual support for foreign tourists and to enhance the customer experience. Therefore, there is a need for increased efficiency in menu creation and improved customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a means for creating menus using a smartphone; an artificial intelligence (AI) means for generating product descriptions from smartphone photos; a means for AI to generate translations; a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a means for AI to explain questions during menu consideration; a means for recommending items according to preferences; a means for uploading product photos taken with a smartphone camera, for AI to automatically recognize the characteristics and features of the products from those photos; and a means for generating product descriptions based on the recognition results. This enables increased efficiency in menu creation and improved customer satisfaction.

A "smartphone" is a multi-functional mobile device that, in addition to the functions of a regular mobile phone, also offers internet connectivity and the ability to use applications.

"Menu creation methods" refer to the methods and tools that restaurant operators use to create menus, particularly those using smartphones.

"Artificial intelligence tools" refer to technologies that use machine learning and data analysis to automatically perform specific tasks.

"Means of generating translation work using artificial intelligence" refers to methods and technologies that use artificial intelligence to translate text into multiple languages.

"Methods for improving the customer experience" refer to methods and tools for improving the convenience and satisfaction customers experience when using a service.

"Methods for using artificial intelligence to explain questions during menu consideration" refers to methods and technologies in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"Methods of recommending based on preferences" refer to methods and technologies that recommend appropriate products and services based on customer preferences and past choices.

"Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

"Methods for generating product descriptions" refers to methods and technologies for creating product descriptions in natural language based on recognized product characteristics and features.

This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, users upload photos of their products taken with their smartphone cameras, and artificial intelligence (AI) automatically recognizes the characteristics and features of the products from these photos. Based on this recognition, the system generates product descriptions. These product descriptions are then displayed to customers on the restaurant's website or application.

Hardware and software to be used

Smartphone: A device used for taking and uploading photos.

Server: Performs data processing and runs AI models.

Artificial intelligence models: Image recognition models and natural language generation models based on TensorFlow and PyTorch (e.g., GPT-3, BERT).

Data processing and data calculation

1. The user takes a photo of the product with their smartphone.

The user launches their smartphone's camera app and takes a picture of the product. For example, they might take a picture of a new dessert, "Chocolate Cake."

2. The user uploads photos to the system.

The user opens a dedicated application, selects the photos they've taken, and presses the upload button. The photos are then sent to the server via the internet.

3. The server receives the photos and inputs them into the AI model.

The server receives the uploaded photos and inputs them into an AI model for image processing. The AI model used here is based on TensorFlow or PyTorch.

4. The server uses an AI model to recognize the characteristics and features of the product.

The server uses an AI model to recognize the characteristics and features of a product from a photograph. For example, it extracts information such as the type of dessert, main ingredients, and visual characteristics.

5. The server generates a product description based on the recognition results.

The server generates product descriptions based on the AI's recognition results. These product descriptions are created using natural language generation technology. The software used includes GPT-3 and BERT.

6. The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Customers can then view them.

Specific examples

A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description, such as "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing," and displays it on the website.

Example of a prompt message

Please upload a photo of your new dessert. AI will recognize the product's characteristics and features from the photo and generate a product description.

The flow of the specific processing in Example 1 will be explained using Figure 11.

Step 1:

The user takes a photo of the product with their smartphone.

Input: Product (e.g., Chocolate cake)

Output: Photos of the photographed product

Specific action: The user launches the camera app on their smartphone and takes a picture of the product. For example, they might take a picture of the new dessert, "Chocolate Cake."

Step 2:

The user uploads a photo to the system.

Input: Photographs of the product

Output: Photo data sent to the server

Specific operation: The user opens a dedicated application, selects the photo they have taken, and presses the upload button. The photo is sent to the server via the internet.

Step 3:

The server receives the photos and inputs them into the AI model.

Input: Photo data sent to the server

Output: Photo data input to the AI model

Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

Step 4:

The server uses an AI model to recognize the characteristics and features of the product.

Input: Photo data entered into the AI model

Output: Characteristics and features of the recognized product (e.g., chocolate cake, cream topping, berry decoration)

Specific operation: The server runs an AI model to extract product characteristics and features from a photograph. For example, it recognizes the type of dessert, main ingredients, and visual features.

Step 5:

The server generates a product description based on the recognition results.

Input: Characteristics and features of the recognized product

Output: Generated product description (Example: "This chocolate cake features rich chocolate and creamy toppings. The berry decorations make it visually appealing.")

Specific operation: The server uses GPT-3 or BERT to generate product descriptions based on the recognition results.

Step 6:

The server displays the product description on the website or application.

Input: Generated product description

Output: Product description displayed on the website or application

Specific operation: The server saves the product description to the website's database, sends the data to the front-end for display, and the customer can view it.

(Application Example 1)

Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Traditional restaurant menu creation was often done manually, which was time-consuming and labor-intensive, and made it difficult to adequately convey the appeal of the products. Furthermore, the lack of multilingual support for foreign tourists and insufficient recommendation features tailored to customer preferences were also problems. In addition, there were limited means to improve the in-store customer experience, such as having to wait for waitstaff or in line at the register, highlighting the need for increased customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an AI method for generating product descriptions from smartphone photos; a method for AI-generated translations; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI-generated explanations of questions during menu consideration; a method for making recommendations based on preferences; a method for recognizing product characteristics and features using an image recognition model; and a method for generating product descriptions based on recognized features using a natural language generation model. This enables efficient menu creation and automatic generation of attractive product descriptions, facilitating multilingual support, improved customer experience, and personalized recommendations.

"A method for creating menus using a smartphone" refers to a method for efficiently creating restaurant menus using a smartphone's camera and applications.

"AI methods for generating product descriptions from smartphone photos" refers to artificial intelligence methods that analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"A method for generating translations using AI" refers to a method of translating product descriptions and menu contents into multiple languages using artificial intelligence.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"An AI-powered solution for questions during menu consideration" refers to a method where artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending based on preferences" refers to a method of recommending appropriate products or menu items based on a customer's past choices and preferences.

"A means of recognizing the characteristics and features of a product using an image recognition model" refers to a method of automatically recognizing the characteristics and features of a product from a photograph using image recognition technology.

"A means of generating product descriptions based on features recognized using a natural language generation model" refers to a method of automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

The system for implementing this invention includes a means for creating menus using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for AI to generate translations, a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register, a means for AI to explain questions when considering menus, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on recognized features using a natural language generation model.

System Program

The server implements the system using the following hardware and software.

Hardware:

Smartphone (with camera)

Server (for hosting AI models)

Software:

TensorFlow: A library for image recognition

OpenAI GPT-3: API for natural language generation

Explanation of the process

Image recognition:

The user takes a photo of a dish with their smartphone camera and uploads it to the server via the application. The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of the dish from the image. For example, features such as "chicken curry, spicy, tomato-based" might be recognized.

Natural language generation:

The server converts the recognized features into a string and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is: "Generate an appealing product description for a dish with the following features: Chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it might generate a description such as: "This spicy chicken curry features juicy chicken simmered in a tomato-based sauce. Aromatic spices will whet your appetite."

Translation work:

The generated product descriptions are translated into multiple languages as needed. The server uses AI to perform translations into multiple languages, such as English and Chinese.

Improving the customer experience:

Customers can use their smartphones to select menu items and place orders in-store. This reduces waiting times at the counter and cashier, enabling smoother service.

Explanation of the question and recommendations:

The AI automatically provides answers to questions customers may have when considering menu options. It also recommends appropriate products and menu items based on the customer's past choices and preferences.

In this way, menu creation becomes more efficient, attractive product descriptions are automatically generated, and multilingual support, improved customer experience, and personalized recommendations become possible.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The input is the photo of the food, and the output is the image data sent to the server.

Step 2:

The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of dishes from uploaded image data. The input is image data, and the output is a list of the recognized characteristics and features. Specifically, the image data is preprocessed, input into the model, and prediction results are obtained.

Step 3:

The server converts the recognized characteristics and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of characteristics and features, and the output is a prompt statement. Specifically, it formats the characteristics and features and generates the prompt statement.

Step 4:

The server receives a response from GPT-3 and generates an attractive product description. The input is a prompt, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and retrieves the product description.

Step 5:

The server translates the generated product description into multiple languages as needed. The input is the product description, and the output is the translated product description. Specifically, it calls a translation API to translate into multiple languages.

Step 6:

Users use their smartphones to select menu items and place orders in the store. Input is generated product descriptions, and output is the user's order information. Specifically, the application displays product descriptions, and the user selects their order.

Step 7:

The server automatically provides answers to questions that arise when users consider menu options, using AI. The input is the user's question, and the output is the AI's answer. Specifically, it analyzes the question and generates an appropriate answer.

Step 8:

The server recommends appropriate products and menu items based on the user's past choices and preferences. The input is the user's past selection data, and the output is the recommended products and menu items. Specifically, it analyzes past selection data and uses a recommendation algorithm to suggest products.

(Example 2)

Next, we will describe Example 2 of the 2nd embodiment. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Traditional systems often required manual translation of product descriptions into multiple languages, resulting in significant time and effort. Furthermore, services for foreign tourists were inadequate, leading to difficulties in understanding product descriptions. Additionally, insufficient efforts to improve the customer experience and address questions during menu selection could potentially lower customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for creating menus using a smart device, means for artificial intelligence (AI) to generate product descriptions from images on the smart device, means for AI to generate translation tasks, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations based on preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into the generation AI model and obtaining translation results, means for saving translation results to a database, means for sending translation results to a terminal, and means for the terminal to display the translation results. This automates the multilingual translation of product descriptions, improves services for foreign tourists, enhances the customer experience, and resolves questions during menu consideration.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that possesses advanced computing power and communication capabilities.

"Menu creation methods" refer to functions and applications used to create menus for restaurants and retail stores using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

"Means of improving the customer experience" refer to features and methods that enhance the convenience and satisfaction customers experience when using products or services.

"A means for AI to explain questions during menu consideration" refers to a function where artificial intelligence automatically provides answers to questions and concerns that arise when customers are considering menu options.

"Preference-based recommendations" refers to a function that recommends appropriate products and services based on a customer's past choices and preferences.

"Methods for translating product descriptions into multiple languages" refers to functions for translating product descriptions into several languages.

A "generative AI model" is an artificial intelligence model trained to perform tasks such as text generation and translation.

A "prompt statement" is an instruction given to a generative AI model to perform a specific task.

"Means for saving translation results to a database" refers to the function of saving translation results generated by a generative AI model to a database.

"Means for sending translation results to the terminal" refers to the function that sends translation results stored in the database to the user's terminal.

"Means for the terminal to display translation results" refers to the function that displays the received translation results on the user's terminal screen.

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translation tasks, means for improving the customer experience, means for artificial intelligence to explain questions when considering menus, means for recommending according to preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into a generation AI model and obtaining translation results, means for saving translation results in a database, means for sending translation results to a terminal, and means for the terminal to display translation results.

Hardware and software to be used

Hardware:

Smart devices (smartphones, tablets, etc.)

Server (a computer with high-performance computing capabilities)

Software:

Generative AI models (e.g., OpenAI's GPT-4)

Database Management System

Communication protocol (e.g., HTTP/HTTPS)

Data processing and data calculation

Product description generation:

The user takes a picture of the product using a smart device and sends the image to the server. The server uses artificial intelligence to generate a product description from the image. Specifically, it uses an image analysis algorithm to recognize the product's characteristics and features, and then generates a product description based on that.

Prompt message generation:

The server generates prompt text using a generative AI model. The prompt text specifies the target language for translation (e.g., English, Chinese, Korean).

Translation work execution:

The server inputs the generated prompt text and product description into the AI model and retrieves the translation results. The AI model then performs multilingual translation based on the input text.

Saving and submitting translation results:

The server saves the acquired translation results to a database. The saved translation results are sent to the user's device, which then displays the translation.

Specific examples

Specific example:

The user enters the Japanese product description: "This product uses high-quality materials."

The server generates the prompt message: "Translate the following product description into English: This product uses high-quality materials."

The server inputs the prompt text and product description into the generated AI model and retrieves the English translation result: "This product uses high-quality materials."

The server saves the translation results to a database and sends them to the terminal.

The device displays the translation results, and the user confirms them.

This system enables multilingual translation of product descriptions through collaboration between the server, terminal, and user.

The flow of the specific processing in Example 2 will be explained using Figure 13.

Step 1:

The user enters the product description.

The user enters the product description in text format using a smart device. The entered product description is then sent from the smart device to the server.

Input: Product description text

Output: Product description text sent to the server

Step 2:

The server receives the product description.

The server receives the product description text sent by the user and temporarily stores it in memory.

Input: Product description text submitted by the user

Output: Product description text saved in memory

Step 3:

The server generates the prompt message.

The server generates prompt messages based on the specified languages for translation. For example, if translation is required for English, Chinese, and Korean, it will generate prompt messages corresponding to each language.

Input: Product description text, target language for translation

Output: Generated prompt message

Step 4:

The server inputs prompt text and product description into the generated AI model.

The server inputs the generated prompt text and product description text into the AI model. The AI model then performs translation based on the input text.

Input: Prompt message, product description text

Output: Data input to the generative AI model

Step 5:

The server retrieves the translation result.

The server retrieves translation results from the generative AI model. The retrieved translation results are separated by language.

Input: Data entered into the generative AI model

Output: Translated text

Step 6:

The server saves the translation results to the database.

The server saves the retrieved translation results to a database. When saving, it associates the original Japanese product description with the translated result.

Input: Translated text, original product description text

Output: Translation results stored in the database

Step 7:

The server sends the translation result to the terminal.

The server sends the saved translation results to the user's device. The transmitted data is provided in a format accessible to the user.

Input: Translation results stored in the database

Output: Translation result sent to the terminal

Step 8:

The device displays the translation result.

The terminal receives the translation results sent from the server and displays them to the user. The user can review the displayed translation results and make corrections or re-translates as needed.

Input: Translation result sent from the server

Output: Translation result displayed on the terminal

(Application Example 2)

Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart glasses 214 will be referred to as a "terminal."

In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing goods, making it difficult for them to understand product descriptions. Furthermore, visually impaired individuals and those with reading and writing difficulties also have limited means of understanding product descriptions. This can lead to a diminished customer experience and potentially impact sales.

In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, and a means for translating product descriptions into multiple languages and displaying and playing them aloud in the user's set language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired individuals, and people with reading and writing difficulties.

"A method for creating menus using a smartphone" refers to a method for creating menus using a smartphone.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

"Methods for generating translation work using artificial intelligence" refers to methods of translating text into multiple languages using artificial intelligence.

"Methods for improving the customer experience" refer to measures taken to improve the customer's experience when purchasing a product.

"A method for using artificial intelligence to explain questions that arise when customers are considering menu options" refers to a method in which artificial intelligence answers questions that customers may have when considering menu options.

"Methods of recommending based on preferences" refer to methods of recommending products and services based on customer preferences.

"A means of translating product descriptions into multiple languages and displaying and playing them in the user's preferred language" refers to a means of translating product descriptions into multiple languages and displaying and playing them in the user's preferred language.

To implement this invention, it is necessary to construct a system that combines a smartphone, artificial intelligence technology, a translation API, and a voice playback function.

First, smartphones have a camera function that allows users to photograph products. When a user photographs a product, the image data is sent to a server. The server then uses image recognition technology to generate a product description. This image recognition technology could utilize, for example, the Google Cloud Vision API.

Next, the generated product description is translated into multiple languages using artificial intelligence on the server. Translation services such as the Google Cloud Translation API are used for this translation. The translated text is then sent to the user's smartphone and displayed based on their language settings.

Furthermore, the translated product descriptions are played back using the smartphone's audio playback function. This makes the product accessible to visually impaired individuals and those with reading and writing difficulties.

As a concrete example, consider a tourist trying to purchase a leather wallet at a physical store in Japan. When the tourist scans the wallet's tag with their smartphone camera, the app automatically recognizes the product description and translates it into the user's chosen language (e.g., English). The translated product description would be displayed as "This is a high-quality leather wallet," and also played aloud.

Examples of prompt messages include the following:

I am developing an application to translate product descriptions into multiple languages. Please translate the following text into the specified languages.

Text: "This is a high-quality leather wallet."

Target language: Japanese

In this way, foreign tourists, visually impaired individuals, and people with reading and writing difficulties will be able to understand product descriptions without encountering language barriers when purchasing items in physical stores.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user takes a picture of the product using their smartphone camera. The input is the product image taken with the smartphone camera. The output is the captured product image data.

Step 2:

The device sends the captured product image data to the server. The input is the product image data. The output is the product image data sent to the server.

Step 3:

The server generates product descriptions using image recognition technology. The input is product image data. The server extracts text information from the images using APIs such as the Google Cloud Vision API and generates the product description. The output is the generated product description text.

Step 4:

The server translates the generated product description text into multiple languages. The input is the product description text. The server uses the Google Cloud Translation API to translate it into the specified languages. The output is the translated product description text.

Step 5:

The server sends the translated product description text to the terminal. The input is the translated product description text. The output is the translated product description text sent to the terminal.

Step 6:

The device displays the translated product description text. The input is the translated product description text. The output is the translated product description displayed on the smartphone screen.

Step 7:

The device plays the translated product description text as audio. The input is the translated product description text. The device uses the smartphone's audio playback function to convert the text to audio and play it. The output is the translated product description played as audio.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Traditional menu creation and customer experience improvement systems fail to adequately answer users' questions when considering menu items and to provide product recommendations tailored to individual preferences. Furthermore, they are insufficient in reducing waiting times and providing multilingual services. This can lead to decreased customer satisfaction and negatively impact store sales.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for the user to input information, means for transmitting the input information to the server, means for the server to analyze the input information using a generated AI model, means for generating recommendations and answers based on the analysis results, means for transmitting the generated recommendations and answers to a terminal, and means for the terminal to display the recommendations and answers to the user. This enables appropriate answers to questions the user may have when considering a menu, and product recommendations tailored to individual preferences. Furthermore, it can reduce waiting times and provide multilingual services, leading to improved customer satisfaction and increased store sales.

A "smart device" is an electronic device used by users to input information and communicate with a server, and includes smartphones and tablets.

"Artificial intelligence tools" refer to algorithms and models that analyze input data and generate appropriate responses or recommendations.

A "generative AI model" refers to a machine learning model that analyzes user input information and generates appropriate responses and recommendations.

"Methods for improving the customer experience" refer to measures that reduce waiting times and allow customers to use services comfortably.

"A means for artificial intelligence to explain questions that arise when considering menu options" refers to a method by which artificial intelligence provides appropriate answers to questions that users may have when considering menu options.

"Methods for recommending products based on preferences" refers to methods for recommending appropriate products based on the user's preferences.

"Means for user information input" refers to the interface through which users input information in formats such as text or voice.

"Means for sending inputted information to the server" refers to the communication means used to send information entered by the user to the server.

"Means for a server to analyze input information using a generated AI model" refers to the means by which a server uses a generated AI model to analyze information received from a user.

"Means for generating recommendations and responses based on analysis results" refers to methods for generating recommendations and responses for users based on the analysis results of a generation AI model.

"Means for sending generated recommendations and responses to the device" refers to the communication methods used by the server to send generated recommendations and responses to the user's device.

"Means by which a device displays recommendations and answers to a user" refers to the means by which a user's device displays recommendations and answers received from a server to the user.

This invention is a system in which a user inputs information using a smart device, a server analyzes that information using a generated AI model, and then generates and provides appropriate recommendations and responses to the user. A specific embodiment of this system is described below.

First, the user inputs information into the system using a smart device such as a smartphone or tablet. For example, the user might input "I like spicy food." This information is entered through the smart device's interface.

Next, the smart device sends the entered information to the server. In this process, the smart device converts the input data into an appropriate format (e.g., JSON format) and sends it to the server via the network.

The server analyzes the received user input using a generation AI model (for example, OpenAI's GPT-4). Specifically, it analyzes information such as "I like spicy food" and searches a database related to spicy food.

The server recommends products tailored to the user's preferences based on the analysis results. For example, it generates a list of spicy dishes and presents it to the user. Furthermore, if the user asks, "What are the ingredients in this dish?", the server generates an answer to that question.

The generated recommendations and responses are sent from the server to the smart device. The server converts the data into the appropriate format and sends it to the smart device over the network.

Smart devices display recommendations and answers received from the server to the user. For example, they might display, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot."

As a concrete example, the following prompt message is shown.

"I like spicy food. Could you recommend some dishes?"

"What are the ingredients in this dish?"

In this way, a system is realized in which the server, smart device, and user work together to improve the customer experience. This system allows users to receive appropriate answers to questions when considering menus and product recommendations tailored to their individual preferences. Furthermore, it enables reduced waiting times and the provision of multilingual services, leading to improved customer satisfaction and increased store sales. The flow of specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters information using a terminal.

The user opens the application on their smartphone or tablet, enters "I like spicy food" into the text box, and presses the submit button. The input data is in text format and represents information about the user's preferences.

Step 2:

The terminal sends the entered information to the server.

The terminal converts the user's input, "I like spicy food," into JSON format and sends it to the server using the HTTPS protocol. The input data is user preference information in text format, and the output data is request data in JSON format.

Step 3:

The server analyzes the input information using a generated AI model.

The server parses the received JSON data and inputs the information "likes spicy food" into a generative AI model. The generative AI model (e.g., GPT-4) searches a database of spicy dishes and generates a list of related dishes. The input data is a request in JSON format, and the output data is the list of dishes resulting from the analysis.

Step 4:

The server generates recommendations and responses based on the analysis results.

The server generates a list of spicy dishes based on the analysis results obtained from the generated AI model. For example, it might list dish names such as "Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is the list of dishes resulting from the analysis, while the output data is a recommendation list presented to the user.

Step 5:

The server sends recommendations and responses generated by the server to the device.

The server converts the generated list of spicy dishes into JSON format and sends it to the terminal using the HTTPS protocol. The input data is a recommendation list, and the output data is a JSON response.

Step 6:

The device displays recommendations and answers to the user.

The terminal parses the received JSON data and displays to the user, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is response data in JSON format, and the output data is text information displayed to the user.

(Application Example 3)

Next, we will describe Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server," and the smart glasses 214 will be referred to as a "terminal."

Traditional food delivery systems have problems such as difficulty for customers to obtain detailed information when choosing a menu and a lack of personalized recommendations. Furthermore, the lack of a means to provide real-time information on ingredients and nutritional content has been a factor in lowering customer satisfaction.

In Application Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting time, a means for artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, a means for recommending dishes based on customer preferences, and a means for providing information on the ingredients of the dishes. As a result, customers can obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

A "menu creation method" is a function for generating a list of dishes and drinks that customers can choose from.

"An artificial intelligence method for generating product descriptions from images" refers to a function that analyzes images taken with a smart device and automatically generates product descriptions based on their content.

"A means of generating translation work using artificial intelligence" refers to a function that automatically translates text between different languages.

"Methods for improving the customer experience by reducing waiting times" refer to features that reduce the waiting time customers experience when using a service.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are choosing from a menu.

"A method of recommending based on preferences" refers to a function that recommends appropriate products and services based on the customer's preferences and past choices.

"A method for recommending dishes based on customer preferences" refers to a function that recommends appropriate dishes based on the preference information entered by the customer.

"Means of providing information on the ingredients of a dish" refers to a function that provides information about the ingredients and components contained in a specific dish.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from images on a smart device, means for generating translations using artificial intelligence, means for improving the customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for making recommendations according to preferences, means for recommending dishes based on customer preferences, and means for providing information on the ingredients of dishes.

System Program

This system's program is implemented using Python and the OpenAI API. The server receives customer input information and uses a generative AI model to provide appropriate recommendations and information.

Explanation of the process

The server receives customer preferences and questions sent from smart devices and generates prompts based on this information. These generated prompts are input into a generation AI model via the OpenAI API, which then generates appropriate answers and recommendations. The generated information is then sent back to the smart device and provided to the customer.

The hardware used will be smart devices such as smartphones and tablets. The software used will be Python and the OpenAI API.

Specific examples

For example, if a customer enters "I like spicy food," the server will generate the following prompt:

"Please recommend some dishes for people who like spicy food."

Entering this prompt into the OpenAI API will cause the generating AI model to recommend spicy dishes such as "Mapo Tofu" and "Kimchi Hot Pot."

Furthermore, when a customer asks, "What are the ingredients in Mapo Tofu?", the server generates the following prompt:

"What are the ingredients in Mapo Tofu?"

When this prompt is entered into the OpenAI API, the generating AI model will provide ingredient information such as "tofu, ground pork, green onions, garlic, ginger, chili bean paste, sweet bean paste, soy sauce, sake, sugar, and chicken broth."

This allows customers to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the application using a smart device and enters their preferences and answers questions. The entered information is sent to the server in text format. Examples of input include "I like spicy food" and "What are the ingredients in Mapo Tofu?".

Step 2:

The server analyzes the input information received from the user and generates an appropriate prompt. For example, in response to the input "I like spicy food," it generates the prompt "Please recommend some dishes for someone who likes spicy food." This prompt is then input into the generation AI model.

Step 3:

The server sends the generated prompt to the OpenAI API and retrieves appropriate answers and recommendations based on the generated AI model. For example, in response to the prompt "What dishes would you recommend for someone who likes spicy food?", answers such as "Mapo Tofu" or "Kimchi Hot Pot" would be generated.

Step 4:

The server analyzes the responses and recommendations obtained from the generative AI model and converts them into a format for user presentation. For example, it formats responses like "Mapo Tofu" and "Kimchi Hot Pot" into a list.

Step 5:

The server sends formatted responses and recommendations to the smart device. The user can then view this information on the smart device's screen.

Step 6:

Users can select menu items based on the information provided or ask further detailed questions. For example, they might re-enter the question, "What are the ingredients in Mapo Tofu?"

Step 7:

The server receives user input again and repeats the same process. Specifically, it generates a prompt again, inputs it into the generation AI model, retrieves the answer, formats it, and provides it to the user.

This allows users to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

Furthermore, an emotion engine for estimating the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

"Example of Morphology 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and recommends products accordingly. Specifically, when a user selects a product, the system recognizes their emotions from their facial expressions and tone of voice, and then uses those emotions to make recommendations.

It recommends products that match the user's preferences. For example, if a user is showing a joyful expression, it will recommend products that are likely to bring that user joy.

"Example of Morphology 2"

Furthermore, the emotion engine generates product descriptions based on the user's emotions. Specifically, it recognizes the user's emotions and selects words that match those emotions to generate a product description. For example, if the user is feeling down, it will select words that will cheer them up to generate a product description. (Example 3)

Furthermore, the emotion engine recognizes the user's emotions and provides services accordingly. Specifically, it recognizes the user's emotions and provides services that match those emotions. For example, if a user is showing anger, it might offer a service such as an apology.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system recommends products that match the recognized emotions.

"Example of Morphology 2"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system selects words that match the recognized emotion and generates a product description.

"Example of Morphology 3"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system provides services that match the recognized emotions.

(Example 1)

Next, we will describe Example 1 of Embodiment 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Traditional restaurant menu creation and customer experience improvement have been time-consuming and labor-intensive. In particular, generating product descriptions from photos and providing personalized recommendations consume significant human resources. Furthermore, multilingual services for foreign tourists are inadequate, highlighting the need for improved customer satisfaction. Additionally, the inability to provide product recommendations based on customer emotions can lead to a decline in the quality of the customer experience. An efficient system is needed to address these challenges.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an artificial intelligence (AI) method for generating product descriptions from smartphone photos; a method for AI to generate translation tasks; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI to explain questions during menu consideration; a method for recommending items according to preferences; and a method for recognizing user emotions and recommending products based on those emotions. This enables more efficient menu creation, product recommendations tailored to customer preferences, enhanced service for foreign tourists through multilingual support, and product recommendations based on customer emotions.

"A method for creating menus using a smartphone" refers to a method for creating restaurant menus using a smartphone's camera and applications.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"Methods for generating translation work using artificial intelligence" refers to methods of translating product descriptions and menu contents into multiple languages using artificial intelligence technology.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"A method for using artificial intelligence to explain questions during menu consideration" refers to a method in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending products based on preferences" refers to a method of recommending appropriate products based on a customer's past choices and preferences.

"A method for recognizing user emotions and recommending products based on those emotions" refers to a method of recognizing emotions by analyzing the user's facial expressions and tone of voice, and then recommending products that match those emotions.

This invention is a system that allows restaurant operators to create menus using smartphones and improve the customer experience. A specific embodiment of this system is described below.

First, the user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they would launch the smartphone's camera app, frame the dessert, and press the shutter button.

Next, the device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in an appropriate format (e.g., JPEG, PNG).

The server analyzes the received photos using image recognition software (e.g., Google Cloud Vision API). The server recognizes the characteristics of the product from the photo (e.g., chocolate cake, fruit parfait). For example, the Google Cloud Vision API might return the label "chocolate cake."

Subsequently, the server generates a product description based on the recognized characteristics. Using a generation AI model (e.g., OpenAI GPT-3), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

The generated product descriptions are displayed on the restaurant's website or application by the server. Specifically, the product descriptions are sent to the front-end in HTML or JSON format and displayed in the user interface.

Furthermore, when a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, the smartphone's camera captures the user's smile, and the microphone records the user's voice tone.

The server analyzes the captured data using an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait."

As a concrete example, consider a scenario where a user takes a photo of a new dessert with their smartphone and uploads it to the system. The server uses the Google Cloud Vision API to analyze the photo and recognize its characteristic as "chocolate cake." The AI then generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

Furthermore, if a user displays an expression of joy while viewing the menu using their smartphone camera, the Microsoft Azure Emotion API recognizes that expression. The server then recommends a "fruit parfait," which is likely to bring joy to the user.

Example of a prompt message:

"Please upload photos of the product taken with your smartphone. Our AI will automatically recognize the product's characteristics and generate a product description. It will also use your camera and microphone to recommend products based on your emotions."

In this way, a system is realized in which servers, terminals, and users work together to efficiently create restaurant menus and recommend products to customers.

The flow of the specific processing in Example 1 will be explained using Figure 17.

Step 1:

The user takes a photo of the product.

The user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they launch the smartphone's camera app, frame the dessert, and press the shutter button. The input is the photograph taken, and the output is the image data stored on the smartphone.

Step 2:

The device uploads the photos to the server.

The device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. The photo data is sent in an appropriate format (e.g., JPEG, PNG). The input is the image data on the smartphone, and the output is the image data sent to the server.

Step 3:

The server analyzes the photo and recognizes the product's characteristics.

The server analyzes the received photograph using image recognition software (e.g., an image recognition API). The server recognizes the characteristics of the product from the photograph (e.g., chocolate cake, fruit parfait). For example, the image recognition API might return the label "chocolate cake." The input is the image data sent to the server, and the output is the recognized product characteristic information.

Step 4:

The server generates the product description.

The server generates a product description based on the recognized characteristics. Using a generative AI model (e.g., a generative AI model), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture." The input is product characteristic information, and the output is the generated product description.

Step 5:

The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Specifically, it sends the product descriptions in HTML or JSON format to the front-end and displays them in the user interface. The input is the generated product description, and the output is the product description displayed on the website or application.

Step 6:

When a user selects a product, the device captures the user's emotions.

When a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, a smartphone's camera captures the user's smile, and the microphone records the user's voice tone. The input is the user's facial expressions and voice tone, and the output is the captured emotional data.

Step 7:

The server analyzes emotions and recommends appropriate products.

The server analyzes the captured data using an emotion engine (e.g., an emotion analysis API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait." The input is the captured emotion data, and the output is the recommended product information.

(Application Example 1)

Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Traditional restaurants faced significant challenges in menu creation and improving the customer experience, requiring considerable time and effort. Specifically, there were multiple obstacles, including menu creation, product description generation, personalized recommendations, and multilingual support. Furthermore, providing service that resonated with customer emotions was difficult, making improving customer satisfaction a challenge.

In Application Example 1, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on the smart device, a means for the AI to generate translations, a means for improving the customer experience, a means for the AI to explain questions during menu consideration, a means for recommending according to preferences, and a means for recognizing customer emotions and recommending products based on those emotions. This enables efficient menu creation, automatic generation of product descriptions, multilingual support, improved customer experience, and emotion-based recommendations.

A "smart device" is a portable electronic device with advanced functions, such as a smartphone or smart glasses.

"Menu creation methods" refer to functions and methods for creating restaurant menus using smart devices.

"AI methods for generating product descriptions" refers to artificial intelligence technology that automatically recognizes the characteristics and features of a product from a photograph taken with a smart device and generates a product description based on that recognition.

"A means of generating translation work using AI" refers to a function that automatically performs multilingual translation using artificial intelligence.

"Methods for improving the customer experience" refer to methods and functions that enhance the customer experience, such as eliminating the need to wait in line with waitstaff or at the cash register.

"An AI-powered solution for questions arising during menu consideration" refers to a function where artificial intelligence automatically explains questions that customers may have when considering menu options.

"A method for recommending products based on preferences" refers to a function that recommends appropriate products based on the customer's preferences.

"A means of recognizing customer emotions and recommending products based on those emotions" refers to a function that recognizes customer emotions from facial expressions, tone of voice, etc., and recommends products that match those emotions.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from photos on a smart device using AI, means for generating translations using AI, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations according to preferences, and means for recognizing customer emotions and recommending products based on those emotions.

Program Processing Description

Hardware

The server will be equipped with a high-performance GPU. Smart devices such as smartphones and smart glasses will be used as terminals.

Software

The server uses the following software:

OpenCV: Image processing library

Keras: A deep learning library

Transformers: Generative AI models (such as GPT-3)

Data processing and calculations

1. Menu creation method:

Take a photo of the food using the device's camera and upload it to the server.

The server uses OpenCV to preprocess images and Keras to recognize product characteristics and features from the images.

Based on the recognition results, product descriptions are generated using Transformers.

2. Methods for AI to generate translation work:

The server uses a generative AI model to translate the generated product descriptions into multiple languages.

In particular, we will implement multilingual support to enhance services for foreign tourists.

3. Means of improving the customer experience:

The terminal allows customers to complete ordering and payment on their smart devices, eliminating the need to wait in line with waitstaff or at the cash register.

4. How AI can explain questions that arise when considering menu options:

The terminal sends questions that arise when customers are considering the menu to the server, and the server automatically generates explanations using AI.

5. Methods for recommending based on preferences:

The server recommends appropriate products based on the customer's past order history and preferences.

6. Means for recognizing customer emotions and recommending products based on those emotions:

The server analyzes the customer's facial expressions and voice tone through the device's camera, and uses Keras to recognize their emotions.

Based on recognized emotions, Transformers are used to recommend appropriate products.

Specific examples and prompt messages

Specific example:

When a customer is wearing smart glasses and looking at a menu, the smart glasses analyze the customer's facial expression and display a message saying, "You seem happy, so we recommend dessert!"

Example prompts for a generative AI model:

"This dish looks like delicious pasta. Please generate a detailed product description."

Thus, the embodiment of this invention involves combining smart devices and AI technology to enable menu creation and improve the customer experience in restaurants.

The flow of the specific processing in Application Example 1 will be explained using Figure 18.

Step 1:

The user takes a photo of the food using a smart device (smartphone or smart glasses). The photo is saved on the device.

Step 2:

The device uploads the captured photos to the server. The server performs image preprocessing using OpenCV. Specifically, it resizes and denoises the images.

Step 3:

The server analyzes pre-processed images using Keras to recognize the characteristics and features of the product. The input is the pre-processed image, and the output is data indicating the product's characteristics and features.

Step 4:

The server generates product descriptions using Transformers based on recognized characteristics and features. The input is data describing the product's characteristics and features, and the output is the generated product description.

Step 5:

The server uses a generative AI model to translate the generated product descriptions into multiple languages. The input is the generated product description, and the output is the translated product description in multiple languages.

Step 6:

The terminal allows customers to complete their orders and payments on their smart devices, eliminating the need to wait in line with waiters or at the cash register. Input is the customer's order information, and output is a notification that the order has been completed.

Step 7:

The terminal sends questions that arise when customers consider the menu to the server. The server automatically generates explanations using AI. The input is the customer's question, and the output is the generated explanation.

Step 8:

The server recommends appropriate products based on the customer's past order history and preferences. The input is the customer's order history and preference data, and the output is the recommended products.

Step 9:

The system analyzes the customer's facial expressions and voice tone through the terminal's camera, and the server uses Keras to recognize their emotions. The input is data on the customer's facial expressions and voice tone, and the output is the recognized emotion.

Step 10:

The server uses Transformers to recommend appropriate products based on the recognized emotions. The input is the recognized emotions, and the output is the recommended products.

(Example 2)

Next, we will describe Example 2 of the 2nd embodiment. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Modern consumers demand quick and accurate understanding of product descriptions, as well as multilingual information. They also require personalized product descriptions that cater to their emotions. However, traditional systems struggle to meet these demands, making the improvement of services, particularly for foreign tourists, a significant challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for having artificial intelligence explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This enables users to quickly and accurately understand product descriptions and provides information in multiple languages. Furthermore, by providing personalized product descriptions tailored to user emotions, customer satisfaction can be improved.

A "smart device" is a portable electronic device that has internet connectivity and can run applications.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Means of improving the customer experience" refer to features that enhance the convenience and satisfaction customers experience when using a service.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function that allows artificial intelligence to provide appropriate answers to questions that arise when customers are considering menu options.

"Preference-based recommendation mechanisms" refer to features that recommend appropriate products and services based on a customer's past choices and preferences.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions from their input and actions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to a function for translating generated product descriptions into multiple languages.

Modes for carrying out the invention

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for generating translations using artificial intelligence, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

Hardware and software to be used

The server generates product descriptions using a generative AI model (e.g., OpenAI's GPT-4). The generated product descriptions are translated into multiple languages using a translation engine (e.g., Google Translate API). To recognize user emotions, an emotion engine (e.g., IBM Watson's Sentiment Analysis API) is used. Smart devices are portable electronic devices with internet connectivity capable of running applications.

Explanation of the program's processing

The server receives text entered by the user through the terminal and generates a product description using a generative AI model. For example, if the user enters "This product is high quality and long-lasting," the server will generate a product description based on this text.

The generated product descriptions are translated into multiple languages using a translation engine. For example, they are translated into major tourist languages such as English, Chinese, and Korean. Specifically, a description like "This product is high quality and long-lasting" is translated.

When a user inputs emotional text or audio data through their device, the server uses an emotion engine to recognize the user's emotions. For example, if a user inputs "I'm feeling down today," the server analyzes this text and recognizes the user's emotion as "down."

The server uses a generative AI model to generate product descriptions that respond to the user's emotions, based on the emotions it recognizes. For example, if the user is feeling down, the server will select uplifting words to generate a product description.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Examples of specific cases and prompt messages

As a concrete example, consider the following scenario.

Scenario 1: Multilingual Translation

If a user enters "This product is high quality and long-lasting" in Japanese, the server will translate this product description into English, Chinese, and Korean.

Scenario 2: Product Description Based on Emotions

If a user enters "I'm feeling down today," the server uses an emotion engine to recognize the user's emotions and generates a product description that will cheer them up.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Example of a prompt message

Examples of prompts for a generative AI model are as follows:

Please translate this Japanese product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'

"Please generate a product description that will cheer up a user who is feeling down. The original description is 'This product is high quality and long-lasting.'"

The above describes the embodiments for carrying out this invention.

The flow of the specific processing in Example 2 will be explained using Figure 19.

Step 1:

The user uses a terminal to input the text that will form the basis of the product description. For example, the user might input, "This product is high quality and long-lasting." The entered text is then sent to the server.

Step 2:

The server uses a generative AI model to generate a product description based on the text entered by the user. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description based on the user's input: 'This product is high quality and long-lasting.'" The generative AI model generates a product description based on this prompt and returns it to the server. The output is the generated product description.

Step 3:

The server sends the generated product description to the translation engine for translation into multiple languages. Specifically, the server sends the following prompt to the translation engine: "Translate this product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'" The translation engine performs the translation based on this prompt and returns it to the server. The output is the product description translated into multiple languages.

Step 4:

The user inputs text or audio data related to their emotions through their device. For example, the user might input, "I'm feeling down today." The input text or audio data is then sent to the server.

Step 5:

The server uses an emotion engine to recognize the user's emotions. Specifically, the server sends the following prompt to the emotion engine: "Analyze the user's input 'I'm feeling down today' and recognize the emotion." Based on this prompt, the emotion engine analyzes the emotion and returns the result to the server. The output is the recognized emotion.

Step 6:

The server uses a generative AI model to generate a product description that responds to the user's emotions, based on the emotions it has perceived. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description that will cheer up the user if they are feeling down. The original description is 'This product is high quality and long-lasting.'" Based on this prompt, the generative AI model generates an emotion-responsive product description and returns it to the server. The output is the emotion-responsive product description.

Step 7:

The server sends a product description to the user's device that corresponds to the generated emotion. The user can then view the uplifting product description through their device. Specifically, if the user enters "I'm feeling down today," the server will provide the user with a product description such as, "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

The above outlines the specific processing flow of this system's program.

(Application Example 2)

Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart glasses 214 will be referred to as a "terminal."

Traditional e-commerce sites often provide product descriptions in a single language, which posed a problem for foreign tourists and multilingual users, making them difficult to understand. Furthermore, the lack of emotionally resonant product descriptions made it difficult to increase user purchase intent. Additionally, users sometimes spent a significant amount of time trying to understand product descriptions, resulting in a less-than-ideal purchasing experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for artificial intelligence to explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This makes it possible to provide product descriptions tailored to user emotions in multiple languages, making them easily understandable to foreign tourists and multilingual users. Furthermore, it can increase user purchasing intent and improve the purchasing experience.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

"Menu creation methods" refer to functions or software that allow users to create menus for products and services using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Customer experience improvement measures" refer to features and software designed to enhance the customer experience when using a product or service.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate explanations for questions that arise when users are considering menu options.

"Preference-based recommendation" refers to a function that recommends appropriate products and services based on the user's preferences and past behavior.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to functions or software for translating generated product descriptions into multiple languages.

This invention will now describe embodiments for carrying out this invention. This system includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

System Configuration

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones, tablets, and smart glasses.

Server: A high-performance computer responsible for data processing and storage.

Software

Artificial intelligence model: Generates product descriptions using the OpenAI API.

Translation software: Multilingual translation is performed using the Google Translate API.

Emotion recognition software: Uses EmotionRecognizer to recognize the user's emotions.

Data processing and data calculation

1. Image Analysis: Images of products taken with a smart device are sent to the server. The server uses an image analysis algorithm to extract product characteristics and features and generate a product description.

2. Emotion Recognition: The smart device's camera and microphone are used to analyze the user's facial expressions and voice tone. EmotionRecognizer software recognizes the user's emotions and sends the results to the server.

3. Product Description Generation: The server generates prompt text based on the user's emotions and uses the OpenAI API to generate a product description appropriate to those emotions.

4. Multilingual Translation: The generated product description is translated into multiple languages using the Google Translate API. The translation results support major languages such as English, Chinese, and Korean.

Specific examples

For example, if a user is feeling down, the smart device's camera captures the user's facial expression, and the EmotionRecognizer recognizes it as "downcast." The server then generates the following prompt:

"Generate a product description that will cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time."

This prompt is sent to the OpenAI API to generate a sentiment-appropriate product description. The generated product description will look like this:

"This product was created to brighten your life, even just a little. It's made from high-quality materials and is built to last."

Next, we will translate this product description into multiple languages using the Google Translate API.

In this way, it becomes possible to provide product descriptions tailored to the user's emotions in multiple languages.

The flow of the specific processing in Application Example 2 will be explained using Figure 20.

Step 1:

The user takes a photo of the product with their smart device.

Input: Product image

Output: Captured image data

Specific operation: The user uses a smart device such as a smartphone or tablet to take a picture of the product they are considering purchasing. This image data is saved on the smart device.

Step 2:

The device sends the captured image to the server.

Input: Captured image data

Output: Image data sent to the server

Specific operation: The smart device transmits the captured image data to the server via the internet. The HTTP protocol and similar protocols are used for transmission.

Step 3:

The server performs image analysis and extracts the product's characteristics and features.

Input: Sent image data

Output: Characteristics and features of the extracted products

Specific operation: The server uses image analysis algorithms to automatically extract product characteristics and features from image data. For example, it recognizes the product's color, shape, brand logo, etc.

Step 4:

The server generates a product description based on the extracted characteristics and features.

Input: Characteristics and features of the extracted products

Output: Generated product description

Specific operation: The server uses a generative AI model to generate product descriptions based on extracted characteristics and features. For example, a description such as "This product is made from high-quality materials and will last a long time" might be generated.

Step 5:

The device recognizes the user's emotions.

Input: User's facial expressions and tone of voice

Output: Recognized user emotions

Specific operation: Using the camera and microphone of a smart device, the EmotionRecognizer software captures the user's facial expressions and voice tone, recognizing the user's emotions. For example, emotions such as "depressed" or "happy" can be recognized.

Step 6:

The server generates prompt messages that respond to the user's emotions.

Input: Recognized user sentiment, generated product description

Output: Prompt message based on emotion

Specific operation: The server generates prompt messages to send to the AI model based on the recognized user's emotions. For example, a prompt message like, "Generate a product description to cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time," might be generated.

Step 7:

The server uses an AI model to generate product descriptions that are appropriate for the user's emotions.

Input: Prompt text that reflects emotion

Output: Product description suitable for emotions

Specific operation: The server uses a generative AI model (e.g., OpenAI's API) to generate emotionally appropriate product descriptions based on the prompt text. For example, a description like, "This product was created to brighten your life a little. It is made from high-quality materials and is durable," might be generated.

Step 8:

The server translates the generated product description into multiple languages.

Input: Generated product description

Output: Product description translated into multiple languages

Specific operation: The server uses the Google Translate API to translate the generated product description into multiple languages, such as English, Chinese, and Korean. For example, a translation might be: "This product is made to brighten your life a little. It is made of high-quality materials and lasts long."

Step 9:

The server sends the product description, translated into multiple languages, to the device.

Input: Product description translated into multiple languages

Output: Translation result sent to the terminal

Specific operation: The server sends product descriptions translated into multiple languages to smart devices via the internet. HTTP protocol and similar protocols are used for transmission.

Step 10:

The device displays product descriptions translated into multiple languages to the user.

Input: Translation result sent to the device

Output: Multilingual product description displayed to the user

Specific operation: The smart device displays the product description, translated into multiple languages, to the user. This allows the user to understand the emotionally relevant product description in multiple languages.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

Traditional customer experience improvement systems often rely on manual processes for menu creation, product descriptions, and translation, resulting in inefficiency. Furthermore, they may fail to adequately address customer questions and provide personalized recommendations, potentially leading to decreased customer satisfaction. Additionally, the lack of features to recognize and respond to customer emotions hinders the improvement of the overall customer experience.

In Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and a means for recognizing user emotions and providing services corresponding to those emotions. This enables quick answers to customer questions when considering menus, product recommendations tailored to preferences, and significantly improves customer satisfaction by providing services that respond to customer emotions.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

"Methods for generating translation work using artificial intelligence" refers to functions that automatically translate text and audio using artificial intelligence.

"Methods for improving customer experience by reducing waiting times" refer to functions that shorten the waiting time customers experience when using a service, thereby providing a smoother experience.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are considering menu options.

"A method for recommending products based on preferences" refers to a function where artificial intelligence recommends appropriate products based on the customer's preferences and past choices.

"A means of recognizing user emotions and providing services that respond to those emotions" refers to a function that recognizes user emotions from their facial expressions and voice, and provides appropriate services that correspond to those emotions.

This invention is a system for improving customer experience and includes means for creating menus using smart devices, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for recommending products according to preferences, and means for recognizing user emotions and providing services that respond to those emotions.

Hardware and software to be used

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones and tablets.

Server: A high-performance server (e.g., a server with an NVIDIA GPU).

Software

Generative AI model: GPT-4 is used as an example.

Emotion recognition engine: Affectiva SDK is used as an example.

Program Processing Description

The server receives information sent from the smart device and performs analysis using a generative AI model. For example, if the user inputs "I like spicy food," the generative AI model generates a list of spicy dishes, and the server sends this list to the smart device. The smart device then displays the received list to the user.

Furthermore, when a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to generate ingredient information and sends it to the smart device. The smart device then displays the received ingredient information to the user.

Furthermore, smart devices use their built-in cameras and microphones to capture the user's facial expressions and voice, and use an emotion recognition engine to recognize the user's emotions. For example, if the user shows an angry expression, the smart device sends that information to a server. The server uses a generative AI model to generate an apology and sends it to the smart device. The smart device then displays the received apology to the user.

Examples of specific cases and prompt messages

Example 1: User enters "I like spicy food."

The user opens the app on their smartphone and enters "I like spicy food."

The server generates a list of spicy dishes using an AI model.

The server generates a list and sends it to the smart device.

A smart device displays a list of spicy dishes to the user.

Example prompt: "Please list dishes you would recommend to a customer who likes spicy food."

Example 2: A user asks, "What are the ingredients in this dish?"

The user opens the tablet app and asks, "What are the ingredients in this dish?"

The server generates ingredient information using an AI model.

The server generates ingredient information and sends it to the smart device.

The smart device displays ingredient information to the user.

Example prompt: "Please tell me the ingredients for this dish."

Example 3: The user shows an angry expression.

The user displays an angry expression while using their smartphone.

Smart devices use cameras to capture the user's facial expressions, which are then analyzed by an emotion recognition engine.

The smart device sends information about the anger it recognizes to the server.

The server generates an apology using an AI model.

The server generates an apology message and sends it to the smart device.

Smart devices display an apology message to the user.

Example prompt: "Generate an apology for when the user is angry."

In this way, the server, smart devices, and users work together to operate the system and improve the customer experience. The flow of a specific process in Example 3 will be explained using Figure 21.

Step 1:

The user inputs information through their device. The user uses a smartphone or tablet to input questions and preferences into the system. For example, they might input "I like spicy food." The input data is saved on the device in text format.

Step 2:

The terminal sends the input information to the server. The terminal sends the user-entered information to the server via the internet. The input data is sent to the server in text format.

Step 3:

The server analyzes the input information using a generative AI model. The server passes the received input information to the generative AI model (e.g., GPT-4) for analysis. The generative AI model generates appropriate responses or recommendations based on the input information. For example, given the information "likes spicy food," it generates a list of spicy dishes. The input data is in text format, and the output data is in list format.

Step 4:

The server generates appropriate answers and recommendations based on the analysis results. The server generates answers and recommendations for the user based on the analysis results obtained from the generating AI model. For example, it might generate a list of spicy dishes. The output data is in list format.

Step 5:

The server sends the generated responses and recommendations to the device. The server sends the generated responses and recommendations to the device via the internet. The output data is sent to the device in list format.

Step 6:

The device displays answers and recommendations to the user. The device displays answers and recommendations received from the server to the user. The user can review the displayed information. The output data is displayed in list format.

Step 7:

The device recognizes the user's emotions using an emotion engine. The device captures the user's facial expressions and voice using its built-in camera and microphone, and recognizes the user's emotions using an emotion engine (e.g., Affectiva SDK). Input data is in image or audio format, and output data is in emotion information format.

Step 8:

The device transmits recognized emotion information to the server. The device transmits the recognized emotion information to the server via the internet. Input data is sent to the server in emotion information format, and output data is also sent in emotion information format.

Step 9:

The server generates appropriate services based on emotional information. The server uses a generative AI model to generate appropriate services based on the received emotional information. For example, if the user shows an angry expression, it will generate an apology. Input data is in emotional information format, and output data is in text format.

Step 10:

The server sends the generated service to the terminal. The server sends the generated service to the terminal via the internet. The output data is sent to the terminal in text format.

Step 11:

The terminal provides services to the user. The terminal delivers services received from the server to the user. For example, it displays an apology. The output data is displayed in text format.

(Application Example 3)

Next, we will describe Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server," and the smart glasses 214 will be referred to as a "terminal."

Traditional food delivery services faced challenges such as users spending a significant amount of time choosing from menus and difficulty in providing services tailored to user preferences and emotions. Furthermore, multilingual support for foreign tourists was insufficient, highlighting the need for improved user experience. There is a demand to address these challenges and provide a more comfortable and personalized service.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing the user's emotions and providing services corresponding to those emotions. This allows users to efficiently select menu items and receive personalized services tailored to their individual preferences and emotions. Furthermore, multilingual support enhances services for foreign tourists.

A "smart device" is a portable electronic device with advanced computing capabilities, such as a smartphone or tablet.

A "menu creation method" refers to a method or device for generating a list of products or services that a user can select.

"An artificial intelligence system for generating product descriptions from images" refers to an artificial intelligence system that uses image recognition technology to automatically analyze the characteristics and features of a product and generates a product description based on that analysis.

"Means of generating translation work using artificial intelligence" refers to an artificial intelligence system that automatically translates text and audio between different languages.

"Customer experience improvement measures" refer to methods and devices that enhance the convenience and satisfaction users experience when using a service.

"A system where artificial intelligence explains questions that arise when considering menu options" refers to a system where artificial intelligence provides appropriate answers to questions that users may have when choosing from a menu.

A "method for recommending products based on preferences" is a system that recommends appropriate products and services based on the user's past choices and input information.

An "emotion recognition system" is a system that analyzes a user's emotions from their facial expressions, tone of voice, etc., and responds accordingly.

The system for carrying out this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing user emotions and providing services corresponding to those emotions.

System Program

The program for this system performs the following operations:

Hardware and software

Hardware:

Smart devices (smartphones, tablets, etc.)

Camera (to capture the user's facial expressions)

Computer (a processing unit for executing programs)

Software:

OpenCV (image processing library)

Keras (deep learning library)

Transformers (a library of generative AI models)

Data processing and data calculation

1. Menu creation method:

The server generates a list of products and services that users can select using their smart devices.

2. Artificial intelligence methods for generating product descriptions from images:

The server analyzes images captured by the smart device's camera and automatically recognizes the product's characteristics and features.

Based on the recognized information, generate a product description.

3. Methods for generating translation tasks using artificial intelligence:

The server automatically translates text and audio between different languages.

In particular, we will provide multilingual support to enhance services for foreign tourists.

4. Measures to improve customer experience:

The server reduces waiting times for users when accessing the service, improving convenience.

5. How artificial intelligence can explain questions that arise during menu planning:

The server uses a generative AI model to provide appropriate answers to questions that arise when users select menu items.

6. Methods for recommending products based on preferences:

The server recommends appropriate products and services based on the user's past choices and input information.

7. Emotion recognition means:

The server analyzes the user's facial expressions and voice tone captured by the camera to recognize their emotions.

Respond appropriately to the recognized emotions.

Specific examples

When a user enters "I like spicy food," the server uses a generative AI model to recommend spicy dishes.

When a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to provide information about the ingredients.

If the user displays an angry expression, the server uses emotion recognition to apologize with the message, "I'm sorry. Is there a problem?"

Example of a prompt message

"I like spicy food. What do you recommend?"

"What are the ingredients in this dish?"

"How would you respond if a user is showing signs of anger?"

In this way, users can efficiently select menu items and receive personalized service tailored to their individual preferences and feelings. Furthermore, multilingual support enhances services for foreign tourists.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The user opens the menu using a smart device.

Input: A menu display request initiated by the user.

Data Processing: The server retrieves menu information from the database and sends it to the smart device.

Output: A menu will appear on your smart device.

Step 2:

The user takes a photo of the food using the camera on their smart device.

Input: An image of a dish taken by the user.

Data Processing: The server receives images and analyzes them using an image recognition algorithm (OpenCV).

Output: The analysis results extract the characteristics and features of the dishes.

Step 3:

The server generates a product description from the image.

Input: Data on the characteristics and features of the dish.

Data Processing: The server uses a generative AI model to generate product descriptions based on characteristics and features.

Output: Product description text is generated and sent to the smart device.

Step 4:

Users enter questions when considering menu options.

Input: The question entered by the user (e.g., "What are the ingredients in this dish?").

Data Processing: The server generates answers to questions using a generative AI model.

Output: The answer text is generated and displayed on the smart device.

Step 5:

The server recommends products based on the user's preferences.

Input: User's past selections and input information (e.g., "I like spicy food").

Data Processing: The server uses a generative AI model to recommend products based on user preferences.

Output: A list of recommended products is displayed on your smart device.

Step 6:

The user captures their facial expressions using the camera on their smart device.

Input: User's facial expression image.

Data Processing: The server analyzes emotions using a facial recognition algorithm (Keras).

Output: As an analysis result, user sentiment data is generated.

Step 7:

The server provides services tailored to the user's emotions.

Input: User emotion data (e.g., anger).

Data Processing: The server generates appropriate responses based on sentiment data (e.g., apology messages).

Output: The corresponding message will be displayed on the smart device.

Step 8:

The server performs the translation.

Input: Text or audio data entered by the user.

Data Processing: The server uses a translation algorithm to translate the input data into multiple languages.

Output: Translated text and audio data are displayed on the smart device.

The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input regarding the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet Search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives prompts containing instructions, as well as inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions provided by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search engine <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which specific processing is performed by the data processing device 12. However, the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[Third Embodiment]

Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

As shown in Figure 5, the data processing system 310 comprises a data processing device 12 and a headset-type terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 comprises a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to the bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. It captures images of the area surrounding the user 20 (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

The communication interface 44 is connected to the network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via the network 54. The exchange of various types of information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the processor 46 performs the reception and output processing. The storage 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the storage 50 and executes the read program 60 on the RAM 48. The reception and output processing is realized by the processor 46 acting as a control unit 46A according to the reception and output program 60 executed on the RAM 48.

Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be explained.

"Example of Morphology 1"

In one embodiment of this invention, a restaurant operator creates a menu using a smartphone. Specifically, the operator uploads photos of products taken with the smartphone's camera, and AI automatically recognizes the characteristics and features of the products from the photos. Based on this recognition, a product description is generated. This product description is then displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Furthermore, in this embodiment of the present invention, the AI generates the translation work. Specifically, the generated product description is translated into multiple languages. This makes the product description easier for foreign tourists to understand. For example, it is possible to translate into major tourist languages such as English, Chinese, and Korean.

"Example of Morphology 3"

Furthermore, in this embodiment of the present invention, the customer experience is also improved. Specifically, when a customer is considering a menu, the AI explains their questions and recommends products according to their preferences. For example, if a customer inputs information such as "I like spicy food," the AI will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI will provide an answer to that question.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: The restaurant operator takes photos of the products using their smartphone camera.

Step 2: Upload the photos you took to the system.

Step 3: The system's AI automatically recognizes the product's characteristics and features from the photograph.

Step 4: The AI generates a product description based on the recognition results.

Step 5: The generated product description is displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Step 1: Obtain the product description generated by the AI.

Step 2: The AI translates the product description into multiple languages.

Step 3: The translated product description is displayed to foreign tourists on the restaurant's website or application.

"Example of Morphology 3"

Step 1: When customers are considering the menu, they input their questions into the AI.

Step 2: The AI generates the answer to that question.

Step 3: The AI recommends products based on the customer's preferences.

Step 4: The AI-generated answers and recommendations are displayed to the customer.

(Example 1)

Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

When restaurant operators create menus, the process of taking photos of products and then recognizing their characteristics and features to generate product descriptions is time-consuming. Furthermore, there is a lack of multilingual support for foreign tourists and other means to enhance the customer experience. Therefore, there is a need for increased efficiency in menu creation and improved customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a means for creating menus using a smartphone; an artificial intelligence (AI) means for generating product descriptions from smartphone photos; a means for AI to generate translations; a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a means for AI to explain questions during menu consideration; a means for recommending items according to preferences; a means for uploading product photos taken with a smartphone camera, for AI to automatically recognize the characteristics and features of the products from those photos; and a means for generating product descriptions based on the recognition results. This enables increased efficiency in menu creation and improved customer satisfaction.

A "smartphone" is a multi-functional mobile device that, in addition to the functions of a regular mobile phone, also offers internet connectivity and the ability to use applications.

"Menu creation methods" refer to the methods and tools that restaurant operators use to create menus, particularly those using smartphones.

"Artificial intelligence tools" refer to technologies that use machine learning and data analysis to automatically perform specific tasks.

"Means of generating translation work using artificial intelligence" refers to methods and technologies that use artificial intelligence to translate text into multiple languages.

"Methods for improving the customer experience" refer to methods and tools for improving the convenience and satisfaction customers experience when using a service.

"Methods for using artificial intelligence to explain questions during menu consideration" refers to methods and technologies in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"Methods of recommending based on preferences" refer to methods and technologies that recommend appropriate products and services based on customer preferences and past choices.

"Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

"Methods for generating product descriptions" refers to methods and technologies for creating product descriptions in natural language based on recognized product characteristics and features.

This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, users upload photos of their products taken with their smartphone cameras, and artificial intelligence (AI) automatically recognizes the characteristics and features of the products from these photos. Based on this recognition, the system generates product descriptions. These product descriptions are then displayed to customers on the restaurant's website or application.

Hardware and software to be used

Smartphone: A device used for taking and uploading photos.

Server: Performs data processing and runs AI models.

Artificial intelligence models: Image recognition models and natural language generation models based on TensorFlow and PyTorch (e.g., GPT-3, BERT).

Data processing and data calculation

1. The user takes a photo of the product with their smartphone.

The user launches their smartphone's camera app and takes a picture of the product. For example, they might take a picture of a new dessert, "Chocolate Cake."

2. The user uploads photos to the system.

The user opens a dedicated application, selects the photos they've taken, and presses the upload button. The photos are then sent to the server via the internet.

3. The server receives the photos and inputs them into the AI model.

The server receives the uploaded photos and inputs them into an AI model for image processing. The AI model used here is based on TensorFlow or PyTorch.

4. The server uses an AI model to recognize the characteristics and features of the product.

The server uses an AI model to recognize the characteristics and features of a product from a photograph. For example, it extracts information such as the type of dessert, main ingredients, and visual characteristics.

5. The server generates a product description based on the recognition results.

The server generates product descriptions based on the AI's recognition results. These product descriptions are created using natural language generation technology. The software used includes GPT-3 and BERT.

6. The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Customers can then view them.

Specific examples

A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description, such as "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing," and displays it on the website.

Example of a prompt message

Please upload a photo of your new dessert. AI will recognize the product's characteristics and features from the photo and generate a product description.

The flow of the specific processing in Example 1 will be explained using Figure 11.

Step 1:

The user takes a photo of the product with their smartphone.

Input: Product (e.g., Chocolate cake)

Output: Photos of the photographed product

Specific action: The user launches the camera app on their smartphone and takes a picture of the product. For example, they might take a picture of the new dessert, "Chocolate Cake."

Step 2:

The user uploads a photo to the system.

Input: Photographs of the product

Output: Photo data sent to the server

Specific operation: The user opens a dedicated application, selects the photo they have taken, and presses the upload button. The photo is sent to the server via the internet.

Step 3:

The server receives the photos and inputs them into the AI model.

Input: Photo data sent to the server

Output: Photo data input to the AI model

Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

Step 4:

The server uses an AI model to recognize the characteristics and features of the product.

Input: Photo data entered into the AI model

Output: Characteristics and features of the recognized product (e.g., chocolate cake, cream topping, berry decoration)

Specific operation: The server runs an AI model to extract product characteristics and features from a photograph. For example, it recognizes the type of dessert, main ingredients, and visual features.

Step 5:

The server generates a product description based on the recognition results.

Input: Characteristics and features of the recognized product

Output: Generated product description (Example: "This chocolate cake features rich chocolate and creamy toppings. The berry decorations make it visually appealing.")

Specific operation: The server uses GPT-3 or BERT to generate product descriptions based on the recognition results.

Step 6:

The server displays the product description on the website or application.

Input: Generated product description

Output: Product description displayed on the website or application

Specific operation: The server saves the product description to the website's database, sends the data to the front-end for display, and the customer can view it.

(Application Example 1)

Next, we will describe Application Example 1 of Configuration Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

Traditional restaurant menu creation was often done manually, which was time-consuming and labor-intensive, and made it difficult to adequately convey the appeal of the products. Furthermore, the lack of multilingual support for foreign tourists and insufficient recommendation features tailored to customer preferences were also problems. In addition, there were limited means to improve the in-store customer experience, such as having to wait for waitstaff or in line at the register, highlighting the need for increased customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an AI method for generating product descriptions from smartphone photos; a method for AI-generated translations; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI-generated explanations of questions during menu consideration; a method for making recommendations based on preferences; a method for recognizing product characteristics and features using an image recognition model; and a method for generating product descriptions based on recognized features using a natural language generation model. This enables efficient menu creation and automatic generation of attractive product descriptions, facilitating multilingual support, improved customer experience, and personalized recommendations.

"A method for creating menus using a smartphone" refers to a method for efficiently creating restaurant menus using a smartphone's camera and applications.

"AI methods for generating product descriptions from smartphone photos" refers to artificial intelligence methods that analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"A method for generating translations using AI" refers to a method of translating product descriptions and menu contents into multiple languages using artificial intelligence.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"An AI-powered solution for questions during menu consideration" refers to a method where artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending based on preferences" refers to a method of recommending appropriate products or menu items based on a customer's past choices and preferences.

"A means of recognizing the characteristics and features of a product using an image recognition model" refers to a method of automatically recognizing the characteristics and features of a product from a photograph using image recognition technology.

"A means of generating product descriptions based on features recognized using a natural language generation model" refers to a method of automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

The system for implementing this invention includes a means for creating menus using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for AI to generate translations, a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register, a means for AI to explain questions when considering menus, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on recognized features using a natural language generation model.

System Program

The server implements the system using the following hardware and software.

Hardware:

Smartphone (with camera)

Server (for hosting AI models)

Software:

TensorFlow: A library for image recognition

OpenAI GPT-3: API for natural language generation

Explanation of the process

Image recognition:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of the food from the image. For example, features such as "chicken curry, spicy, tomato-based" might be recognized.

Natural language generation:

The server converts the recognized features into a string and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is: "Generate an appealing product description for a dish with the following features: Chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it might generate a description such as: "This spicy chicken curry features juicy chicken simmered in a tomato-based sauce. Aromatic spices will whet your appetite."

Translation work:

The generated product descriptions are translated into multiple languages as needed. The server uses AI to perform translations into multiple languages, such as English and Chinese.

Improving the customer experience:

Customers can use their smartphones to select menu items and place orders in-store. This reduces waiting times at the counter and cashier, enabling smoother service.

Explanation of the question and recommendations:

The AI automatically provides answers to questions customers may have when considering menu options. It also recommends appropriate products and menu items based on the customer's past choices and preferences.

In this way, menu creation becomes more efficient, attractive product descriptions are automatically generated, and multilingual support, improved customer experience, and personalized recommendations become possible.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The input is the photo of the food, and the output is the image data sent to the server.

Step 2:

The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of dishes from uploaded image data. The input is image data, and the output is a list of the recognized characteristics and features. Specifically, the image data is preprocessed, input into the model, and prediction results are obtained.

Step 3:

The server converts the recognized characteristics and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of characteristics and features, and the output is a prompt statement. Specifically, it formats the characteristics and features and generates the prompt statement.

Step 4:

The server receives a response from GPT-3 and generates an attractive product description. The input is a prompt, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and retrieves the product description.

Step 5:

The server translates the generated product description into multiple languages as needed. The input is the product description, and the output is the translated product description. Specifically, it calls a translation API to translate into multiple languages.

Step 6:

Users use their smartphones to select menu items and place orders in the store. Input is generated product descriptions, and output is the user's order information. Specifically, the application displays product descriptions, and the user selects their order.

Step 7:

The server automatically provides answers to questions that arise when users consider menu options, using AI. The input is the user's question, and the output is the AI's answer. Specifically, it analyzes the question and generates an appropriate answer.

Step 8:

The server recommends appropriate products and menu items based on the user's past choices and preferences. The input is the user's past selection data, and the output is the recommended products and menu items. Specifically, it analyzes past selection data and uses a recommendation algorithm to suggest products.

(Example 2)

Next, we will describe Embodiment 2 of Embodiment Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

Traditional systems often required manual translation of product descriptions into multiple languages, resulting in significant time and effort. Furthermore, services for foreign tourists were inadequate, leading to difficulties in understanding product descriptions. Additionally, insufficient efforts to improve the customer experience and address questions during menu selection could potentially lower customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for creating menus using a smart device, means for artificial intelligence (AI) to generate product descriptions from images on the smart device, means for AI to generate translation tasks, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations based on preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into the generation AI model and obtaining translation results, means for saving translation results to a database, means for sending translation results to a terminal, and means for the terminal to display the translation results. This automates the multilingual translation of product descriptions, improves services for foreign tourists, enhances the customer experience, and resolves questions during menu consideration.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that possesses advanced computing power and communication capabilities.

"Menu creation methods" refer to functions and applications used to create menus for restaurants and retail stores using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

"Means of improving the customer experience" refer to features and methods that enhance the convenience and satisfaction customers experience when using products or services.

"A means for AI to explain questions during menu consideration" refers to a function where artificial intelligence automatically provides answers to questions and concerns that arise when customers are considering menu options.

"Preference-based recommendations" refers to a function that recommends appropriate products and services based on a customer's past choices and preferences.

"Methods for translating product descriptions into multiple languages" refers to functions for translating product descriptions into several languages.

A "generative AI model" is an artificial intelligence model trained to perform tasks such as text generation and translation.

A "prompt statement" is an instruction given to a generative AI model to perform a specific task.

"Means for saving translation results to a database" refers to the function of saving translation results generated by a generative AI model to a database.

"Means for sending translation results to the terminal" refers to the function that sends translation results stored in the database to the user's terminal.

"Means for the terminal to display translation results" refers to the function that displays the received translation results on the user's terminal screen.

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translation tasks, means for improving the customer experience, means for artificial intelligence to explain questions when considering menus, means for recommending according to preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into a generation AI model and obtaining translation results, means for saving translation results in a database, means for sending translation results to a terminal, and means for the terminal to display translation results.

Hardware and software to be used

Hardware:

Smart devices (smartphones, tablets, etc.)

Server (a computer with high-performance computing capabilities)

Software:

Generative AI models (e.g., OpenAI's GPT-4)

Database Management System

Communication protocol (e.g., HTTP/HTTPS)

Data processing and data calculation

Product description generation:

The user takes a picture of the product using a smart device and sends the image to the server. The server uses artificial intelligence to generate a product description from the image. Specifically, it uses an image analysis algorithm to recognize the product's characteristics and features, and then generates a product description based on that.

Prompt message generation:

The server generates prompt text using a generative AI model. The prompt text specifies the target language for translation (e.g., English, Chinese, Korean).

Execution of translation work:

The server inputs the generated prompt text and product description into the AI model and retrieves the translation result. The AI model then performs multilingual translation based on the input text.

Saving and sending translation results:

The server saves the acquired translation results to a database. The saved translation results are sent to the user's device, which then displays the translation.

Specific examples

Specific example:

The user enters the Japanese product description: "This product uses high-quality materials."

The server generates the prompt message: "Translate the following product description into English: This product uses high-quality materials."

The server inputs the prompt text and product description into the generated AI model and retrieves the English translation result: "This product uses high-quality materials."

The server saves the translation results to a database and sends them to the terminal.

The device displays the translation results, and the user confirms them.

This system enables multilingual translation of product descriptions through collaboration between the server, terminal, and user.

The flow of the specific processing in Example 2 will be explained using Figure 13.

Step 1:

The user enters the product description.

The user enters the product description in text format using a smart device. The entered product description is then sent from the smart device to the server.

Input: Product description text

Output: Product description text sent to the server

Step 2:

The server receives the product description.

The server receives the product description text sent by the user and temporarily stores it in memory.

Input: Product description text submitted by the user

Output: Product description text saved in memory

Step 3:

The server generates the prompt message.

The server generates prompt messages based on the specified languages for translation. For example, if translation is required for English, Chinese, and Korean, it will generate prompt messages corresponding to each language.

Input: Product description text, target language for translation

Output: Generated prompt message

Step 4:

The server inputs prompt text and product description into the generated AI model.

The server inputs the generated prompt text and product description text into the AI model. The AI model then performs translation based on the input text.

Input: Prompt message, product description text

Output: Data input to the generative AI model

Step 5:

The server retrieves the translation result.

The server retrieves translation results from the generative AI model. The retrieved translation results are separated by language.

Input: Data entered into the generative AI model

Output: Translated text

Step 6:

The server saves the translation results to the database.

The server saves the retrieved translation results to a database. When saving, it associates the original Japanese product description with the translated result.

Input: Translated text, original product description text

Output: Translation results stored in the database

Step 7:

The server sends the translation result to the terminal.

The server sends the saved translation results to the user's device. The transmitted data is provided in a format accessible to the user.

Input: Translation results stored in the database

Output: Translation result sent to the terminal

Step 8:

The device displays the translation result.

The terminal receives the translation results sent from the server and displays them to the user. The user can review the displayed translation results and make corrections or re-translates as needed.

Input: Translation result sent from the server

Output: Translation result displayed on the terminal

(Application Example 2)

Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing goods, making it difficult for them to understand product descriptions. Furthermore, visually impaired individuals and those with reading and writing difficulties also have limited means of understanding product descriptions. This can lead to a diminished customer experience and potentially impact sales.

In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, and a means for translating product descriptions into multiple languages and displaying and playing them aloud in the user's set language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired individuals, and people with reading and writing difficulties.

"A method for creating menus using a smartphone" refers to a method for creating menus using a smartphone.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

"Methods for generating translation work using artificial intelligence" refers to methods of translating text into multiple languages using artificial intelligence.

"Methods for improving the customer experience" refer to measures taken to improve the customer's experience when purchasing a product.

"A method for using artificial intelligence to explain questions that arise when customers are considering menu options" refers to a method in which artificial intelligence answers questions that customers may have when considering menu options.

"Methods of recommending based on preferences" refer to methods of recommending products and services based on customer preferences.

"A means of translating product descriptions into multiple languages and displaying and playing them in the user's preferred language" refers to a means of translating product descriptions into multiple languages and displaying and playing them in the language specified by the user.

To implement this invention, it is necessary to construct a system that combines a smartphone, artificial intelligence technology, a translation API, and a voice playback function.

First, smartphones have a camera function that allows users to photograph products. When a user photographs a product, the image data is sent to a server. The server then generates a product description using image recognition technology. This image recognition technology might include, for example, the Google Cloud Vision API.

Next, the generated product description is translated into multiple languages using artificial intelligence on the server. Translation services such as the Google Cloud Translation API are used for this translation. The translated text is then sent to the user's smartphone and displayed based on their language settings.

Furthermore, the translated product descriptions are played back using the smartphone's audio playback function. This makes the product accessible to visually impaired individuals and those with reading and writing difficulties.

As a concrete example, consider a tourist trying to purchase a leather wallet at a physical store in Japan. When the tourist scans the wallet's tag with their smartphone camera, the app automatically recognizes the product description and translates it into the user's chosen language (e.g., English). The translated product description would be displayed as "This is a high-quality leather wallet," and also played aloud.

Examples of prompt messages include the following:

I am developing an application to translate product descriptions into multiple languages. Please translate the following text into the specified languages.

Text: "This is a high-quality leather wallet."

Target language: Japanese

In this way, foreign tourists, visually impaired individuals, and people with reading and writing difficulties will be able to understand product descriptions without encountering language barriers when purchasing items in physical stores.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user takes a picture of the product using their smartphone camera. The input is the product image captured by the smartphone camera. The output is the captured product image data.

Step 2:

The device sends the captured product image data to the server. The input is the product image data. The output is the product image data sent to the server.

Step 3:

The server generates product descriptions using image recognition technology. The input is product image data. The server extracts text information from the images using APIs such as Google Cloud Vision API and generates the product description. The output is the generated product description text.

Step 4:

The server translates the generated product description text into multiple languages. The input is the product description text. The server uses the Google Cloud Translation API to translate it into the specified languages. The output is the translated product description text.

Step 5:

The server sends the translated product description text to the terminal. The input is the translated product description text. The output is the translated product description text sent to the terminal.

Step 6:

The device displays the translated product description text. The input is the translated product description text. The output is the translated product description displayed on the smartphone screen.

Step 7:

The device plays the translated product description text as audio. The input is the translated product description text. The device uses the smartphone's audio playback function to convert the text to audio and play it. The output is the translated product description played as audio.

(Example 3)

Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

Traditional menu creation and customer experience improvement systems fail to adequately answer users' questions when considering menu items and to provide product recommendations tailored to individual preferences. Furthermore, they are insufficient in reducing waiting times and providing multilingual services. This can lead to decreased customer satisfaction and negatively impact store sales.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for the user to input information, means for transmitting the input information to the server, means for the server to analyze the input information using a generated AI model, means for generating recommendations and answers based on the analysis results, means for transmitting the generated recommendations and answers to a terminal, and means for the terminal to display the recommendations and answers to the user. This enables appropriate answers to questions the user may have when considering a menu, and product recommendations tailored to individual preferences. Furthermore, it can reduce waiting times and provide multilingual services, leading to improved customer satisfaction and increased store sales.

A "smart device" is an electronic device used by users to input information and communicate with a server, and includes smartphones and tablets.

"Artificial intelligence tools" refer to algorithms and models that analyze input data and generate appropriate responses or recommendations.

A "generative AI model" refers to a machine learning model that analyzes user input information and generates appropriate responses and recommendations.

"Methods for improving the customer experience" refer to measures that reduce waiting times and allow customers to use services comfortably.

"A means for artificial intelligence to explain questions that arise when considering menu options" refers to a method by which artificial intelligence provides appropriate answers to questions that users may have when considering menu options.

"Methods for recommending products based on preferences" refers to methods for recommending appropriate products based on the user's preferences.

"Means for user information input" refers to the interface through which users input information in formats such as text or voice.

"Means for sending inputted information to the server" refers to the communication means used to send information entered by the user to the server.

"Means of analyzing input information using a server-generated AI model" refers to the means by which a server uses a generated AI model to analyze information received from a user.

"Means for generating recommendations and responses based on analysis results" refers to methods for generating recommendations and responses for users based on the analysis results of a generation AI model.

"Means for sending generated recommendations and responses to the device" refers to the communication methods used by the server to send generated recommendations and responses to the user's device.

"Means by which a device displays recommendations and answers to a user" refers to the means by which a user's device displays recommendations and answers received from a server to the user.

This invention is a system in which a user inputs information using a smart device, a server analyzes that information using a generated AI model, and then generates and provides appropriate recommendations and answers to the user. Specific embodiments of this system are described below.

First, the user inputs information into the system using a smart device such as a smartphone or tablet. For example, the user might input "I like spicy food." This information is entered through the smart device's interface.

Next, the smart device sends the entered information to the server. In this process, the smart device converts the input data into an appropriate format (e.g., JSON format) and sends it to the server via the network.

The server analyzes the received user input using a generation AI model (for example, OpenAI's GPT-4). Specifically, it analyzes information such as "I like spicy food" and searches a database related to spicy food.

The server recommends products tailored to the user's preferences based on the analysis results. For example, it generates a list of spicy dishes and presents it to the user. Furthermore, if the user asks, "What are the ingredients in this dish?", the server generates an answer to that question.

The generated recommendations and responses are sent from the server to the smart device. The server converts the data into the appropriate format and sends it to the smart device over the network.

Smart devices display recommendations and answers received from the server to the user. For example, they might display, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot."

As a concrete example, the following prompt message is shown.

"I like spicy food. Could you recommend some dishes?"

"What are the ingredients in this dish?"

In this way, a system is realized in which the server, smart device, and user work together to improve the customer experience. This system allows users to receive appropriate answers to questions when considering menus and product recommendations tailored to their individual preferences. Furthermore, it enables reduced waiting times and the provision of multilingual services, leading to improved customer satisfaction and increased store sales. The flow of specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters information using a terminal.

The user opens the application on their smartphone or tablet, enters "I like spicy food" into the text box, and presses the submit button. The input data is in text format and represents information about the user's preferences.

Step 2:

The terminal sends the entered information to the server.

The terminal converts the user's input, "I like spicy food," into JSON format and sends it to the server using the HTTPS protocol. The input data is user preference information in text format, and the output data is request data in JSON format.

Step 3:

The server analyzes the input information using a generated AI model.

The server parses the received JSON data and inputs the information "likes spicy food" into a generative AI model. The generative AI model (e.g., GPT-4) searches a database of spicy dishes and generates a list of related dishes. The input data is a request in JSON format, and the output data is the list of dishes resulting from the analysis.

Step 4:

The server generates recommendations and responses based on the analysis results.

The server generates a list of spicy dishes based on the analysis results obtained from the generated AI model. For example, it might list dish names such as "Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is the list of dishes resulting from the analysis, while the output data is a recommendation list presented to the user.

Step 5:

The server sends recommendations and responses generated by the server to the device.

The server converts the generated list of spicy dishes into JSON format and sends it to the terminal using the HTTPS protocol. The input data is a recommendation list, and the output data is a JSON response.

Step 6:

The device displays recommendations and answers to the user.

The terminal parses the received JSON data and displays to the user, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is response data in JSON format, and the output data is text information displayed to the user.

(Application Example 3)

Next, we will describe Application Example 3 of Configuration Example 3. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

Traditional food delivery systems have problems such as difficulty for customers to obtain detailed information when choosing a menu and a lack of personalized recommendations. Furthermore, the lack of a means to provide real-time information on ingredients and nutritional content has been a factor in lowering customer satisfaction.

In Application Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting time, a means for artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, a means for recommending dishes based on customer preferences, and a means for providing information on the ingredients of the dishes. As a result, customers can obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

A "menu creation method" is a function for generating a list of dishes and drinks that customers can choose from.

"An artificial intelligence method for generating product descriptions from images" refers to a function that analyzes images taken with a smart device and automatically generates product descriptions based on their content.

"A means of generating translation work using artificial intelligence" refers to a function that automatically translates text between different languages.

"Methods for improving the customer experience by reducing waiting times" refer to features that reduce the waiting time customers experience when using a service.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are choosing from a menu.

"A method of recommending based on preferences" refers to a function that recommends appropriate products and services based on the customer's preferences and past choices.

"A method for recommending dishes based on customer preferences" refers to a function that recommends appropriate dishes based on the preference information entered by the customer.

"Means of providing information on the ingredients of a dish" refers to a function that provides information about the ingredients and components contained in a specific dish.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from images on a smart device, means for generating translations using artificial intelligence, means for improving the customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for making recommendations according to preferences, means for recommending dishes based on customer preferences, and means for providing information on the ingredients of dishes.

System Program

This system's program is implemented using Python and the OpenAI API. The server receives customer input information and uses a generative AI model to provide appropriate recommendations and information.

Explanation of the process

The server receives customer preferences and questions sent from smart devices and generates prompts based on this information. These generated prompts are input into a generation AI model via the OpenAI API, which then generates appropriate answers and recommendations. The generated information is then sent back to the smart device and provided to the customer.

The hardware used will be smart devices such as smartphones and tablets. The software used will be Python and the OpenAI API.

Specific examples

For example, if a customer enters "I like spicy food," the server will generate the following prompt:

"Please recommend some dishes for people who like spicy food."

Entering this prompt into the OpenAI API will cause the generating AI model to recommend spicy dishes such as "Mapo Tofu" and "Kimchi Hot Pot."

Furthermore, when a customer asks, "What are the ingredients in Mapo Tofu?", the server generates the following prompt:

"What are the ingredients in Mapo Tofu?"

When this prompt is entered into the OpenAI API, the generating AI model will provide ingredient information such as "tofu, ground pork, green onions, garlic, ginger, chili bean paste, sweet bean paste, soy sauce, sake, sugar, and chicken broth."

This allows customers to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the application using a smart device and enters their preferences and answers questions. The entered information is sent to the server in text format. Examples of input include "I like spicy food" and "What are the ingredients in Mapo Tofu?".

Step 2:

The server analyzes the input information received from the user and generates an appropriate prompt. For example, in response to the input "I like spicy food," it generates the prompt "Please recommend some dishes for someone who likes spicy food." This prompt is then input into the generation AI model.

Step 3:

The server sends the generated prompt to the OpenAI API and retrieves appropriate answers and recommendations based on the generated AI model. For example, in response to the prompt "What dishes would you recommend for someone who likes spicy food?", answers such as "Mapo Tofu" or "Kimchi Hot Pot" would be generated.

Step 4:

The server analyzes the responses and recommendations obtained from the generative AI model and converts them into a format for user presentation. For example, it formats responses like "Mapo Tofu" and "Kimchi Hot Pot" into a list.

Step 5:

The server sends formatted responses and recommendations to the smart device. The user can then view this information on the smart device's screen.

Step 6:

Users can select menu items based on the information provided or ask further detailed questions. For example, they might re-enter the question, "What are the ingredients in Mapo Tofu?"

Step 7:

The server receives user input again and repeats the same process. Specifically, it generates a prompt again, inputs it into the generation AI model, retrieves the answer, formats it, and provides it to the user.

This allows users to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

Furthermore, an emotion engine for estimating the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

"Example of Morphology 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and recommends products accordingly. Specifically, when a user is selecting a product, the system recognizes their emotions from their facial expressions and tone of voice, and recommends products that match those emotions. For example, if a user shows a joyful expression, the system recommends products that are likely to make that user happy.

"Example of Morphology 2"

Furthermore, the emotion engine generates product descriptions based on the user's emotions. Specifically, it recognizes the user's emotions and selects words that match those emotions to generate a product description. For example, if the user is feeling down, it will select words that will cheer them up to generate a product description. (Example 3)

Furthermore, the emotion engine recognizes the user's emotions and provides services accordingly. Specifically, it recognizes the user's emotions and provides services that match those emotions. For example, if a user is showing anger, it might offer a service such as an apology.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system recommends products that match the recognized emotions.

"Example of Morphology 2"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system selects words that match the recognized emotion and generates a product description.

"Example of Morphology 3"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system provides services that match the recognized emotions.

(Example 1)

Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

Traditional restaurant menu creation and customer experience improvement have been time-consuming and labor-intensive. In particular, generating product descriptions from photos and providing personalized recommendations consume significant human resources. Furthermore, multilingual services for foreign tourists are inadequate, highlighting the need for improved customer satisfaction. Additionally, the inability to provide product recommendations based on customer emotions can lead to a decline in the quality of the customer experience. An efficient system is needed to address these challenges.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an artificial intelligence (AI) method for generating product descriptions from smartphone photos; a method for AI to generate translation tasks; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI to explain questions during menu consideration; a method for recommending items according to preferences; and a method for recognizing user emotions and recommending products based on those emotions. This enables more efficient menu creation, product recommendations tailored to customer preferences, enhanced service for foreign tourists through multilingual support, and product recommendations based on customer emotions.

"A method for creating menus using a smartphone" refers to a method for creating restaurant menus using a smartphone's camera and applications.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"Methods for generating translation work using artificial intelligence" refers to methods of translating product descriptions and menu contents into multiple languages using artificial intelligence technology.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"A method for using artificial intelligence to explain questions during menu consideration" refers to a method in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending products based on preferences" refers to a method of recommending appropriate products based on a customer's past choices and preferences.

"A method for recognizing user emotions and recommending products based on those emotions" refers to a method of recognizing emotions by analyzing the user's facial expressions and tone of voice, and then recommending products that match those emotions.

This invention is a system that allows restaurant operators to create menus using smartphones and improve the customer experience. A specific embodiment of this system is described below.

First, the user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they would launch the smartphone's camera app, frame the dessert, and press the shutter button.

Next, the device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in an appropriate format (e.g., JPEG, PNG).

The server analyzes the received photos using image recognition software (e.g., Google Cloud Vision API). The server recognizes the characteristics of the product from the photo (e.g., chocolate cake, fruit parfait). For example, the Google Cloud Vision API might return the label "chocolate cake."

Subsequently, the server generates a product description based on the recognized characteristics. Using a generation AI model (e.g., OpenAI GPT-3), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

The generated product descriptions are displayed on the restaurant's website or application by the server. Specifically, the product descriptions are sent to the front-end in HTML or JSON format and displayed in the user interface.

Furthermore, when a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, the smartphone's camera captures the user's smile, and the microphone records the user's voice tone.

The server analyzes the captured data using an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait."

As a concrete example, consider a scenario where a user takes a photo of a new dessert with their smartphone and uploads it to the system. The server uses the Google Cloud Vision API to analyze the photo and recognize its characteristic as "chocolate cake." The AI then generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

Furthermore, if a user displays an expression of joy while viewing the menu using their smartphone camera, the Microsoft Azure Emotion API recognizes that expression. The server then recommends a "fruit parfait," which is likely to bring joy to the user.

Example of a prompt message:

"Please upload photos of the product taken with your smartphone. Our AI will automatically recognize the product's characteristics and generate a product description. It will also use your camera and microphone to recommend products based on your emotions."

In this way, a system is realized in which servers, terminals, and users work together to efficiently create restaurant menus and recommend products to customers.

The flow of the specific processing in Example 1 will be explained using Figure 17.

Step 1:

The user takes a photo of the product.

The user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they launch the smartphone's camera app, frame the dessert, and press the shutter button. The input is the photograph taken, and the output is the image data stored on the smartphone.

Step 2:

The device uploads the photos to the server.

The device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. The photo data is sent in an appropriate format (e.g., JPEG, PNG). The input is the image data on the smartphone, and the output is the image data sent to the server.

Step 3:

The server analyzes the photo and recognizes the product's characteristics.

The server analyzes the received photograph using image recognition software (e.g., an image recognition API). The server recognizes the characteristics of the product from the photograph (e.g., chocolate cake, fruit parfait). For example, the image recognition API might return the label "chocolate cake." The input is the image data sent to the server, and the output is the recognized product characteristic information.

Step 4:

The server generates the product description.

The server generates a product description based on the recognized characteristics. Using a generative AI model (e.g., a generative AI model), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture." The input is product characteristic information, and the output is the generated product description.

Step 5:

The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Specifically, it sends the product descriptions in HTML or JSON format to the front-end and displays them in the user interface. The input is the generated product description, and the output is the product description displayed on the website or application.

Step 6:

When a user selects a product, the device captures the user's emotions.

When a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, a smartphone's camera captures the user's smile, and the microphone records the user's voice tone. The input is the user's facial expressions and voice tone, and the output is the captured emotional data.

Step 7:

The server analyzes emotions and recommends appropriate products.

The server analyzes the captured data using an emotion engine (e.g., an emotion analysis API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait." The input is the captured emotion data, and the output is the recommended product information.

(Application Example 1)

Next, we will describe Application Example 1 of Configuration Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

Traditional restaurants faced significant challenges in menu creation and improving the customer experience, requiring considerable time and effort. Specifically, there were multiple obstacles, including menu creation, product description generation, personalized recommendations, and multilingual support. Furthermore, providing service that resonated with customer emotions was difficult, making improving customer satisfaction a challenge.

In Application Example 1, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on the smart device, a means for the AI to generate translations, a means for improving the customer experience, a means for the AI to explain questions during menu consideration, a means for recommending according to preferences, and a means for recognizing customer emotions and recommending products based on those emotions. This enables efficient menu creation, automatic generation of product descriptions, multilingual support, improved customer experience, and emotion-based recommendations.

A "smart device" is a portable electronic device with advanced functions, such as a smartphone or smart glasses.

"Menu creation methods" refer to functions and methods for creating restaurant menus using smart devices.

"AI methods for generating product descriptions" refers to artificial intelligence technology that automatically recognizes the characteristics and features of a product from a photograph taken with a smart device and generates a product description based on that recognition.

"A means of generating translation work using AI" refers to a function that automatically performs multilingual translation using artificial intelligence.

"Methods for improving the customer experience" refer to methods and functions that enhance the customer experience, such as eliminating the need to wait in line with waitstaff or at the cash register.

"An AI-powered solution for questions arising during menu consideration" refers to a function where artificial intelligence automatically explains questions that customers may have when considering menu options.

"A method for recommending products based on preferences" refers to a function that recommends appropriate products based on the customer's preferences.

"A means of recognizing customer emotions and recommending products based on those emotions" refers to a function that recognizes customer emotions from facial expressions, tone of voice, etc., and recommends products that match those emotions.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from photos on a smart device using AI, means for generating translations using AI, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations according to preferences, and means for recognizing customer emotions and recommending products based on those emotions.

Program Processing Description

Hardware

The server will be equipped with a high-performance GPU. Smart devices such as smartphones and smart glasses will be used as terminals.

Software

The server uses the following software:

OpenCV: Image processing library

Keras: A deep learning library

Transformers: Generative AI models (such as GPT-3)

Data processing and calculations

1. Menu creation method:

Take a photo of the food using the device's camera and upload it to the server.

The server uses OpenCV to preprocess images and Keras to recognize product characteristics and features from the images.

Based on the recognition results, product descriptions are generated using Transformers.

2. Methods for AI to generate translation work:

The server uses a generative AI model to translate the generated product descriptions into multiple languages.

In particular, we will implement multilingual support to enhance services for foreign tourists.

3. Means of improving the customer experience:

The terminal allows customers to complete ordering and payment on their smart devices, eliminating the need to wait in line with waitstaff or at the cash register.

4. How AI can explain questions that arise when considering menu options:

The terminal sends questions that arise when customers are considering the menu to the server, and the server automatically generates explanations using AI.

5. Methods for recommending based on preferences:

The server recommends appropriate products based on the customer's past order history and preferences.

6. Means for recognizing customer emotions and recommending products based on those emotions:

The server analyzes the customer's facial expressions and voice tone through the device's camera, and uses Keras to recognize their emotions.

Based on recognized emotions, Transformers are used to recommend appropriate products.

Specific examples and prompt messages

Specific example:

When a customer is wearing smart glasses and looking at a menu, the smart glasses analyze the customer's facial expression and display a message saying, "You seem happy, so we recommend dessert!"

Example prompts for a generative AI model:

"This dish looks like delicious pasta. Please generate a detailed product description."

Thus, the embodiment of this invention involves combining smart devices and AI technology to enable menu creation and improve the customer experience in restaurants.

The flow of the specific processing in Application Example 1 will be explained using Figure 18.

Step 1:

The user takes a photo of the food using a smart device (smartphone or smart glasses). The photo is saved on the device.

Step 2:

The device uploads the captured photos to the server. The server performs image preprocessing using OpenCV. Specifically, it resizes and denoises the images.

Step 3:

The server analyzes pre-processed images using Keras to recognize the characteristics and features of the product. The input is the pre-processed image, and the output is data indicating the product's characteristics and features.

Step 4:

The server generates product descriptions using Transformers based on recognized characteristics and features. The input is data describing the product's characteristics and features, and the output is the generated product description.

Step 5:

The server uses a generative AI model to translate the generated product descriptions into multiple languages. The input is the generated product description, and the output is the translated product description in multiple languages.

Step 6:

The terminal allows customers to complete their orders and payments on their smart devices, eliminating the need to wait in line with waiters or at the cash register. Input is the customer's order information, and output is a notification that the order has been completed.

Step 7:

The terminal sends questions that arise when customers consider the menu to the server. The server automatically generates explanations using AI. The input is the customer's question, and the output is the generated explanation.

Step 8:

The server recommends appropriate products based on the customer's past order history and preferences. The input is the customer's order history and preference data, and the output is the recommended products.

Step 9:

The system analyzes the customer's facial expressions and voice tone through the terminal's camera, and the server uses Keras to recognize their emotions. The input is data on the customer's facial expressions and voice tone, and the output is the recognized emotion.

Step 10:

The server uses Transformers to recommend appropriate products based on the recognized emotions. The input is the recognized emotions, and the output is the recommended products.

(Example 2)

Next, we will describe Embodiment 2 of Embodiment Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

Modern consumers demand quick and accurate understanding of product descriptions, as well as multilingual information. They also require personalized product descriptions that cater to their emotions. However, traditional systems struggle to meet these demands, making the improvement of services, particularly for foreign tourists, a significant challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for having artificial intelligence explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This enables users to quickly and accurately understand product descriptions and provides information in multiple languages. Furthermore, by providing personalized product descriptions tailored to user emotions, customer satisfaction can be improved.

A "smart device" is a portable electronic device that has internet connectivity and can run applications.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Means of improving the customer experience" refer to features that enhance the convenience and satisfaction customers experience when using a service.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function that allows artificial intelligence to provide appropriate answers to questions that arise when customers are considering menu options.

"Preference-based recommendation mechanisms" refer to features that recommend appropriate products and services based on a customer's past choices and preferences.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions from their input and actions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to a function for translating generated product descriptions into multiple languages.

Modes for carrying out the invention

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for generating translations using artificial intelligence, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

Hardware and software to be used

The server generates product descriptions using a generative AI model (e.g., OpenAI's GPT-4). The generated product descriptions are translated into multiple languages using a translation engine (e.g., Google Translate API). To recognize user emotions, an emotion engine (e.g., IBM Watson's Sentiment Analysis API) is used. Smart devices are portable electronic devices with internet connectivity capable of running applications.

Explanation of the program's processing

The server receives text entered by the user through the terminal and generates a product description using a generative AI model. For example, if the user enters "This product is high quality and long-lasting," the server will generate a product description based on this text.

The generated product descriptions are translated into multiple languages using a translation engine. For example, they are translated into major tourist languages such as English, Chinese, and Korean. Specifically, a description like "This product is high quality and long-lasting" is translated.

When a user inputs emotional text or audio data through their device, the server uses an emotion engine to recognize the user's emotions. For example, if a user inputs "I'm feeling down today," the server analyzes this text and recognizes the user's emotion as "down."

The server uses a generative AI model to generate product descriptions that respond to the user's emotions, based on the emotions it recognizes. For example, if the user is feeling down, the server will select uplifting words to generate a product description.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Examples of specific cases and prompt messages

As a concrete example, consider the following scenario.

Scenario 1: Multilingual Translation

If a user enters "This product is high quality and long-lasting" in Japanese, the server will translate this product description into English, Chinese, and Korean.

Scenario 2: Product Description Based on Emotions

If a user enters "I'm feeling down today," the server uses an emotion engine to recognize the user's emotions and generates a product description that will cheer them up.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Example of a prompt message

Examples of prompts for a generative AI model are as follows:

Please translate this Japanese product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'

"Please generate a product description that will cheer up a user who is feeling down. The original description is 'This product is high quality and long-lasting.'"

The above describes the embodiments for carrying out this invention.

The flow of the specific processing in Example 2 will be explained using Figure 19.

Step 1:

The user uses a terminal to input the text that will form the basis of the product description. For example, the user might input, "This product is high quality and long-lasting." The entered text is then sent to the server.

Step 2:

The server uses a generative AI model to generate a product description based on the text entered by the user. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description based on the user's input: 'This product is high quality and long-lasting.'" The generative AI model generates a product description based on this prompt and returns it to the server. The output is the generated product description.

Step 3:

The server sends the generated product description to the translation engine for translation into multiple languages. Specifically, the server sends the following prompt to the translation engine: "Translate this product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'" The translation engine performs the translation based on this prompt and returns it to the server. The output is the product description translated into multiple languages.

Step 4:

The user inputs text or audio data related to their emotions through their device. For example, the user might input, "I'm feeling down today." The input text or audio data is then sent to the server.

Step 5:

The server uses an emotion engine to recognize the user's emotions. Specifically, the server sends the following prompt to the emotion engine: "Analyze the user's input 'I'm feeling down today' and recognize the emotion." Based on this prompt, the emotion engine analyzes the emotion and returns the result to the server. The output is the recognized emotion.

Step 6:

The server uses a generative AI model to generate a product description that responds to the user's emotions, based on the emotions it has perceived. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description that will cheer up the user if they are feeling down. The original description is 'This product is high quality and long-lasting.'" Based on this prompt, the generative AI model generates an emotion-responsive product description and returns it to the server. The output is the emotion-responsive product description.

Step 7:

The server sends a product description to the user's device that corresponds to the generated emotion. The user can then view the uplifting product description through their device. Specifically, if the user enters "I'm feeling down today," the server will provide the user with a product description such as, "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

The above outlines the specific processing flow of this system's program.

(Application Example 2)

Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

Traditional e-commerce sites often provide product descriptions in a single language, which posed a problem for foreign tourists and multilingual users, making them difficult to understand. Furthermore, the lack of emotionally resonant product descriptions made it difficult to increase user purchase intent. Additionally, users sometimes spent a significant amount of time trying to understand product descriptions, resulting in a less-than-ideal purchasing experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for artificial intelligence to explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This makes it possible to provide product descriptions tailored to user emotions in multiple languages, making them easily understandable to foreign tourists and multilingual users. Furthermore, it can increase user purchasing intent and improve the purchasing experience.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

"Menu creation methods" refer to functions or software that allow users to create menus for products and services using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Customer experience improvement measures" refer to features and software designed to enhance the customer experience when using a product or service.

"A means for AI to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate explanations for questions that arise when users are considering menu options.

"Preference-based recommendation" refers to a function that recommends appropriate products and services based on the user's preferences and past behavior.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to functions or software for translating generated product descriptions into multiple languages.

This invention will now describe embodiments for carrying out this invention. This system includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

System Configuration

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones, tablets, and smart glasses.

Server: A high-performance computer responsible for data processing and storage.

Software

Artificial intelligence model: Generates product descriptions using the OpenAI API.

Translation software: Multilingual translation is performed using the Google Translate API.

Emotion recognition software: Uses EmotionRecognizer to recognize the user's emotions.

Data processing and data calculation

1. Image Analysis: Images of products taken with a smart device are sent to the server. The server uses an image analysis algorithm to extract product characteristics and features and generate a product description.

2. Emotion Recognition: The smart device's camera and microphone are used to analyze the user's facial expressions and voice tone. EmotionRecognizer software recognizes the user's emotions and sends the results to the server.

3. Product Description Generation: The server generates prompt text based on the user's emotions and uses the OpenAI API to generate a product description appropriate to those emotions.

4. Multilingual Translation: The generated product description is translated into multiple languages using the Google Translate API. The translation results support major languages such as English, Chinese, and Korean.

Specific examples

For example, if a user is feeling down, the smart device's camera captures the user's facial expression, and the EmotionRecognizer recognizes it as "downcast." The server then generates the following prompt:

"Generate a product description that will cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time."

This prompt is sent to the OpenAI API to generate a sentiment-appropriate product description. The generated product description will look like this:

"This product was created to brighten your life, even just a little. It's made from high-quality materials and is built to last."

Next, we will translate this product description into multiple languages using the Google Translate API.

In this way, it becomes possible to provide product descriptions tailored to the user's emotions in multiple languages.

The flow of the specific processing in Application Example 2 will be explained using Figure 20.

Step 1:

The user takes a photo of the product with their smart device.

Input: Product image

Output: Captured image data

Specific operation: The user uses a smart device such as a smartphone or tablet to take a picture of the product they are considering purchasing. This image data is saved on the smart device.

Step 2:

The device sends the captured image to the server.

Input: Captured image data

Output: Image data sent to the server

Specific operation: The smart device transmits the captured image data to the server via the internet. The HTTP protocol and similar protocols are used for transmission.

Step 3:

The server performs image analysis and extracts the product's characteristics and features.

Input: Sent image data

Output: Characteristics and features of the extracted products

Specific operation: The server uses image analysis algorithms to automatically extract product characteristics and features from image data. For example, it recognizes the product's color, shape, brand logo, etc.

Step 4:

The server generates a product description based on the extracted characteristics and features.

Input: Characteristics and features of the extracted products

Output: Generated product description

Specific operation: The server uses a generative AI model to generate product descriptions based on extracted characteristics and features. For example, a description such as "This product is made from high-quality materials and will last a long time" might be generated.

Step 5:

The device recognizes the user's emotions.

Input: User's facial expressions and tone of voice

Output: Recognized user emotions

Specific operation: Using the camera and microphone of a smart device, the EmotionRecognizer software captures the user's facial expressions and voice tone, recognizing the user's emotions. For example, emotions such as "depressed" or "happy" can be recognized.

Step 6:

The server generates prompt messages that respond to the user's emotions.

Input: Recognized user sentiment, generated product description

Output: Prompt message based on emotion

Specific operation: The server generates prompt messages to send to the AI model based on the recognized user's emotions. For example, a prompt message like, "Generate a product description to cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time," might be generated.

Step 7:

The server uses an AI model to generate product descriptions that are appropriate for the user's emotions.

Input: Prompt text that responds to emotion

Output: Product description suitable for emotions

Specific operation: The server uses a generative AI model (e.g., OpenAI's API) to generate emotionally appropriate product descriptions based on the prompt text. For example, a description like, "This product was created to brighten your life a little. It is made from high-quality materials and is durable," might be generated.

Step 8:

The server translates the generated product description into multiple languages.

Input: Generated product description

Output: Product description translated into multiple languages

Specific operation: The server uses the Google Translate API to translate the generated product description into multiple languages, such as English, Chinese, and Korean. For example, a translation might be: "This product is made to brighten your life a little. It is made of high-quality materials and lasts long."

Step 9:

The server sends the product description, translated into multiple languages, to the device.

Input: Product description translated into multiple languages

Output: Translation result sent to the terminal

Specific operation: The server sends product descriptions translated into multiple languages to smart devices via the internet. HTTP protocol and similar protocols are used for transmission.

Step 10:

The device displays product descriptions translated into multiple languages to the user.

Input: Translation result sent to the device

Output: Multilingual product description displayed to the user

Specific operation: The smart device displays the product description, translated into multiple languages, to the user. This allows the user to understand the emotionally resonant product description in multiple languages.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

Traditional customer experience improvement systems often rely on manual processes for menu creation, product descriptions, and translation, resulting in inefficiency. Furthermore, they may fail to adequately address customer questions and provide personalized recommendations, potentially leading to decreased customer satisfaction. Additionally, the lack of features to recognize and respond to customer emotions hinders the improvement of the overall customer experience.

In Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and a means for recognizing user emotions and providing services corresponding to those emotions. This enables quick answers to customer questions when considering menus, product recommendations tailored to preferences, and significantly improves customer satisfaction by providing services that respond to customer emotions.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"Methods for generating translation work using artificial intelligence" refers to functions that automatically translate text and audio using artificial intelligence.

"Methods for improving customer experience by reducing waiting times" refer to functions that shorten the waiting time customers experience when using a service, thereby providing a smoother experience.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are considering menu options.

"A method for recommending products based on preferences" refers to a function where artificial intelligence recommends appropriate products based on the customer's preferences and past choices.

"A means of recognizing user emotions and providing services that respond to those emotions" refers to a function that recognizes user emotions from their facial expressions and voice, and provides appropriate services that correspond to those emotions.

This invention is a system for improving customer experience and includes means for creating menus using smart devices, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for recommending products according to preferences, and means for recognizing user emotions and providing services that respond to those emotions.

Hardware and software to be used

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones and tablets.

Server: A high-performance server (e.g., a server with an NVIDIA GPU).

Software

Generative AI model: GPT-4 is used as an example.

Emotion recognition engine: Affectiva SDK is used as an example.

Program Processing Description

The server receives information sent from the smart device and performs analysis using a generative AI model. For example, if the user inputs "I like spicy food," the generative AI model generates a list of spicy dishes, and the server sends this list to the smart device. The smart device then displays the received list to the user.

Furthermore, when a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to generate ingredient information and sends it to the smart device. The smart device then displays the received ingredient information to the user.

Furthermore, smart devices use their built-in cameras and microphones to capture the user's facial expressions and voice, and use an emotion recognition engine to recognize the user's emotions. For example, if the user shows an angry expression, the smart device sends that information to a server. The server uses a generative AI model to generate an apology and sends it to the smart device. The smart device then displays the received apology to the user.

Examples of specific cases and prompt messages

Example 1: User enters "I like spicy food."

The user opens the app on their smartphone and enters "I like spicy food."

The server generates a list of spicy dishes using an AI model.

The server generates a list and sends it to the smart device.

A smart device displays a list of spicy dishes to the user.

Example prompt: "Please list dishes you would recommend to a customer who likes spicy food."

Example 2: A user asks, "What are the ingredients in this dish?"

The user opens the tablet app and asks, "What are the ingredients in this dish?"

The server generates ingredient information using an AI model.

The server generates ingredient information and sends it to the smart device.

The smart device displays ingredient information to the user.

Example prompt: "Please tell me the ingredients for this dish."

Example 3: The user shows an angry expression.

The user displays an angry expression while using their smartphone.

Smart devices use cameras to capture the user's facial expressions, which are then analyzed by an emotion recognition engine.

The smart device sends information about the anger it recognizes to the server.

The server generates an apology using an AI model.

The server generates an apology message and sends it to the smart device.

Smart devices display an apology message to the user.

Example prompt: "Generate an apology for when the user is angry."

In this way, the server, smart devices, and users work together to operate the system and improve the customer experience. The flow of a specific process in Example 3 will be explained using Figure 21.

Step 1:

The user inputs information through their device. The user uses a smartphone or tablet to input questions and preferences into the system. For example, they might input "I like spicy food." The input data is saved on the device in text format.

Step 2:

The terminal sends the input information to the server. The terminal sends the user-entered information to the server via the internet. The input data is sent to the server in text format.

Step 3:

The server analyzes the input information using a generative AI model. The server passes the received input information to the generative AI model (e.g., GPT-4) for analysis. The generative AI model generates appropriate responses or recommendations based on the input information. For example, given the information "likes spicy food," it generates a list of spicy dishes. The input data is in text format, and the output data is in list format.

Step 4:

The server generates appropriate answers and recommendations based on the analysis results. The server generates answers and recommendations for the user based on the analysis results obtained from the generating AI model. For example, it might generate a list of spicy dishes. The output data is in list format.

Step 5:

The server sends the generated responses and recommendations to the device. The server sends the generated responses and recommendations to the device via the internet. The output data is sent to the device in list format.

Step 6:

The device displays answers and recommendations to the user. The device displays answers and recommendations received from the server to the user. The user can review the displayed information. The output data is displayed in list format.

Step 7:

The device recognizes the user's emotions using an emotion engine. The device captures the user's facial expressions and voice using its built-in camera and microphone, and recognizes the user's emotions using an emotion engine (e.g., Affectiva SDK). Input data is in image or audio format, and output data is in emotion information format.

Step 8:

The device transmits recognized emotion information to the server. The device transmits the recognized emotion information to the server via the internet. Input data is sent to the server in emotion information format, and output data is also sent in emotion information format.

Step 9:

The server generates appropriate services based on emotional information. The server uses a generative AI model to generate appropriate services based on the received emotional information. For example, if the user shows an angry expression, it will generate an apology. Input data is in emotional information format, and output data is in text format.

Step 10:

The server sends the generated service to the terminal. The server sends the generated service to the terminal via the internet. The output data is sent to the terminal in text format.

Step 11:

The terminal provides services to the user. The terminal delivers services received from the server to the user. For example, it displays an apology. The output data is displayed in text format.

(Application Example 3)

Next, we will describe Application Example 3 of Configuration Example 3. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

Traditional food delivery services faced challenges such as users spending a significant amount of time choosing from menus and difficulty in providing services tailored to user preferences and emotions. Furthermore, multilingual support for foreign tourists was insufficient, highlighting the need for improved user experience. There is a demand to address these challenges and provide a more comfortable and personalized service.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing the user's emotions and providing services corresponding to those emotions. This allows users to efficiently select menu items and receive personalized services tailored to their individual preferences and emotions. Furthermore, multilingual support enhances services for foreign tourists.

A "smart device" is a portable electronic device with advanced computing capabilities, such as a smartphone or tablet.

A "menu creation method" refers to a method or device for generating a list of products or services that a user can select.

"An artificial intelligence method for generating product descriptions from images" refers to an artificial intelligence system that uses image recognition technology to automatically analyze the characteristics and features of a product and generates a product description based on that analysis.

"Means of generating translation work using artificial intelligence" refers to an artificial intelligence system that automatically translates text and audio between different languages.

"Customer experience improvement measures" refer to methods and devices that enhance the convenience and satisfaction users experience when using a service.

"A system where artificial intelligence explains questions that arise when considering menu options" refers to a system where artificial intelligence provides appropriate answers to questions that users may have when choosing from a menu.

A "method for recommending products based on preferences" is a system that recommends appropriate products and services based on the user's past choices and input information.

An "emotion recognition system" is a system that analyzes a user's emotions from their facial expressions, tone of voice, etc., and responds accordingly.

The system for carrying out this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing user emotions and providing services corresponding to those emotions.

System Program

The program for this system performs the following operations:

Hardware and software

Hardware:

Smart devices (smartphones, tablets, etc.)

Camera (to capture the user's facial expressions)

Computer (a processing unit for executing programs)

Software:

OpenCV (image processing library)

Keras (deep learning library)

Transformers (a library of generative AI models)

Data processing and data calculation

1. Menu creation method:

The server generates a list of products and services that users can select using their smart devices.

2. Artificial intelligence methods for generating product descriptions from images:

The server analyzes images captured by the smart device's camera and automatically recognizes the product's characteristics and features.

Based on the recognized information, generate a product description.

3. Methods for generating translation tasks using artificial intelligence:

The server automatically translates text and audio between different languages.

In particular, we will provide multilingual support to enhance services for foreign tourists.

4. Measures to improve customer experience:

The server reduces waiting times for users when accessing the service, improving convenience.

5. How artificial intelligence can explain questions that arise during menu planning:

The server uses a generative AI model to provide appropriate answers to questions that arise when users select menu items.

6. Methods for recommending products based on preferences:

The server recommends appropriate products and services based on the user's past choices and input information.

7. Emotion recognition means:

The server analyzes the user's facial expressions and voice tone captured by the camera to recognize their emotions.

Respond appropriately to the recognized emotions.

Specific examples

When a user enters "I like spicy food," the server uses a generative AI model to recommend spicy dishes.

When a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to provide information about the ingredients.

If the user displays an angry expression, the server uses emotion recognition to apologize with the message, "I'm sorry. Is there a problem?"

Example of a prompt message

"I like spicy food. What do you recommend?"

"What are the ingredients in this dish?"

"How would you respond if a user is showing signs of anger?"

In this way, users can efficiently select menu items and receive personalized service tailored to their individual preferences and feelings. Furthermore, multilingual support enhances services for foreign tourists.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The user opens the menu using a smart device.

Input: A menu display request initiated by the user.

Data Processing: The server retrieves menu information from the database and sends it to the smart device.

Output: A menu will appear on your smart device.

Step 2:

The user takes a photo of the food using the camera on their smart device.

Input: An image of a dish taken by the user.

Data Processing: The server receives images and analyzes them using an image recognition algorithm (OpenCV).

Output: The analysis results extract the characteristics and features of the dishes.

Step 3:

The server generates a product description from the image.

Input: Data on the characteristics and features of the dish.

Data Processing: The server uses a generative AI model to generate product descriptions based on characteristics and features.

Output: Product description text is generated and sent to the smart device.

Step 4:

Users enter questions when considering menu options.

Input: The question entered by the user (e.g., "What are the ingredients in this dish?").

Data Processing: The server generates answers to questions using a generative AI model.

Output: The answer text is generated and displayed on the smart device.

Step 5:

The server recommends products based on the user's preferences.

Input: User's past selections and input information (e.g., "I like spicy food").

Data Processing: The server uses a generative AI model to recommend products based on user preferences.

Output: A list of recommended products is displayed on your smart device.

Step 6:

The user captures their facial expressions using the camera on their smart device.

Input: User's facial expression image.

Data Processing: The server analyzes emotions using a facial recognition algorithm (Keras).

Output: As an analysis result, user sentiment data is generated.

Step 7:

The server provides services tailored to the user's emotions.

Input: User emotion data (e.g., anger).

Data Processing: The server generates appropriate responses based on sentiment data (e.g., apology messages).

Output: The corresponding message will be displayed on the smart device.

Step 8:

The server performs the translation.

Input: Text or audio data entered by the user.

Data Processing: The server uses a translation algorithm to translate the input data into multiple languages.

Output: Translated text and audio data are displayed on the smart device.

The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input regarding the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet Search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives prompts containing instructions, as well as inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions provided by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search engine <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example of a configuration in which specific processing is performed by the data processing device 12 was given, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[Fourth Embodiment]

Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to the bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. It captures images of the area surrounding the user 20 (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

The communication interface 44 is connected to the network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via the network 54. The exchange of various types of information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and movements of the robot 414 are controlled by the motors of the arms, hands, and feet.

It can be more controlled. Some of the emotions of robot 414 can be expressed by controlling these motors. Furthermore, the facial expressions of robot 414 can also be expressed by controlling the illumination state of the LEDs in its eyes.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In robot 414, the processor 46 performs the reception and output processing. The storage 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the storage 50 and executes the read program 60 on the RAM 48. The reception and output processing is realized by the processor 46 acting as a control unit 46A according to the reception and output program 60 executed on the RAM 48.

Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be explained.

"Example of Morphology 1"

In one embodiment of this invention, a restaurant operator creates a menu using a smartphone. Specifically, the operator uploads photos of products taken with the smartphone's camera, and AI automatically recognizes the characteristics and features of the products from the photos. Based on this recognition, a product description is generated. This product description is then displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Furthermore, in this embodiment of the present invention, the AI generates the translation work. Specifically, the generated product description is translated into multiple languages. This makes the product description easier for foreign tourists to understand. For example, it is possible to translate into major tourist languages such as English, Chinese, and Korean.

"Example of Morphology 3"

Furthermore, in this embodiment of the present invention, the customer experience is also improved. Specifically, when a customer is considering a menu, the AI explains their questions and recommends products according to their preferences. For example, if a customer inputs information such as "I like spicy food," the AI will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI will provide an answer to that question.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: The restaurant operator takes photos of the products using their smartphone camera.

Step 2: Upload the photos you took to the system.

Step 3: The system's AI automatically recognizes the product's characteristics and features from the photograph.

Step 4: The AI generates a product description based on the recognition results.

Step 5: The generated product description is displayed to customers on the restaurant's website or application.

"Example of Morphology 2"

Step 1: Obtain the product description generated by the AI.

Step 2: The AI translates the product description into multiple languages.

Step 3: The translated product description is displayed to foreign tourists on the restaurant's website or application.

"Example of Morphology 3"

Step 1: When customers are considering the menu, they input their questions into the AI.

Step 2: The AI generates the answer to that question.

Step 3: The AI recommends products based on the customer's preferences.

Step 4: The AI-generated answers and recommendations are displayed to the customer.

(Example 1)

Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 will be referred to as the "terminal."

When restaurant operators create menus, the process of taking photos of products and then recognizing their characteristics and features to generate product descriptions is time-consuming. Furthermore, there is a lack of means to provide multilingual support for foreign tourists and to enhance the customer experience. Therefore, there is a need for increased efficiency in menu creation and improved customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a means for creating menus using a smartphone; an artificial intelligence (AI) means for generating product descriptions from smartphone photos; a means for AI to generate translations; a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a means for AI to explain questions during menu consideration; a means for recommending items according to preferences; a means for uploading product photos taken with a smartphone camera, for AI to automatically recognize the characteristics and features of the products from those photos; and a means for generating product descriptions based on the recognition results. This enables increased efficiency in menu creation and improved customer satisfaction.

A "smartphone" is a multi-functional mobile device that, in addition to the functions of a regular mobile phone, also offers internet connectivity and the ability to use applications.

"Menu creation methods" refer to the methods and tools that restaurant operators use to create menus, particularly those using smartphones.

"Artificial intelligence tools" refer to technologies that use machine learning and data analysis to automatically perform specific tasks.

"Means of generating translation work using artificial intelligence" refers to methods and technologies that use artificial intelligence to translate text into multiple languages.

"Methods for improving the customer experience" refer to methods and tools for improving the convenience and satisfaction customers experience when using a service.

"Methods for using artificial intelligence to explain questions during menu consideration" refers to methods and technologies in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"Methods of recommending based on preferences" refer to methods and technologies that recommend appropriate products and services based on customer preferences and past choices.

"Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

"Methods for generating product descriptions" refers to methods and technologies for creating product descriptions in natural language based on recognized product characteristics and features.

This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, users upload photos of their products taken with their smartphone cameras, and artificial intelligence (AI) automatically recognizes the characteristics and features of the products from these photos. Based on this recognition, the system generates product descriptions. These product descriptions are then displayed to customers on the restaurant's website or application.

Hardware and software to be used

Smartphone: A device used for taking and uploading photos.

Server: Performs data processing and runs AI models.

Artificial intelligence models: Image recognition models and natural language generation models based on TensorFlow and PyTorch (e.g., GPT-3, BERT).

Data processing and data calculation

1. The user takes a photo of the product with their smartphone.

The user launches their smartphone's camera app and takes a picture of the product. For example, they might take a picture of a new dessert, "Chocolate Cake."

2. The user uploads photos to the system.

The user opens a dedicated application, selects the photos they've taken, and presses the upload button. The photos are then sent to the server via the internet.

3. The server receives the photos and inputs them into the AI model.

The server receives the uploaded photos and inputs them into an AI model for image processing. The AI model used here is based on TensorFlow or PyTorch.

4. The server uses an AI model to recognize the characteristics and features of the product.

The server uses an AI model to recognize the characteristics and features of a product from a photograph. For example, it extracts information such as the type of dessert, main ingredients, and visual characteristics.

5. The server generates a product description based on the recognition results.

The server generates product descriptions based on the AI's recognition results. These product descriptions are created using natural language generation technology. The software used includes GPT-3 and BERT.

6. The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Customers can then view them.

Specific examples

A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description, such as "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing," and displays it on the website.

Example of a prompt message

Please upload a photo of your new dessert. AI will recognize the product's characteristics and features from the photo and generate a product description.

The flow of the specific processing in Example 1 will be explained using Figure 11.

Step 1:

The user takes a photo of the product with their smartphone.

Input: Product (e.g., Chocolate cake)

Output: Photos of the photographed product

Specific action: The user launches the camera app on their smartphone and takes a picture of the product. For example, they might take a picture of the new dessert, "Chocolate Cake."

Step 2:

The user uploads a photo to the system.

Input: Photographs of the product

Output: Photo data sent to the server

Specific operation: The user opens a dedicated application, selects the photo they have taken, and presses the upload button. The photo is sent to the server via the internet.

Step 3:

The server receives the photos and inputs them into the AI model.

Input: Photo data sent to the server

Output: Photo data input to the AI model

Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

Step 4:

The server uses an AI model to recognize the characteristics and features of the product.

Input: Photo data entered into the AI model

Output: Characteristics and features of the recognized product (e.g., chocolate cake, cream topping, berry decoration)

Specific operation: The server runs an AI model to extract product characteristics and features from a photograph. For example, it recognizes the type of dessert, main ingredients, and visual features.

Step 5:

The server generates a product description based on the recognition results.

Input: Characteristics and features of the recognized product

Output: Generated product description (Example: "This chocolate cake features rich chocolate and creamy toppings. The berry decorations make it visually appealing.")

Specific operation: The server uses GPT-3 or BERT to generate product descriptions based on the recognition results.

Step 6:

The server displays the product description on the website or application.

Input: Generated product description

Output: Product description displayed on the website or application

Specific operation: The server saves the product description to the website's database, sends the data to the front-end for display, and the customer can view it.

(Application Example 1)

Next, we will describe Application Example 1 of Configuration Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 as the "terminal."

Traditional restaurant menu creation was often done manually, which was time-consuming and labor-intensive, and made it difficult to adequately convey the appeal of the products. Furthermore, the lack of multilingual support for foreign tourists and insufficient recommendation features tailored to customer preferences were also problems. In addition, there were limited means to improve the in-store customer experience, such as having to wait for waitstaff or in line at the register, highlighting the need for increased customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an AI method for generating product descriptions from smartphone photos; a method for AI-generated translations; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI-generated explanations of questions during menu consideration; a method for making recommendations based on preferences; a method for recognizing product characteristics and features using an image recognition model; and a method for generating product descriptions based on recognized features using a natural language generation model. This enables efficient menu creation and automatic generation of attractive product descriptions, facilitating multilingual support, improved customer experience, and personalized recommendations.

"A method for creating menus using a smartphone" refers to a method for efficiently creating restaurant menus using a smartphone's camera and applications.

"AI methods for generating product descriptions from smartphone photos" refers to artificial intelligence methods that analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"A method for generating translations using AI" refers to a method of translating product descriptions and menu contents into multiple languages using artificial intelligence.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"An AI-powered solution for questions during menu consideration" refers to a method where artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending based on preferences" refers to a method of recommending appropriate products or menu items based on a customer's past choices and preferences.

"A means of recognizing the characteristics and features of a product using an image recognition model" refers to a method of automatically recognizing the characteristics and features of a product from a photograph using image recognition technology.

"A means of generating product descriptions based on features recognized using a natural language generation model" refers to a method of automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

The system for implementing this invention includes a means for creating menus using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for AI to generate translations, a means for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register, a means for AI to explain questions when considering menus, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on recognized features using a natural language generation model.

System Program

The server implements the system using the following hardware and software.

Hardware:

Smartphone (with camera)

Server (for hosting AI models)

Software:

TensorFlow: A library for image recognition

OpenAI GPT-3: API for natural language generation

Explanation of the process

Image recognition:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of the food from the image. For example, features such as "chicken curry, spicy, tomato-based" might be recognized.

Natural language generation:

The server converts the recognized features into a string and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is: "Generate an appealing product description for a dish with the following features: Chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it might generate a description such as: "This spicy chicken curry features juicy chicken simmered in a tomato-based sauce. Aromatic spices will whet your appetite."

Translation work:

The generated product descriptions are translated into multiple languages as needed. The server uses AI to perform translations into multiple languages, such as English and Chinese.

Improving the customer experience:

Customers can use their smartphones to select menu items and place orders in-store. This reduces waiting times at the counter and cashier, enabling smoother service.

Explanation of the question and recommendations:

The AI automatically provides answers to questions customers may have when considering menu options. It also recommends appropriate products and menu items based on the customer's past choices and preferences.

In this way, menu creation becomes more efficient, attractive product descriptions are automatically generated, and multilingual support, improved customer experience, and personalized recommendations become possible.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user takes a photo of the food with their smartphone camera and uploads it to the server via the application. The input is the photo of the food, and the output is the image data sent to the server.

Step 2:

The server uses a TensorFlow ResNet50 model to recognize the characteristics and features of dishes from uploaded image data. The input is image data, and the output is a list of the recognized characteristics and features. Specifically, the image data is preprocessed, input into the model, and prediction results are obtained.

Step 3:

The server converts the recognized characteristics and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of characteristics and features, and the output is a prompt statement. Specifically, it formats the characteristics and features and generates the prompt statement.

Step 4:

The server receives a response from GPT-3 and generates an attractive product description. The input is a prompt, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and retrieves the product description.

Step 5:

The server translates the generated product description into multiple languages as needed. The input is the product description, and the output is the translated product description. Specifically, it calls a translation API to translate into multiple languages.

Step 6:

Users use their smartphones to select menu items and place orders in the store. Input is generated product descriptions, and output is the user's order information. Specifically, the application displays product descriptions, and the user selects their order.

Step 7:

The server automatically provides answers to questions that arise when users consider menu options, using AI. The input is the user's question, and the output is the AI's answer. Specifically, it analyzes the question and generates an appropriate answer.

Step 8:

The server recommends appropriate products and menu items based on the user's past choices and preferences. The input is the user's past selection data, and the output is the recommended products and menu items. Specifically, it analyzes past selection data and uses a recommendation algorithm to suggest products.

(Example 2)

Next, we will describe Embodiment 2 of Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 will be referred to as the "terminal."

Traditional systems often required manual translation of product descriptions into multiple languages, resulting in significant time and effort. Furthermore, services for foreign tourists were inadequate, leading to difficulties in understanding product descriptions. Additionally, insufficient efforts to improve the customer experience and address questions during menu selection could potentially lower customer satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for creating menus using a smart device, means for artificial intelligence (AI) to generate product descriptions from images on the smart device, means for AI to generate translation tasks, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations based on preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into the generation AI model and obtaining translation results, means for saving translation results to a database, means for sending translation results to a terminal, and means for the terminal to display the translation results. This automates the multilingual translation of product descriptions, improves services for foreign tourists, enhances the customer experience, and resolves questions during menu consideration.

A "smart device" is a portable electronic device, such as a smartphone or tablet, that possesses advanced computing power and communication capabilities.

"Menu creation methods" refer to functions and applications used to create menus for restaurants and retail stores using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

"Means of improving the customer experience" refer to features and methods that enhance the convenience and satisfaction customers experience when using products or services.

"A means for AI to explain questions during menu consideration" refers to a function where artificial intelligence automatically provides answers to questions and concerns that arise when customers are considering menu options.

"Preference-based recommendations" refers to a function that recommends appropriate products and services based on a customer's past choices and preferences.

"Methods for translating product descriptions into multiple languages" refers to functions for translating product descriptions into several languages.

A "generative AI model" is an artificial intelligence model trained to perform tasks such as text generation and translation.

A "prompt statement" is an instruction given to a generative AI model to perform a specific task.

"Means for saving translation results to a database" refers to the function of saving translation results generated by a generative AI model to a database.

"Means for sending translation results to the terminal" refers to the function that sends translation results stored in the database to the user's terminal.

"Means for the terminal to display translation results" refers to the function that displays the received translation results on the user's terminal screen.

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translation tasks, means for improving the customer experience, means for artificial intelligence to explain questions when considering menus, means for recommending according to preferences, means for translating product descriptions into multiple languages, means for generating prompt sentences using a generation AI model, means for inputting prompt sentences and product descriptions into a generation AI model and obtaining translation results, means for saving translation results in a database, means for sending translation results to a terminal, and means for the terminal to display translation results.

Hardware and software to be used

Hardware:

Smart devices (smartphones, tablets, etc.)

Server (a computer with high-performance computing capabilities)

Software:

Generative AI models (e.g., OpenAI's GPT-4)

Database Management System

Communication protocol (e.g., HTTP/HTTPS)

Data processing and data calculation

Product description generation:

The user takes a picture of the product using a smart device and sends the image to the server. The server uses artificial intelligence to generate a product description from the image. Specifically, it uses an image analysis algorithm to recognize the product's characteristics and features, and then generates a product description based on that.

Prompt message generation:

The server generates prompt text using a generative AI model. The prompt text specifies the target language for translation (e.g., English, Chinese, Korean).

Translation work execution:

The server inputs the generated prompt text and product description into the AI model and retrieves the translation result. The AI model then performs multilingual translation based on the input text.

Saving and sending translation results:

The server saves the acquired translation results to a database. The saved translation results are sent to the user's device, which then displays the translation.

Specific examples

Specific example:

The user enters the Japanese product description: "This product uses high-quality materials."

The server generates the prompt message: "Translate the following product description into English: This product uses high-quality materials."

The server inputs the prompt text and product description into the generated AI model and retrieves the English translation result: "This product uses high-quality materials."

The server saves the translation results to a database and sends them to the terminal.

The device displays the translation results, and the user confirms them.

This system enables multilingual translation of product descriptions through collaboration between the server, terminal, and user.

The flow of the specific processing in Example 2 will be explained using Figure 13.

Step 1:

The user enters the product description.

The user enters the product description in text format using a smart device. The entered product description is then sent from the smart device to the server.

Input: Product description text

Output: Product description text sent to the server

Step 2:

The server receives the product description.

The server receives the product description text sent by the user and temporarily stores it in memory.

Input: Product description text submitted by the user

Output: Product description text saved in memory

Step 3:

The server generates the prompt message.

The server generates prompt messages based on the specified languages for translation. For example, if translation is required for English, Chinese, and Korean, it will generate prompt messages corresponding to each language.

Input: Product description text, target language for translation

Output: Generated prompt message

Step 4:

The server inputs prompt text and product description into the generated AI model.

The server inputs the generated prompt text and product description text into the AI model. The AI model then performs translation based on the input text.

Input: Prompt message, product description text

Output: Data input to the generative AI model

Step 5:

The server retrieves the translation result.

The server retrieves translation results from the generative AI model. The retrieved translation results are separated by language.

Input: Data entered into the generative AI model

Output: Translated text

Step 6:

The server saves the translation results to the database.

The server saves the retrieved translation results to a database. When saving, it associates the original Japanese product description with the translated result.

Input: Translated text, original product description text

Output: Translation results stored in the database

Step 7:

The server sends the translation result to the terminal.

The server sends the saved translation results to the user's device. The transmitted data is provided in a format accessible to the user.

Input: Translation results stored in the database

Output: Translation result sent to the terminal

Step 8:

The device displays the translation result.

The terminal receives the translation results sent from the server and displays them to the user. The user can review the displayed translation results and make corrections or re-translates as needed.

Input: Translation result sent from the server

Output: Translation result displayed on the terminal

(Application Example 2)

Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the robot 414 as a "terminal."

In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing goods, making it difficult for them to understand product descriptions. Furthermore, visually impaired individuals and those with reading and writing difficulties also have limited means of understanding product descriptions. This can lead to a diminished customer experience and potentially impact sales.

In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, and a means for translating product descriptions into multiple languages and displaying and playing them aloud in the user's set language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired individuals, and people with reading and writing difficulties.

"A method for creating menus using a smartphone" refers to a method for creating menus using a smartphone.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

"Methods for generating translation work using artificial intelligence" refers to methods of translating text into multiple languages using artificial intelligence.

"Methods for improving the customer experience" refer to measures taken to improve the customer's experience when purchasing a product.

"A method for using artificial intelligence to explain questions that arise when customers are considering menu options" refers to a method in which artificial intelligence answers questions that customers may have when considering menu options.

"Methods of recommending based on preferences" refer to methods of recommending products and services based on customer preferences.

"A means of translating product descriptions into multiple languages and displaying and playing audio in the user's preferred language" refers to a means of translating product descriptions into multiple languages and displaying and playing audio in the language specified by the user.

To implement this invention, it is necessary to construct a system that combines a smartphone, artificial intelligence technology, a translation API, and a voice playback function.

First, smartphones have a camera function that allows users to photograph products. When a user photographs a product, the image data is sent to a server. The server then uses image recognition technology to generate a product description. This image recognition technology could include, for example, the Google Cloud Vision API.

Next, the generated product description is translated into multiple languages using artificial intelligence on the server. Translation services such as the Google Cloud Translation API are used for this translation. The translated text is then sent to the user's smartphone and displayed based on their language settings.

Furthermore, the translated product descriptions are played back using the smartphone's audio playback function. This makes the product accessible to visually impaired individuals and those with reading and writing difficulties.

As a concrete example, consider a tourist trying to purchase a leather wallet at a physical store in Japan. When the tourist scans the wallet's tag with their smartphone camera, the app automatically recognizes the product description and translates it into the user's chosen language (e.g., English). The translated product description would be displayed as "This is a high-quality leather wallet," and also played aloud.

Examples of prompt messages include the following:

I am developing an application to translate product descriptions into multiple languages. Please translate the following text into the specified languages.

Text: "This is a high-quality leather wallet."

Target language: Japanese

In this way, foreign tourists, visually impaired individuals, and people with reading and writing difficulties will be able to understand product descriptions without encountering language barriers when purchasing items in physical stores.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user takes a picture of the product using their smartphone camera. The input is the product image taken with the smartphone camera. The output is the captured product image data.

Step 2:

The device sends the captured product image data to the server. The input is the product image data. The output is the product image data sent to the server.

Step 3:

The server generates product descriptions using image recognition technology. The input is product image data. The server extracts text information from the images using APIs such as Google Cloud Vision API and generates the product description. The output is the generated product description text.

Step 4:

The server translates the generated product description text into multiple languages. The input is the product description text. The server uses the Google Cloud Translation API to translate it into the specified languages. The output is the translated product description text.

Step 5:

The server sends the translated product description text to the terminal. The input is the translated product description text. The output is the translated product description text sent to the terminal.

Step 6:

The device displays the translated product description text. The input is the translated product description text. The output is the translated product description displayed on the smartphone screen.

Step 7:

The device plays the translated product description text as audio. The input is the translated product description text. The device uses the smartphone's audio playback function to convert the text to audio and play it. The output is the translated product description played as audio.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 will be referred to as the "terminal."

Traditional menu creation and customer experience improvement systems fail to adequately answer users' questions when considering menu items and to provide product recommendations tailored to individual preferences. Furthermore, they are insufficient in reducing waiting times and providing multilingual services. This can lead to decreased customer satisfaction and negatively impact store sales.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for the user to input information, means for transmitting the input information to the server, means for the server to analyze the input information using a generated AI model, means for generating recommendations and answers based on the analysis results, means for transmitting the generated recommendations and answers to a terminal, and means for the terminal to display the recommendations and answers to the user. This enables appropriate answers to questions the user may have when considering a menu, and product recommendations tailored to individual preferences. Furthermore, it can reduce waiting times and provide multilingual services, leading to improved customer satisfaction and increased store sales.

A "smart device" is an electronic device used by users to input information and communicate with a server, and includes smartphones and tablets.

"Artificial intelligence tools" refer to algorithms and models that analyze input data and generate appropriate responses or recommendations.

A "generative AI model" refers to a machine learning model that analyzes user input information and generates appropriate responses and recommendations.

"Methods for improving the customer experience" refer to measures that reduce waiting times and allow customers to use services comfortably.

"A means for artificial intelligence to explain questions that arise when considering menu options" refers to a method by which artificial intelligence provides appropriate answers to questions that users may have when considering menu options.

"Methods for recommending products based on preferences" refers to methods for recommending appropriate products based on the user's preferences.

"Means of user information input" refers to the interface through which users input information in formats such as text or voice.

"Means for sending inputted information to the server" refers to the communication means used to send information entered by the user to the server.

"Means for a server to analyze input information using a generated AI model" refers to the means by which a server uses a generated AI model to analyze information received from a user.

"Means for generating recommendations and responses based on analysis results" refers to methods for generating recommendations and responses for users based on the analysis results of a generation AI model.

"Means for sending generated recommendations and responses to the device" refers to the communication methods used by the server to send generated recommendations and responses to the user's device.

"Means by which a device displays recommendations and answers to a user" refers to the means by which a user's device displays recommendations and answers received from a server to the user.

This invention is a system in which a user inputs information using a smart device, a server analyzes that information using a generated AI model, and then generates and provides appropriate recommendations and responses to the user. A specific embodiment of this system is described below.

First, the user inputs information into the system using a smart device such as a smartphone or tablet. For example, the user might input "I like spicy food." This information is entered through the smart device's interface.

Next, the smart device sends the entered information to the server. In this process, the smart device converts the input data into an appropriate format (e.g., JSON format) and sends it to the server via the network.

The server analyzes the received user input using a generation AI model (for example, OpenAI's GPT-4). Specifically, it analyzes information such as "I like spicy food" and searches a database related to spicy food.

The server recommends products tailored to the user's preferences based on the analysis results. For example, it generates a list of spicy dishes and presents it to the user. Furthermore, if the user asks, "What are the ingredients in this dish?", the server generates an answer to that question.

The generated recommendations and responses are sent from the server to the smart device. The server converts the data into the appropriate format and sends it to the smart device over the network.

Smart devices display recommendations and answers received from the server to the user. For example, they might display, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot."

As a concrete example, the following prompt message is shown.

"I like spicy food. Could you recommend some dishes?"

"What are the ingredients in this dish?"

In this way, a system is realized in which the server, smart device, and user work together to improve the customer experience. This system allows users to receive appropriate answers to questions when considering menus and product recommendations tailored to their individual preferences. Furthermore, it enables reduced waiting times and the provision of multilingual services, leading to improved customer satisfaction and increased store sales. The flow of specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters information using a terminal.

The user opens the application on their smartphone or tablet, enters "I like spicy food" into the text box, and presses the submit button. The input data is in text format and represents information about the user's preferences.

Step 2:

The terminal sends the entered information to the server.

The terminal converts the user's input, "I like spicy food," into JSON format and sends it to the server using the HTTPS protocol. The input data is user preference information in text format, and the output data is request data in JSON format.

Step 3:

The server analyzes the input information using a generated AI model.

The server parses the received JSON data and inputs the information "likes spicy food" into a generative AI model. The generative AI model (e.g., GPT-4) searches a database of spicy dishes and generates a list of related dishes. The input data is a request in JSON format, and the output data is the list of dishes resulting from the analysis.

Step 4:

The server generates recommendations and responses based on the analysis results.

The server generates a list of spicy dishes based on the analysis results obtained from the generated AI model. For example, it might list dish names such as "Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is the list of dishes resulting from the analysis, while the output data is a recommendation list presented to the user.

Step 5:

The server sends recommendations and responses generated by the server to the device.

The server converts the generated list of spicy dishes into JSON format and sends it to the terminal using the HTTPS protocol. The input data is a recommendation list, and the output data is a JSON response.

Step 6:

The device displays recommendations and answers to the user.

The terminal parses the received JSON data and displays to the user, "Recommended spicy dishes are Mapo Tofu, Kimchi Stew, and Sichuan-style Mala Hot Pot." The input data is response data in JSON format, and the output data is text information displayed to the user.

(Application Example 3)

Next, we will explain Application Example 3 of Configuration Example 3. In the following explanation, the data processing device 12 will be referred to as a "server," and the robot 414 will be referred to as a "terminal."

Traditional food delivery systems have problems such as difficulty for customers to obtain detailed information when choosing a menu and a lack of personalized recommendations. Furthermore, the lack of a means to provide real-time information on ingredients and nutritional content has been a factor in lowering customer satisfaction.

In Application Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting time, a means for artificial intelligence to explain questions during menu consideration, a means for recommending according to preferences, a means for recommending dishes based on customer preferences, and a means for providing information on the ingredients of the dishes. As a result, customers can obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

A "menu creation method" is a function for generating a list of dishes and drinks that customers can choose from.

"An artificial intelligence method for generating product descriptions from images" refers to a function that analyzes images taken with a smart device and automatically generates product descriptions based on their content.

"A means of generating translation work using artificial intelligence" refers to a function that automatically translates text between different languages.

"Methods for improving the customer experience by reducing waiting times" refer to features that reduce the waiting time customers experience when using a service.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are choosing from a menu.

"A method of recommending based on preferences" refers to a function that recommends appropriate products and services based on the customer's preferences and past choices.

"A method for recommending dishes based on customer preferences" refers to a function that recommends appropriate dishes based on the preference information entered by the customer.

"Means of providing information on the ingredients of a dish" refers to a function that provides information about the ingredients and components contained in a specific dish.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from images on a smart device, means for generating translations using artificial intelligence, means for improving the customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for making recommendations according to preferences, means for recommending dishes based on customer preferences, and means for providing information on the ingredients of dishes.

System Program

This system's program is implemented using Python and the OpenAI API. The server receives customer input information and uses a generative AI model to provide appropriate recommendations and information.

Explanation of the process

The server receives customer preferences and questions sent from smart devices and generates prompts based on this information. These generated prompts are input into a generation AI model via the OpenAI API, which then generates appropriate answers and recommendations. The generated information is then sent back to the smart device and provided to the customer.

The hardware used will be smart devices such as smartphones and tablets. The software used will be Python and the OpenAI API.

Specific examples

For example, if a customer enters "I like spicy food," the server will generate the following prompt:

"Please recommend some dishes for people who like spicy food."

Entering this prompt into the OpenAI API will cause the generating AI model to recommend spicy dishes such as "Mapo Tofu" and "Kimchi Hot Pot."

Furthermore, when a customer asks, "What are the ingredients in the mapo tofu?", the server generates the following prompt:

"What are the ingredients in Mapo Tofu?"

When this prompt is entered into the OpenAI API, the generating AI model will provide ingredient information such as "tofu, ground pork, green onions, garlic, ginger, chili bean paste, sweet bean paste, soy sauce, sake, sugar, and chicken broth."

This allows customers to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the application using a smart device and enters their preferences and answers questions. The entered information is sent to the server in text format. Examples of input include "I like spicy food" and "What are the ingredients in Mapo Tofu?".

Step 2:

The server analyzes the input information received from the user and generates an appropriate prompt. For example, in response to the input "I like spicy food," it generates the prompt "Please recommend some dishes for someone who likes spicy food." This prompt is then input into the generation AI model.

Step 3:

The server sends the generated prompt to the OpenAI API and retrieves appropriate answers and recommendations based on the generated AI model. For example, in response to the prompt "What dishes would you recommend for someone who likes spicy food?", answers such as "Mapo Tofu" or "Kimchi Hot Pot" would be generated.

Step 4:

The server analyzes the responses and recommendations obtained from the generative AI model and converts them into a format for user presentation. For example, it formats responses like "Mapo Tofu" and "Kimchi Hot Pot" into a list.

Step 5:

The server sends formatted responses and recommendations to the smart device. The user can then view this information on the smart device's screen.

Step 6:

Users can select menu items based on the information provided or ask further detailed questions. For example, they might re-enter the question, "What are the ingredients in Mapo Tofu?"

Step 7:

The server receives user input again and repeats the same process. Specifically, it generates a prompt again, inputs it into the generation AI model, retrieves the answer, formats it, and provides it to the user.

This allows users to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

Furthermore, an emotion engine for estimating the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

"Example of Morphology 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and recommends products accordingly. Specifically, when a user is selecting a product, the system recognizes their emotions from their facial expressions and tone of voice, and recommends products that match those emotions. For example, if a user shows a joyful expression, the system recommends products that are likely to make that user happy.

"Example of Morphology 2"

Furthermore, the emotion engine generates product descriptions based on the user's emotions. Specifically, it recognizes the user's emotions and selects words appropriate to those emotions to generate the product description. For example, if the user is feeling down, it will select words to cheer them up and generate a product description. (Example 3)

Furthermore, the emotion engine recognizes the user's emotions and provides services accordingly. Specifically, it recognizes the user's emotions and provides services that match those emotions. For example, if a user is showing anger, it might offer a service such as an apology.

The following describes the processing flow for each example form.

"Example of Morphology 1"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system recommends products that match the recognized emotions.

"Example of Morphology 2"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system selects words that match the recognized emotion and generates a product description.

"Example of Morphology 3"

Step 1: When a user selects a product, the system captures the user's facial expressions and voice tone.

Step 2: The emotion engine recognizes the user's emotions from the captured information.

Step 3: The system provides services that match the recognized emotions.

(Example 1)

Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 will be referred to as the "terminal."

Traditional restaurant menu creation and customer experience improvement have been time-consuming and labor-intensive. In particular, generating product descriptions from photos and providing personalized recommendations consume significant human resources. Furthermore, multilingual services for foreign tourists are inadequate, highlighting the need for improved customer satisfaction. Additionally, the inability to provide product recommendations based on customer emotions can lead to a decline in the quality of the customer experience. An efficient system is needed to address these challenges.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

This invention includes a server comprising: a menu creation method using a smartphone; an artificial intelligence (AI) method for generating product descriptions from smartphone photos; a method for AI to generate translation tasks; a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register; a method for AI to explain questions during menu consideration; a method for recommending items according to preferences; and a method for recognizing user emotions and recommending products based on those emotions. This enables more efficient menu creation, product recommendations tailored to customer preferences, enhanced service for foreign tourists through multilingual support, and product recommendations based on customer emotions.

"A method for creating menus using a smartphone" refers to a method for creating restaurant menus using a smartphone's camera and applications.

"An artificial intelligence method for generating product descriptions from smartphone photos" refers to a method that uses artificial intelligence technology to analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

"Methods for generating translation work using artificial intelligence" refers to methods of translating product descriptions and menu contents into multiple languages using artificial intelligence technology.

"Methods to improve the customer experience by eliminating the need to wait for waitstaff or in line at the register" refers to methods that allow customers to receive service smoothly without having to wait for waitstaff or in line at the register.

"A method for using artificial intelligence to explain questions during menu consideration" refers to a method in which artificial intelligence automatically provides answers to questions that arise when customers are considering menu options.

"A method of recommending products based on preferences" refers to a method of recommending appropriate products based on a customer's past choices and preferences.

"A method for recognizing user emotions and recommending products based on those emotions" refers to a method of recognizing emotions by analyzing the user's facial expressions and tone of voice, and then recommending products that match those emotions.

This invention is a system that allows restaurant operators to create menus using smartphones and improve the customer experience. A specific embodiment of this system is described below.

First, the user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they would launch the smartphone's camera app, frame the dessert, and press the shutter button.

Next, the device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in an appropriate format (e.g., JPEG, PNG).

The server analyzes the received photos using image recognition software (e.g., Google Cloud Vision API). The server recognizes the characteristics of the product from the photo (e.g., chocolate cake, fruit parfait). For example, the Google Cloud Vision API might return the label "chocolate cake."

Subsequently, the server generates a product description based on the recognized characteristics. Using a generation AI model (e.g., OpenAI GPT-3), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

The generated product descriptions are displayed on the restaurant's website or application by the server. Specifically, the product descriptions are sent to the front-end in HTML or JSON format and displayed in the user interface.

Furthermore, when a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, the smartphone's camera captures the user's smile, and the microphone records the user's voice tone.

The server analyzes the captured data using an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait."

As a concrete example, consider a scenario where a user takes a photo of a new dessert with their smartphone and uploads it to the system. The server uses the Google Cloud Vision API to analyze the photo and recognize its characteristic as "chocolate cake." The AI then generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture."

Furthermore, if a user displays an expression of joy while viewing the menu using their smartphone camera, the Microsoft Azure Emotion API recognizes that expression. The server then recommends a "fruit parfait," which is likely to bring joy to the user.

Example of a prompt message:

"Please upload photos of the product taken with your smartphone. Our AI will automatically recognize the product's characteristics and generate a product description. It will also use your camera and microphone to recommend products based on your emotions."

In this way, a system is realized in which servers, terminals, and users work together to efficiently create restaurant menus and recommend products to customers.

The flow of the specific processing in Example 1 will be explained using Figure 17.

Step 1:

The user takes a photo of the product.

The user takes a picture of the product using their smartphone camera. For example, to take a picture of a new dessert, they launch the smartphone's camera app, frame the dessert, and press the shutter button. The input is the photograph taken, and the output is the image data stored on the smartphone.

Step 2:

The device uploads the photos to the server.

The device uploads the captured photos to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. The photo data is sent in an appropriate format (e.g., JPEG, PNG). The input is the image data on the smartphone, and the output is the image data sent to the server.

Step 3:

The server analyzes the photo and recognizes the product's characteristics.

The server analyzes the received photograph using image recognition software (e.g., an image recognition API). The server recognizes the characteristics of the product from the photograph (e.g., chocolate cake, fruit parfait). For example, the image recognition API might return the label "chocolate cake." The input is the image data sent to the server, and the output is the recognized product characteristic information.

Step 4:

The server generates the product description.

The server generates a product description based on the recognized characteristics. Using a generative AI model (e.g., a generative AI model), it generates a product description such as, "A rich chocolate cake. It's not too sweet and has a moist texture." The input is product characteristic information, and the output is the generated product description.

Step 5:

The server displays the product description on the website or application.

The server displays the generated product descriptions on the restaurant's website or application. Specifically, it sends the product descriptions in HTML or JSON format to the front-end and displays them in the user interface. The input is the generated product description, and the output is the product description displayed on the website or application.

Step 6:

When a user selects a product, the device captures the user's emotions.

When a user selects a product, the device uses its camera and microphone to capture the user's facial expressions and voice tone. For example, a smartphone's camera captures the user's smile, and the microphone records the user's voice tone. The input is the user's facial expressions and voice tone, and the output is the captured emotional data.

Step 7:

The server analyzes emotions and recommends appropriate products.

The server analyzes the captured data using an emotion engine (e.g., an emotion analysis API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that are suitable for the user. For example, if the user shows a joyful expression, the server will recommend a "fruit parfait." The input is the captured emotion data, and the output is the recommended product information.

(Application Example 1)

Next, we will describe Application Example 1 of Configuration Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 as the "terminal."

Traditional restaurants faced significant challenges in menu creation and improving the customer experience, requiring considerable time and effort. Specifically, there were multiple obstacles, including menu creation, product description generation, personalized recommendations, and multilingual support. Furthermore, providing service that resonated with customer emotions was difficult, making improving customer satisfaction a challenge.

In Application Example 1, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on the smart device, a means for the AI to generate translations, a means for improving the customer experience, a means for the AI to explain questions during menu consideration, a means for recommending according to preferences, and a means for recognizing customer emotions and recommending products based on those emotions. This enables efficient menu creation, automatic generation of product descriptions, multilingual support, improved customer experience, and emotion-based recommendations.

A "smart device" is a portable electronic device with advanced functions, such as a smartphone or smart glasses.

"Menu creation methods" refer to functions and methods for creating restaurant menus using smart devices.

"AI methods for generating product descriptions" refers to artificial intelligence technology that automatically recognizes the characteristics and features of a product from a photograph taken with a smart device and generates a product description based on that recognition.

"A means of generating translation work using AI" refers to a function that automatically performs multilingual translation using artificial intelligence.

"Methods for improving the customer experience" refer to methods and functions that enhance the customer experience, such as eliminating the need to wait in line with waitstaff or at the cash register.

"An AI-powered solution for questions during menu consideration" refers to a function where artificial intelligence automatically explains questions that arise when customers are considering menu options.

"A method for recommending products based on preferences" refers to a function that recommends appropriate products based on the customer's preferences.

"A means of recognizing customer emotions and recommending products based on those emotions" refers to a function that recognizes customer emotions from facial expressions, tone of voice, etc., and recommends products that match those emotions.

The system for carrying out this invention includes means for creating menus using a smart device, means for generating product descriptions from photos on a smart device using AI, means for generating translations using AI, means for improving the customer experience, means for AI to explain questions during menu consideration, means for making recommendations according to preferences, and means for recognizing customer emotions and recommending products based on those emotions.

Program Processing Description

Hardware

The server will be equipped with a high-performance GPU. Smart devices such as smartphones and smart glasses will be used as terminals.

Software

The server uses the following software:

OpenCV: Image processing library

Keras: A deep learning library

Transformers: Generative AI models (such as GPT-3)

Data processing and calculations

1. Menu creation method:

Take a photo of the food using the device's camera and upload it to the server.

The server uses OpenCV to preprocess images and Keras to recognize product characteristics and features from the images.

Based on the recognition results, product descriptions are generated using Transformers.

2. Methods for AI to generate translation work:

The server uses a generative AI model to translate the generated product descriptions into multiple languages.

In particular, we will implement multilingual support to enhance services for foreign tourists.

3. Means of improving the customer experience:

The terminal allows customers to complete ordering and payment on their smart devices, eliminating the need to wait in line with waitstaff or at the cash register.

4. How AI can explain questions that arise when considering menu options:

The terminal sends questions that arise when customers are considering the menu to the server, and the server automatically generates explanations using AI.

5. Methods for recommending based on preferences:

The server recommends appropriate products based on the customer's past order history and preferences.

6. Means for recognizing customer emotions and recommending products based on those emotions:

The server analyzes the customer's facial expressions and voice tone through the device's camera, and uses Keras to recognize their emotions.

Based on recognized emotions, Transformers are used to recommend appropriate products.

Specific examples and prompt messages

Specific example:

When a customer is wearing smart glasses and looking at a menu, the smart glasses analyze the customer's facial expression and display a message saying, "You seem happy, so we recommend dessert!"

Example prompts for a generative AI model:

"This dish looks like delicious pasta. Please generate a detailed product description."

Thus, the embodiment of this invention combines smart devices and AI technology to enable menu creation and improve the customer experience in restaurants.

The flow of the specific processing in Application Example 1 will be explained using Figure 18.

Step 1:

The user takes a photo of the food using a smart device (smartphone or smart glasses). The photo is saved on the device.

Step 2:

The device uploads the captured photos to the server. The server performs image preprocessing using OpenCV. Specifically, it resizes and denoises the images.

Step 3:

The server analyzes pre-processed images using Keras to recognize the characteristics and features of the product. The input is the pre-processed image, and the output is data indicating the product's characteristics and features.

Step 4:

The server generates product descriptions using Transformers based on recognized characteristics and features. The input is data describing the product's characteristics and features, and the output is the generated product description.

Step 5:

The server uses a generative AI model to translate the generated product descriptions into multiple languages. The input is the generated product description, and the output is the translated product description in multiple languages.

Step 6:

The terminal allows customers to complete their orders and payments on their smart devices, eliminating the need to wait in line with waiters or at the cash register. Input is the customer's order information, and output is a notification that the order has been completed.

Step 7:

The terminal sends questions that arise when customers consider the menu to the server. The server automatically generates explanations using AI. The input is the customer's question, and the output is the generated explanation.

Step 8:

The server recommends appropriate products based on the customer's past order history and preferences. The input is the customer's order history and preference data, and the output is the recommended products.

Step 9:

The system analyzes the customer's facial expressions and voice tone through the terminal's camera, and the server uses Keras to recognize their emotions. The input is data on the customer's facial expressions and voice tone, and the output is the recognized emotion.

Step 10:

The server uses Transformers to recommend appropriate products based on the recognized emotions. The input is the recognized emotions, and the output is the recommended products.

(Example 2)

Next, we will describe Embodiment 2 of Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 will be referred to as the "terminal."

Modern consumers demand quick and accurate understanding of product descriptions, as well as multilingual information. They also require personalized product descriptions that cater to their emotions. However, traditional systems struggle to meet these demands, making the improvement of services, particularly for foreign tourists, a significant challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for having artificial intelligence explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This enables users to quickly and accurately understand product descriptions and provides information in multiple languages. Furthermore, by providing personalized product descriptions tailored to user emotions, customer satisfaction can be improved.

A "smart device" is a portable electronic device that has internet connectivity and can run applications.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Means of improving the customer experience" refer to features that enhance the convenience and satisfaction customers experience when using a service.

"A means for artificial intelligence to explain questions during menu consideration" refers to a function that allows artificial intelligence to provide appropriate answers to questions that arise when customers are considering menu options.

"Preference-based recommendation mechanisms" refer to features that recommend appropriate products and services based on a customer's past choices and preferences.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions from their input and actions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to a function for translating generated product descriptions into multiple languages.

Modes for carrying out the invention

This invention is a system that includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for generating translations using artificial intelligence, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

Hardware and software to be used

The server generates product descriptions using a generative AI model (e.g., OpenAI's GPT-4). The generated product descriptions are translated into multiple languages using a translation engine (e.g., Google Translate API). To recognize user emotions, an emotion engine (e.g., IBM Watson's Sentiment Analysis API) is used. Smart devices are portable electronic devices with internet connectivity capable of running applications.

Explanation of the program's processing

The server receives text entered by the user through the terminal and generates a product description using a generative AI model. For example, if the user enters "This product is high quality and long-lasting," the server will generate a product description based on this text.

The generated product descriptions are translated into multiple languages using a translation engine. For example, they are translated into major tourist languages such as English, Chinese, and Korean. Specifically, a description like "This product is high quality and long-lasting" is translated.

When a user inputs emotional text or audio data through their device, the server uses an emotion engine to recognize the user's emotions. For example, if a user inputs "I'm feeling down today," the server analyzes this text and recognizes the user's emotion as "down."

The server uses a generative AI model to generate product descriptions that respond to the user's emotions, based on the emotions it recognizes. For example, if the user is feeling down, the server will select uplifting words to generate a product description.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Examples of specific cases and prompt messages

As a concrete example, consider the following scenario.

Scenario 1: Multilingual Translation

If a user enters "This product is high quality and long-lasting" in Japanese, the server will translate this product description into English, Chinese, and Korean.

Scenario 2: Product Description Based on Emotions

If a user enters "I'm feeling down today," the server uses an emotion engine to recognize the user's emotions and generates a product description that will cheer them up.

Original description: "This product is high quality and long-lasting."

Description tailored to the emotion: "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

Example of a prompt message

Examples of prompts for a generative AI model are as follows:

Please translate this Japanese product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'

"Please generate a product description that will cheer up a user who is feeling down. The original description is 'This product is high quality and long-lasting.'"

The above describes the embodiments for carrying out this invention.

The flow of the specific processing in Example 2 will be explained using Figure 19.

Step 1:

The user uses a terminal to input the text that will form the basis of the product description. For example, the user might input, "This product is high quality and long-lasting." The entered text is then sent to the server.

Step 2:

The server uses a generative AI model to generate a product description based on the text entered by the user. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description based on the user's input: 'This product is high quality and long-lasting.'" The generative AI model generates a product description based on this prompt and returns it to the server. The output is the generated product description.

Step 3:

The server sends the generated product description to the translation engine for translation into multiple languages. Specifically, the server sends the following prompt to the translation engine: "Translate this product description into English, Chinese, and Korean: 'This product is high quality and long-lasting.'" The translation engine performs the translation based on this prompt and returns it to the server. The output is the product description translated into multiple languages.

Step 4:

The user inputs text or audio data related to their emotions through their device. For example, the user might input, "I'm feeling down today." The input text or audio data is then sent to the server.

Step 5:

The server uses an emotion engine to recognize the user's emotions. Specifically, the server sends the following prompt to the emotion engine: "Analyze the user's input 'I'm feeling down today' and recognize the emotion." Based on this prompt, the emotion engine analyzes the emotion and returns the result to the server. The output is the recognized emotion.

Step 6:

The server uses a generative AI model to generate a product description that responds to the user's emotions, based on the emotions it has perceived. Specifically, the server sends the following prompt to the generative AI model: "Generate a product description that will cheer up the user if they are feeling down. The original description is 'This product is high quality and long-lasting.'" Based on this prompt, the generative AI model generates an emotion-responsive product description and returns it to the server. The output is the emotion-responsive product description.

Step 7:

The server sends a product description to the user's device that corresponds to the generated emotion. The user can then view the uplifting product description through their device. Specifically, if the user enters "I'm feeling down today," the server will provide the user with a product description such as, "This product is high quality and will brighten your everyday life. It's durable, so you can use it for a long time!"

The above outlines the specific processing flow of this system's program.

(Application Example 2)

Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the robot 414 will be referred to as a "terminal."

Traditional e-commerce sites often provide product descriptions in a single language, which posed a problem for foreign tourists and multilingual users, making them difficult to understand. Furthermore, the lack of emotionally resonant product descriptions made it difficult to increase user purchase intent. Additionally, users sometimes spent a significant amount of time trying to understand product descriptions, resulting in a less-than-ideal purchasing experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

This invention includes a server comprising: a menu creation means using a smart device; an artificial intelligence means for generating product descriptions from images on the smart device; a means for generating translation tasks using artificial intelligence; a means for improving the customer experience; a means for artificial intelligence to explain questions during menu consideration; a means for making recommendations based on preferences; a means for recognizing user emotions and generating product descriptions corresponding to those emotions; and a means for translating the generated product descriptions into multiple languages. This makes it possible to provide product descriptions tailored to user emotions in multiple languages, making them easily understandable to foreign tourists and multilingual users. Furthermore, it can increase user purchasing intent and improve the purchasing experience.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

"Menu creation methods" refer to functions or software that allow users to create menus for products and services using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"A means of generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

"Customer experience improvement measures" refer to features and software designed to enhance the customer experience when using a product or service.

"A means for AI to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate explanations for questions that arise when users are considering menu options.

"Preference-based recommendation" refers to a function that recommends appropriate products and services based on the user's preferences and past behavior.

"A means of recognizing user emotions and generating product descriptions that correspond to those emotions" refers to artificial intelligence technology that analyzes user emotions and generates product descriptions appropriate to those emotions.

"Means for translating generated product descriptions into multiple languages" refers to functions or software for translating generated product descriptions into multiple languages.

This invention will now describe embodiments for carrying out this invention. This system includes means for creating menus using a smart device, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving the customer experience, means for artificial intelligence to explain questions during menu consideration, means for making recommendations based on preferences, means for recognizing user emotions and generating product descriptions corresponding to those emotions, and means for translating the generated product descriptions into multiple languages.

System Configuration

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones, tablets, and smart glasses.

Server: A high-performance computer responsible for data processing and storage.

Software

Artificial intelligence model: Generates product descriptions using the OpenAI API.

Translation software: Multilingual translation is performed using the Google Translate API.

Emotion recognition software: Uses EmotionRecognizer to recognize the user's emotions.

Data processing and data calculation

1. Image Analysis: Images of products taken with a smart device are sent to the server. The server uses an image analysis algorithm to extract product characteristics and features and generate a product description.

2. Emotion Recognition: The smart device's camera and microphone are used to analyze the user's facial expressions and voice tone. EmotionRecognizer software recognizes the user's emotions and sends the results to the server.

3. Product Description Generation: The server generates prompt text based on the user's emotions and uses the OpenAI API to generate a product description appropriate to those emotions.

4. Multilingual Translation: The generated product description is translated into multiple languages using the Google Translate API. The translation results support major languages such as English, Chinese, and Korean.

Specific examples

For example, if a user is feeling down, the smart device's camera captures the user's facial expression, and the EmotionRecognizer recognizes it as "downcast." The server then generates the following prompt:

"Generate a product description that will cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time."

This prompt is sent to the OpenAI API to generate a sentiment-appropriate product description. The generated product description will look like this:

"This product was created to brighten your life, even just a little. It's made from high-quality materials and is built to last."

Next, we will translate this product description into multiple languages using the Google Translate API.

In this way, it becomes possible to provide product descriptions in multiple languages that cater to the user's emotions.

The flow of the specific processing in Application Example 2 will be explained using Figure 20.

Step 1:

The user takes a photo of the product with their smart device.

Input: Product image

Output: Captured image data

Specific operation: The user uses a smart device such as a smartphone or tablet to take a picture of the product they are considering purchasing. This image data is saved on the smart device.

Step 2:

The device sends the captured image to the server.

Input: Captured image data

Output: Image data sent to the server

Specific operation: The smart device transmits the captured image data to the server via the internet. The HTTP protocol and similar protocols are used for transmission.

Step 3:

The server performs image analysis and extracts the product's characteristics and features.

Input: Sent image data

Output: Characteristics and features of the extracted products

Specific operation: The server uses image analysis algorithms to automatically extract product characteristics and features from image data. For example, it recognizes the product's color, shape, brand logo, etc.

Step 4:

The server generates a product description based on the extracted characteristics and features.

Input: Characteristics and features of the extracted products

Output: Generated product description

Specific operation: The server uses a generative AI model to generate product descriptions based on extracted characteristics and features. For example, a description such as "This product is made from high-quality materials and will last a long time" might be generated.

Step 5:

The device recognizes the user's emotions.

Input: User's facial expressions and tone of voice

Output: Recognized user emotions

Specific operation: Using the camera and microphone of a smart device, the EmotionRecognizer software captures the user's facial expressions and voice tone, recognizing the user's emotions. For example, emotions such as "depressed" or "happy" can be recognized.

Step 6:

The server generates prompt messages that respond to the user's emotions.

Input: Recognized user sentiment, generated product description

Output: Prompt message based on emotion

Specific operation: The server generates prompt messages to send to the AI model based on the recognized user's emotions. For example, a prompt message like, "Generate a product description to cheer up a user who is feeling down: This product is made from high-quality materials and will last a long time," might be generated.

Step 7:

The server uses an AI model to generate product descriptions that are appropriate for the user's emotions.

Input: Prompt text that responds to emotion

Output: Product description suitable for emotions

Specific operation: The server uses a generative AI model (e.g., OpenAI's API) to generate emotionally appropriate product descriptions based on the prompt text. For example, a description like, "This product was created to brighten your life a little. It is made from high-quality materials and is durable," might be generated.

Step 8:

The server translates the generated product description into multiple languages.

Input: Generated product description

Output: Product description translated into multiple languages

Specific operation: The server uses the Google Translate API to translate the generated product description into multiple languages, such as English, Chinese, and Korean. For example, a translation might be: "This product is made to brighten your life a little. It is made of high-quality materials and lasts long."

Step 9:

The server sends the product description, translated into multiple languages, to the device.

Input: Product description translated into multiple languages

Output: Translation result sent to the terminal

Specific operation: The server sends product descriptions translated into multiple languages to smart devices via the internet. HTTP protocol and similar protocols are used for transmission.

Step 10:

The device displays product descriptions translated into multiple languages to the user.

Input: Translation result sent to the device

Output: Multilingual product description displayed to the user

Specific operation: The smart device displays the product description, translated into multiple languages, to the user. This allows the user to understand the emotionally relevant product description in multiple languages.

(Example 3)

Next, we will describe Embodiment 3 of Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the robot 414 will be referred to as the "terminal."

Traditional customer experience improvement systems often rely on manual processes for menu creation, product descriptions, and translation, resulting in inefficiency. Furthermore, they may fail to adequately address customer questions and provide personalized recommendations, potentially leading to decreased customer satisfaction. Additionally, the lack of features to recognize and respond to customer emotions hinders the improvement of the overall customer experience.

In Example 3, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and a means for recognizing user emotions and providing services corresponding to those emotions. This enables quick answers to customer questions when considering menus, product recommendations tailored to preferences, and significantly improves customer satisfaction by providing services that respond to customer emotions.

A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

"Menu creation method" refers to a function that allows restaurants and service businesses to create menus using smart devices.

"An artificial intelligence method for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with smart devices and automatically generates product descriptions based on those images.

"Methods for generating translation work using artificial intelligence" refers to functions that automatically translate text and audio using artificial intelligence.

"Methods for improving customer experience by reducing waiting times" refer to functions that shorten the waiting time customers experience when using a service, thereby providing a smoother experience.

"A means for artificial intelligence to explain questions when considering menu options" refers to a function where artificial intelligence provides appropriate answers to questions that arise when customers are considering menu options.

"A method for recommending products based on preferences" refers to a function where artificial intelligence recommends appropriate products based on the customer's preferences and past choices.

"A means of recognizing user emotions and providing services that respond to those emotions" refers to a function that recognizes user emotions from their facial expressions and voice, and provides appropriate services that correspond to those emotions.

This invention is a system for improving customer experience and includes means for creating menus using smart devices, artificial intelligence means for generating product descriptions from images, means for artificial intelligence to generate translations, means for improving customer experience by reducing waiting times, means for artificial intelligence to explain questions during menu consideration, means for recommending products according to preferences, and means for recognizing user emotions and providing services that respond to those emotions.

Hardware and software to be used

Hardware

Smart devices: Portable electronic devices with internet connectivity, such as smartphones and tablets.

Server: A high-performance server (e.g., a server with an NVIDIA GPU).

Software

Generative AI model: GPT-4 is used as an example.

Emotion recognition engine: Affectiva SDK is used as an example.

Program Processing Description

The server receives information sent from the smart device and performs analysis using a generative AI model. For example, if the user inputs "I like spicy food," the generative AI model generates a list of spicy dishes, and the server sends this list to the smart device. The smart device then displays the received list to the user.

Furthermore, when a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to generate ingredient information and sends it to the smart device. The smart device then displays the received ingredient information to the user.

Furthermore, smart devices use their built-in cameras and microphones to capture the user's facial expressions and voice, and use an emotion recognition engine to recognize the user's emotions. For example, if the user shows an angry expression, the smart device sends that information to a server. The server uses a generative AI model to generate an apology and sends it to the smart device. The smart device then displays the received apology to the user.

Examples of specific cases and prompt messages

Example 1: User enters "I like spicy food."

The user opens the app on their smartphone and enters "I like spicy food."

The server generates a list of spicy dishes using an AI model.

The server generates a list and sends it to the smart device.

A smart device displays a list of spicy dishes to the user.

Example prompt: "Please list dishes you would recommend to a customer who likes spicy food."

Example 2: A user asks, "What are the ingredients in this dish?"

The user opens the tablet app and asks, "What are the ingredients in this dish?"

The server generates ingredient information using an AI model.

The server generates ingredient information and sends it to the smart device.

The smart device displays ingredient information to the user.

Example prompt: "Please tell me the ingredients for this dish."

Example 3: The user shows an angry expression.

The user displays an angry expression while using their smartphone.

Smart devices use cameras to capture the user's facial expressions, which are then analyzed by an emotion recognition engine.

The smart device sends information about the anger it recognizes to the server.

The server generates an apology using an AI model.

The server generates an apology message and sends it to the smart device.

Smart devices display an apology message to the user.

Example prompt: "Generate an apology for when the user is angry."

In this way, the server, smart devices, and users work together to operate the system and improve the customer experience. The flow of a specific process in Example 3 will be explained using Figure 21.

Step 1:

The user inputs information through their device. The user uses a smartphone or tablet to input questions and preferences into the system. For example, they might input "I like spicy food." The input data is saved on the device in text format.

Step 2:

The terminal sends the input information to the server. The terminal sends the user-entered information to the server via the internet. The input data is sent to the server in text format.

Step 3:

The server analyzes the input information using a generative AI model. The server passes the received input information to the generative AI model (e.g., GPT-4) for analysis. The generative AI model generates appropriate responses or recommendations based on the input information. For example, given the information "likes spicy food," it generates a list of spicy dishes. The input data is in text format, and the output data is in list format.

Step 4:

The server generates appropriate answers and recommendations based on the analysis results. The server generates answers and recommendations for the user based on the analysis results obtained from the generating AI model. For example, it might generate a list of spicy dishes. The output data is in list format.

Step 5:

The server sends the generated responses and recommendations to the device. The server sends the generated responses and recommendations to the device via the internet. The output data is sent to the device in list format.

Step 6:

The device displays answers and recommendations to the user. The device displays answers and recommendations received from the server to the user. The user can review the displayed information. The output data is displayed in list format.

Step 7:

The device recognizes the user's emotions using an emotion engine. The device captures the user's facial expressions and voice using its built-in camera and microphone, and recognizes the user's emotions using an emotion engine (e.g., Affectiva SDK). Input data is in image or audio format, and output data is in emotion information format.

Step 8:

The device transmits recognized emotion information to the server. The device transmits the recognized emotion information to the server via the internet. Input data is sent to the server in emotion information format, and output data is also sent in emotion information format.

Step 9:

The server generates appropriate services based on emotional information. The server uses a generative AI model to generate appropriate services based on the received emotional information. For example, if the user shows an angry expression, it will generate an apology. Input data is in emotional information format, and output data is in text format.

Step 10:

The server sends the generated service to the terminal. The server sends the generated service to the terminal via the internet. The output data is sent to the terminal in text format.

Step 11:

The terminal provides services to the user. The terminal delivers services received from the server to the user. For example, it displays an apology. The output data is displayed in text format.

(Application Example 3)

Next, we will explain Application Example 3 of Configuration Example 3. In the following explanation, the data processing device 12 will be referred to as a "server," and the robot 414 will be referred to as a "terminal."

Traditional food delivery services faced challenges such as users spending a significant amount of time choosing from menus and difficulty in providing services tailored to user preferences and emotions. Furthermore, multilingual support for foreign tourists was insufficient, highlighting the need for improved user experience. There is a demand to address these challenges and provide a more comfortable and personalized service.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for the artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for the artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing the user's emotions and providing services corresponding to those emotions. This allows users to efficiently select menu items and receive personalized services tailored to their individual preferences and emotions. Furthermore, multilingual support enhances services for foreign tourists.

A "smart device" is a portable electronic device with advanced computing capabilities, such as a smartphone or tablet.

A "menu creation method" refers to a method or device for generating a list of products or services that a user can select.

"An artificial intelligence method for generating product descriptions from images" refers to an artificial intelligence system that uses image recognition technology to automatically analyze the characteristics and features of a product and generates a product description based on that analysis.

"Means of generating translation work using artificial intelligence" refers to an artificial intelligence system that automatically translates text and audio between different languages.

"Customer experience improvement measures" refer to methods and devices that enhance the convenience and satisfaction users experience when using a service.

"A system where artificial intelligence explains questions that arise when considering menu options" refers to a system where artificial intelligence provides appropriate answers to questions that users may have when choosing from a menu.

A "method for recommending products based on preferences" is a system that recommends appropriate products and services based on the user's past choices and input information.

An "emotion recognition system" is a system that analyzes a user's emotions from their facial expressions, tone of voice, etc., and responds accordingly.

The system for carrying out this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for artificial intelligence to generate translation tasks, a means for improving the customer experience by reducing waiting times, a means for artificial intelligence to explain questions during menu consideration, a means for recommending products according to preferences, and an emotion recognition means for recognizing user emotions and providing services corresponding to those emotions.

System Program

The program for this system performs the following operations:

Hardware and software

Hardware:

Smart devices (smartphones, tablets, etc.)

Camera (to capture the user's facial expressions)

Computer (a processing unit for executing programs)

Software:

OpenCV (image processing library)

Keras (deep learning library)

Transformers (a library of generative AI models)

Data processing and data calculation

1. Menu creation method:

The server generates a list of products and services that users can select using their smart devices.

2. Artificial intelligence methods for generating product descriptions from images:

The server analyzes images captured by the smart device's camera and automatically recognizes the product's characteristics and features.

Based on the recognized information, generate a product description.

3. Methods for generating translation tasks using artificial intelligence:

The server automatically translates text and audio between different languages.

In particular, we will provide multilingual support to enhance services for foreign tourists.

4. Measures to improve customer experience:

The server reduces waiting times for users when accessing the service, improving convenience.

5. How artificial intelligence can explain questions that arise during menu planning:

The server uses a generative AI model to provide appropriate answers to questions that arise when users select menu items.

6. Methods for recommending products based on preferences:

The server recommends appropriate products and services based on the user's past choices and input information.

7. Emotion recognition means:

The server analyzes the user's facial expressions and voice tone captured by the camera to recognize their emotions.

Respond appropriately to the recognized emotions.

Specific examples

When a user enters "I like spicy food," the server uses a generative AI model to recommend spicy dishes.

When a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to provide information about the ingredients.

If the user displays an angry expression, the server uses emotion recognition to apologize with the message, "I'm sorry. Is there a problem?"

Example of a prompt message

"I like spicy food. What do you recommend?"

"What are the ingredients in this dish?"

"How would you respond if a user is showing signs of anger?"

In this way, users can efficiently select menu items and receive personalized service tailored to their individual preferences and feelings. Furthermore, multilingual support enhances services for foreign tourists.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The user opens the menu using a smart device.

Input: A menu display request initiated by the user.

Data Processing: The server retrieves menu information from the database and sends it to the smart device.

Output: A menu will appear on your smart device.

Step 2:

The user takes a photo of the food using the camera on their smart device.

Input: An image of a dish taken by the user.

Data Processing: The server receives images and analyzes them using an image recognition algorithm (OpenCV).

Output: The analysis results extract the characteristics and features of the dishes.

Step 3:

The server generates a product description from the image.

Input: Data on the characteristics and features of the dish.

Data Processing: The server uses a generative AI model to generate product descriptions based on characteristics and features.

Output: Product description text is generated and sent to the smart device.

Step 4:

Users enter questions when considering menu options.

Input: The question entered by the user (e.g., "What are the ingredients in this dish?").

Data Processing: The server generates answers to questions using a generative AI model.

Output: The answer text is generated and displayed on the smart device.

Step 5:

The server recommends products based on the user's preferences.

Input: User's past selections and input information (e.g., "I like spicy food").

Data Processing: The server uses a generative AI model to recommend products based on user preferences.

Output: A list of recommended products is displayed on your smart device.

Step 6:

The user captures their facial expressions using the camera on their smart device.

Input: User's facial expression image.

Data Processing: The server analyzes emotions using a facial recognition algorithm (Keras).

Output: As an analysis result, user sentiment data is generated.

Step 7:

The server provides services tailored to the user's emotions.

Input: User emotion data (e.g., anger).

Data Processing: The server generates appropriate responses based on sentiment data (e.g., apology messages).

Output: The corresponding message will be displayed on the smart device.

Step 8:

The server performs the translation.

Input: Text or audio data entered by the user.

Data Processing: The server uses a translation algorithm to translate the input data into multiple languages.

Output: Translated text and audio data are displayed on the smart device.

The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input regarding the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet Search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives prompts containing instructions, as well as inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions provided by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search engine <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12. However, the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, namely an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Furthermore, the emotions of "pleasure" are located on the upper side of the concentric circles, and emotions of "displeasure" are located on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calmer impression.

The inner part of the Emotion Map 400 represents inner thoughts, while the outer part represents actions. Therefore, the further you go from the outer edge of the Emotion Map 400, the more visible (expressed through actions) your emotions become.

Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort; when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort; when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a Brain Physiological Signal Analysis System for Speech Emotion Recognition and Emotion, Tokushima University, Doctoral Dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map displays emotions belonging to the "response" domain, where sensation is dominant. The right half of the emotion map displays emotions belonging to the "situation" domain, where situational awareness is dominant.

The emotion map defines two emotions that promote learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. This is when the robot experiences negative emotions such as, "I never want to feel this way again," or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. This is when the robot feels positive emotions such as, "I want more," or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassurance," "peace of mind," and "reassurance" have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, involving multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing according to the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, nor is it necessary to store the entirety of the specific processing program 56 in the storage device 32. It is acceptable to store only a portion of the specific processing program 56.

The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

The hardware resource performing a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource performing the specific process may consist of a single processor.

Examples of configurations using a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, which functions as a hardware resource for executing specific processing. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources for executing specific processing, on a single IC chip, as exemplified by SoC (System-on-a-chip). Thus, specific processing is realized using one or more of the above-mentioned types of processors as hardware resources.

Furthermore, the hardware structure of these various processors can, more specifically, utilize electrical circuits combining circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be removed, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and represent only one example of the technology. For example, the above descriptions of the structure, function, operation, and effects are examples of the structure, function, operation, and effects of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technology presented. Furthermore, to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other information that do not require special explanation to enable the implementation of the technology have been omitted from the descriptions and illustrations presented above.

All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

The following is further disclosed regarding the embodiments described above.

(Claim 1)

A system including a method for creating menus using a smartphone, an AI method for generating product descriptions from smartphone photos, an AI method for generating translations, a method for improving the customer experience by eliminating the need to wait for waitstaff or in line at the register, an AI method for explaining questions during menu consideration, and a method for recommending items according to customer preferences.

(Claim 2)

The system according to claim 1, wherein the AI means for generating product descriptions from smartphone photos automatically recognizes the characteristics and features of the product and generates a product description based on that recognition.

(Claim 3)

The system according to claim 1, wherein the means for generating the aforementioned translation work using AI supports multiple languages and particularly enhances services for foreign tourists.

(Claim 4)

The system according to claim 1, further comprising an emotion engine that recognizes the user's emotions.

(Claim 5)

The system according to claim 4, wherein the emotion engine recognizes the user's emotions and recommends products according to those emotions.

(Claim 6)

The system according to claim 4, wherein the emotion engine recognizes the user's emotions and generates a product description in accordance with those emotions.

"Example 1"

(Claim 1)
A method for generating product descriptions from photos taken with a smartphone,
A means for inputting instructions for translating the aforementioned product description into a generating AI model and generating the translation result of the aforementioned product description,
A means for inputting the user's questions when considering menus into the aforementioned AI generation model and generating answers to those questions,
A system including means for analyzing the user's input and generating recommendation information.

(Claim 2)
The system according to claim 1, wherein, when generating the product description, the generating AI model automatically recognizes the characteristics and features of the product from the photograph and generates the product description based on the recognition result.

(Claim 3)
The system according to claim 1, wherein the generating AI model that generates the translation results generates the translation results for each of the multiple languages.

The descriptions of each means in claims 1 to 3 of "Example 1" described above can be similarly applied to the following examples and applications.

"Application Example 1"

(Claim 1)
A method for creating menus using a smartphone,
An AI method for generating product descriptions from smartphone photos,
A method for AI to generate translation tasks,
A means of improving the customer experience by eliminating the need to wait in line with waitstaff or at the cash register,
A method for AI to explain questions that arise when considering menu options,
A method for recommending based on preferences,
A means of recognizing the characteristics and features of a product using an image recognition model,
A means for generating product descriptions based on features recognized using a natural language generation model,
A system that includes this.

(Claim 2)
The system according to claim 1, wherein the AI means for generating product descriptions from photos taken with a smartphone automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which the AI generates the translation work supports multiple languages and particularly enhances services for foreign tourists.

"Example 2"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means to improve the customer experience,
A means of explaining questions that arise when considering menu options using artificial intelligence,
Recommended methods according to your preference,
Methods for translating product descriptions into multiple languages,
A means for generating prompt sentences using a generative AI model,
A method for inputting prompt text and product description into a generative AI model and obtaining translation results,
A means of saving the translation results to a database,
A means of sending the translation result to the terminal,
A system that includes means for the terminal to display the translation result.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Application Example 2"

(Claim 1)
A method for creating menus using a smartphone,
An artificial intelligence method for generating product descriptions from smartphone photos,
Methods for generating translation tasks using artificial intelligence,
Means of improving the customer experience,
A means of explaining questions that arise when considering menu options using artificial intelligence,
A method for recommending based on preferences,
A means of translating product descriptions into multiple languages and displaying and playing audio in the user's preferred language,
A system that includes this.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from photos taken with a smartphone automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Example 3"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means of improving the customer experience to reduce waiting times,
A means of explaining questions that arise when considering menu options using artificial intelligence,
A method for recommending products according to preferences,
The means by which users input information,
A means of sending the input information to the server,
A means for the server to analyze input information using a generated AI model,
A means of generating recommendations and answers based on the analysis results,
A means of sending the generated recommendations and responses to the device,
A system that includes means by which a device displays recommendations or answers to the user.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Application Example 3"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means of improving the customer experience to reduce waiting times,
A means of explaining questions that arise when considering menu options using artificial intelligence,
A method for recommending based on preferences,
A means of recommending dishes based on customer preferences,
A system that includes means for providing information on the ingredients of a dish.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Example 1 of combining an emotion engine"

(Claim 1)
A method for creating menus using a smartphone,
An artificial intelligence method for generating product descriptions from smartphone photos,
Methods for generating translation tasks using artificial intelligence,
A means of improving the customer experience by eliminating the need to wait in line with waitstaff or at the cash register,
A means of explaining questions that arise when considering menu options using artificial intelligence,
A method for recommending based on preferences,
A means of recognizing user emotions and recommending products based on those emotions,
A system that includes this.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from photos taken with a smartphone automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Application Example 1 of Combining Emotional Engines"

(Claim 1)
A method for creating menus using smart devices,
An AI method for generating product descriptions from photos taken with smart devices,
A method for AI to generate translation tasks,
Means of improving the customer experience,
A method for AI to explain questions that arise when considering menu options,
A method for recommending based on preferences,
A means of recognizing customer emotions and recommending products based on those emotions,
A system that includes this.

(Claim 2)
The system according to claim 1, wherein the AI means for generating product descriptions from photographs of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which the AI generates the translation work supports multiple languages and particularly enhances services for foreign tourists.

"Example 2 of combining an emotion engine"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means to improve the customer experience,
A means of explaining questions that arise when considering menu options using artificial intelligence,
Methods of recommendation according to preference,
A means of recognizing user emotions and generating product descriptions that correspond to those emotions,
A system that includes means for translating generated product descriptions into multiple languages.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Application Example 2 of Combining Emotional Engines"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means of improving the customer experience,
A means of explaining questions that arise when considering menu options using artificial intelligence,
Methods of recommendation according to preference,
A means of recognizing user emotions and generating product descriptions that correspond to those emotions,
A means of translating the generated product description into multiple languages,
A system that includes this.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Example 3 of combining an emotion engine"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means to improve the customer experience by reducing waiting times,
A means of explaining questions that arise when considering menu options using artificial intelligence,
A method for recommending products according to preferences,
A system that includes means for recognizing a user's emotions and providing services that respond to those emotions.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

"Application Example 3 of Combining Emotional Engines"

(Claim 1)
A method for creating menus using smart devices,
An artificial intelligence method for generating product descriptions from images on smart devices,
Methods for generating translation tasks using artificial intelligence,
Means to improve the customer experience by reducing waiting times,
A means of explaining questions that arise when considering menu options using artificial intelligence,
A method for recommending products according to preferences,
An emotion recognition means that recognizes the user's emotions and provides services corresponding to those emotions,
A system that includes this.

(Claim 2)
The system according to claim 1, wherein the artificial intelligence means for generating product descriptions from images of the smart device automatically recognizes the characteristics and features of the product and generates a product description based on that.

(Claim 3)
The system according to claim 1, wherein the means by which artificial intelligence generates the aforementioned translation work supports multiple languages and particularly enhances services for foreign tourists.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset terminal 414 Robot

Claims (1)

A means for automatically recognizing the characteristics or features of a product from image data of the product taken with a smartphone by using an image analysis algorithm,
A means for converting the aforementioned characteristics or features into a string and inputting it as a prompt to the generating AI model, thereby causing the generating AI model to generate a product description;
A means for recognizing the user's emotions by analyzing the user's facial expressions or voice data using an emotion engine,
A means for causing the generating AI model to generate a product description corresponding to the emotion by inputting a prompt sentence containing the emotion and the product description text into the generating AI model,
A means for inputting instructions to the generating AI model to translate the product description corresponding to the aforementioned emotion into multiple languages, and for generating the translated result of the product description corresponding to the aforementioned emotion,
A means for transmitting the translated results of the emotion-appropriate product description, translated into multiple languages, to the smartphone,
A means for inputting the user's questions during menu consideration into the generating AI model, thereby causing the generating AI model to generate answers to those questions,
A means for analyzing the user's past selection data using a recommendation algorithm and generating recommendation information,
A system that includes this.