JP7825006B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

Today, consumers need to select and purchase the foods they need based on their health condition and diet goals. However, these choices depend on individual knowledge and experience, making the right choice can often be difficult. Furthermore, purchasing unnecessary products results in unnecessary expenses, and many consumers feel a financial burden.

With this system, consumers enter their height, weight, and goals, and then submit their purchase receipt after shopping. The system then identifies items that it determines are unnecessary for that purchase, and visualizes the money saved and weight loss that would result if the unnecessary items were not purchased. This allows consumers to select and purchase products that are appropriate for their health condition and diet goals, thereby reducing unnecessary spending.

1 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 10 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. FIG. 10 is a sequence diagram showing a processing flow of the data processing system in Application Example 1 of Form Example 1. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a third embodiment of the third form example. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used by the processor as work memory.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a symbol is an interface that includes a communication processor, an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be just A, just B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A, a microphone 38B, etc., and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or finger) to receive user input via the indicator. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 is equipped with a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible by the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"
This embodiment of the present invention provides a dedicated input form as a means for consumers to enter their height, weight, and diet goals. This input form is provided on a website or smartphone application, and allows consumers to directly input their height, weight, diet goals, etc.

"Example 2"
Next, as a means for consumers to submit their purchase receipts after shopping, the system provides a function for them to take an image of the receipt using their smartphone camera and upload it to the application. This function allows consumers to communicate purchase information to the system without having to submit a physical receipt directly.

"Example 3"
Then, as a means of correcting items that are deemed unnecessary for this shopping trip, the system automatically detects unnecessary items based on the consumer's height, weight, and diet goal entered by the consumer, as well as the submitted receipt information. For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary items.

"Example 4"
Finally, to visualize the money saved and weight lost by not purchasing unnecessary products, the system calculates the total cost of the detected unnecessary products and the estimated weight loss if those products were not consumed, and displays this information to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their health management and savings.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Consumers enter their height, weight, diet goals, etc. into a dedicated input form. For example, they might enter a height of 170 cm, a weight of 65 kg, and a diet goal of "restricting carbohydrates."

Step 2: Once the information is complete, the consumer presses the "Submit" button to send the information to the system.

"Example 2"

Step 1: The consumer makes a purchase and obtains a receipt for the items purchased.

Step 2: The consumer takes a picture of the receipt using their smartphone camera.

Step 3: The captured image is uploaded to the application and the system reads the receipt information.

"Example 3"

Step 1: The system automatically detects unwanted items based on the consumer's height, weight, and goals entered, as well as the submitted receipt information.

Step 2: For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary items.

"Example 4"

Step 1: The system calculates the total value of the detected unwanted products and the estimated weight loss that would occur if the product were not consumed.

Step 2: The calculation results are displayed to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their health management and savings.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional diet support systems required users to input their height, weight, and diet goals, as well as submit purchase receipts after shopping and identify unwanted items. However, these systems were time-consuming for users, making them difficult to use on an ongoing basis. Furthermore, they lacked the ability to individually suggest appropriate diet plans based on the data entered by the user, making them unable to provide effective diet support.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for receiving and saving the data entered by the customer, means for generating prompt sentences for the generative AI model based on the saved data, and means for the generative AI model to suggest a diet plan. This allows users to use the system continuously without any hassle, and makes it possible to suggest appropriate diet plans individually.

"Customer" refers to the user of the system.

"Height and Weight" refers to the customer's current height and weight data.

"Goals" refers to the diet and health goals set by the customer.

"Input form" refers to the interface where customers enter data such as height, weight, and goals.

"Purchase receipt" refers to the detailed list of purchases that a customer receives after making a purchase.

"Means for correcting" refers to the function that determines unnecessary products based on the customer's height, weight, and goals, and removes them from the list.

"Visualization means" refers to a function that visually displays the money you can save and the weight you will lose if you don't buy unnecessary products.

"Means for receiving and storing data" refers to the function by which the server receives data entered by the customer and stores it in a database, etc.

"Generative AI model" refers to an artificial intelligence model that generates an appropriate diet plan based on input data.

"Prompt sentence" refers to an instruction sentence generated for a generative AI model based on input data.

"Diet Plan" refers to a specific action plan proposed by the generative AI model to help customers achieve their goals.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is installed on a website or smartphone application, and a generative AI model proposes an appropriate diet plan based on the data entered by the customer.

Hardware and software used

Hardware: Servers (e.g., AWS (registered trademark) EC2), user devices (e.g., smartphones, PCs)

Software: Web servers (e.g., Apache (registered trademark), Nginx), databases (e.g., MySQL (registered trademark), PostgreSQL), smartphone applications (e.g., iOS apps, ANDROID (registered trademark) apps)

Specific system operation

Server

The server provides a dedicated input form on a website or smartphone application. This input form allows customers to enter their height, weight, diet goals, etc. The server receives the input data and stores it in a database. The stored data is used as prompts for the generative AI model.

Device

The device displays the websites and smartphone applications that the user accesses. When the user enters their height, weight, and diet goals into an input form, the device sends the data to a server. The device then displays the data the user has entered in real time, allowing them to check it.

User

Users access a website or smartphone application and enter their height, weight, and diet goals into a dedicated input form. Once the user has completed the input, the data is sent to the server. The user can then review the data they have entered and make any necessary corrections.

Specific examples

Example 1: Website use

A user visits a website and enters the following information into a form:

Height: 170 cm
Weight: 70 kg
Diet goal: Lose 5 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

"The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Based on this information, please suggest an appropriate diet plan."

Example 2: Use in a smartphone application

A user opens a smartphone application and enters the following information into the input form:

Height: 160 cm
Weight: 60 kg
Diet goal: Lose 3 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

"The user's height is 160 cm, weight is 60 kg, and their diet goal is to lose 3 kg. Based on this information, please suggest an appropriate diet plan."

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application and accesses an input form. The input form is an interface where the user can enter their height, weight, and diet goals. Once the input form is displayed, the user is ready to enter the required information.

Step 2:

The user enters the data.

The user enters their height, weight, and diet goal into the input form. For example, the user might enter "Height: 170 cm", "Weight: 70 kg", and "Diet goal: Lose 5 kg". Once the input is complete, the user taps the "Submit" button. The input data is saved on the device in JSON format.

Step 3:

The device sends the data to the server.

The device sends the data entered by the user to the server. Specifically, the device uses an HTTP POST request to send the input data to the server. At this time, the data is sent in JSON format. The input data (height, weight, diet goal) is sent to the server.

Step 4:

The server receives and stores the data.

The server receives the data sent from the device. The received data is stored in a database. For example, the server connects to a MySQL database and executes the following SQL query:

sql

INSERT INTO user_data (height, weight, goal) VALUES (170, 70, '5 kg weight loss');

This process saves the user's input data in the database.

Step 5:

The server generates a prompt for the generative AI model.

The server generates prompts for the generative AI model based on the stored data. For example, it generates prompts like the following:

"The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Based on this information, please suggest an appropriate diet plan."

This prompt is sent to the generative AI model.

Step 6:

Generative AI models suggest diet plans.

The generative AI model generates a diet plan based on the prompt it receives. For example, the generative AI model might suggest the following diet plan:

1. Do 30 minutes of aerobic exercise every day.

2. Limit your daily calorie intake to 1,500 kcal

3. Strength training three times a week

This diet plan is sent to the user's device via the server and displayed to the user.

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional food delivery systems struggled to provide meal plans based on customers' health status and diet goals, making it impossible to provide services tailored to individual needs. Furthermore, problems arose, such as customers buying unnecessary items and making meal choices that hindered the achievement of diet goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height, weight, and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for generating an appropriate meal plan based on the customer's height, weight, and goals, and means for presenting the generated meal plan to the customer. This makes it possible to provide an appropriate meal plan based on the customer's health condition and diet goals, and support the achievement of diet goals while reducing unnecessary spending.

"Customer" refers to the consumer who uses the service.

"Height and Weight" refers to the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that a customer wishes to achieve.

"Purchase receipt" refers to the detailed list of purchases received by a customer when making a purchase.

"Correction means" refers to a method or system for evaluating products purchased by customers and identifying unwanted products.

"Visualization means" refers to a method or system that visually displays the money saved or weight lost.

"Meal Plan" refers to the meal content and schedule suggested based on the client's health and diet goals.

"Means for generating" refers to a method or system for creating a meal plan based on customer input data.

"Presentation means" refers to the method or system for showing the generated meal plan to the customer.

The following system configuration is proposed as an embodiment of this invention.

System Configuration

The system includes a smartphone application used by customers, a server, and a generative AI model. The smartphone application provides an interface for customers to input their height, weight, and diet goals. The server receives the input data and sends prompts to the generative AI model to generate an appropriate meal plan. The generated meal plan is then presented to the customer again via the smartphone application.

Hardware and software used

Hardware: Smartphones, servers

Software: Smartphone applications, generative AI models (e.g., OpenAI (registered trademark) GPT-3 (registered trademark))

Data processing and calculation

1. Data entry: Customers use a smartphone application to enter their height, weight, and diet goals.

2. Data transmission: The smartphone application sends the entered data to the server.

3. Prompt generation: Based on the data received by the server, a prompt sentence is generated to be sent to the generative AI model.

4. Meal plan generation: The generative AI model receives the prompt and generates an appropriate meal plan.

5. Data reception and presentation: The server receives the generated meal plan and presents it to the customer via the smartphone application.

Specific examples

For example, if a customer enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg," the server will send the following prompt text to the generative AI model:

Example prompt

"Please suggest an appropriate meal plan based on the following: height: 170 cm, weight: 70 kg, goal: to lose 5 kg."

Based on this prompt, the generative AI model generates a balanced meal plan. For example, it suggests specific meal options such as "Breakfast - oatmeal, Lunch - salad, Dinner - chicken breast and vegetables." This meal plan is presented to the customer via a smartphone application.

In this way, we can provide customers with appropriate meal plans based on their health condition and diet goals, helping them achieve their diet goals while reducing unnecessary spending.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goal. The entered data is temporarily saved within the smartphone application. An example of input data is: height 170cm, weight 70kg, diet goal "lose 5kg."

Step 2:

The smartphone application sends the entered data to the server. The data sent includes the user's height, weight, and diet goals. The server analyzes the received data and proceeds to the next processing step.

Step 3:

The server generates a prompt based on the received data. Specifically, it combines the user's height, weight, and diet goal to create a prompt to send to the generative AI model. For example, a prompt might be generated that reads, "Based on height: 170 cm, weight: 70 kg, goal: to lose 5 kg, please suggest an appropriate meal plan."

Step 4:

The server sends the generated prompt text to the generative AI model. The generative AI model performs data calculations based on the received prompt text and generates an appropriate meal plan. The generative AI model uses, for example, OpenAI GPT-3.

Step 5:

The generative AI model generates a meal plan based on the prompt and sends the results back to the server. An example of a generated meal plan is "Breakfast - Oatmeal, Lunch - Salad, Dinner - Chicken Breast and Vegetables."

Step 6:

The server analyzes the meal plan received from the generative AI model and sends it to the smartphone application. The transmitted data includes the generated meal plan.

Step 7:

The smartphone application presents the meal plan received from the server to the user. The user can review and implement the proposed meal plan through the application.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Under the previous system, customers had to physically submit receipts for their purchases, which was time-consuming and risked loss or damage. Furthermore, there were insufficient ways to identify unnecessary purchases and visualize savings and weight loss. This made it difficult for customers to effectively manage their shopping to achieve their goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to take an image of their purchase receipt after shopping and upload it to the application so that they can submit it, means for correcting items that are determined to be unnecessary for this purchase, and means for visualizing the amount of money saved and weight loss that would be achieved if the unnecessary items were not purchased. This saves the customer the trouble of submitting a physical receipt and allows them to easily communicate their purchase information to the system. Furthermore, by identifying unnecessary items and visualizing the amount of money saved and weight loss that would be achieved, customers can effectively manage their shopping toward their goals.

"Customer" refers to an individual who uses the system to shop and manages their purchase information.

"Height and Weight" refers to the customer's current physical height and weight.

"Goal" refers to a specific physical or financial goal that the client wishes to achieve.

"Purchase receipt" means a paper or electronic record issued to a customer when they make a purchase that contains details of the items purchased.

"Taking an image" refers to the act of taking a photo of a physical receipt using a device such as a smartphone or camera.

"Application" refers to a software program that runs on a device such as a smartphone or tablet.

"Uploading" refers to the act of sending captured images to a server via the Internet.

"Server" refers to a computer system that receives, processes, and stores data over a network.

"Analyzing an image" refers to the act of using image recognition technology to extract the contents of a photographed receipt as digital data.

"Unnecessary products" refer to products that are determined not to be necessary for the customer to achieve their goals.

"Savings" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss you would expect if you had not purchased unnecessary items.

"Visualizing" refers to the act of displaying data visually so that customers can easily understand it.

"Displaying graphically" refers to the act of displaying data in a visual format such as a graph or chart.

This invention provides a system that allows customers to take an image of their purchase receipt after shopping and upload it to an application in order to submit it. A specific embodiment of this system is described below.

The user takes a picture of the purchase receipt using the smartphone camera. The image is then uploaded using a dedicated application (e.g., "ReceiptUploader"). The application has the function of sending the image taken by the user to the server.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google® Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted.

The analyzed data is stored in a database by the server. The stored data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database.

Users can check their purchase history using the application. By opening the "Purchase History" section within the application, data retrieved from the server is displayed. Users can view a list of past purchases.

As a concrete example, consider a scenario after a user has shopped at a supermarket. The user takes a picture of the receipt using their smartphone camera and opens the "ReceiptUploader" application. Within the application, the user taps the "Upload" button, selects the image of the receipt, and uploads it. The application sends the image to a server, which uses the Google Cloud Vision API to analyze the contents of the receipt. The analyzed data is stored in a database, and the user can open the "Purchase History" section within the application to view information about recent purchases.

Examples of prompts to input into a generative AI model include:

"Please describe the functionality that allows users to take a picture of a receipt using their smartphone camera and upload it to the application. Please include the names of any specific hardware or software."

Using this prompt, the generative AI model can generate sentences that explain in detail how the system will operate.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user takes a picture of the receipt.

The user uses their smartphone camera to take an image of the receipt for the item they purchased. When taking the photo, they are careful to capture the entire receipt clearly. The input is the physical receipt, and the output is a digital image.

Step 2:

The device uploads the image to the application.

The device (smartphone) uploads the captured image of the receipt to the "ReceiptUploader" application. The user opens the application, taps the "Upload" button, and selects the captured image. The input is a digital image, and the output is the transmission of image data to the server.

Step 3:

The server receives and analyzes the image.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted. The input is image data, and the output is analyzed text data.

Step 4:

The server stores the analysis results in a database.

The server saves the analyzed data in a database. The saved data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database. The input is the analyzed text data, and the output is saved in the database.

Step 5:

The user checks their purchase history.

Users can use the "ReceiptUploader" application to check their purchase history. When they open the "Purchase History" section in the application, data retrieved from the server is displayed. Users can view a list of past purchases. The input is data retrieved from the database, and the output is displayed in the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional shopping systems required customers to physically submit receipts for their purchases, which was time-consuming and made managing purchase information cumbersome. Furthermore, there was a lack of systems that could identify unnecessary products based on customers' health goals and visualize their savings and health benefits.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application; a means for correcting items determined to be unnecessary for the current purchase; a means for visualizing the money saved and weight lost if the unnecessary items were not purchased; a means for analyzing the contents of the purchase receipt and automatically extracting purchase information; a means for rewarding points to the customer based on the extracted purchase information; and a means for allowing the rewarded points to be used for the next purchase. This saves the customer the trouble of submitting a physical receipt, simplifies management of purchase information, and enables visualization of point rewards and savings. It also identifies unnecessary items based on the customer's health goals and visualizes savings and health benefits.

"Customer" refers to a consumer who purchases goods or services.

"Height and weight" refers to basic physical information that refers to the customer's physical height and weight.

A "goal" refers to a specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record detailing a transaction issued to a customer when they purchase goods.

A "smartphone" is a type of mobile phone that can connect to the Internet and use applications.

A "camera" is a device used to take images and videos.

An "application" is a software program that runs on a smartphone or computer.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Unnecessary products" are products that are deemed not necessary for achieving the customer's goals.

"Correction" is the act of checking the content and making corrections or improvements.

"Money saved" is the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" refers to the visual display of data or information.

"Analysis" is the act of examining data or information in detail to clarify its structure and meaning.

"Purchase information" refers to detailed information about the products purchased by the customer.

"Automatic extraction" means that the system extracts data without human intervention.

"Point redemption" is the act of awarding points to customers based on the purchase amount.

"Can be used on next purchase" means that the customer can use the points the next time they purchase a product.

The following describes in detail the mode for implementing this invention.

First, a smartphone application is used by customers to enter their current height, weight, and goals. The customer launches the application and enters their height, weight, and goals. This registers the customer's health goals in the system.

Next, we provide a way for customers to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application. Customers take a photo of their receipt and use the upload function within the application to send the image to the server.

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. Next, it uses pytesseract to extract text from the preprocessed image. It then analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text.

Based on the analyzed purchase information, the application provides a means for rewarding customers with points. The server calculates points based on the purchase amount and adds them to the customer's account. Rewarded points are managed within the application so they can be used for the next purchase.

Furthermore, as a means of filtering out items that may be deemed unnecessary for this shopping trip, an algorithm is implemented to determine unnecessary items based on the customer's height, weight, and goals. The server compares purchase information with the customer's health goals to identify unnecessary items.

To visualize the money saved and weight lost by not buying unnecessary products, the server calculates the amount saved and weight lost and displays them graphically. This allows customers to visually see the savings and health benefits.

As a concrete example, consider the case where a user purchases a drink at a convenience store, takes a photo of the receipt, and uploads it to the app. The app extracts "drink" and "150 yen" from the receipt and rewards 15 points. Furthermore, if the drink is deemed an unnecessary item based on the user's health goals, the amount of money saved and the weight lost will be displayed.

Examples of prompts for a generative AI model are as follows:

"A user has uploaded a receipt image taken with their smartphone camera. Please create a program that extracts purchase information from this image and rewards points. 10% of the purchase amount will be rewarded as points."

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goals. The entered data is temporarily saved within the application and sent to the server. The entered data is used as the basis for setting the user's health goals.

Step 2:

After shopping, the user takes a photo of the purchase receipt with their smartphone camera and uploads it to the application. The captured receipt image is temporarily saved within the application and then sent to the server. The input data is the receipt image.

Step 3:

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. The input data is the receipt image, and the output data is the preprocessed image.

Step 4:

The server extracts text from the preprocessed image using pytesseract. The input data is the preprocessed image, and the output data is the extracted text. Specifically, it uses OCR (optical character recognition) technology to read the text information in the image.

Step 5:

The server analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text. The input data is the extracted text, and the output data is the analyzed purchase information. Specifically, it uses a text analysis algorithm to extract the required information.

Step 6:

The server rewards the customer with points based on the analyzed purchase information. The input data is the analyzed purchase information, and the output data is the rewarded points. Specifically, it calculates points based on the purchase amount and adds them to the customer's account.

Step 7:

The server determines unnecessary products based on the customer's height, weight, and goals. The input data is the customer's health goals and purchase information, and the output data is a list of unnecessary products. Specifically, it runs an algorithm that compares the health goals with the purchase information and identifies unnecessary products.

Step 8:

The server calculates the amount of money saved and weight lost if unnecessary items are not purchased, and displays this graphically. The input data is a list of unnecessary items and purchase information, and the output data is the amount saved and weight lost. Specifically, it calculates the savings and health benefits and displays them visually to the user.

Step 9:

The user uses the points they have earned on their next purchase. The input data is the points earned, and the output data is the balance after the points have been used. Specific operations include using the points to discount the purchase amount and updating the balance.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Today's consumers find it difficult to balance health management and saving money. Consumers with diet or health goals in particular have difficulty determining which products are unnecessary when shopping. Furthermore, there are few ways to specifically assess the savings and health benefits of not purchasing unnecessary products. For this reason, a system that helps consumers effectively manage their health and save money is needed.

The identification processing by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for converting the submitted receipt information into text data using optical character recognition technology, means for determining unnecessary products based on the customer's height, weight, and goals, means for calculating the amount of money saved and the weight lost if the unnecessary products are not purchased, and means for graphically displaying the calculation results. This allows consumers to easily identify unnecessary products when shopping and specifically understand the amount of money saved and the health benefits.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and weight" refers to the customer's current physical measurements and is used as basic data for health management.

"Goals" are specific health and diet objectives set by customers, and include, for example, "carbohydrate restriction" or "calorie restriction."

A "receipt" is a detailed list of purchased items that a customer receives after shopping, listing the product names and prices.

"Optical character recognition technology" is a technology that extracts character information from image data and is used to convert receipt images into text data.

"Text data" refers to text information converted using optical character recognition technology, and includes information such as product names and prices.

"Unnecessary products" are products that are determined not to be necessary to purchase from the perspective of health management or dieting, based on the customer's height, weight, and goals.

"Amount saved" is the total amount of money you can save by not purchasing unnecessary items.

"Weight loss" is the amount of weight loss predicted if the unnecessary product is not consumed.

"Displaying graphically" means displaying the calculation results in the form of graphs, charts, etc. to make them easier to understand visually.

Form for implementing the invention

This invention is a system that enables consumers to effectively manage their health and save money. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. A device for users to enter their height, weight, and goals.

2. A terminal where users can submit receipts after shopping.

3. Hardware and software for the server to receive and process data

Hardware and software used

Device: Mobile devices such as smartphones and tablets

Server: High-performance data processing server

OCR technology: Optical character recognition technology such as Google® Cloud Vision API

Database: Relational database such as MySQL or PostgreSQL

Data visualization libraries: D3.js, Chart.js, etc.

Program processing

The server receives the height, weight, and goal information entered by the user, as well as the submitted receipt. Using this data, the server processes and calculates the data to detect unwanted products. Specifically, the process is carried out as follows:

1. Data reception and analysis:

The server receives the user's input height, weight, and goal (e.g., carbohydrate restriction).

The server converts the information on the submitted receipt into text data using OCR technology.

2. Detecting unwanted products:

The server detects unnecessary products based on the goal. For example, if the goal is to "limit carbohydrates," the server will list high-carbohydrate products on the receipt, such as "bread" and "sweets," as unnecessary products.

3. Calculate savings and weight loss:

The server calculates the total price of the unwanted items detected.

The server calculates the estimated weight loss that would occur if the unnecessary products were not consumed.

4. Visualizing the results:

The server displays the calculation results to the consumer in the form of graphs, charts, etc.

Specific examples

Let's say a user is 170cm tall, weighs 70kg, sets their goal to "restrict carbohydrates," and submits a receipt. The receipt includes "bread (300 yen)," "sweets (200 yen)," and "vegetables (100 yen)." The system detects "bread" and "sweets" as unnecessary items and calculates that a total of 500 yen can be saved. The system also calculates that the estimated weight loss would be 0.5kg if these items were not consumed, and displays this in a graph.

Example prompt

"Height 170cm, weight 70kg, goal is carbohydrate restriction. Detect unnecessary items based on the following receipt information and calculate the amount saved and weight loss. Receipt information: bread (300 yen), sweets (200 yen), vegetables (100 yen)." The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

Users enter their height, weight, and goal, then submit a receipt.

Input: The user opens the app on their smartphone and enters their height (170cm), weight (70kg), and goal of "restricting carbohydrates." After shopping, they also take a photo of the receipt with their smartphone camera and upload it to the server via the app.

Output: User input data and receipt image are sent to the server.

Step 2:

The server receives the input data and receipt information.

Input: Height, weight, goal, and receipt image submitted by the user.

Output: User input data and receipt image stored in a database on the server.

Step 3:

The server converts the receipt information into text data using OCR technology.

Input: Receipt image stored on the server.

Data processing: The server calls the Google Cloud Vision API, analyzes the receipt image, and extracts the product name and price.

Output: Converted text data (e.g., "Bread (300 yen)", "Sweets (200 yen)", "Vegetables (100 yen)").

Step 4:

The server detects unwanted items based on your goals.

Input: User's goal (e.g., "Limit carbohydrates") and converted text data.

Data calculation: The server performs rule-based filtering based on predefined food categories and goals to list unwanted products.

Output: List of unwanted products (e.g., "bread", "sweets").

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

Input: List of unwanted items and their respective prices.

Data calculation: The server calculates the total price of the unnecessary products and also calculates the estimated weight loss if the unnecessary products were not consumed. The model used here is a simulation model based on the relationship between calories consumed and weight loss.

Output: Amount saved (e.g., 500 yen) and estimated weight loss (e.g., 0.5 kg).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them to the user.

Enter: Amount saved and estimated weight loss.

Data processing: The server uses data visualization libraries such as D3.js and Chart.js to visualize the calculation results in graphs and charts.

Output: Graphs and charts displayed on the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Today's consumers find it difficult to simultaneously manage their health and save money. In particular, consumers with dietary goals need to identify and avoid unnecessary products from their purchases in order to achieve health and financial benefits. However, doing this manually is time-consuming and inaccurate. Therefore, a system is needed that allows consumers to easily identify unnecessary products and efficiently manage their savings and health.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for analyzing receipt information and automatically detecting unnecessary items based on diet goals, means for calculating the amount of money saved and estimated weight loss if the unnecessary items are not purchased and displaying them in graphs and charts, and an application installed on a smartphone. This allows consumers to easily identify unnecessary items and efficiently save money and manage their health.

"Customer" refers to consumers who use the system.

"Height and Weight" is data indicating the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

A "purchase receipt" is paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unwanted products" refer to products that are deemed inappropriate or unnecessary for the customer's goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Receipt information" refers to data such as the product name, price, and quantity listed on the purchase receipt.

"Diet Goals" refers to health goals set by a customer, such as weight loss or specific nutritional restrictions.

"Automatically detect" means that the system identifies unwanted products programmatically without human intervention.

"Graphs and charts" refer to diagrams used to visually display data.

"Applications installed on smartphones" refers to software programs that run on smartphones.

This invention is a system that allows consumers to efficiently manage their health and save money. Specific embodiments for implementing this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A mobile device such as a smartphone.

2. Server: A central server that processes and analyzes data.

3. Application: A dedicated application installed on a smartphone.

Program processing

User device

The user device has the following functions:

User information input: Users enter their height, weight, and diet goals through the application.

Receipt scanning: The user scans the receipt for the item they purchased with their smartphone camera and obtains the image data.

Server

The server performs the following process:

Analyzing receipt information: The acquired receipt image is converted into text data using OCR (optical character recognition) technology. This process uses the pytesseract library.

Detection of unnecessary products: Automatically detect unnecessary products from receipt text data based on the user's diet goals. For example, for a user aiming to limit carbohydrate intake, high-carbohydrate products such as "bread" and "sweets" will be detected.

Calculation of savings and weight loss: Calculates the total cost of the detected unnecessary products and the estimated weight loss if you did not consume those products. Weight loss is calculated based on the calories required to lose 1 kg of weight (approximately 7700 kcal).

application

The application has the following features:

Visualization of results: Calculated savings and estimated weight loss are displayed in graphs and charts, allowing users to visually see the effects of their health management and savings.

Specific examples

For example, if a user sets a goal of "restricting carbohydrates" and the receipt includes "bread" and "sweets," these items will be detected as unnecessary products. Let's say the total amount of the detected unnecessary products is 500 yen, and the estimated weight loss is 0.2 kg.

Example prompt

Please enter your user information:

Height: 170 cm
Weight: 70 kg
Diet goal: Carbohydrate restriction

Scan your receipt:

(Upload receipt image)

Unwanted items detected...

Unwanted items detected:

Bread: 200 yen Sweets: 300 yen Savings: 500 yen

Estimated weight loss: 0.2kg

The results are displayed in a graph.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goals.

Input: User's height, weight, and diet goal

Output: User information data

Specific operation: The user enters their height, weight, and diet goal (e.g., carbohydrate restriction) into the application's input form and presses the submit button. This generates user information data and sends it to the server.

Step 2:

After shopping, the user scans the receipt for the purchased items using their smartphone camera.

Input: Receipt image

Output: Receipt image data

Specific operation: The user uses the application's camera function to take a photo of the purchase receipt. The captured receipt image is saved within the application and sent to the server.

Step 3:

The server receives the receipt image and converts it into text data using OCR technology.

Input: Receipt image data

Output: Receipt text data

Specific operation: The server uses the pytesseract library to extract text from the received receipt image data. The extracted text data is saved as receipt text data.

Step 4:

The server automatically detects unnecessary items from receipt text data based on the user's diet goals.

Input: User information data, receipt text data

Output: List of unwanted items

Specific operation: The server analyzes receipt text data based on the user's diet goal (e.g., carbohydrate restriction). Products containing specific keywords (e.g., "bread," "sweets") are listed as unwanted items.

Step 5:

The server calculates the amount of money you could save and the estimated weight loss if you didn't buy unnecessary items.

Input: List of unwanted items

Output: Money saved, estimated weight loss

Specific operation: The server calculates the savings amount by adding up the prices of each item based on the list of unwanted items. It also calculates the estimated weight loss amount based on the calorie information of each item. The weight loss amount is calculated based on the calories required to lose 1 kg (approximately 7700 kcal).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them on the user's smartphone.

Input: Amount saved, estimated weight loss

Output: Image data for graphs and charts

Specific operation: Based on the calculation results, the server generates graphs and charts showing the amount of savings and estimated weight loss. Image data of the generated graphs and charts is sent to the user's smartphone and displayed within the application.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and determines which products are unnecessary based on those emotions. Specifically, when a user is feeling stressed, it may be possible to avoid unnecessary purchases of unhealthy foods such as sweets.

It also detects foods as being nutritious and recommends healthy foods when the user is relaxing.

"Example 2"

Furthermore, the emotion engine graphically displays the money saved and weight lost based on the user's emotions. Specifically, it displays the amount of money saved and the amount of weight lost if the user refrains from purchasing unnecessary products when feeling stressed. This allows users to visually understand the relationship between their emotions, health management, and saving money.

"Example 3"

For example, if a user buys chocolate when feeling stressed, the emotion engine will detect the chocolate as an unnecessary item and display a graph of the amount of money saved and weight loss they would have experienced if they had not bought the chocolate. This helps users recognize their tendency to buy unhealthy foods when they are stressed and understand the resulting financial and health impacts.

The processing flow for each example is explained below.

"Example 1"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine determines which products are unnecessary based on the user's emotions and notifies the user of the results.

"Example 2"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine graphically displays the money saved and weight lost based on the user's emotions.

"Example 3"

Step 1: Buy chocolate when the user is feeling stressed.

Step 2: The emotion engine recognizes the user's emotions and detects chocolate as an unwanted item.

Step 3: The emotion engine displays a graph of the amount of money saved and weight loss you would have experienced if you had not bought the chocolate.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

The previous system allowed customers to enter their height, weight, and goals, and to submit receipts after shopping, but it lacked the ability to identify unnecessary products based on the customer's emotions and recommend healthy products. This made it difficult for customers to select appropriate products when they were stressed or relaxed. It also lacked a way to visualize the money saved and weight lost by not purchasing unnecessary items. This made it difficult to raise customers' awareness of health management and saving money.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for recognizing the customer's emotions and determining unnecessary items based on those emotions, and means for recommending healthy products based on the customer's emotions. This enables the customer to select appropriate products according to their emotional state, thereby increasing their awareness of health management and saving money.

"Customer" refers to an individual who uses the system to enter their height, weight, and diet goals, and then submits a purchase receipt after shopping.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to specific numerical values or conditions related to dieting or health that a customer wishes to achieve.

"Input means" refers to the interface that allows customers to input their height, weight, and goals into the system.

"Purchase receipt" refers to the paper or electronic data containing the purchase details of a product that a customer receives when making a purchase.

"Submission means" refers to the interface through which customers submit purchase receipts to the system.

"Correction method" refers to the function by which the system determines unnecessary products based on customer input data and emotional data and notifies the customer.

"Visualization means" refers to the system's ability to graphically display the money saved and weight lost if unnecessary items are not purchased.

"Emotion recognition means" refers to the system's ability to analyze customer emotions and determine which products are unnecessary based on those emotions.

"Healthy product recommendation means" refers to the system's ability to recommend healthy products based on customer sentiment.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is set up on a website or smartphone application, and allows customers to directly input their height, weight, diet goals, etc. In addition, by combining it with an emotion engine, it has the function of determining unnecessary products based on the customer's emotions and recommending healthy products.

The server places an input form on a website or smartphone application. This input form is created using HTML and CSS, and JavaScript (registered trademark) is used to validate the input data in real time. When a user enters data into the input form, the server receives the data and stores it in a database (e.g., MySQL or PostgreSQL).

Next, the server uses an emotion engine (for example, Microsoft® Azure®'s Emotion API or Google Cloud's Natural Language API) to analyze the user's emotions. The emotion engine recognizes the user's current emotions based on the user's input data and past behavioral data. For example, if the user is feeling stressed, the server will use that information to determine that unhealthy foods such as sweets are unnecessary. On the other hand, if the user is relaxed, the server will recommend healthy foods.

As a concrete example, suppose a user uses a smartphone application to enter their height as 170cm, weight as 70kg, and diet goal as "lose 5kg." This data is sent to the server and stored in a database. The emotion engine then analyzes the user's emotions, and if it determines that the user is feeling stressed, the server displays a message such as "Avoid sweet snacks such as chocolate and cookies."

Example of a prompt to input to a generative AI model:

A user enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg." The emotion engine analyzes the user's emotions and determines that the user is feeling stressed. In this case, what products would be deemed unnecessary and what products would be recommended?

By inputting this prompt into a generative AI model, specific product recommendations and a determination of unnecessary products can be obtained. In this way, the system can make appropriate product recommendations based on the user's input data and emotional data.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application. The device sends a request to the server through the browser or application to retrieve an input form. The server generates the input form using HTML and CSS and sends it to the device. The device renders the received HTML and CSS and displays the input form to the user.

Input: User access request

Output: Display the input form

Step 2:

Users enter their height, weight, and diet goals.

The user enters their height, weight, and diet goals into the displayed input form. The device uses JavaScript to validate the format of the input data in real time to ensure there are no errors.

Input: User's height, weight, and diet goal

Output: Validated input data

Step 3:

The server receives the input data and stores it in the database.

When the user presses the send button, the device sends the input data in JSON format to the server. The server parses the received data and saves it in a MySQL database. Specifically, it adds the data to the database using the INSERT statement.

Input: JSON format input data

Output: Data stored in the database

Step 4:

The server calls the emotion engine and analyzes the user's emotions.

The server calls an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) based on the stored data. The data is sent to the API, which analyzes the user's emotions. The API determines that the user is feeling stressed and returns the result to the server.

Input: Data stored in the database

Output: Sentiment analysis results from the emotion engine

Step 5:

The server determines which products are unnecessary based on emotions and decides which products to recommend.

The server determines which products are unnecessary and which are recommended based on emotional data obtained from the emotion engine. For example, if the user is feeling stressed, the server will determine that "sweets such as chocolate and cookies should be avoided" and decide to "recommend healthy foods such as fruits and nuts."

Input: Sentiment analysis results from the emotion engine

Output: List of unwanted and recommended products

Step 6:

The server displays the results to the user.

The server generates HTML to display the results of the judgment to the user and sends it to the device. The device renders the received HTML and displays unwanted products and recommended products to the user. The user checks the displayed results and decides on their next course of action.

Input: List of unwanted and recommended products

Output: The result displayed to the user

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional food delivery systems don't recommend foods based on the user's health status or emotions, which can lead to users choosing unhealthy foods when they're stressed. Furthermore, there's a lack of support for users to choose foods that align with their diet goals, making it difficult to diet effectively.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for the customer to enter their current height and weight and goals, a means for the customer to submit a purchase receipt after shopping, a means for correcting items determined to be unnecessary in this purchase, a means for visualizing the money saved and weight lost if the unnecessary items are not purchased, and a means for recognizing the customer's emotions and determining unnecessary items based on those emotions. This makes it possible to recommend foods based on the user's health condition and emotions, allowing the user to avoid unhealthy foods when feeling stressed and to choose foods that are in line with their diet goals.

"Customer" refers to a consumer who uses a service or product.

"Height and weight" are numbers that indicate an individual's physical height and weight.

"Goal" refers to the specific objective or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record of a transaction issued when a customer purchases a product.

"Unnecessary products" refer to products that are deemed inappropriate for the customer's health condition or goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight Loss" is a numerical value that represents the weight loss a customer can achieve by avoiding unnecessary products.

"Means for recognizing emotions" refers to technologies or devices that detect a customer's emotional state and make decisions based on that.

"Visualization means" refers to technology or devices that visually display the money saved or weight lost.

A system for implementing this invention is configured as follows: First, a dedicated form is provided for users to enter their height, weight, and diet goals. This form is provided on a website or smartphone application, allowing users to easily enter information.

Next, we present a system that combines an emotion engine. This system recognizes the user's emotions in real time and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products, and when the user is relaxed, it recommends healthy foods.

The following hardware and software are used to implement this system:

Hardware: Smartphone, tablet, or PC

Software: Python, EmotionRecognizer library, FoodRecommender library

The server receives the user's input data such as height, weight, and diet goal, and uses the EmotionRecognizer library to recognize the user's emotions in real time. It then uses the FoodRecommender library to recommend appropriate foods based on the user's input data and the recognized emotions.

For example, if a user enters their height as 170cm, weight as 65kg, and diet goal as "lose weight," and the emotion engine recognizes that the user is feeling stressed, the system will avoid unhealthy foods (e.g., sweet snacks) and instead recommend healthy foods (e.g., salads and fruit).

Examples of prompt statements include:

"If a user is 170cm tall, weighs 65kg, and their diet goal is to lose weight, and the emotion engine determines that the user is feeling stressed, what foods would you recommend?"

In this way, food recommendations can be made based on the user's health status and emotions, helping users avoid unhealthy foods when they are stressed and choosing foods that align with their diet goals.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The user uses the terminal to input their height, weight, and diet goal. The input data is sent to the server. The input data includes the user's height (e.g., 170 cm), weight (e.g., 65 kg), and diet goal (e.g., "lose weight").

Step 2:

The server stores the received user input data and uses the EmotionRecognizer library to recognize the user's emotions in real time. For emotion recognition, the user's facial expressions and voice data are used as input. The output is the user's emotional state (e.g., stressed, relaxed).

Step 3:

The server uses the FoodRecommender library to recommend appropriate foods based on the recognized emotional state and user input data. Input data includes the user's height, weight, diet goals, and emotional state. The output is a list of recommended foods (e.g., salads, fruits).

Step 4:

The server sends a list of recommended foods to the user's device, where the user can check the list of recommended foods on their device. Specifically, if the user is feeling stressed, unhealthy foods will be avoided and healthy foods will be recommended.

Step 5:

A user makes a purchase and submits the purchase receipt to the server using a terminal. The server analyzes the submitted receipt data and generates a list of the purchased items. Input data includes receipt images or electronic data. The output is a list of the purchased items.

Step 6:

The server compares the list of purchased items with the user's diet goals and emotional state to determine which items are unnecessary. Specific operations include detecting items that are inappropriate for the user's goals (e.g., sweet snacks). Input data includes the list of purchased items, the user's diet goals, and their emotional state. The output is a list of unnecessary items.

Step 7:

The server calculates and visualizes the money saved and weight lost if unnecessary items are not purchased. Specific operations include graphically displaying the amount saved and weight lost. The input data includes a list of unnecessary items. The output is a graphical representation of the amount saved and weight lost.

Step 8:

The server sends the visualized data to the user's device, where the user can check the amount of money saved and the weight lost on the device. Specifically, the user can visually understand the benefits they can gain by avoiding unnecessary products.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

With conventional health management systems, customers had to submit physical proof of purchase after shopping, which was time-consuming. Furthermore, the amount of savings and weight loss were not visualized taking into account the customer's emotional state, making it difficult for customers to understand the relationship between their emotions, health management, and savings.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current physical information and goals, means for the customer to submit a proof of purchase after shopping, means for the customer to capture an image of the proof of purchase using the camera function of their mobile device and upload it to the application, means for the server to convert the contents of the proof of purchase into text data using image processing technology, means for the server to save the converted text data, means for the customer to input emotional data, means for the server to analyze the customer's emotional data using emotion analysis technology, means for the server to determine unnecessary products based on the analysis results and calculate the amount of savings and weight loss, and means for the server to graphically display the calculation results. This allows customers to communicate their purchase information to the system without having to directly submit a physical proof of purchase, and further enables them to visually understand the relationship between their emotions, health management, and savings.

"Customer" refers to an individual who uses the system to manage their own health and save money.

"Physical information" refers to physical data such as the customer's height, weight, and body fat percentage.

"Goals" refers to the health or savings goals the customer wants to achieve.

"Proof of purchase" refers to a document that proves a purchase, such as a receipt or invoice that a customer receives after shopping.

"Mobile device" refers to portable electronic devices such as smartphones and tablets.

"Photography function" refers to the camera function built into the mobile device.

"Application" refers to a software program that runs on a mobile device.

"Image processing technology" refers to the technology of analyzing image data and extracting necessary information.

"Text data" refers to text information extracted using image processing technology.

"Emotional data" refers to data that indicates a customer's emotional state.

"Emotion analysis technology" refers to technology that analyzes emotional data and evaluates a customer's emotional state.

"Unnecessary products" refer to products that are determined not to be necessary for the customer's health or savings goals.

"Amount saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight you would lose if you had not purchased unnecessary items.

"Displaying graphically" refers to displaying the calculation results in a visually easy-to-understand manner.

Form for implementing the invention

This invention is a system in which customers submit a purchase certificate after shopping and visualize the amount of savings and weight loss based on emotional data. A specific embodiment of this system is described below.

System Configuration

This system primarily uses the following hardware and software:

Mobile devices (smartphones and tablets)

Camera function

application

Server

Image processing software (e.g., OpenCV, Tesseract)

Emotion analysis technology

Program processing

After shopping, the user uses the camera function on their mobile device to take a picture of the purchase certificate. The image is then uploaded to the server via the application. The server then uses image processing software (e.g., OpenCV or Tesseract) to extract text from the image and convert information such as the name, price, and quantity of the purchased item into text data. The server then stores this text data in a database.

Next, the user enters their emotional data through the application. The emotional data may be entered manually by the user or automatically obtained from the wearable device. The server then analyzes the emotional data using emotion analysis technology to assess whether the user is feeling stressed and to what extent.

Based on the results of the emotion analysis, the server calculates the amount of money saved and the amount of weight lost if the user refrains from buying unnecessary items when feeling stressed. For example, if the user refrains from buying chocolate when feeling stressed, the calculation is based on the price (200 yen) and calories (300 kcal). The calculation results are displayed graphically by the server, and the user can check them through the application.

Specific examples

As a concrete example, consider the case where a user decides not to buy chocolate when they are feeling stressed. The user takes a photo of the receipt with their smartphone camera and uploads it to the application. The server uses image processing software to convert the contents of the receipt into text data and stores it in a database. Next, the user enters emotional data into the application, which the server analyzes using emotion analysis technology. Based on the analysis results, the server calculates the amount of savings and weight loss based on the price of the chocolate (200 yen) and calories (300 kcal). Finally, the server displays the calculation results graphically, and the user can check them through the application.

Example prompt

Example of a prompt to input to a generative AI model:

"Generate a program that calculates and graphically displays the amount of money saved and weight loss a user would achieve if they avoided buying unnecessary items when they were feeling stressed."

In this way, users can visually understand the relationship between their emotions, health management, and saving money.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user takes a photo of the purchase certificate using their smartphone camera.

Enter: Proof of purchase

Output: Image of proof of purchase

Specific operation: The user activates the smartphone camera, frames the purchase certificate, and presses the capture button. The captured image is saved to the smartphone's gallery.

Step 2:

Upload the image of the purchase certificate taken by the device to the application.

Input: Image of proof of purchase

Output: Image data sent to the server

Specific operation: The user launches the application, selects an image of the purchase certificate, and presses the upload button. The device then sends the selected image to the server.

Step 3:

The server uses image processing technology to convert the contents of the purchase certificate into text data.

Input: Image data of purchase certificate

Output: Text data

Specific operation: The server analyzes the received image data using image processing software (e.g., OpenCV, Tesseract) and extracts text information from the image. The extracted text information is converted into text data such as the name, price, and quantity of the purchased item.

Step 4:

The server stores the converted text data in a database.

Input: Text data

Output: Purchase information stored in the database

Specific operation: The server saves the converted text data in a database. The saved data includes the purchase date and time, store name, product information, etc.

Step 5:

The user inputs emotional data into the application.

Input: Emotion data

Output: Emotion data sent to the server

Specific operation: The user inputs their emotional state through the application. Emotional data includes information such as stress level and mood. It may also be obtained automatically from the wearable device.

Step 6:

The server analyzes the user's emotional data using emotion analysis technology.

Input: Emotion data

Output: Analysis results

Specific operation: The server uses emotion analysis technology to analyze emotional data and evaluate whether the user is feeling stressed and to what extent. The analysis results are displayed as numbers and graphs.

Step 7:

The server determines which items are unnecessary based on the analysis results and calculates the amount of money saved and the weight lost.

Input: Analysis results, text data

Output: Savings and weight loss data

Specific operation: Based on the results of emotion analysis and purchase information, the server calculates the amount of money saved and the amount of weight lost if the user does not buy unnecessary items when feeling stressed. For example, the calculation is based on the price (200 yen) and calories (300 kcal) of chocolate.

Step 8:

The server displays the calculation results graphically.

Input: Savings and weight loss data

Output: Graphical display

Specific operation: The server generates data for graphically displaying the calculation results and provides it to the user through the application. The user can visually check the amount of savings and weight loss through the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional shopping support systems required customers to physically submit receipts for purchased items, which was a time-consuming process. Furthermore, the shopping support did not take into account the customer's emotions or health status, leading to unnecessary purchases when they were feeling stressed. This led to wasteful spending and negative health effects, which was an issue.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to the application; a means for correcting items that are determined to be unnecessary in this purchase; a means for visualizing the money saved and weight lost if the customer does not buy unnecessary items; and a means for analyzing the customer's emotions and graphically displaying the amount of money saved and weight lost if the customer does not buy unnecessary items when they are feeling stressed. This saves the customer the trouble of submitting physical receipts, and enables shopping support that takes their emotions and health state into consideration.

"Customer" refers to a consumer who purchases goods or services.

"Height and Weight" refers to the customer's physical height and weight.

"Goal" refers to the specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a proof of purchase issued when a customer purchases a product.

A "smartphone camera" is a photographic device built into a smartphone.

An "application" is a software program that runs on a smartphone or other device.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Correction" is the act of checking the content and correcting or deleting unnecessary parts.

"Unnecessary items" are items that customers have determined they do not need to purchase.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" is the act of visually displaying data or information.

"Analyzing emotions" is the process of assessing and understanding a customer's emotional state.

"Stress" refers to mental or physical tension or strain.

"Graphically displayed" means displaying information in a visual format such as a graph or chart.

This invention provides a system that allows customers to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to an application. It also includes a function that analyzes the customer's emotions and graphically displays the amount of money saved and the amount of weight lost if they avoid buying unnecessary items when they are feeling stressed.

System Configuration

1. Hardware:

Smartphone: Use the camera function to take a photo of your purchase receipt.

Server: Serves as the backend for the application, storing and processing data.

2. Software:

Application: Installed on a smartphone, it takes photos of receipts, uploads them, performs sentiment analysis, and displays them graphically.

OpenCV: A library for loading and preprocessing receipt images.

Tesseract OCR: Optical character recognition software for extracting text from receipt images.

EmotionRecognizer: Emotion recognition software for analyzing customer emotions.

Matplotlib: A library for graphically displaying savings and weight loss.

System Operation

1. Take a photo of your receipt and upload it:

Users take a photo of their purchase receipt using their smartphone camera.

The captured images are uploaded to the server via the application.

2. Text extraction:

The server preprocesses the image using OpenCV and extracts text from the receipt using Tesseract OCR.

3. Sentiment analysis:

The server analyzes the user's emotions using EmotionRecognizer.

When you're feeling stressed, calculate how much money you'll save and how much weight you'll lose if you don't buy unnecessary items.

4. Graphical display:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

The application will display this visually to the user.

Specific examples

For example, if a user avoids buying unnecessary items when feeling stressed, the application will graphically display a savings of 1,000 yen and a weight loss of 0.5 kg.

Example prompt

Create a program that calculates and graphically displays the savings and weight loss that a user could achieve by not purchasing unnecessary items when they are feeling stressed. Take a photo of the receipt with a smartphone camera and extract the text using Tesseract OCR. Analyze the user's emotions using EmotionRecognizer and display the results using Matplotlib.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

Users take a photo of their purchase receipt using their smartphone camera.

Input: Physical image of purchase receipt

Output: Receipt image saved on your smartphone

Specific operation: The user launches the smartphone's camera app and takes a photo of the purchase receipt. The captured image is saved to the smartphone's storage.

Step 2:

The device will upload the captured receipt image to the application.

Input: Receipt image saved on your smartphone

Output: Receipt image uploaded to the server

Specific operation: The user launches the application, selects the captured receipt image, and presses the upload button. The image is sent to the server via the Internet.

Step 3:

The server preprocesses the receipt image using OpenCV and extracts text using Tesseract OCR.

Input: Receipt image uploaded to the server

Output: Extracted text data

Specific operation: The server uses OpenCV to remove noise from the image and highlight the text. Then, it uses Tesseract OCR to extract the text from the image.

Step 4:

The server analyzes the user's emotions using EmotionRecognizer.

Input: User emotional data (e.g., the emotional state the user enters into the application)

Output: Analyzed emotional state

Specific operation: The server uses EmotionRecognizer to analyze the user's emotional data and determine whether they are under stress.

Step 5:

When you feel stressed, the server calculates the amount of money you would save and the amount of weight you would lose if you didn't buy unnecessary items.

Input: Analyzed emotional state, extracted text data

Output: Savings and weight loss data

Specific operation: If the emotional state is stressed, the server identifies unnecessary items from the extracted text data and calculates the amount of savings and weight loss that would result if the item were not purchased.

Step 6:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

Input: Savings and weight loss data

Output: Image data for graphical display

Specific operation: The server uses Matplotlib to generate images that visually display the amount of savings and weight loss in the form of bar graphs, etc., and sends the image data to the application.

Step 7:

The terminal displays the graphical data received from the server to the user.

Input: Graphical image data sent from the server

Output: Graphical information displayed on a smartphone screen

Specific operation: The application displays the image data received from the server on the smartphone screen, allowing the user to visually check the amount of savings and weight loss.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Today's consumers aim to balance health management with saving money, but they often end up buying unnecessary products in their daily shopping. These unnecessary products not only go against their health goals but also lead to wasteful spending. However, it is not easy for consumers to identify and avoid these unnecessary products. Therefore, there is a need for a system that automatically detects unnecessary products based on consumers' health goals and visualizes the savings and health impact.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for customers to input their current height and weight and goals, means for customers to upload an image of their purchase receipt after shopping, means for converting the receipt image into digital data using optical character recognition technology, means for analyzing product information from the digital data and detecting unnecessary products, means for calculating the total price of unnecessary products and estimated weight loss, and means for visualizing the calculation results in graphs and charts. This allows consumers to automatically detect unnecessary products based on their own health goals and specifically understand the amount of money they can save and the impact on their health.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and weight" refers to the customer's basic physical data and is fundamental information for setting health goals.

"Goals" refer to the health and weight management goals that a customer wishes to achieve, such as carbohydrate restriction or calorie restriction.

"Receipt image" refers to photo data of the purchase details that a customer obtains after shopping.

"Optical character recognition technology" is a technology that converts text information in an image into digital data, and is also known as OCR (Optical Character Recognition).

"Digital data" means text information converted using optical character recognition technology.

"Product information" refers to data such as the product name, price, and nutritional information listed on the receipt.

"Unnecessary products" are products that are contrary to the customer's health goals and should be avoided.

"Total amount" is the total price of the unwanted items detected.

"Estimated weight loss" is the amount of weight loss predicted if the unnecessary product was not consumed.

"Graphs and charts" are diagrams that visually display calculation results.

This invention is a system that allows consumers to efficiently manage their own health and save money. Specific embodiments of this system are described below.

First, the user accesses the system's web page using a terminal. The user enters their height, weight, and health goals (e.g., carbohydrate restriction). This data is sent to the server and stored in a database.

Next, the user uploads an image of the receipt obtained after shopping from their device to the system. The uploaded receipt image is sent to the server, which then uses optical character recognition technology such as Google Cloud Vision API to convert the receipt image into digital data. This digital data includes the text information on the receipt.

The server analyzes the converted digital data and extracts product names and their nutritional information. For example, if the receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

The server then calculates the total cost of the unnecessary items, as well as the estimated weight loss that would occur if these items were not consumed. A standard calorie expenditure model is used to calculate weight loss. The results are visualized as graphs and charts using a data visualization library such as D3.js.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

As a concrete example, consider the case where a user purchases chocolate when feeling stressed. The emotion engine analyzes the user's emotional state and determines that stress is high. In this case, it detects chocolate as an unnecessary item and displays a graph of the amount of money saved and weight loss that would have been achieved if the user had not purchased the chocolate. This allows the user to recognize their tendency to purchase unhealthy foods when they are stressed and understand the resulting financial and health impacts.

Example prompt:

Detect the foods the user purchases when they are stressed, calculate the amount of money saved and the weight loss they would have lost if they had not purchased those foods, and display the results in a graph.

This system allows users to specifically understand both their own health management and savings. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

Users enter their height, weight, and goals.

Using a terminal, a user accesses the system's web page and enters their height (e.g., 170 cm), weight (e.g., 70 kg), and health goal (e.g., carbohydrate restriction) into the form. The entered data is sent to the server and stored in a database.

Input: height, weight, health goals

Output: User information stored in the database

Step 2:

The user uploads an image of the receipt.

Users use their device to upload images of recent purchase receipts to the system. The uploaded images are then sent to the server.

Input: Receipt image

Output: Receipt image saved on the server

Step 3:

The server performs OCR processing on the receipt image.

The server uses optical character recognition (OCR) technology to convert the receipt image into digital data. Specifically, it calls the Google Cloud Vision API to perform the OCR process.

Input: Receipt image

Output: Digital data (text information)

Step 4:

The server analyzes product information and detects unnecessary products.

The server analyzes the digital data obtained through OCR processing and extracts product names and their nutritional information. For example, if a receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

Input: Digital data (text information), user's health goals

Output: List of unwanted items

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

The server calculates the total value of the unwanted products detected, as well as the estimated weight loss that would occur if the user did not consume these products. A standard calorie expenditure model is used to calculate weight loss.

Input: List of unwanted items, product price, calorie information

Output: Total amount, estimated weight loss

Step 6:

The server visualizes the calculation results in graphs and charts.

The server uses a data visualization library such as D3.js to generate graphs and charts showing the total price of unwanted items and estimated weight loss.

Input: Total amount, estimated weight loss

Output: Graphs and charts

Step 7:

The user checks the results.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

Input: Graphs and charts

Output: User understanding and recognition

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Today's consumers are expected to manage their health and save money at the same time, but it can be difficult to determine which products are unnecessary when shopping. Furthermore, there is a lack of ways to grasp the specific amount of savings or weight loss that can be achieved by not purchasing unnecessary items. For this reason, there is a need for a system that allows consumers to make healthy choices and realize the savings they are making.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for customers to enter their current height and weight and goals, means for customers to submit purchase receipts after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and the amount of weight lost, and means for displaying the detection results in a graph. This allows consumers to easily identify unnecessary items when shopping and specifically understand both health management and savings.

"Customer" refers to a consumer who uses the system to make purchases.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

"Purchase receipt" refers to the paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unnecessary items" refer to items that should be removed from the shopping list based on the customer's height, weight, and goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Visualization methods" refers to ways to display the money saved and weight lost in visual form, such as graphs or charts.

"Means of detection" refers to a method for identifying unwanted products based on the customer's height, weight, and goals.

"Means of calculation" refers to a method for calculating the amount of money saved and weight loss that would be achieved by not purchasing unnecessary items.

"Graphical display" refers to a method of displaying detection results in a visually easy-to-understand format.

A system for implementing this invention includes means for a customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and weight lost, and means for displaying the detection results in a graph.

System Configuration

The system is implemented as a smartphone application. The application is built using Python and Flask, and performs data processing on the server side. The smartphone sends the user's height, weight, and goals, as well as receipt information, to the server.

Data processing and calculation

The server receives the data sent by the user and performs the following processing:

1. Detecting unwanted products:

The server identifies unnecessary items on the receipt based on the user's height, weight, and goals. For example, if the user's goal is to "limit carbohydrates," it will detect "bread" and "sweets" on the receipt as unnecessary items.

2. Calculate savings and weight loss:

The server calculates the amount of money saved and the amount of weight lost if the unnecessary items were not purchased. For example, it adds up the prices of the unnecessary items and calculates the amount of weight lost based on the calories in those items.

3. Visualizing the results:

The server generates the calculation results in the form of graphs and charts and sends them to a smartphone application. Users can visually check the results through the application.

Hardware and software used

Hardware: Smartphone

Software: Python, Flask

Specific examples

If a user's goal is to "limit carbohydrates" and the receipt includes "bread" and "sweets," the system will detect these as unnecessary items. The savings will be calculated as 350 yen (bread 150 yen + sweets 200 yen), and the weight loss will be 0.064 kg (500 kcal / 7700 kcal). These results are displayed as a graph on the smartphone application.

Example prompt

If a user is aiming to limit carbohydrates, detect unnecessary items in their receipts and calculate savings and weight loss. Use the following data:

Height: 170cm
Weight: 70kg
Goal: Low-carb Receipt items: ["bread", "sweets", "vegetables"]

The flow of the specific processing in Application Example 3 is explained using Figure 22.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goal.

Input: height, weight, goal

Output: Height, weight, and goal data

Specific action: The user enters their height, weight, and goal into the application's input form and presses the submit button.

Step 2:

After shopping, users scan their purchase receipt with their smartphone and upload it to the application.

Input: Purchase receipt image

Output: Receipt text data

Specific operation: The user takes a photo of the receipt using their smartphone camera and uploads it to the application. The application uses OCR technology to extract text data from the image.

Step 3:

The server receives the height, weight, goal, and receipt text data sent by the user.

Input: Height, weight, goal, and receipt text data

Output: Save received data

Specific operation: The server receives the HTTP request and stores the submitted data in the database.

Step 4:

The server detects unwanted items on the receipt based on the user's goals.

Input: Height, weight, goal, and receipt text data

Output: List of unwanted items

Specific operation: The server references the list of unwanted items related to the goal and identifies the unwanted items by matching them with the text data on the receipt.

Step 5:

The server calculates the amount of savings and weight loss you would have achieved if you had not purchased unnecessary items.

Input: List of unwanted items

Output: Savings, weight loss

Specific operation: The server adds up the prices of the unwanted items and calculates the amount of weight loss based on the calories of the unwanted items. For example, if the total price of the unwanted items is 350 yen and the total calories are 500 kcal, the amount of weight loss will be calculated as 0.064 kg (500 kcal / 7700 kcal).

Step 6:

The server generates the calculation results in the form of graphs and charts and sends them to the smartphone application.

Input: Amount saved, weight loss

Output: Image data for graphs and charts

Specific operation: The server generates graphs and charts based on the calculation results and sends them as image data to the smartphone application.

Step 7:

Users can visually check the savings and weight loss results through a smartphone application.

Input: Image data for graphs and charts

Output: Visual results

What it does: The user opens the app and looks at the graphs and charts that are displayed, allowing them to understand the specific savings and weight loss they could have achieved by not purchasing unnecessary items.

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

This embodiment of the present invention provides a dedicated input form as a means for consumers to enter their height, weight, and diet goals. This input form is provided on a website or smartphone application, allowing consumers to directly input their height, weight, diet goals, etc.

"Example 2"

Next, as a means for consumers to submit their purchase receipts after shopping, we provide a feature that allows them to take an image of the receipt using their smartphone camera and upload it to the application. This feature allows consumers to communicate purchase information to the system without having to directly submit a physical receipt.

"Example 3"

Then, as a means of correcting items that are deemed unnecessary for this shopping trip, the system automatically detects unnecessary items based on the height, weight, and goal entered by the consumer, as well as the information on the submitted receipt. For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary.

"Example 4"

Finally, as a means of visualizing the money saved and weight loss that would result from not purchasing unnecessary products, the system calculates the total cost of the detected unnecessary products and the estimated amount of weight loss that would result from not consuming those products, and displays this information to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their own health management and savings.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Consumers enter their height, weight, diet goals, etc. into a dedicated input form. For example, they might enter a height of 170 cm, a weight of 65 kg, and a diet goal of "restricting carbohydrates."

Step 2: Once the information is complete, the consumer presses the "Submit" button to send the information to the system.

"Example 2"

Step 1: The consumer makes a purchase and obtains a receipt for the items purchased.

Step 2: The consumer takes a picture of the receipt using their smartphone camera.

Step 3: The captured image is uploaded to the application and the system reads the receipt information.

"Example 3"

Step 1: The system automatically detects unwanted items based on the consumer's height, weight, and goals entered, as well as the submitted receipt information.

Step 2: For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary items.

"Example 4"

Step 1: The system calculates the total value of the detected unwanted products and the estimated weight loss that would occur if the product were not consumed.

Step 2: The calculation results are displayed to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their health management and savings.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional diet support systems required users to not only input their height, weight, and diet goals, but also submit purchase receipts after shopping to identify unnecessary items. However, these systems were time-consuming for users, making continued use difficult. Furthermore, they lacked the functionality to suggest an appropriate individual diet plan based on the data entered by the user, which meant they were unable to provide effective diet support.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for receiving and saving the data entered by the customer, means for generating prompt sentences for the generative AI model based on the saved data, and means for the generative AI model to suggest a diet plan. This allows users to use the system continuously without any hassle, and makes it possible to suggest appropriate diet plans individually.

"Customer" refers to the user of the system.

"Height and Weight" refers to the customer's current height and weight data.

"Goals" refers to the diet and health goals set by the customer.

"Input form" refers to the interface where customers enter data such as height, weight, and goals.

"Purchase receipt" refers to the detailed list of purchases that a customer receives after making a purchase.

"Means for correcting" refers to the function that determines unnecessary products based on the customer's height, weight, and goals, and removes them from the list.

"Visualization means" refers to a function that visually displays the money you can save and the weight you will lose if you don't buy unnecessary products.

"Means for receiving and storing data" refers to the function by which the server receives data entered by the customer and stores it in a database, etc.

"Generative AI model" refers to an artificial intelligence model that generates an appropriate diet plan based on input data.

"Prompt sentence" refers to an instruction sentence generated for a generative AI model based on input data.

"Diet Plan" refers to a specific action plan proposed by the generative AI model to help customers achieve their goals.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is installed on a website or smartphone application, and a generative AI model proposes an appropriate diet plan based on the data entered by the customer.

Hardware and software used

Hardware: Servers (e.g., AWS (registered trademark) EC2), user devices (e.g., smartphones, PCs)

Software: Web servers (e.g., Apache, Nginx), databases (e.g., MySQL, PostgreSQL), smartphone applications (e.g., iOS apps, Android (registered trademark) apps)

Specific system operation

Server

The server provides a dedicated input form on a website or smartphone application. This input form allows customers to enter their height, weight, diet goals, etc. The server receives the input data and stores it in a database. The stored data is used as prompts for the generative AI model.

Device

The device displays the websites and smartphone applications that the user accesses. When the user enters their height, weight, and diet goals into an input form, the device sends the data to a server. The device then displays the data the user has entered in real time, allowing them to check it.

User

Users access a website or smartphone application and enter their height, weight, and diet goals into a dedicated input form. Once the user has completed the input, the data is sent to the server. The user can then review the data they have entered and make any necessary corrections.

Specific examples

Example 1: Website use

A user visits a website and enters the following information into a form:

Height: 170 cm
Weight: 70 kg
Diet goal: Lose 5 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

"The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Based on this information, please suggest an appropriate diet plan."

Example 2: Use in a smartphone application

A user opens a smartphone application and enters the following information into the input form:

Height: 160 cm
Weight: 60 kg
Diet goal: Lose 3 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

"The user's height is 160 cm, weight is 60 kg, and their diet goal is to lose 3 kg. Based on this information, please suggest an appropriate diet plan."

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application and accesses an input form. The input form is an interface where the user can enter their height, weight, and diet goals. Once the input form is displayed, the user is ready to enter the required information.

Step 2:

The user enters the data.

The user enters their height, weight, and diet goal into the input form. For example, the user might enter "Height: 170 cm", "Weight: 70 kg", and "Diet goal: Lose 5 kg". Once the input is complete, the user taps the "Submit" button. The input data is saved on the device in JSON format.

Step 3:

The device sends the data to the server.

The device sends the data entered by the user to the server. Specifically, the device uses an HTTP POST request to send the input data to the server. At this time, the data is sent in JSON format. The input data (height, weight, diet goal) is sent to the server.

Step 4:

The server receives and stores the data.

The server receives the data sent from the device. The received data is stored in a database. For example, the server connects to a MySQL database and executes the following SQL query:

sql

INSERT INTO user_data (height, weight, goal) VALUES (170, 70, '5 kg weight loss');

This process saves the user's input data in the database.

Step 5:

The server generates a prompt for the generative AI model.

The server generates prompts for the generative AI model based on the stored data. For example, it generates prompts like the following:

The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Please suggest an appropriate diet plan based on this information.

This prompt is sent to the generative AI model.

Step 6:

Generative AI models suggest diet plans.

The generative AI model generates a diet plan based on the prompt it receives. For example, the generative AI model might suggest the following diet plan:

1. Do 30 minutes of aerobic exercise every day.

2. Limit your daily calorie intake to 1,500 kcal

3. Strength training three times a week

This diet plan is sent to the user's device via the server and displayed to the user.

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional food delivery systems struggled to provide meal plans based on customers' health status and diet goals, making it impossible to provide services tailored to individual needs. Furthermore, problems arose, such as customers buying unnecessary items and making meal choices that hindered the achievement of diet goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height, weight, and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for generating an appropriate meal plan based on the customer's height, weight, and goals, and means for presenting the generated meal plan to the customer. This makes it possible to provide an appropriate meal plan based on the customer's health condition and diet goals, and support the achievement of diet goals while reducing unnecessary spending.

"Customer" refers to the consumer who uses the service.

"Height and Weight" refers to the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that a customer wishes to achieve.

"Purchase receipt" refers to the detailed list of purchases received by a customer when making a purchase.

"Correction means" refers to a method or system for evaluating products purchased by customers and identifying unwanted products.

"Visualization means" refers to a method or system that visually displays the money saved or weight lost.

"Meal Plan" refers to the meal content and schedule suggested based on the client's health and diet goals.

"Means for generating" refers to a method or system for creating a meal plan based on customer input data.

"Presentation means" refers to the method or system for showing the generated meal plan to the customer.

The following system configuration is proposed as an embodiment of this invention.

System Configuration

The system includes a smartphone application used by customers, a server, and a generative AI model. The smartphone application provides an interface for customers to input their height, weight, and diet goals. The server receives the input data and sends prompts to the generative AI model to generate an appropriate meal plan. The generated meal plan is then presented to the customer again via the smartphone application.

Hardware and software used

Hardware: Smartphones, servers

Software: Smartphone applications, generative AI models (e.g., OpenAI GPT-3)

Data processing and calculation

1. Data entry: Customers use a smartphone application to enter their height, weight, and diet goals.

2. Data transmission: The smartphone application sends the entered data to the server.

3. Prompt generation: Based on the data received by the server, a prompt sentence is generated to be sent to the generative AI model.

4. Meal plan generation: The generative AI model receives the prompt and generates an appropriate meal plan.

5. Data reception and presentation: The server receives the generated meal plan and presents it to the customer via the smartphone application.

Specific examples

For example, if a customer enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg," the server will send the following prompt text to the generative AI model:

Example prompt

Please suggest an appropriate meal plan based on the following: height: 170 cm, weight: 70 kg, goal: 5 kg weight loss.

Based on this prompt, the generative AI model generates a balanced meal plan. For example, it suggests specific meal options such as "Breakfast - oatmeal, Lunch - salad, Dinner - chicken breast and vegetables." This meal plan is presented to the customer via a smartphone application.

In this way, we can provide customers with appropriate meal plans based on their health condition and diet goals, helping them achieve their diet goals while reducing unnecessary spending.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goal. The entered data is temporarily saved within the smartphone application. An example of input data is: height 170cm, weight 70kg, diet goal "lose 5kg."

Step 2:

The smartphone application sends the entered data to the server. The data sent includes the user's height, weight, and diet goals. The server analyzes the received data and proceeds to the next processing step.

Step 3:

The server generates a prompt based on the received data. Specifically, it combines the user's height, weight, and diet goal to create a prompt to send to the generative AI model. For example, a prompt might be generated that reads, "Based on height: 170 cm, weight: 70 kg, goal: to lose 5 kg, please suggest an appropriate meal plan."

Step 4:

The server sends the generated prompt text to the generative AI model. The generative AI model performs data calculations based on the received prompt text and generates an appropriate meal plan. The generative AI model uses, for example, OpenAI GPT-3.

Step 5:

The generative AI model generates a meal plan based on the prompt and sends the results back to the server. An example of a generated meal plan is "Breakfast - Oatmeal, Lunch - Salad, Dinner - Chicken Breast and Vegetables."

Step 6:

The server analyzes the meal plan received from the generative AI model and sends it to the smartphone application. The transmitted data includes the generated meal plan.

Step 7:

The smartphone application presents the meal plan received from the server to the user. The user can review and implement the proposed meal plan through the application.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Under the previous system, customers had to physically submit receipts for their purchases, which was time-consuming and risked loss or damage. Furthermore, there were insufficient ways to identify unnecessary purchases and visualize savings and weight loss. This made it difficult for customers to effectively manage their shopping to achieve their goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to take an image of their purchase receipt after shopping and upload it to the application so that they can submit it, means for correcting items that are determined to be unnecessary for this purchase, and means for visualizing the amount of money saved and weight loss that would be achieved if the unnecessary items were not purchased. This saves the customer the trouble of submitting a physical receipt and allows them to easily communicate their purchase information to the system. Furthermore, by identifying unnecessary items and visualizing the amount of money saved and weight loss that would be achieved, customers can effectively manage their shopping toward their goals.

"Customer" refers to an individual who uses the system to shop and manages their purchase information.

"Height and Weight" refers to the customer's current physical height and weight.

"Goal" refers to a specific physical or financial goal that the client wishes to achieve.

"Purchase receipt" means a paper or electronic record issued to a customer when they make a purchase that contains details of the items purchased.

"Taking an image" refers to the act of taking a photo of a physical receipt using a device such as a smartphone or camera.

"Application" refers to a software program that runs on a device such as a smartphone or tablet.

"Uploading" refers to the act of sending captured images to a server via the Internet.

"Server" refers to a computer system that receives, processes, and stores data over a network.

"Analyzing an image" refers to the act of using image recognition technology to extract the contents of a photographed receipt as digital data.

"Unnecessary products" refer to products that are determined not to be necessary for the customer to achieve their goals.

"Savings" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss you would expect if you had not purchased unnecessary items.

"Visualizing" refers to the act of displaying data visually so that customers can easily understand it.

"Displaying graphically" refers to the act of displaying data in a visual format such as a graph or chart.

This invention provides a system that allows customers to take an image of their purchase receipt after shopping and upload it to an application in order to submit it. A specific embodiment of this system is described below.

The user takes a picture of the purchase receipt using the smartphone camera. The image is then uploaded using a dedicated application (e.g., "ReceiptUploader"). The application has the function of sending the image taken by the user to the server.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted.

The analyzed data is stored in a database by the server. The stored data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database.

Users can check their purchase history using the application. By opening the "Purchase History" section within the application, data retrieved from the server is displayed. Users can view a list of past purchases.

As a concrete example, consider a scenario after a user has shopped at a supermarket. The user takes a picture of the receipt using their smartphone camera and opens the "ReceiptUploader" application. Within the application, the user taps the "Upload" button, selects the image of the receipt, and uploads it. The application sends the image to a server, which uses the Google Cloud Vision API to analyze the contents of the receipt. The analyzed data is stored in a database, and the user can open the "Purchase History" section within the application to view information about recent purchases.

Examples of prompts to input into a generative AI model include:

"Please describe the functionality that allows users to take a picture of a receipt using their smartphone camera and upload it to the application. Please include the names of any specific hardware or software."

Using this prompt, the generative AI model can generate sentences that explain in detail how the system will operate.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user takes a picture of the receipt.

The user uses their smartphone camera to take an image of the receipt for the item they purchased. When taking the photo, they are careful to capture the entire receipt clearly. The input is the physical receipt, and the output is a digital image.

Step 2:

The device uploads the image to the application.

The device (smartphone) uploads the captured image of the receipt to the "ReceiptUploader" application. The user opens the application, taps the "Upload" button, and selects the captured image. The input is a digital image, and the output is the transmission of image data to the server.

Step 3:

The server receives and analyzes the image.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted. The input is image data, and the output is analyzed text data.

Step 4:

The server stores the analysis results in a database.

The server saves the analyzed data in a database. The saved data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database. The input is the analyzed text data, and the output is saved in the database.

Step 5:

The user checks their purchase history.

Users can use the "ReceiptUploader" application to check their purchase history. When they open the "Purchase History" section in the application, data retrieved from the server is displayed. Users can view a list of past purchases. The input is data retrieved from the database, and the output is displayed in the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional shopping systems required customers to physically submit receipts for their purchases, which was time-consuming and made managing purchase information cumbersome. Furthermore, there was a lack of systems that could identify unnecessary products based on customers' health goals and visualize their savings and health benefits.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application; a means for correcting items determined to be unnecessary for the current purchase; a means for visualizing the money saved and weight lost if the unnecessary items were not purchased; a means for analyzing the contents of the purchase receipt and automatically extracting purchase information; a means for rewarding points to the customer based on the extracted purchase information; and a means for allowing the rewarded points to be used for the next purchase. This saves the customer the trouble of submitting a physical receipt, simplifies management of purchase information, and enables visualization of point rewards and savings. It also identifies unnecessary items based on the customer's health goals and visualizes savings and health benefits.

"Customer" refers to a consumer who purchases goods or services.

"Height and weight" refers to basic physical information that refers to the customer's physical height and weight.

A "goal" refers to a specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record detailing a transaction issued to a customer when they purchase goods.

A "smartphone" is a type of mobile phone that can connect to the Internet and use applications.

A "camera" is a device used to take images and videos.

An "application" is a software program that runs on a smartphone or computer.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Unnecessary products" are products that are deemed not necessary for achieving the customer's goals.

"Correction" is the act of checking the content and making corrections or improvements.

"Money saved" is the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" refers to the visual display of data or information.

"Analysis" is the act of examining data or information in detail to clarify its structure and meaning.

"Purchase information" refers to detailed information about the products purchased by the customer.

"Automatic extraction" means that the system extracts data without human intervention.

"Point redemption" is the act of awarding points to customers based on the purchase amount.

"Can be used on next purchase" means that the customer can use the points the next time they purchase a product.

The following describes in detail the mode for implementing this invention.

First, a smartphone application is used by the customer to enter their current height, weight, and goals. The customer launches the application and enters their height, weight, and goals. This registers the customer's health goals in the system.

Next, we provide a way for customers to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application. Customers take a photo of their receipt and use the upload function within the application to send the image to the server.

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. Next, it uses pytesseract to extract text from the preprocessed image. It then analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text.

Based on the analyzed purchase information, the application provides a means for rewarding customers with points. The server calculates points based on the purchase amount and adds them to the customer's account. Rewarded points are managed within the application so they can be used for the next purchase.

Furthermore, as a means of filtering out items that may be deemed unnecessary for this shopping trip, an algorithm is implemented to determine unnecessary items based on the customer's height, weight, and goals. The server compares purchase information with the customer's health goals to identify unnecessary items.

To visualize the money saved and weight lost by not buying unnecessary products, the server calculates the amount saved and weight lost and displays them graphically. This allows customers to visually see the savings and health benefits.

As a concrete example, consider the case where a user purchases a drink at a convenience store, takes a photo of the receipt, and uploads it to the app. The app extracts "drink" and "150 yen" from the receipt and rewards 15 points. Furthermore, if the drink is deemed an unnecessary item based on the user's health goals, the amount of money saved and the weight lost will be displayed.

Examples of prompts for a generative AI model are as follows:

"A user has uploaded a receipt image taken with their smartphone camera. Please create a program that extracts purchase information from this image and rewards points. 10% of the purchase amount will be rewarded as points."

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goals. The entered data is temporarily saved within the application and sent to the server. The entered data is used as the basis for setting the user's health goals.

Step 2:

After shopping, the user takes a photo of the purchase receipt with their smartphone camera and uploads it to the application. The captured receipt image is temporarily saved within the application and then sent to the server. The input data is the receipt image.

Step 3:

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. The input data is the receipt image, and the output data is the preprocessed image.

Step 4:

The server extracts text from the preprocessed image using pytesseract. The input data is the preprocessed image, and the output data is the extracted text. Specifically, it uses OCR (optical character recognition) technology to read the text information in the image.

Step 5:

The server analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text. The input data is the extracted text, and the output data is the analyzed purchase information. Specifically, it uses a text analysis algorithm to extract the required information.

Step 6:

The server rewards the customer with points based on the analyzed purchase information. The input data is the analyzed purchase information, and the output data is the rewarded points. Specifically, it calculates points based on the purchase amount and adds them to the customer's account.

Step 7:

The server determines unnecessary products based on the customer's height, weight, and goals. The input data is the customer's health goals and purchase information, and the output data is a list of unnecessary products. Specifically, it runs an algorithm that compares the health goals with the purchase information and identifies unnecessary products.

Step 8:

The server calculates the amount of money saved and weight lost if unnecessary items are not purchased, and displays this graphically. The input data is a list of unnecessary items and purchase information, and the output data is the amount saved and weight lost. Specifically, it calculates the savings and health benefits and displays them visually to the user.

Step 9:

The user uses the points they have earned on their next purchase. The input data is the points earned, and the output data is the balance after the points have been used. Specific operations include using the points to discount the purchase amount and updating the balance.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Today's consumers find it difficult to balance health management and saving money. Consumers with diet or health goals in particular have difficulty determining which products are unnecessary when shopping. Furthermore, there are few ways to specifically assess the savings and health benefits of not purchasing unnecessary products. For this reason, a system that helps consumers effectively manage their health and save money is needed.

The identification processing by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for converting the submitted receipt information into text data using optical character recognition technology, means for determining unnecessary products based on the customer's height, weight, and goals, means for calculating the amount of money saved and the weight lost if the unnecessary products are not purchased, and means for graphically displaying the calculation results. This allows consumers to easily identify unnecessary products when shopping and specifically understand the amount of money saved and the health benefits.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and Weight" refers to the customer's current physical measurements and is used as basic data for health management.

"Goals" are specific health and diet objectives set by customers, and include, for example, "carbohydrate restriction" or "calorie restriction."

A "receipt" is a detailed list of purchased items that a customer receives after shopping, listing the product names and prices.

"Optical character recognition technology" is a technology that extracts character information from image data and is used to convert receipt images into text data.

"Text data" refers to text information converted using optical character recognition technology, and includes information such as product names and prices.

"Unnecessary products" are products that are determined not to be necessary to purchase from the perspective of health management or dieting, based on the customer's height, weight, and goals.

"Amount saved" is the total amount of money you can save by not purchasing unnecessary items.

"Weight loss" is the amount of weight loss predicted if the unnecessary product is not consumed.

"Displaying graphically" means displaying the calculation results in the form of graphs, charts, etc. to make them easier to understand visually.

Form for implementing the invention

This invention is a system that enables consumers to effectively manage their health and save money. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. A device for users to enter their height, weight, and goals.

2. A terminal where users can submit receipts after shopping.

3. Hardware and software for the server to receive and process data

Hardware and software used

Device: Mobile devices such as smartphones and tablets

Server: High-performance data processing server

OCR technology: Optical character recognition technology such as Google Cloud Vision API

Database: Relational database such as MySQL or PostgreSQL

Data visualization libraries: D3.js, Chart.js, etc.

Program processing

The server receives the height, weight, and goal information entered by the user, as well as the submitted receipt. Using this data, the server processes and calculates the data to detect unwanted products. Specifically, the process is carried out as follows:

1. Data reception and analysis:

The server receives the user's input height, weight, and goal (e.g., carbohydrate restriction).

The server converts the information on the submitted receipt into text data using OCR technology.

2. Detecting unwanted products:

The server detects unnecessary products based on the goal. For example, if the goal is to "limit carbohydrates," the server will list high-carbohydrate products on the receipt, such as "bread" and "sweets," as unnecessary products.

3. Calculate savings and weight loss:

The server calculates the total price of the unwanted items detected.

The server calculates the estimated weight loss that would occur if the unnecessary products were not consumed.

4. Visualizing the results:

The server displays the calculation results to the consumer in the form of graphs, charts, etc.

Specific examples

Let's say a user is 170cm tall, weighs 70kg, sets their goal to "restrict carbohydrates," and submits a receipt. The receipt includes "bread (300 yen)," "sweets (200 yen)," and "vegetables (100 yen)." The system detects "bread" and "sweets" as unnecessary items and calculates that a total of 500 yen can be saved. The system also calculates that the estimated weight loss would be 0.5kg if these items were not consumed, and displays this in a graph.

Example prompt

"Height 170cm, weight 70kg, goal is carbohydrate restriction. Detect unnecessary items based on the following receipt information and calculate the amount saved and weight loss. Receipt information: bread (300 yen), sweets (200 yen), vegetables (100 yen)." The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

Users enter their height, weight, and goal, then submit a receipt.

Input: The user opens the app on their smartphone and enters their height (170cm), weight (70kg), and goal of "restricting carbohydrates." After shopping, they also take a photo of the receipt with their smartphone camera and upload it to the server via the app.

Output: User input data and receipt image are sent to the server.

Step 2:

The server receives the input data and receipt information.

Input: Height, weight, goal, and receipt image submitted by the user.

Output: User input data and receipt image stored in a database on the server.

Step 3:

The server converts the receipt information into text data using OCR technology.

Input: Receipt image stored on the server.

Data processing: The server calls the Google Cloud Vision API, analyzes the receipt image, and extracts the product name and price.

Output: Converted text data (e.g., "Bread (300 yen)", "Sweets (200 yen)", "Vegetables (100 yen)").

Step 4:

The server detects unwanted items based on your goals.

Input: User's goal (e.g., "Limit carbohydrates") and converted text data.

Data calculation: The server performs rule-based filtering based on predefined food categories and goals to list unwanted products.

Output: List of unwanted products (e.g., "bread", "sweets").

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

Input: List of unwanted items and their respective prices.

Data calculation: The server calculates the total price of the unnecessary products and also calculates the estimated weight loss if the unnecessary products were not consumed. The model used here is a simulation model based on the relationship between calories consumed and weight loss.

Output: Amount saved (e.g., 500 yen) and estimated weight loss (e.g., 0.5 kg).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them to the user.

Enter: Amount saved and estimated weight loss.

Data processing: The server uses data visualization libraries such as D3.js and Chart.js to visualize the calculation results in graphs and charts.

Output: Graphs and charts displayed on the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Today's consumers find it difficult to simultaneously manage their health and save money. In particular, consumers with dietary goals need to identify and avoid unnecessary products from their purchases in order to achieve health and financial benefits. However, doing this manually is time-consuming and inaccurate. Therefore, a system is needed that allows consumers to easily identify unnecessary products and efficiently manage their savings and health.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for analyzing receipt information and automatically detecting unnecessary items based on diet goals, means for calculating the amount of money saved and estimated weight loss if the unnecessary items are not purchased and displaying them in graphs and charts, and an application installed on a smartphone. This allows consumers to easily identify unnecessary items and efficiently save money and manage their health.

"Customer" refers to consumers who use the system.

"Height and Weight" is data indicating the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

A "purchase receipt" is paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unwanted products" refer to products that are deemed inappropriate or unnecessary for the customer's goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Receipt information" refers to data such as the product name, price, and quantity listed on the purchase receipt.

"Diet Goals" refers to health goals set by a customer, such as weight loss or specific nutritional restrictions.

"Automatically detect" means that the system identifies unwanted products programmatically without human intervention.

"Graphs and charts" refer to diagrams used to visually display data.

"Applications installed on smartphones" refers to software programs that run on smartphones.

This invention is a system that allows consumers to efficiently manage their health and save money. Specific embodiments for implementing this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A mobile device such as a smartphone.

2. Server: A central server that processes and analyzes data.

3. Application: A dedicated application installed on a smartphone.

Program processing

User device

The user device has the following functions:

User information input: Users enter their height, weight, and diet goals through the application.

Receipt scanning: The user scans the receipt for the item they purchased with their smartphone camera and obtains the image data.

Server

The server performs the following process:

Analyzing receipt information: The acquired receipt image is converted into text data using OCR (optical character recognition) technology. This process uses the pytesseract library.

Detection of unnecessary products: Automatically detect unnecessary products from receipt text data based on the user's diet goals. For example, for a user aiming to limit carbohydrate intake, high-carbohydrate products such as "bread" and "sweets" will be detected.

Calculation of savings and weight loss: Calculates the total cost of the detected unnecessary products and the estimated weight loss if you did not consume those products. Weight loss is calculated based on the calories required to lose 1 kg of weight (approximately 7700 kcal).

application

The application has the following features:

Visualization of results: Calculated savings and estimated weight loss are displayed in graphs and charts, allowing users to visually see the effects of their health management and savings.

Specific examples

For example, if a user sets a goal of "restricting carbohydrates" and the receipt includes "bread" and "sweets," these items will be detected as unnecessary products. Suppose the total amount of the detected unnecessary products is 500 yen, and the estimated weight loss is 0.2 kg.

Example prompt

Please enter your user information:

Height: 170 cm

Weight: 70 kg

Diet goal: Carbohydrate restriction

Scan your receipt:

(Upload receipt image)

Unwanted items detected...

Unwanted items detected:

Bread: 200 yen Sweets: 300 yen Savings: 500 yen

Estimated weight loss: 0.2kg

The results are displayed in a graph.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goals.

Input: User's height, weight, and diet goal

Output: User information data

Specific operation: The user enters their height, weight, and diet goal (e.g., carbohydrate restriction) into the application's input form and presses the submit button. This generates user information data and sends it to the server.

Step 2:

After shopping, the user scans the receipt for the purchased items using their smartphone camera.

Input: Receipt image

Output: Receipt image data

Specific operation: The user uses the application's camera function to take a photo of the purchase receipt. The captured receipt image is saved within the application and sent to the server.

Step 3:

The server receives the receipt image and converts it into text data using OCR technology.

Input: Receipt image data

Output: Receipt text data

Specific operation: The server uses the pytesseract library to extract text from the received receipt image data. The extracted text data is saved as receipt text data.

Step 4:

The server automatically detects unnecessary items from receipt text data based on the user's diet goals.

Input: User information data, receipt text data

Output: List of unwanted items

Specific operation: The server analyzes receipt text data based on the user's diet goal (e.g., carbohydrate restriction). Products containing specific keywords (e.g., "bread," "sweets") are listed as unwanted items.

Step 5:

The server calculates the amount of money you could save and the estimated weight loss if you didn't buy unnecessary items.

Input: List of unwanted items

Output: Money saved, estimated weight loss

Specific operation: The server calculates the savings amount by adding up the prices of each item based on the list of unwanted items. It also calculates the estimated weight loss amount based on the calorie information of each item. The weight loss amount is calculated based on the calories required to lose 1 kg (approximately 7700 kcal).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them on the user's smartphone.

Input: Amount saved, estimated weight loss

Output: Image data for graphs and charts

Specific operation: Based on the calculation results, the server generates graphs and charts showing the amount of savings and estimated weight loss. Image data of the generated graphs and charts is sent to the user's smartphone and displayed within the application.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products. When the user is relaxed, it recommends healthy foods.

"Example 2"

Furthermore, the emotion engine graphically displays the money saved and weight lost based on the user's emotions. Specifically, it displays the amount of money saved and the amount of weight lost if the user refrains from purchasing unnecessary products when feeling stressed. This allows users to visually understand the relationship between their emotions, health management, and saving money.

"Example 3"

For example, if a user buys chocolate when feeling stressed, the emotion engine will detect the chocolate as an unnecessary item and display a graph of the amount of money saved and weight loss they would have experienced if they had not bought the chocolate. This helps users recognize their tendency to buy unhealthy foods when they are stressed and understand the resulting financial and health impacts.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine determines which products are unnecessary based on the user's emotions and notifies the user of the results.

"Example 2"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine graphically displays the money saved and weight lost based on the user's emotions.

"Example 3"

Step 1: Buy chocolate when the user is feeling stressed.

Step 2: The emotion engine recognizes the user's emotions and detects chocolate as an unwanted item.

Step 3: The emotion engine displays a graph of the amount of money saved and weight loss you would have experienced if you had not bought the chocolate.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

The previous system allowed customers to enter their height, weight, and goals, and to submit receipts after shopping, but it lacked the ability to identify unnecessary products based on the customer's emotions and recommend healthy products. This made it difficult for customers to select appropriate products when they were stressed or relaxed. It also lacked a way to visualize the money saved and weight lost by not purchasing unnecessary items. This made it difficult to raise customers' awareness of health management and saving money.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for recognizing the customer's emotions and determining unnecessary items based on those emotions, and means for recommending healthy products based on the customer's emotions. This enables the customer to select appropriate products according to their emotional state, thereby increasing their awareness of health management and saving money.

"Customer" refers to an individual who uses the system to enter their height, weight, and diet goals, and then submits a purchase receipt after shopping.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to specific numerical values or conditions related to dieting or health that a customer wishes to achieve.

"Input means" refers to the interface that allows customers to input their height, weight, and goals into the system.

"Purchase receipt" refers to the paper or electronic data containing the purchase details of a product that a customer receives when making a purchase.

"Submission means" refers to the interface through which customers submit purchase receipts to the system.

"Correction method" refers to the function by which the system determines unnecessary products based on customer input data and emotional data and notifies the customer.

"Visualization means" refers to the system's ability to graphically display the money saved and weight lost if unnecessary items are not purchased.

"Emotion recognition means" refers to the system's ability to analyze customer emotions and determine which products are unnecessary based on those emotions.

"Healthy product recommendation means" refers to the system's ability to recommend healthy products based on customer sentiment.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is set up on a website or smartphone application, and allows customers to directly input their height, weight, diet goals, etc. In addition, by combining it with an emotion engine, it has the function of determining unnecessary products based on the customer's emotions and recommending healthy products.

The server places an input form on a website or smartphone application. This input form is created using HTML and CSS, and JavaScript is used to validate the input data in real time. When a user enters data into the input form, the server receives the data and stores it in a database (for example, MySQL or PostgreSQL).

Next, the server uses an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) to analyze the user's emotions. The emotion engine recognizes the user's current emotions based on the user's input data and past behavioral data. For example, if the user is feeling stressed, the server will use that information to determine that unhealthy foods such as sweets are unnecessary. On the other hand, if the user is relaxed, the server will recommend healthy foods.

As a concrete example, suppose a user uses a smartphone application to enter their height as 170cm, weight as 70kg, and diet goal as "lose 5kg." This data is sent to the server and stored in a database. The emotion engine then analyzes the user's emotions, and if it determines that the user is feeling stressed, the server displays a message such as "Avoid sweet snacks such as chocolate and cookies."

Example of a prompt to input to a generative AI model:

A user enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg." The emotion engine analyzes the user's emotions and determines that the user is feeling stressed. In this case, what products would be deemed unnecessary and what products would be recommended?

By inputting this prompt into a generative AI model, specific product recommendations and a determination of unnecessary products can be obtained. In this way, the system can make appropriate product recommendations based on the user's input data and emotional data.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application. The device sends a request to the server through the browser or application to retrieve an input form. The server generates the input form using HTML and CSS and sends it to the device. The device renders the received HTML and CSS and displays the input form to the user.

Input: User access request

Output: Display the input form

Step 2:

Users enter their height, weight, and diet goals.

The user enters their height, weight, and diet goals into the displayed input form. The device uses JavaScript to validate the format of the input data in real time to ensure there are no errors.

Input: User's height, weight, and diet goal

Output: Validated input data

Step 3:

The server receives the input data and stores it in the database.

When the user presses the send button, the device sends the input data in JSON format to the server. The server parses the received data and saves it in a MySQL database. Specifically, it adds the data to the database using the INSERT statement.

Input: JSON format input data

Output: Data stored in the database

Step 4:

The server calls the emotion engine and analyzes the user's emotions.

The server calls an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) based on the stored data. The data is sent to the API, which analyzes the user's emotions. The API determines that the user is feeling stressed and returns the result to the server.

Input: Data stored in the database

Output: Sentiment analysis results from the emotion engine

Step 5:

The server determines which products are unnecessary based on emotions and decides which products to recommend.

The server determines which products are unnecessary and which are recommended based on emotional data obtained from the emotion engine. For example, if the user is feeling stressed, the server will determine that "sweets such as chocolate and cookies should be avoided" and decide to "recommend healthy foods such as fruits and nuts."

Input: Sentiment analysis results from the emotion engine

Output: List of unwanted and recommended products

Step 6:

The server displays the results to the user.

The server generates HTML to display the results of the judgment to the user and sends it to the device. The device renders the received HTML and displays unwanted products and recommended products to the user. The user checks the displayed results and decides on their next course of action.

Input: List of unwanted and recommended products

Output: The result displayed to the user

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional food delivery systems don't recommend foods based on the user's health status or emotions, which can lead to users choosing unhealthy foods when they're stressed. Furthermore, there's a lack of support for users to choose foods that align with their diet goals, making it difficult to diet effectively.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for the customer to enter their current height and weight and goals, a means for the customer to submit a purchase receipt after shopping, a means for correcting items determined to be unnecessary in this purchase, a means for visualizing the money saved and weight lost if the unnecessary items are not purchased, and a means for recognizing the customer's emotions and determining unnecessary items based on those emotions. This makes it possible to recommend foods based on the user's health condition and emotions, allowing the user to avoid unhealthy foods when feeling stressed and to choose foods that are in line with their diet goals.

"Customer" refers to a consumer who uses a service or product.

"Height and weight" are numbers that indicate an individual's physical height and weight.

"Goal" refers to the specific objective or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record of a transaction issued when a customer purchases a product.

"Unnecessary products" refer to products that are deemed inappropriate for the customer's health condition or goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight Loss" is a numerical value that represents the weight loss a customer can achieve by avoiding unnecessary products.

"Means for recognizing emotions" refers to technologies or devices that detect a customer's emotional state and make decisions based on that.

"Visualization means" refers to technology or devices that visually display the money saved or weight lost.

A system for implementing this invention is configured as follows: First, a dedicated form is provided for users to enter their height, weight, and diet goals. This form is provided on a website or smartphone application, allowing users to easily enter information.

Next, we present a system that combines an emotion engine. This system recognizes the user's emotions in real time and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products, and when the user is relaxed, it recommends healthy foods.

The following hardware and software are used to implement this system:

Hardware: Smartphone, tablet, or PC

Software: Python, EmotionRecognizer library, FoodRecommender library

The server receives the user's input data such as height, weight, and diet goal, and uses the EmotionRecognizer library to recognize the user's emotions in real time. It then uses the FoodRecommender library to recommend appropriate foods based on the user's input data and the recognized emotions.

For example, if a user enters their height as 170cm, weight as 65kg, and diet goal as "lose weight," and the emotion engine recognizes that the user is feeling stressed, the system will avoid unhealthy foods (e.g., sweet snacks) and instead recommend healthy foods (e.g., salads and fruit).

Examples of prompt statements include:

"If a user is 170cm tall, weighs 65kg, and their diet goal is to lose weight, and the emotion engine determines that the user is feeling stressed, what foods would you recommend?"

In this way, food recommendations can be made based on the user's health status and emotions, helping users avoid unhealthy foods when they are stressed and choosing foods that align with their diet goals.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The user uses the terminal to input their height, weight, and diet goal. The input data is sent to the server. The input data includes the user's height (e.g., 170 cm), weight (e.g., 65 kg), and diet goal (e.g., "lose weight").

Step 2:

The server stores the received user input data and uses the EmotionRecognizer library to recognize the user's emotions in real time. For emotion recognition, the user's facial expressions and voice data are used as input. The output is the user's emotional state (e.g., stressed, relaxed).

Step 3:

The server uses the FoodRecommender library to recommend appropriate foods based on the recognized emotional state and user input data. Input data includes the user's height, weight, diet goals, and emotional state. The output is a list of recommended foods (e.g., salads, fruits).

Step 4:

The server sends a list of recommended foods to the user's device, where the user can check the list of recommended foods on their device. Specifically, if the user is feeling stressed, unhealthy foods will be avoided and healthy foods will be recommended.

Step 5:

A user makes a purchase and submits the purchase receipt to the server using a terminal. The server analyzes the submitted receipt data and generates a list of the purchased items. Input data includes receipt images or electronic data. The output is a list of the purchased items.

Step 6:

The server compares the list of purchased items with the user's diet goals and emotional state to determine which items are unnecessary. Specific operations include detecting items that are inappropriate for the user's goals (e.g., sweet snacks). Input data includes the list of purchased items, the user's diet goals, and their emotional state. The output is a list of unnecessary items.

Step 7:

The server calculates and visualizes the money saved and weight lost if unnecessary items are not purchased. Specific operations include graphically displaying the amount saved and weight lost. The input data includes a list of unnecessary items. The output is a graphical representation of the amount saved and weight lost.

Step 8:

The server sends the visualized data to the user's device, where the user can check the amount of money saved and the weight lost on the device. Specifically, the user can visually understand the benefits they can gain by avoiding unnecessary products.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

With conventional health management systems, customers had to submit physical proof of purchase after shopping, which was time-consuming. Furthermore, the amount of savings and weight loss were not visualized taking into account the customer's emotional state, making it difficult for customers to understand the relationship between their emotions, health management, and savings.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current physical information and goals, means for the customer to submit a proof of purchase after shopping, means for the customer to capture an image of the proof of purchase using the camera function of their mobile device and upload it to the application, means for the server to convert the contents of the proof of purchase into text data using image processing technology, means for the server to save the converted text data, means for the customer to input emotional data, means for the server to analyze the customer's emotional data using emotion analysis technology, means for the server to determine unnecessary products based on the analysis results and calculate the amount of savings and weight loss, and means for the server to graphically display the calculation results. This allows customers to communicate their purchase information to the system without having to directly submit a physical proof of purchase, and further enables them to visually understand the relationship between their emotions, health management, and savings.

"Customer" refers to an individual who uses the system to manage their own health and save money.

"Physical information" refers to physical data such as the customer's height, weight, and body fat percentage.

"Goals" refers to the health or savings goals the customer wants to achieve.

"Proof of purchase" refers to a document that proves a purchase, such as a receipt or invoice that a customer receives after shopping.

"Mobile device" refers to portable electronic devices such as smartphones and tablets.

"Photography function" refers to the camera function built into the mobile device.

"Application" refers to a software program that runs on a mobile device.

"Image processing technology" refers to the technology of analyzing image data and extracting necessary information.

"Text data" refers to text information extracted using image processing technology.

"Emotional data" refers to data that indicates a customer's emotional state.

"Emotion analysis technology" refers to technology that analyzes emotional data and evaluates a customer's emotional state.

"Unnecessary products" refer to products that are determined not to be necessary for the customer's health or savings goals.

"Amount saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight you would lose if you had not purchased unnecessary items.

"Displaying graphically" refers to displaying the calculation results in a visually easy-to-understand manner.

Form for implementing the invention

This invention is a system in which customers submit a purchase certificate after shopping and visualize the amount of savings and weight loss based on emotional data. A specific embodiment of this system is described below.

System Configuration

This system primarily uses the following hardware and software:

Mobile devices (smartphones and tablets)

Camera function

application

Server

Image processing software (e.g., OpenCV, Tesseract)

Emotion analysis technology

Program processing

After shopping, the user uses the camera function on their mobile device to take a picture of the purchase certificate. The image is then uploaded to the server via the application. The server then uses image processing software (e.g., OpenCV or Tesseract) to extract text from the image and convert information such as the name, price, and quantity of the purchased item into text data. The server then stores this text data in a database.

Next, the user enters their emotional data through the application. The emotional data may be entered manually by the user or automatically obtained from the wearable device. The server then analyzes the emotional data using emotion analysis technology to assess whether the user is feeling stressed and to what extent.

Based on the results of the emotion analysis, the server calculates the amount of money saved and the amount of weight lost if the user refrains from buying unnecessary items when feeling stressed. For example, if the user refrains from buying chocolate when feeling stressed, the calculation is based on the price (200 yen) and calories (300 kcal). The calculation results are displayed graphically by the server, and the user can check them through the application.

Specific examples

As a concrete example, consider the case where a user decides not to buy chocolate when they are feeling stressed. The user takes a photo of the receipt with their smartphone camera and uploads it to the application. The server uses image processing software to convert the contents of the receipt into text data and stores it in a database. Next, the user enters emotional data into the application, which the server analyzes using emotion analysis technology. Based on the analysis results, the server calculates the amount of savings and weight loss based on the price of the chocolate (200 yen) and calories (300 kcal). Finally, the server displays the calculation results graphically, and the user can check them through the application.

Example prompt

Example of a prompt to input to a generative AI model:

"Generate a program that calculates and graphically displays the amount of money saved and weight loss a user would achieve if they avoided buying unnecessary items when they were feeling stressed."

In this way, users can visually understand the relationship between their emotions, health management, and saving money.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user takes a photo of the purchase certificate using their smartphone camera.

Enter: Proof of purchase

Output: Image of proof of purchase

Specific operation: The user activates the smartphone camera, frames the purchase certificate, and presses the capture button. The captured image is saved to the smartphone's gallery.

Step 2:

Upload the image of the purchase certificate taken by the device to the application.

Input: Image of proof of purchase

Output: Image data sent to the server

Specific operation: The user launches the application, selects an image of the purchase certificate, and presses the upload button. The device then sends the selected image to the server.

Step 3:

The server uses image processing technology to convert the contents of the purchase certificate into text data.

Input: Image data of purchase certificate

Output: Text data

Specific operation: The server analyzes the received image data using image processing software (e.g., OpenCV, Tesseract) and extracts text information from the image. The extracted text information is converted into text data such as the name, price, and quantity of the purchased item.

Step 4:

The server stores the converted text data in a database.

Input: Text data

Output: Purchase information stored in the database

Specific operation: The server saves the converted text data in a database. The saved data includes the purchase date and time, store name, product information, etc.

Step 5:

The user inputs emotional data into the application.

Input: Emotion data

Output: Emotion data sent to the server

Specific operation: The user inputs their emotional state through the application. Emotional data includes information such as stress level and mood. It may also be obtained automatically from the wearable device.

Step 6:

The server analyzes the user's emotional data using emotion analysis technology.

Input: Emotion data

Output: Analysis results

Specific operation: The server uses emotion analysis technology to analyze emotional data and evaluate whether the user is feeling stressed and to what extent. The analysis results are displayed as numbers and graphs.

Step 7:

The server determines which items are unnecessary based on the analysis results and calculates the amount of money saved and the weight lost.

Input: Analysis results, text data

Output: Savings and weight loss data

Specific operation: Based on the results of emotion analysis and purchase information, the server calculates the amount of money saved and the amount of weight lost if the user does not buy unnecessary items when feeling stressed. For example, the calculation is based on the price (200 yen) and calories (300 kcal) of chocolate.

Step 8:

The server displays the calculation results graphically.

Input: Savings and weight loss data

Output: Graphical display

Specific operation: The server generates data for graphically displaying the calculation results and provides it to the user through the application. The user can visually check the amount of savings and weight loss through the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional shopping support systems required customers to physically submit receipts for purchased items, which was a time-consuming process. Furthermore, the shopping support did not take into account the customer's emotions or health status, leading to unnecessary purchases when they were feeling stressed. This led to wasteful spending and negative health effects, which was an issue.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to the application; a means for correcting items that are determined to be unnecessary in this purchase; a means for visualizing the money saved and weight lost if the customer does not buy unnecessary items; and a means for analyzing the customer's emotions and graphically displaying the amount of money saved and weight lost if the customer does not buy unnecessary items when they are feeling stressed. This saves the customer the trouble of submitting physical receipts, and enables shopping support that takes their emotions and health state into consideration.

"Customer" refers to a consumer who purchases goods or services.

"Height and Weight" refers to the customer's physical height and weight.

"Goal" refers to the specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a proof of purchase issued when a customer purchases a product.

A "smartphone camera" is a photographic device built into a smartphone.

An "application" is a software program that runs on a smartphone or other device.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Correction" is the act of checking the content and correcting or deleting unnecessary parts.

"Unnecessary items" are items that customers have determined they do not need to purchase.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" is the act of visually displaying data or information.

"Analyzing emotions" is the process of assessing and understanding a customer's emotional state.

"Stress" refers to mental or physical tension or strain.

"Graphically displayed" means displaying information in a visual format such as a graph or chart.

This invention provides a system that allows customers to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to an application. It also includes a function that analyzes the customer's emotions and graphically displays the amount of money saved and the amount of weight lost if they avoid buying unnecessary items when they are feeling stressed.

System Configuration

1. Hardware:

Smartphone: Use the camera function to take a photo of your purchase receipt.

Server: Serves as the backend for the application, storing and processing data.

2. Software:

Application: Installed on a smartphone, it takes photos of receipts, uploads them, performs sentiment analysis, and displays them graphically.

OpenCV: A library for loading and preprocessing receipt images.

Tesseract OCR: Optical character recognition software for extracting text from receipt images.

EmotionRecognizer: Emotion recognition software for analyzing customer emotions.

Matplotlib: A library for graphically displaying savings and weight loss.

System Operation

1. Take a photo of your receipt and upload it:

Users take a photo of their purchase receipt using their smartphone camera.

The captured images are uploaded to the server via the application.

2. Text extraction:

The server preprocesses the image using OpenCV and extracts text from the receipt using Tesseract OCR.

3. Sentiment analysis:

The server analyzes the user's emotions using EmotionRecognizer.

When you're feeling stressed, calculate how much money you'll save and how much weight you'll lose if you don't buy unnecessary items.

4. Graphical display:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

The application will display this visually to the user.

Specific examples

For example, if a user avoids buying unnecessary items when feeling stressed, the application will graphically display a savings of 1,000 yen and a weight loss of 0.5 kg.

Example prompt

Create a program that calculates and graphically displays the savings and weight loss that a user could achieve by not purchasing unnecessary items when they are feeling stressed. Take a photo of the receipt with a smartphone camera and extract the text using Tesseract OCR. Analyze the user's emotions using EmotionRecognizer and display the results using Matplotlib.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

Users take a photo of their purchase receipt using their smartphone camera.

Input: Physical image of purchase receipt

Output: Receipt image saved on your smartphone

Specific operation: The user launches the smartphone's camera app and takes a photo of the purchase receipt. The captured image is saved to the smartphone's storage.

Step 2:

The device will upload the captured receipt image to the application.

Input: Receipt image saved on your smartphone

Output: Receipt image uploaded to the server

Specific operation: The user launches the application, selects the captured receipt image, and presses the upload button. The image is sent to the server via the Internet.

Step 3:

The server preprocesses the receipt image using OpenCV and extracts text using Tesseract OCR.

Input: Receipt image uploaded to the server

Output: Extracted text data

Specific operation: The server uses OpenCV to remove noise from the image and highlight the text. Then, it uses Tesseract OCR to extract the text from the image.

Step 4:

The server analyzes the user's emotions using EmotionRecognizer.

Input: User emotional data (e.g., the emotional state the user enters into the application)

Output: Analyzed emotional state

Specific operation: The server uses EmotionRecognizer to analyze the user's emotional data and determine whether they are under stress.

Step 5:

When you feel stressed, the server calculates the amount of money you would save and the amount of weight you would lose if you didn't buy unnecessary items.

Input: Analyzed emotional state, extracted text data

Output: Savings and weight loss data

Specific operation: If the emotional state is stressed, the server identifies unnecessary items from the extracted text data and calculates the amount of savings and weight loss that would result if the item were not purchased.

Step 6:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

Input: Savings and weight loss data

Output: Image data for graphical display

Specific operation: The server uses Matplotlib to generate images that visually display the amount of savings and weight loss in the form of bar graphs, etc., and sends the image data to the application.

Step 7:

The terminal displays the graphical data received from the server to the user.

Input: Graphical image data sent from the server

Output: Graphical information displayed on a smartphone screen

Specific operation: The application displays the image data received from the server on the smartphone screen, allowing the user to visually check the amount of savings and weight loss.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Today's consumers aim to balance health management with saving money, but they often end up buying unnecessary products in their daily shopping. These unnecessary products not only go against their health goals but also lead to wasteful spending. However, it is not easy for consumers to identify and avoid these unnecessary products. Therefore, there is a need for a system that automatically detects unnecessary products based on consumers' health goals and visualizes the savings and health impact.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for customers to input their current height and weight and goals, means for customers to upload an image of their purchase receipt after shopping, means for converting the receipt image into digital data using optical character recognition technology, means for analyzing product information from the digital data and detecting unnecessary products, means for calculating the total price of unnecessary products and estimated weight loss, and means for visualizing the calculation results in graphs and charts. This allows consumers to automatically detect unnecessary products based on their own health goals and specifically understand the amount of money they can save and the impact on their health.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and weight" refers to the customer's basic physical data and is fundamental information for setting health goals.

"Goals" refer to the health and weight management goals that a customer wishes to achieve, such as carbohydrate restriction or calorie restriction.

"Receipt image" refers to photo data of the purchase details that a customer obtains after shopping.

"Optical character recognition technology" is a technology that converts text information in an image into digital data, and is also known as OCR (Optical Character Recognition).

"Digital data" means text information converted using optical character recognition technology.

"Product information" refers to data such as the product name, price, and nutritional information listed on the receipt.

"Unnecessary products" are products that are contrary to the customer's health goals and should be avoided.

"Total amount" is the total price of the unwanted items detected.

"Estimated weight loss" is the amount of weight loss predicted if the unnecessary product was not consumed.

"Graphs and charts" are diagrams that visually display calculation results.

This invention is a system that allows consumers to efficiently manage their own health and save money. Specific embodiments of this system are described below.

First, the user accesses the system's web page using a terminal. The user enters their height, weight, and health goals (e.g., carbohydrate restriction). This data is sent to the server and stored in a database.

Next, the user uploads an image of the receipt obtained after shopping from their device to the system. The uploaded receipt image is sent to the server, which then uses optical character recognition technology such as Google Cloud Vision API to convert the receipt image into digital data. This digital data includes the text information on the receipt.

The server analyzes the converted digital data and extracts product names and their nutritional information. For example, if a receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

The server then calculates the total cost of the unnecessary items, as well as the estimated weight loss that would occur if these items were not consumed. A standard calorie expenditure model is used to calculate weight loss. The results are visualized as graphs and charts using a data visualization library such as D3.js.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary products.

As a concrete example, consider the case where a user purchases chocolate when feeling stressed. The emotion engine analyzes the user's emotional state and determines that stress is high. In this case, it detects chocolate as an unnecessary item and displays a graph of the amount of money saved and weight loss that would have been achieved if the user had not purchased the chocolate. This allows the user to recognize their tendency to purchase unhealthy foods when they are stressed and understand the resulting financial and health impacts.

Example prompt:

Detect the foods the user purchases when they are stressed, calculate the amount of money saved and the weight loss they would have lost if they had not purchased those foods, and display the results in a graph.

This system allows users to specifically understand both their own health management and savings. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

Users enter their height, weight, and goals.

Using a terminal, a user accesses the system's web page and enters their height (e.g., 170 cm), weight (e.g., 70 kg), and health goal (e.g., carbohydrate restriction) into the form. The entered data is sent to the server and stored in a database.

Input: height, weight, health goals

Output: User information stored in the database

Step 2:

The user uploads an image of the receipt.

Users use their device to upload images of recent purchase receipts to the system. The uploaded images are then sent to the server.

Input: Receipt image

Output: Receipt image saved on the server

Step 3:

The server performs OCR processing on the receipt image.

The server uses optical character recognition (OCR) technology to convert the receipt image into digital data. Specifically, it calls the Google Cloud Vision API to perform the OCR process.

Input: Receipt image

Output: Digital data (text information)

Step 4:

The server analyzes product information and detects unnecessary products.

The server analyzes the digital data obtained through OCR processing and extracts product names and their nutritional information. For example, if a receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

Input: Digital data (text information), user's health goals

Output: List of unwanted items

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

The server calculates the total value of the unwanted products detected, as well as the estimated weight loss that would occur if the user did not consume these products. A standard calorie expenditure model is used to calculate weight loss.

Input: List of unwanted items, product price, calorie information

Output: Total amount, estimated weight loss

Step 6:

The server visualizes the calculation results in graphs and charts.

The server uses a data visualization library such as D3.js to generate graphs and charts showing the total price of unwanted items and estimated weight loss.

Input: Total amount, estimated weight loss

Output: Graphs and charts

Step 7:

The user checks the results.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

Input: Graphs and charts

Output: User understanding and recognition

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Today's consumers are expected to manage their health and save money at the same time, but it can be difficult to determine which products are unnecessary when shopping. Furthermore, there is a lack of ways to grasp the specific amount of savings or weight loss that can be achieved by not purchasing unnecessary items. For this reason, there is a need for a system that allows consumers to make healthy choices and realize the savings they are making.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for customers to enter their current height and weight and goals, means for customers to submit purchase receipts after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and the amount of weight lost, and means for displaying the detection results in a graph. This allows consumers to easily identify unnecessary items when shopping and specifically understand both health management and savings.

"Customer" refers to a consumer who uses the system to make purchases.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

"Purchase receipt" refers to the paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unnecessary items" refer to items that should be removed from the shopping list based on the customer's height, weight, and goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Visualization methods" refers to ways to display the money saved and weight lost in visual form, such as graphs or charts.

"Means of detection" refers to a method for identifying unwanted products based on the customer's height, weight, and goals.

"Means of calculation" refers to a method for calculating the amount of money saved and weight loss that would be achieved by not purchasing unnecessary items.

"Means of displaying in a graph" refers to a method of displaying detection results in a visually easy-to-understand format.

A system for implementing this invention includes means for a customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and weight lost, and means for displaying the detection results in a graph.

System Configuration

The system is implemented as a smartphone application. The application is built using Python and Flask, and performs data processing on the server side. The smartphone sends the user's height, weight, and goals, as well as receipt information, to the server.

Data processing and calculation

The server receives the data sent by the user and performs the following processing:

1. Detecting unwanted products:

The server identifies unnecessary items on the receipt based on the user's height, weight, and goals. For example, if the user's goal is to "limit carbohydrates," it will detect "bread" and "sweets" on the receipt as unnecessary items.

2. Calculate savings and weight loss:

The server calculates the amount of money saved and the amount of weight lost if the unnecessary items were not purchased. For example, it adds up the prices of the unnecessary items and calculates the amount of weight lost based on the calories in those items.

3. Visualizing the results:

The server generates the calculation results in the form of graphs and charts and sends them to a smartphone application. Users can visually check the results through the application.

Hardware and software used

Hardware: Smartphone

Software: Python, Flask

Specific examples

If a user's goal is to "limit carbohydrates" and the receipt includes "bread" and "sweets," the system will detect these as unnecessary items. The savings will be calculated as 350 yen (bread 150 yen + sweets 200 yen), and the weight loss will be 0.064 kg (500 kcal / 7700 kcal). These results are displayed as a graph on the smartphone application.

Example prompt

If a user is aiming to limit carbohydrates, detect unnecessary items in their receipts and calculate savings and weight loss. Use the following data:

Height: 170cm

Weight: 70kg

Goal: Carbohydrate restriction

Receipt items: ["Bread", "Sweets", "Vegetables"]

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goal.

Input: height, weight, goal

Output: Height, weight, and goal data

Specific action: The user enters their height, weight, and goal into the application's input form and presses the submit button.

Step 2:

After shopping, users scan their purchase receipt with their smartphone and upload it to the application.

Input: Purchase receipt image

Output: Receipt text data

Specific operation: The user takes a photo of the receipt using their smartphone camera and uploads it to the application. The application uses OCR technology to extract text data from the image.

Step 3:

The server receives the height, weight, goal, and receipt text data sent by the user.

Input: Height, weight, goal, and receipt text data

Output: Save received data

Specific operation: The server receives the HTTP request and stores the submitted data in the database.

Step 4:

The server detects unwanted items on the receipt based on the user's goals.

Input: Height, weight, goal, and receipt text data

Output: List of unwanted items

Specific operation: The server references the list of unwanted items related to the goal and identifies the unwanted items by matching them with the text data on the receipt.

Step 5:

The server calculates the amount of savings and weight loss you would have achieved if you had not purchased unnecessary items.

Input: List of unwanted items

Output: Savings, weight loss

Specific operation: The server adds up the prices of the unwanted items and calculates the amount of weight loss based on the calories of the unwanted items. For example, if the total price of the unwanted items is 350 yen and the total calories are 500 kcal, the amount of weight loss will be calculated as 0.064 kg (500 kcal / 7700 kcal).

Step 6:

The server generates the calculation results in the form of graphs and charts and sends them to the smartphone application.

Input: Amount saved, weight loss

Output: Image data for graphs and charts

Specific operation: The server generates graphs and charts based on the calculation results and sends them as image data to the smartphone application.

Step 7:

Users can visually check the savings and weight loss results through a smartphone application.

Input: Image data for graphs and charts

Output: Visual results

What it does: The user opens the app and looks at the graphs and charts that are displayed, allowing them to understand the specific savings and weight loss they could have achieved by not purchasing unnecessary items.

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives the voice of the user 20, and

and accepts instructions, etc. from the processor 46. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

This embodiment of the present invention provides a dedicated input form as a means for consumers to enter their height, weight, and diet goals. This input form is provided on a website or smartphone application, and allows consumers to directly input their height, weight, diet goals, etc.

"Example 2"

Next, as a means for consumers to submit their purchase receipts after shopping, we provide a feature that allows them to take an image of the receipt using their smartphone camera and upload it to the application. This feature allows consumers to communicate purchase information to the system without having to directly submit a physical receipt.

"Example 3"

And as a way to correct items that you may not need in this shopping trip, the system uses consumer

Unnecessary items are automatically detected based on the height, weight, and goal entered by the user, as well as the information on the submitted receipt. For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary items.

"Example 4"

Finally, as a means of visualizing the money saved and weight loss that would result from not purchasing unnecessary products, the system calculates the total cost of the detected unnecessary products and the estimated amount of weight loss that would result from not consuming those products, and displays this information to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their own health management and savings.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Consumers enter their height, weight, diet goals, etc. into a dedicated input form. For example, they might enter a height of 170 cm, a weight of 65 kg, and a diet goal of "restricting carbohydrates."

Step 2: Once the information is complete, the consumer presses the "Submit" button to send the information to the system.

"Example 2"

Step 1: The consumer makes a purchase and obtains a receipt for the items purchased.

Step 2: The consumer takes a picture of the receipt using their smartphone camera.

Step 3: The captured image is uploaded to the application and the system reads the receipt information.

"Example 3"

Step 1: The system automatically detects unwanted items based on the consumer's height, weight, and goals entered, as well as the submitted receipt information.

Step 2: For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary items.

"Example 4"

Step 1: The system calculates the total value of the detected unwanted products and the estimated weight loss that would occur if the product were not consumed.

Step 2: The calculation results are displayed to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their health management and savings.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional diet support systems required users to input their height, weight, and diet goals, as well as submit purchase receipts after shopping and identify unwanted items. However, these systems were time-consuming for users, making them difficult to use on an ongoing basis. Furthermore, they lacked the ability to individually suggest appropriate diet plans based on the data entered by the user, making them unable to provide effective diet support.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for receiving and saving the data entered by the customer, means for generating prompt sentences for the generative AI model based on the saved data, and means for the generative AI model to suggest a diet plan. This allows users to use the system continuously without any hassle, and makes it possible to suggest appropriate diet plans individually.

"Customer" refers to the user of the system.

"Height and Weight" refers to the customer's current height and weight data.

"Goals" refers to the diet and health goals set by the customer.

"Input form" refers to the interface where customers enter data such as height, weight, and goals.

"Purchase receipt" refers to the detailed list of purchases that a customer receives after making a purchase.

"Means for correcting" refers to the function that determines unnecessary products based on the customer's height, weight, and goals, and removes them from the list.

"Visualization means" refers to a function that visually displays the money you can save and the weight you will lose if you don't buy unnecessary products.

"Means for receiving and storing data" refers to the function by which the server receives data entered by the customer and stores it in a database, etc.

"Generative AI model" refers to an artificial intelligence model that generates an appropriate diet plan based on input data.

"Prompt sentence" refers to an instruction sentence generated for a generative AI model based on input data.

"Diet Plan" refers to a specific action plan proposed by the generative AI model to help customers achieve their goals.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is installed on a website or smartphone application, and a generative AI model proposes an appropriate diet plan based on the data entered by the customer.

Hardware and software used

Hardware: Servers (e.g., AWS EC2), user devices (e.g., smartphones, PCs)

Software: Web servers (e.g., Apache, Nginx), databases (e.g., MySQL, PostgreSQL), smartphone applications (e.g., iOS apps, Android apps)

Specific system operation

Server

The server provides a dedicated input form on a website or smartphone application. This input form allows customers to enter their height, weight, diet goals, etc. The server receives the input data and stores it in a database. The stored data is used as prompts for the generative AI model.

Device

The device displays the websites and smartphone applications that the user accesses. When the user enters their height, weight, and diet goals into an input form, the device sends the data to a server. The device then displays the data the user has entered in real time, allowing them to check it.

User

Users access a website or smartphone application and enter their height, weight, and diet goals into a dedicated input form. Once the user has completed the input, the data is sent to the server. The user can then review the data they have entered and make any necessary corrections.

Specific examples

Example 1: Website use

A user visits a website and enters the following information into a form:

Height: 170 cm

Weight: 70 kg

Diet goal: Lose 5 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Please suggest an appropriate diet plan based on this information.

Example 2: Use in a smartphone application

A user opens a smartphone application and enters the following information into the input form:

Height: 160 cm

Weight: 60 kg

Diet goal: Lose 3 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

The user's height is 160 cm, weight is 60 kg, and their diet goal is to lose 3 kg. Please suggest an appropriate diet plan based on this information.

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application and accesses an input form. The input form is an interface where the user can enter their height, weight, and diet goals. Once the input form is displayed, the user is ready to enter the required information.

Step 2:

The user enters the data.

The user enters their height, weight, and diet goal into the input form. For example, the user might enter "Height: 170 cm", "Weight: 70 kg", and "Diet goal: Lose 5 kg". Once the input is complete, the user taps the "Submit" button. The input data is saved on the device in JSON format.

Step 3:

The device sends the data to the server.

The device sends the data entered by the user to the server. Specifically, the device uses an HTTP POST request to send the input data to the server. At this time, the data is sent in JSON format. The input data (height, weight, diet goal) is sent to the server.

Step 4:

The server receives and stores the data.

The server receives the data sent from the device. The received data is stored in a database. For example, the server connects to a MySQL database and executes the following SQL query:

sql

INSERT INTO user_data (height, weight, goal) VALUES (170, 70, '5 kg weight loss');

This process saves the user's input data in the database.

Step 5:

The server generates a prompt for the generative AI model.

The server generates prompts for the generative AI model based on the stored data. For example, it generates prompts like the following:

The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Please suggest an appropriate diet plan based on this information.

This prompt is sent to the generative AI model.

Step 6:

Generative AI models suggest diet plans.

The generative AI model generates a diet plan based on the prompt it receives. For example, the generative AI model might suggest the following diet plan:

1. Do 30 minutes of aerobic exercise every day.

2. Limit your daily calorie intake to 1,500 kcal

3. Strength training three times a week

This diet plan is sent to the user's device via the server and displayed to the user.

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional food delivery systems struggled to provide meal plans based on customers' health status and diet goals, making it impossible to provide services tailored to individual needs. Furthermore, problems arose, such as customers buying unnecessary items and making meal choices that hindered the achievement of diet goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height, weight, and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for generating an appropriate meal plan based on the customer's height, weight, and goals, and means for presenting the generated meal plan to the customer. This makes it possible to provide an appropriate meal plan based on the customer's health condition and diet goals, and support the achievement of diet goals while reducing unnecessary spending.

"Customer" refers to the consumer who uses the service.

"Height and Weight" refers to the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that a customer wishes to achieve.

"Purchase receipt" refers to the detailed list of purchases received by a customer when making a purchase.

"Correction means" refers to a method or system for evaluating products purchased by customers and identifying unwanted products.

"Visualization means" refers to a method or system that visually displays the money saved or weight lost.

"Meal Plan" refers to the meal content and schedule suggested based on the client's health and diet goals.

"Means for generating" refers to a method or system for creating a meal plan based on customer input data.

"Presentation means" refers to the method or system for showing the generated meal plan to the customer.

The following system configuration is proposed as an embodiment of this invention.

System Configuration

The system includes a smartphone application used by customers, a server, and a generative AI model. The smartphone application provides an interface for customers to input their height, weight, and diet goals. The server receives the input data and sends prompts to the generative AI model to generate an appropriate meal plan. The generated meal plan is then presented to the customer again via the smartphone application.

Hardware and software used

Hardware: Smartphones, servers

Software: Smartphone applications, generative AI models (e.g., OpenAI GPT-3)

Data processing and calculation

1. Data entry: Customers use a smartphone application to enter their height, weight, and diet goals.

2. Data transmission: The smartphone application sends the entered data to the server.

3. Prompt generation: Based on the data received by the server, a prompt sentence is generated to be sent to the generative AI model.

4. Meal plan generation: The generative AI model receives the prompt and generates an appropriate meal plan.

5. Data reception and presentation: The server receives the generated meal plan and presents it to the customer via the smartphone application.

Specific examples

For example, if a customer enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg," the server will send the following prompt text to the generative AI model:

Example prompt

Please suggest an appropriate meal plan based on the following: height: 170 cm, weight: 70 kg, goal: 5 kg weight loss.

Based on this prompt, the generative AI model generates a balanced meal plan. For example, it suggests specific meal options such as "Breakfast - oatmeal, Lunch - salad, Dinner - chicken breast and vegetables." This meal plan is presented to the customer via a smartphone application.

In this way, we can provide customers with appropriate meal plans based on their health condition and diet goals, helping them achieve their diet goals while reducing unnecessary spending.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goal. The entered data is temporarily saved within the smartphone application. An example of input data is: height 170cm, weight 70kg, diet goal "lose 5kg."

Step 2:

The smartphone application sends the entered data to the server. The data sent includes the user's height, weight, and diet goals. The server analyzes the received data and proceeds to the next processing step.

Step 3:

The server generates a prompt based on the received data. Specifically, it combines the user's height, weight, and diet goal to create a prompt to send to the generative AI model. For example, a prompt might be generated that reads, "Based on height: 170 cm, weight: 70 kg, goal: to lose 5 kg, please suggest an appropriate meal plan."

Step 4:

The server sends the generated prompt text to the generative AI model. The generative AI model performs data calculations based on the received prompt text and generates an appropriate meal plan. The generative AI model uses, for example, OpenAI GPT-3.

Step 5:

The generative AI model generates a meal plan based on the prompt and sends the results back to the server. An example of a generated meal plan is "Breakfast - Oatmeal, Lunch - Salad, Dinner - Chicken Breast and Vegetables."

Step 6:

The server analyzes the meal plan received from the generative AI model and sends it to the smartphone application. The transmitted data includes the generated meal plan.

Step 7:

The smartphone application presents the meal plan received from the server to the user. The user can review and implement the proposed meal plan through the application.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Under the previous system, customers had to physically submit receipts for their purchases, which was time-consuming and risked loss or damage. Furthermore, there were insufficient ways to identify unnecessary purchases and visualize savings and weight loss. This made it difficult for customers to effectively manage their shopping to achieve their goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to take an image of their purchase receipt after shopping and upload it to the application so that they can submit it, means for correcting items that are determined to be unnecessary for this purchase, and means for visualizing the amount of money saved and weight loss that would be achieved if the unnecessary items were not purchased. This saves the customer the trouble of submitting a physical receipt and allows them to easily communicate their purchase information to the system. Furthermore, by identifying unnecessary items and visualizing the amount of money saved and weight loss that would be achieved, customers can effectively manage their shopping toward their goals.

"Customer" refers to an individual who uses the system to shop and manages their purchase information.

"Height and Weight" refers to the customer's current physical height and weight.

"Goal" refers to a specific physical or financial goal that the client wishes to achieve.

"Purchase receipt" means a paper or electronic record issued to a customer when they make a purchase that contains details of the items purchased.

"Taking an image" refers to the act of taking a photo of a physical receipt using a device such as a smartphone or camera.

"Application" refers to a software program that runs on a device such as a smartphone or tablet.

"Uploading" refers to the act of sending captured images to a server via the Internet.

"Server" refers to a computer system that receives, processes, and stores data over a network.

"Analyzing an image" refers to the act of using image recognition technology to extract the contents of a photographed receipt as digital data.

"Unnecessary products" refer to products that are determined not to be necessary for the customer to achieve their goals.

"Savings" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss you would expect if you had not purchased unnecessary items.

"Visualizing" refers to the act of displaying data visually so that customers can easily understand it.

"Displaying graphically" refers to the act of displaying data in a visual format such as a graph or chart.

This invention provides a system that allows customers to take an image of their purchase receipt after shopping and upload it to an application in order to submit it. A specific embodiment of this system is described below.

The user takes a picture of the purchase receipt using the smartphone camera. The image is then uploaded using a dedicated application (e.g., "ReceiptUploader"). The application has the function of sending the image taken by the user to the server.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted.

The analyzed data is stored in a database by the server. The stored data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database.

Users can check their purchase history using the application. By opening the "Purchase History" section within the application, data retrieved from the server is displayed. Users can view a list of past purchases.

As a concrete example, consider a scenario after a user has shopped at a supermarket. The user takes a picture of the receipt using their smartphone camera and opens the "ReceiptUploader" application. Within the application, the user taps the "Upload" button, selects the image of the receipt, and uploads it. The application sends the image to a server, which uses the Google Cloud Vision API to analyze the contents of the receipt. The analyzed data is stored in a database, and the user can open the "Purchase History" section within the application to view information about recent purchases.

Examples of prompts to input into a generative AI model include:

"Please describe the functionality that allows users to take a picture of a receipt using their smartphone camera and upload it to the application. Please include the names of any specific hardware or software."

Using this prompt, the generative AI model can generate sentences that explain in detail how the system will operate.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user takes a picture of the receipt.

The user uses their smartphone camera to take an image of the receipt for the item they purchased. When taking the photo, they are careful to capture the entire receipt clearly. The input is the physical receipt, and the output is a digital image.

Step 2:

The device uploads the image to the application.

The device (smartphone) uploads the captured image of the receipt to the "ReceiptUploader" application. The user opens the application, taps the "Upload" button, and selects the captured image. The input is a digital image, and the output is the transmission of image data to the server.

Step 3:

The server receives and analyzes the image.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted. The input is image data, and the output is analyzed text data.

Step 4:

The server stores the analysis results in a database.

The server saves the analyzed data in a database. The saved data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database. The input is the analyzed text data, and the output is saved in the database.

Step 5:

The user checks their purchase history.

Users can use the "ReceiptUploader" application to check their purchase history. When they open the "Purchase History" section in the application, data retrieved from the server is displayed. Users can view a list of past purchases. The input is data retrieved from the database, and the output is displayed in the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional shopping systems required customers to physically submit receipts for their purchases, which was time-consuming and made managing purchase information cumbersome. Furthermore, there was a lack of systems that could identify unnecessary products based on customers' health goals and visualize their savings and health benefits.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application; a means for correcting items determined to be unnecessary for the current purchase; a means for visualizing the money saved and weight lost if the unnecessary items were not purchased; a means for analyzing the contents of the purchase receipt and automatically extracting purchase information; a means for rewarding points to the customer based on the extracted purchase information; and a means for allowing the rewarded points to be used for the next purchase. This saves the customer the trouble of submitting a physical receipt, simplifies management of purchase information, and enables visualization of point rewards and savings. It also identifies unnecessary items based on the customer's health goals and visualizes savings and health benefits.

"Customer" refers to a consumer who purchases goods or services.

"Height and weight" refers to basic physical information that refers to the customer's physical height and weight.

A "goal" refers to a specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record detailing a transaction issued to a customer when they purchase goods.

A "smartphone" is a type of mobile phone that can connect to the Internet and use applications.

A "camera" is a device used to take images and videos.

An "application" is a software program that runs on a smartphone or computer.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Unnecessary products" are products that are deemed not necessary for achieving the customer's goals.

"Correction" is the act of checking the content and making corrections or improvements.

"Money saved" is the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" refers to the visual display of data or information.

"Analysis" is the act of examining data or information in detail to clarify its structure and meaning.

"Purchase information" refers to detailed information about the products purchased by the customer.

"Automatic extraction" means that the system extracts data without human intervention.

"Point redemption" is the act of awarding points to customers based on the purchase amount.

"Can be used on next purchase" means that the customer can use the points the next time they purchase a product.

The following describes in detail the mode for implementing this invention.

First, a smartphone application is used by the customer to enter their current height, weight, and goals. The customer launches the application and enters their height, weight, and goals. This registers the customer's health goals in the system.

Next, we provide a way for customers to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application. Customers take a photo of their receipt and use the upload function within the application to send the image to the server.

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. Next, it uses pytesseract to extract text from the preprocessed image. It then analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text.

Based on the analyzed purchase information, the application provides a means for rewarding customers with points. The server calculates points based on the purchase amount and adds them to the customer's account. Rewarded points are managed within the application so they can be used for the next purchase.

Furthermore, as a means of filtering out items that may be deemed unnecessary for this shopping trip, an algorithm is implemented to determine unnecessary items based on the customer's height, weight, and goals. The server compares purchase information with the customer's health goals to identify unnecessary items.

To visualize the money saved and weight lost by not buying unnecessary products, the server calculates the amount saved and weight lost and displays them graphically. This allows customers to visually see the savings and health benefits.

As a concrete example, consider the case where a user purchases a drink at a convenience store, takes a photo of the receipt, and uploads it to the app. The app extracts "drink" and "150 yen" from the receipt and rewards 15 points. Furthermore, if the drink is deemed an unnecessary item based on the user's health goals, the amount of money saved and the weight lost will be displayed.

Examples of prompts for a generative AI model are as follows:

"A user has uploaded a receipt image taken with their smartphone camera. Please create a program that extracts purchase information from this image and rewards points. 10% of the purchase amount will be rewarded as points."

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goals. The entered data is temporarily saved within the application and sent to the server. The entered data is used as the basis for setting the user's health goals.

Step 2:

After shopping, the user takes a photo of the purchase receipt with their smartphone camera and uploads it to the application. The captured receipt image is temporarily saved within the application and then sent to the server. The input data is the receipt image.

Step 3:

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. The input data is the receipt image, and the output data is the preprocessed image.

Step 4:

The server extracts text from the preprocessed image using pytesseract. The input data is the preprocessed image, and the output data is the extracted text. Specifically, it uses OCR (optical character recognition) technology to read the text information in the image.

Step 5:

The server analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text. The input data is the extracted text, and the output data is the analyzed purchase information. Specifically, it uses a text analysis algorithm to extract the required information.

Step 6:

The server rewards the customer with points based on the analyzed purchase information. The input data is the analyzed purchase information, and the output data is the rewarded points. Specifically, it calculates points based on the purchase amount and adds them to the customer's account.

Step 7:

The server determines unnecessary products based on the customer's height, weight, and goals. The input data is the customer's health goals and purchase information, and the output data is a list of unnecessary products. Specifically, it runs an algorithm that compares the health goals with the purchase information and identifies unnecessary products.

Step 8:

The server calculates the amount of money saved and weight lost if unnecessary items are not purchased, and displays this graphically. The input data is a list of unnecessary items and purchase information, and the output data is the amount saved and weight lost. Specifically, it calculates the savings and health benefits and displays them visually to the user.

Step 9:

The user uses the points they have earned on their next purchase. The input data is the points earned, and the output data is the balance after the points have been used. Specific operations include using the points to discount the purchase amount and updating the balance.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Today's consumers find it difficult to balance health management and saving money. Consumers with diet or health goals in particular have difficulty determining which products are unnecessary when shopping. Furthermore, there are few ways to specifically assess the savings and health benefits of not purchasing unnecessary products. For this reason, a system that helps consumers effectively manage their health and save money is needed.

The identification processing by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for converting the submitted receipt information into text data using optical character recognition technology, means for determining unnecessary products based on the customer's height, weight, and goals, means for calculating the amount of money saved and the weight lost if the unnecessary products are not purchased, and means for graphically displaying the calculation results. This allows consumers to easily identify unnecessary products when shopping and specifically understand the amount of money saved and the health benefits.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and Weight" refers to the customer's current physical measurements and is used as basic data for health management.

"Goals" are specific health and diet objectives set by customers, and include, for example, "carbohydrate restriction" or "calorie restriction."

A "receipt" is a detailed list of purchased items that a customer receives after shopping, listing the product names and prices.

"Optical character recognition technology" is a technology that extracts character information from image data and is used to convert receipt images into text data.

"Text data" refers to text information converted using optical character recognition technology, and includes information such as product names and prices.

"Unnecessary products" are products that are determined not to be necessary to purchase from the perspective of health management or dieting, based on the customer's height, weight, and goals.

"Amount saved" is the total amount of money you can save by not purchasing unnecessary items.

"Weight loss" is the amount of weight loss predicted if the unnecessary product is not consumed.

"Displaying graphically" means displaying the calculation results in the form of graphs, charts, etc. to make them easier to understand visually.

Form for implementing the invention

This invention is a system that enables consumers to effectively manage their health and save money. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. A device for users to enter their height, weight, and goals.

2. A terminal where users can submit receipts after shopping.

3. Hardware and software for the server to receive and process data

Hardware and software used

Device: Mobile devices such as smartphones and tablets

Server: High-performance data processing server

OCR technology: Optical character recognition technology such as Google Cloud Vision API

Database: Relational database such as MySQL or PostgreSQL

Data visualization libraries: D3.js, Chart.js, etc.

Program processing

The server receives the height, weight, and goal information entered by the user, as well as the submitted receipt. Using this data, the server processes and calculates the data to detect unwanted products. Specifically, the process is carried out as follows:

1. Data reception and analysis:

The server receives the user's input height, weight, and goal (e.g., carbohydrate restriction).

The server converts the information on the submitted receipt into text data using OCR technology.

2. Detecting unwanted products:

The server detects unnecessary products based on the goal. For example, if the goal is to "limit carbohydrates," the server will list high-carbohydrate products on the receipt, such as "bread" and "sweets," as unnecessary products.

3. Calculate savings and weight loss:

The server calculates the total price of the unwanted items detected.

The server calculates the estimated weight loss that would occur if the unnecessary products were not consumed.

4. Visualizing the results:

The server displays the calculation results to the consumer in the form of graphs, charts, etc.

Specific examples

Let's say a user is 170cm tall, weighs 70kg, sets their goal to "restrict carbohydrates," and submits a receipt. The receipt includes "bread (300 yen)," "sweets (200 yen)," and "vegetables (100 yen)." The system detects "bread" and "sweets" as unnecessary items and calculates that a total of 500 yen can be saved. The system also calculates that the estimated weight loss would be 0.5kg if these items were not consumed, and displays this in a graph.

Example prompt

"Height 170cm, weight 70kg, goal is carbohydrate restriction. Detect unnecessary items based on the following receipt information and calculate the amount saved and weight loss. Receipt information: bread (300 yen), sweets (200 yen), vegetables (100 yen)." The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

Users enter their height, weight, and goal, then submit a receipt.

Input: The user opens the app on their smartphone and enters their height (170cm), weight (70kg), and goal of "restricting carbohydrates." After shopping, they also take a photo of the receipt with their smartphone camera and upload it to the server via the app.

Output: User input data and receipt image are sent to the server.

Step 2:

The server receives the input data and receipt information.

Input: Height, weight, goal, and receipt image submitted by the user.

Output: User input data and receipt image stored in a database on the server.

Step 3:

The server converts the receipt information into text data using OCR technology.

Input: Receipt image stored on the server.

Data processing: The server calls the Google Cloud Vision API, analyzes the receipt image, and extracts the product name and price.

Output: Converted text data (e.g., "Bread (300 yen)", "Sweets (200 yen)", "Vegetables (100 yen)").

Step 4:

The server detects unwanted items based on your goals.

Input: User's goal (e.g., "Limit carbohydrates") and converted text data.

Data calculation: The server performs rule-based filtering based on predefined food categories and goals to list unwanted products.

Output: List of unwanted products (e.g., "bread", "sweets").

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

Input: List of unwanted items and their respective prices.

Data calculation: The server calculates the total price of the unnecessary products and also calculates the estimated weight loss if the unnecessary products were not consumed. The model used here is a simulation model based on the relationship between calories consumed and weight loss.

Output: Amount saved (e.g., 500 yen) and estimated weight loss (e.g., 0.5 kg).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them to the user.

Enter: Amount saved and estimated weight loss.

Data processing: The server uses data visualization libraries such as D3.js and Chart.js to visualize the calculation results in graphs and charts.

Output: Graphs and charts displayed on the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Configuration Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Today's consumers find it difficult to simultaneously manage their health and save money. In particular, consumers with dietary goals need to identify and avoid unnecessary products from their purchases in order to achieve health and financial benefits. However, doing this manually is time-consuming and inaccurate. Therefore, a system is needed that allows consumers to easily identify unnecessary products and efficiently manage their savings and health.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for analyzing receipt information and automatically detecting unnecessary items based on diet goals, means for calculating the amount of money saved and estimated weight loss if the unnecessary items are not purchased and displaying them in graphs and charts, and an application installed on a smartphone. This allows consumers to easily identify unnecessary items and efficiently save money and manage their health.

"Customer" refers to consumers who use the system.

"Height and Weight" is data indicating the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

A "purchase receipt" is paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unwanted products" refer to products that are deemed inappropriate or unnecessary for the customer's goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Receipt information" refers to data such as the product name, price, and quantity listed on the purchase receipt.

"Diet Goals" refers to health goals set by a customer, such as weight loss or specific nutritional restrictions.

"Automatically detect" means that the system identifies unwanted products programmatically without human intervention.

"Graphs and charts" refer to diagrams used to visually display data.

"Applications installed on smartphones" refers to software programs that run on smartphones.

This invention is a system that allows consumers to efficiently manage their health and save money. Specific embodiments for implementing this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A mobile device such as a smartphone.

2. Server: A central server that processes and analyzes data.

3. Application: A dedicated application installed on a smartphone.

Program processing

User device

The user device has the following functions:

User information input: Users enter their height, weight, and diet goals through the application.

Receipt scanning: The user scans the receipt for the item they purchased with their smartphone camera and obtains the image data.

Server

The server performs the following process:

Analyzing receipt information: The acquired receipt image is converted into text data using OCR (optical character recognition) technology. This process uses the pytesseract library.

Detection of unnecessary products: Automatically detect unnecessary products from receipt text data based on the user's diet goals. For example, for a user aiming to limit carbohydrate intake, high-carbohydrate products such as "bread" and "sweets" will be detected.

Calculation of savings and weight loss: Calculates the total cost of the detected unnecessary products and the estimated weight loss if you did not consume those products. Weight loss is calculated based on the calories required to lose 1 kg of weight (approximately 7700 kcal).

application

The application has the following features:

Visualization of results: Calculated savings and estimated weight loss are displayed in graphs and charts, allowing users to visually see the effects of their health management and savings.

Specific examples

For example, if a user sets a goal of "restricting carbohydrates" and the receipt includes "bread" and "sweets," these items will be detected as unnecessary products. Suppose the total amount of the detected unnecessary products is 500 yen, and the estimated weight loss is 0.2 kg.

Example prompt

Please enter your user information:

Height: 170 cm

Weight: 70 kg

Diet goal: Carbohydrate restriction

Scan your receipt:

(Upload receipt image)

Unwanted items detected...

Unwanted items detected:

Bread: 200 yen

Sweets: 300 yen

Savings: ¥500

Estimated weight loss: 0.2kg

The results are displayed in a graph.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goals.

Input: User's height, weight, and diet goal

Output: User information data

Specific operation: The user enters their height, weight, and diet goal (e.g., carbohydrate restriction) into the application's input form and presses the submit button. This generates user information data and sends it to the server.

Step 2:

After shopping, the user scans the receipt for the purchased items using their smartphone camera.

Input: Receipt image

Output: Receipt image data

Specific operation: The user uses the application's camera function to take a photo of the purchase receipt. The captured receipt image is saved within the application and sent to the server.

Step 3:

The server receives the receipt image and converts it into text data using OCR technology.

Input: Receipt image data

Output: Receipt text data

Specific operation: The server uses the pytesseract library to extract text from the received receipt image data. The extracted text data is saved as receipt text data.

Step 4:

The server automatically detects unnecessary items from receipt text data based on the user's diet goals.

Input: User information data, receipt text data

Output: List of unwanted items

Specific operation: The server analyzes receipt text data based on the user's diet goal (e.g., carbohydrate restriction). Products containing specific keywords (e.g., "bread," "sweets") are listed as unwanted items.

Step 5:

The server calculates the amount of money you could save and the estimated weight loss if you didn't buy unnecessary items.

Input: List of unwanted items

Output: Money saved, estimated weight loss

Specific operation: The server calculates the savings amount by adding up the prices of each item based on the list of unwanted items. It also calculates the estimated weight loss amount based on the calorie information of each item. The weight loss amount is calculated based on the calories required to lose 1 kg (approximately 7700 kcal).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them on the user's smartphone.

Input: Amount saved, estimated weight loss

Output: Image data for graphs and charts

Specific operation: Based on the calculation results, the server generates graphs and charts showing the amount of savings and estimated weight loss. Image data of the generated graphs and charts is sent to the user's smartphone and displayed within the application.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products. When the user is relaxed, it recommends healthy foods.

"Example 2"

Furthermore, the emotion engine graphically displays the money saved and weight lost based on the user's emotions. Specifically, it displays the amount of money saved and the amount of weight lost if the user refrains from purchasing unnecessary products when feeling stressed. This allows users to visually understand the relationship between their emotions, health management, and saving money.

"Example 3"

For example, if a user buys chocolate while feeling stressed, their emotions

The engine detects chocolate as an unnecessary item and displays a graph of the savings and weight loss that would occur if the chocolate were not purchased. This helps users recognize their tendency to purchase unhealthy foods when they are stressed and understand the resulting financial and health impacts.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine determines which products are unnecessary based on the user's emotions and notifies the user of the results.

"Example 2"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine graphically displays the money saved and weight lost based on the user's emotions.

"Example 3"

Step 1: Buy chocolate when the user is feeling stressed.

Step 2: The emotion engine recognizes the user's emotions and detects chocolate as an unwanted item.

Step 3: The emotion engine displays a graph of the amount of money saved and weight loss you would have experienced if you had not bought the chocolate.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

The previous system allowed customers to enter their height, weight, and goals, and to submit receipts after shopping, but it lacked the ability to identify unnecessary products based on the customer's emotions and recommend healthy products. This made it difficult for customers to select appropriate products when they were stressed or relaxed. It also lacked a way to visualize the money saved and weight lost by not purchasing unnecessary items. This made it difficult to raise customers' awareness of health management and saving money.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for recognizing the customer's emotions and determining unnecessary items based on those emotions, and means for recommending healthy products based on the customer's emotions. This enables the customer to select appropriate products according to their emotional state, thereby increasing their awareness of health management and saving money.

"Customer" refers to an individual who uses the system to enter their height, weight, and diet goals, and then submits a purchase receipt after shopping.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to specific numerical values or conditions related to dieting or health that a customer wishes to achieve.

"Input means" refers to the interface that allows customers to input their height, weight, and goals into the system.

"Purchase receipt" refers to the paper or electronic data containing the purchase details of a product that a customer receives when making a purchase.

"Submission means" refers to the interface through which customers submit purchase receipts to the system.

"Correction method" refers to the function by which the system determines unnecessary products based on customer input data and emotional data and notifies the customer.

"Visualization means" refers to the system's ability to graphically display the money saved and weight lost if unnecessary items are not purchased.

"Emotion recognition means" refers to the system's ability to analyze customer emotions and determine which products are unnecessary based on those emotions.

"Healthy product recommendation means" refers to the system's ability to recommend healthy products based on customer sentiment.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is set up on a website or smartphone application, and allows customers to directly input their height, weight, diet goals, etc. In addition, by combining it with an emotion engine, it has the function of determining unnecessary products based on the customer's emotions and recommending healthy products.

The server places an input form on a website or smartphone application. This input form is created using HTML and CSS, and JavaScript is used to validate the input data in real time. When a user enters data into the input form, the server receives the data and stores it in a database (for example, MySQL or PostgreSQL).

Next, the server uses an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) to analyze the user's emotions. The emotion engine recognizes the user's current emotions based on the user's input data and past behavioral data. For example, if the user is feeling stressed, the server will use that information to determine that unhealthy foods such as sweets are unnecessary. On the other hand, if the user is relaxed, the server will recommend healthy foods.

As a concrete example, suppose a user uses a smartphone application to enter their height as 170cm, weight as 70kg, and diet goal as "lose 5kg." This data is sent to the server and stored in a database. The emotion engine then analyzes the user's emotions, and if it determines that the user is feeling stressed, the server displays a message such as "Avoid sweet snacks such as chocolate and cookies."

Example of a prompt to input to a generative AI model:

A user enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg." The emotion engine analyzes the user's emotions and determines that the user is feeling stressed. In this case, what products would be deemed unnecessary and what products would be recommended?

By inputting this prompt into a generative AI model, specific product recommendations and a determination of unnecessary products can be obtained. In this way, the system can make appropriate product recommendations based on the user's input data and emotional data.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application. The device sends a request to the server through the browser or application to retrieve an input form. The server generates the input form using HTML and CSS and sends it to the device. The device renders the received HTML and CSS and displays the input form to the user.

Input: User access request

Output: Display the input form

Step 2:

Users enter their height, weight, and diet goals.

The user enters their height, weight, and diet goals into the displayed input form. The device uses JavaScript to validate the format of the input data in real time to ensure there are no errors.

Input: User's height, weight, and diet goal

Output: Validated input data

Step 3:

The server receives the input data and stores it in the database.

When the user presses the send button, the device sends the input data in JSON format to the server. The server parses the received data and saves it in a MySQL database. Specifically, it adds the data to the database using the INSERT statement.

Input: JSON format input data

Output: Data stored in the database

Step 4:

The server calls the emotion engine and analyzes the user's emotions.

The server calls an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) based on the stored data. The data is sent to the API, which analyzes the user's emotions. The API determines that the user is feeling stressed and returns the result to the server.

Input: Data stored in the database

Output: Sentiment analysis results from the emotion engine

Step 5:

The server determines which products are unnecessary based on emotions and decides which products to recommend.

The server determines which products are unnecessary and which are recommended based on emotional data obtained from the emotion engine. For example, if the user is feeling stressed, the server will determine that "sweets such as chocolate and cookies should be avoided" and decide to "recommend healthy foods such as fruits and nuts."

Input: Sentiment analysis results from the emotion engine

Output: List of unwanted and recommended products

Step 6:

The server displays the results to the user.

The server generates HTML to display the results of the judgment to the user and sends it to the device. The device renders the received HTML and displays unwanted products and recommended products to the user. The user checks the displayed results and decides on their next course of action.

Input: List of unwanted and recommended products

Output: The result displayed to the user

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional food delivery systems don't recommend foods based on the user's health status or emotions, which can lead to users choosing unhealthy foods when they're stressed. Furthermore, there's a lack of support for users to choose foods that align with their diet goals, making it difficult to diet effectively.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for the customer to enter their current height and weight and goals, a means for the customer to submit a purchase receipt after shopping, a means for correcting items determined to be unnecessary in this purchase, a means for visualizing the money saved and weight lost if the unnecessary items are not purchased, and a means for recognizing the customer's emotions and determining unnecessary items based on those emotions. This makes it possible to recommend foods based on the user's health condition and emotions, allowing the user to avoid unhealthy foods when feeling stressed and to choose foods that are in line with their diet goals.

"Customer" refers to a consumer who uses a service or product.

"Height and weight" are numbers that indicate an individual's physical height and weight.

"Goal" refers to the specific objective or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record of a transaction issued when a customer purchases a product.

"Unnecessary products" refer to products that are deemed inappropriate for the customer's health condition or goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight Loss" is a numerical value that represents the weight loss a customer can achieve by avoiding unnecessary products.

"Means for recognizing emotions" refers to technologies or devices that detect a customer's emotional state and make decisions based on that.

"Visualization means" refers to technology or devices that visually display the money saved or weight lost.

A system for implementing this invention is configured as follows: First, a dedicated form is provided for users to enter their height, weight, and diet goals. This form is provided on a website or smartphone application, allowing users to easily enter information.

Next, we present a system that combines an emotion engine. This system recognizes the user's emotions in real time and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products, and when the user is relaxed, it recommends healthy foods.

The following hardware and software are used to implement this system:

Hardware: Smartphone, tablet, or PC

Software: Python, EmotionRecognizer library, FoodRecommender library

The server receives the user's input data such as height, weight, and diet goal, and uses the EmotionRecognizer library to recognize the user's emotions in real time. It then uses the FoodRecommender library to recommend appropriate foods based on the user's input data and the recognized emotions.

For example, if a user enters their height as 170cm, weight as 65kg, and diet goal as "lose weight," and the emotion engine recognizes that the user is feeling stressed, the system will avoid unhealthy foods (e.g., sweet snacks) and instead recommend healthy foods (e.g., salads and fruit).

Examples of prompt statements include:

"If a user is 170cm tall, weighs 65kg, and their diet goal is to lose weight, and the emotion engine determines that the user is feeling stressed, what foods would you recommend?"

In this way, food recommendations can be made based on the user's health status and emotions, helping users avoid unhealthy foods when they are stressed and choosing foods that align with their diet goals.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The user uses the terminal to input their height, weight, and diet goal. The input data is sent to the server. The input data includes the user's height (e.g., 170 cm), weight (e.g., 65 kg), and diet goal (e.g., "lose weight").

Step 2:

The server stores the received user input data and uses the EmotionRecognizer library to recognize the user's emotions in real time. For emotion recognition, the user's facial expressions and voice data are used as input. The output is the user's emotional state (e.g., stressed, relaxed).

Step 3:

The server uses the FoodRecommender library to recommend appropriate foods based on the recognized emotional state and user input data. Input data includes the user's height, weight, diet goals, and emotional state. The output is a list of recommended foods (e.g., salads, fruits).

Step 4:

The server sends a list of recommended foods to the user's device, where the user can check the list of recommended foods on their device. Specifically, if the user is feeling stressed, unhealthy foods will be avoided and healthy foods will be recommended.

Step 5:

A user makes a purchase and submits the purchase receipt to the server using a terminal. The server analyzes the submitted receipt data and generates a list of the purchased items. Input data includes receipt images or electronic data. The output is a list of the purchased items.

Step 6:

The server compares the list of purchased items with the user's diet goals and emotional state to determine which items are unnecessary. Specific operations include detecting items that are inappropriate for the user's goals (e.g., sweet snacks). Input data includes the list of purchased items, the user's diet goals, and their emotional state. The output is a list of unnecessary items.

Step 7:

The server calculates and visualizes the money saved and weight lost if unnecessary items are not purchased. Specific operations include graphically displaying the amount saved and weight lost. The input data includes a list of unnecessary items. The output is a graphical representation of the amount saved and weight lost.

Step 8:

The server sends the visualized data to the user's device, where the user can check the amount of money saved and the weight lost on the device. Specifically, the user can visually understand the benefits they can gain by avoiding unnecessary products.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

With conventional health management systems, customers had to submit physical proof of purchase after shopping, which was time-consuming. Furthermore, the amount of savings and weight loss were not visualized taking into account the customer's emotional state, making it difficult for customers to understand the relationship between their emotions, health management, and savings.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current physical information and goals, means for the customer to submit a proof of purchase after shopping, means for the customer to capture an image of the proof of purchase using the camera function of their mobile device and upload it to the application, means for the server to convert the contents of the proof of purchase into text data using image processing technology, means for the server to save the converted text data, means for the customer to input emotional data, means for the server to analyze the customer's emotional data using emotion analysis technology, means for the server to determine unnecessary products based on the analysis results and calculate the amount of savings and weight loss, and means for the server to graphically display the calculation results. This allows customers to communicate their purchase information to the system without having to directly submit a physical proof of purchase, and further enables them to visually understand the relationship between their emotions, health management, and savings.

"Customer" refers to an individual who uses the system to manage their own health and save money.

"Physical information" refers to physical data such as the customer's height, weight, and body fat percentage.

"Goals" refers to the health or savings goals the customer wants to achieve.

"Proof of purchase" refers to a document that proves a purchase, such as a receipt or invoice that a customer receives after shopping.

"Mobile device" refers to portable electronic devices such as smartphones and tablets.

"Photography function" refers to the camera function built into the mobile device.

"Application" refers to a software program that runs on a mobile device.

"Image processing technology" refers to the technology of analyzing image data and extracting necessary information.

"Text data" refers to text information extracted using image processing technology.

"Emotional data" refers to data that indicates a customer's emotional state.

"Emotion analysis technology" refers to technology that analyzes emotional data and evaluates a customer's emotional state.

"Unnecessary products" refer to products that are determined not to be necessary for the customer's health or savings goals.

"Amount saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight you would lose if you had not purchased unnecessary items.

"Displaying graphically" refers to displaying the calculation results in a visually easy-to-understand manner.

Form for implementing the invention

This invention is a system in which customers submit a purchase certificate after shopping and visualize the amount of savings and weight loss based on emotional data. A specific embodiment of this system is described below.

System Configuration

This system primarily uses the following hardware and software:

Mobile devices (smartphones and tablets)

Camera function

application

Server

Image processing software (e.g., OpenCV, Tesseract)

Emotion analysis technology

Program processing

After shopping, the user uses the camera function on their mobile device to take a picture of the purchase certificate. The image is then uploaded to the server via the application. The server then uses image processing software (e.g., OpenCV or Tesseract) to extract text from the image and convert information such as the name, price, and quantity of the purchased item into text data. The server then stores this text data in a database.

Next, the user enters their emotional data through the application. The emotional data may be entered manually by the user or automatically obtained from the wearable device. The server then analyzes the emotional data using emotion analysis technology to assess whether the user is feeling stressed and to what extent.

Based on the results of the emotion analysis, the server calculates the amount of money saved and the amount of weight lost if the user refrains from buying unnecessary items when feeling stressed. For example, if the user refrains from buying chocolate when feeling stressed, the calculation is based on the price (200 yen) and calories (300 kcal). The calculation results are displayed graphically by the server, and the user can check them through the application.

Specific examples

As a concrete example, consider the case where a user decides not to buy chocolate when they are feeling stressed. The user takes a photo of the receipt with their smartphone camera and uploads it to the application. The server uses image processing software to convert the contents of the receipt into text data and stores it in a database. Next, the user enters emotional data into the application, which the server analyzes using emotion analysis technology. Based on the analysis results, the server calculates the amount of savings and weight loss based on the price of the chocolate (200 yen) and calories (300 kcal). Finally, the server displays the calculation results graphically, and the user can check them through the application.

Example prompt

Example of a prompt to input to a generative AI model:

"Generate a program that calculates and graphically displays the amount of money saved and weight loss a user would achieve if they avoided buying unnecessary items when they were feeling stressed."

In this way, users can visually understand the relationship between their emotions, health management, and saving money.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user takes a photo of the purchase certificate using their smartphone camera.

Enter: Proof of purchase

Output: Image of proof of purchase

Specific operation: The user activates the smartphone camera, frames the purchase certificate, and presses the capture button. The captured image is saved to the smartphone's gallery.

Step 2:

Upload the image of the purchase certificate taken by the device to the application.

Input: Image of proof of purchase

Output: Image data sent to the server

Specific operation: The user launches the application, selects an image of the purchase certificate, and presses the upload button. The device then sends the selected image to the server.

Step 3:

The server uses image processing technology to convert the contents of the purchase certificate into text data.

Input: Image data of purchase certificate

Output: Text data

Specific operation: The server analyzes the received image data using image processing software (e.g., OpenCV, Tesseract) and extracts text information from the image. The extracted text information is converted into text data such as the name, price, and quantity of the purchased item.

Step 4:

The server stores the converted text data in a database.

Input: Text data

Output: Purchase information stored in the database

Specific operation: The server saves the converted text data in a database. The saved data includes the purchase date and time, store name, product information, etc.

Step 5:

The user inputs emotional data into the application.

Input: Emotion data

Output: Emotion data sent to the server

Specific operation: The user inputs their emotional state through the application. Emotional data includes information such as stress level and mood. It may also be obtained automatically from the wearable device.

Step 6:

The server analyzes the user's emotional data using emotion analysis technology.

Input: Emotion data

Output: Analysis results

Specific operation: The server uses emotion analysis technology to analyze emotional data and evaluate whether the user is feeling stressed and to what extent. The analysis results are displayed as numbers and graphs.

Step 7:

The server determines which items are unnecessary based on the analysis results and calculates the amount of money saved and the weight lost.

Input: Analysis results, text data

Output: Savings and weight loss data

Specific operation: Based on the results of emotion analysis and purchase information, the server calculates the amount of money saved and the amount of weight lost if the user does not buy unnecessary items when feeling stressed. For example, the calculation is based on the price (200 yen) and calories (300 kcal) of chocolate.

Step 8:

The server displays the calculation results graphically.

Input: Savings and weight loss data

Output: Graphical display

Specific operation: The server generates data for graphically displaying the calculation results and provides it to the user through the application. The user can visually check the amount of savings and weight loss through the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional shopping support systems required customers to physically submit receipts for purchased items, which was a time-consuming process. Furthermore, the shopping support did not take into account the customer's emotions or health status, leading to unnecessary purchases when they were feeling stressed. This led to wasteful spending and negative health effects, which was an issue.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to the application; a means for correcting items that are determined to be unnecessary in this purchase; a means for visualizing the money saved and weight lost if the customer does not buy unnecessary items; and a means for analyzing the customer's emotions and graphically displaying the amount of money saved and weight lost if the customer does not buy unnecessary items when they are feeling stressed. This saves the customer the trouble of submitting physical receipts, and enables shopping support that takes their emotions and health state into consideration.

"Customer" refers to a consumer who purchases goods or services.

"Height and Weight" refers to the customer's physical height and weight.

"Goal" refers to the specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a proof of purchase issued when a customer purchases a product.

A "smartphone camera" is a photographic device built into a smartphone.

An "application" is a software program that runs on a smartphone or other device.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Correction" is the act of checking the content and correcting or deleting unnecessary parts.

"Unnecessary items" are items that customers have determined they do not need to purchase.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" is the act of visually displaying data or information.

"Analyzing emotions" is the process of assessing and understanding a customer's emotional state.

"Stress" refers to mental or physical tension or strain.

"Graphically displayed" means displaying information in a visual format such as a graph or chart.

This invention provides a system that allows customers to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to an application. It also includes a function that analyzes the customer's emotions and graphically displays the amount of money saved and the amount of weight lost if they avoid buying unnecessary items when they are feeling stressed.

System Configuration

1. Hardware:

Smartphone: Use the camera function to take a photo of your purchase receipt.

Server: Serves as the backend for the application, storing and processing data.

2. Software:

Application: Installed on a smartphone, it takes photos of receipts, uploads them, performs sentiment analysis, and displays them graphically.

OpenCV: A library for loading and preprocessing receipt images.

Tesseract OCR: Optical character recognition software for extracting text from receipt images.

EmotionRecognizer: Emotion recognition software for analyzing customer emotions.

Matplotlib: A library for graphically displaying savings and weight loss.

System Operation

1. Take a photo of your receipt and upload it:

Users take a photo of their purchase receipt using their smartphone camera.

The captured images are uploaded to the server via the application.

2. Text extraction:

The server preprocesses the image using OpenCV and extracts text from the receipt using Tesseract OCR.

3. Sentiment analysis:

The server analyzes the user's emotions using EmotionRecognizer.

When you're feeling stressed, calculate how much money you'll save and how much weight you'll lose if you don't buy unnecessary items.

4. Graphical display:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

The application will display this visually to the user.

Specific examples

For example, if a user avoids buying unnecessary items when feeling stressed, the application will graphically display a savings of 1,000 yen and a weight loss of 0.5 kg.

Example prompt

Create a program that calculates and graphically displays the savings and weight loss that a user could achieve by not purchasing unnecessary items when they are feeling stressed. Take a photo of the receipt with a smartphone camera and extract the text using Tesseract OCR. Analyze the user's emotions using EmotionRecognizer and display the results using Matplotlib.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

Users take a photo of their purchase receipt using their smartphone camera.

Input: Physical image of purchase receipt

Output: Receipt image saved on your smartphone

Specific operation: The user launches the smartphone's camera app and takes a photo of the purchase receipt. The captured image is saved to the smartphone's storage.

Step 2:

The device will upload the captured receipt image to the application.

Input: Receipt image saved on your smartphone

Output: Receipt image uploaded to the server

Specific operation: The user launches the application, selects the captured receipt image, and presses the upload button. The image is sent to the server via the Internet.

Step 3:

The server preprocesses the receipt image using OpenCV and extracts text using Tesseract OCR.

Input: Receipt image uploaded to the server

Output: Extracted text data

Specific operation: The server uses OpenCV to remove noise from the image and highlight the text. Then, it uses Tesseract OCR to extract the text from the image.

Step 4:

The server analyzes the user's emotions using EmotionRecognizer.

Input: User emotional data (e.g., the emotional state the user enters into the application)

Output: Analyzed emotional state

Specific operation: The server uses EmotionRecognizer to analyze the user's emotional data and determine whether they are under stress.

Step 5:

When you feel stressed, the server calculates the amount of money you would save and the amount of weight you would lose if you didn't buy unnecessary items.

Input: Analyzed emotional state, extracted text data

Output: Savings and weight loss data

Specific operation: If the emotional state is stressed, the server identifies unnecessary items from the extracted text data and calculates the amount of savings and weight loss that would result if the item were not purchased.

Step 6:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

Input: Savings and weight loss data

Output: Image data for graphical display

Specific operation: The server uses Matplotlib to generate images that visually display the amount of savings and weight loss in the form of bar graphs, etc., and sends the image data to the application.

Step 7:

The terminal displays the graphical data received from the server to the user.

Input: Graphical image data sent from the server

Output: Graphical information displayed on a smartphone screen

Specific operation: The application displays the image data received from the server on the smartphone screen, allowing the user to visually check the amount of savings and weight loss.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Today's consumers aim to balance health management with saving money, but they often end up buying unnecessary products in their daily shopping. These unnecessary products not only go against their health goals but also lead to wasteful spending. However, it is not easy for consumers to identify and avoid these unnecessary products. Therefore, there is a need for a system that automatically detects unnecessary products based on consumers' health goals and visualizes the savings and health impact.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for customers to input their current height and weight and goals, means for customers to upload an image of their purchase receipt after shopping, means for converting the receipt image into digital data using optical character recognition technology, means for analyzing product information from the digital data and detecting unnecessary products, means for calculating the total price of unnecessary products and estimated weight loss, and means for visualizing the calculation results in graphs and charts. This allows consumers to automatically detect unnecessary products based on their own health goals and specifically understand the amount of money they can save and the impact on their health.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and weight" refers to the customer's basic physical data and is fundamental information for setting health goals.

"Goals" refer to the health and weight management goals that a customer wishes to achieve, such as carbohydrate restriction or calorie restriction.

"Receipt image" refers to photo data of the purchase details that a customer obtains after shopping.

"Optical character recognition technology" is a technology that converts text information in an image into digital data, and is also known as OCR (Optical Character Recognition).

"Digital data" means text information converted using optical character recognition technology.

"Product information" refers to data such as the product name, price, and nutritional information listed on the receipt.

"Unnecessary products" are products that are contrary to the customer's health goals and should be avoided.

"Total amount" is the total price of the unwanted items detected.

"Estimated weight loss" is the amount of weight loss predicted if the unnecessary product was not consumed.

"Graphs and charts" are diagrams that visually display calculation results.

This invention is a system that allows consumers to efficiently manage their own health and save money. Specific embodiments of this system are described below.

First, the user accesses the system's web page using a terminal. The user enters their height, weight, and health goals (e.g., carbohydrate restriction). This data is sent to the server and stored in a database.

Next, the user uploads an image of the receipt obtained after shopping from their device to the system. The uploaded receipt image is sent to the server, which then uses optical character recognition technology such as Google Cloud Vision API to convert the receipt image into digital data. This digital data includes the text information on the receipt.

The server analyzes the converted digital data and extracts product names and their nutritional information. For example, if the receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

The server then calculates the total cost of the unnecessary items, as well as the estimated weight loss that would occur if these items were not consumed. A standard calorie expenditure model is used to calculate weight loss. The results are visualized as graphs and charts using a data visualization library such as D3.js.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

As a concrete example, consider the case where a user purchases chocolate when feeling stressed. The emotion engine analyzes the user's emotional state and determines that stress is high. In this case, it detects chocolate as an unnecessary item and displays a graph of the amount of money saved and weight loss that would have been achieved if the user had not purchased the chocolate. This allows the user to recognize their tendency to purchase unhealthy foods when they are stressed and understand the resulting financial and health impacts.

Example prompt:

Detect the foods the user purchases when they are stressed, calculate the amount of money saved and the weight loss they would have lost if they had not purchased those foods, and display the results in a graph.

This system allows users to specifically understand both their own health management and savings. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

Users enter their height, weight, and goals.

Using a terminal, a user accesses the system's web page and enters their height (e.g., 170 cm), weight (e.g., 70 kg), and health goal (e.g., carbohydrate restriction) into the form. The entered data is sent to the server and stored in a database.

Input: height, weight, health goals

Output: User information stored in the database

Step 2:

The user uploads an image of the receipt.

Users use their device to upload images of recent purchase receipts to the system. The uploaded images are then sent to the server.

Input: Receipt image

Output: Receipt image saved on the server

Step 3:

The server performs OCR processing on the receipt image.

The server uses optical character recognition (OCR) technology to convert the receipt image into digital data. Specifically, it calls the Google Cloud Vision API to perform the OCR process.

Input: Receipt image

Output: Digital data (text information)

Step 4:

The server analyzes product information and detects unnecessary products.

The server analyzes the digital data obtained through OCR processing and extracts product names and their nutritional information. For example, if a receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

Input: Digital data (text information), user's health goals

Output: List of unwanted items

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

The server calculates the total value of the unwanted products detected, as well as the estimated weight loss that would occur if the user did not consume these products. A standard calorie expenditure model is used to calculate weight loss.

Input: List of unwanted items, product price, calorie information

Output: Total amount, estimated weight loss

Step 6:

The server visualizes the calculation results in graphs and charts.

The server uses a data visualization library such as D3.js to generate graphs and charts showing the total price of unwanted items and estimated weight loss.

Input: Total amount, estimated weight loss

Output: Graphs and charts

Step 7:

The user checks the results.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

Input: Graphs and charts

Output: User understanding and recognition

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Today's consumers are expected to manage their health and save money at the same time, but it can be difficult to determine which products are unnecessary when shopping. Furthermore, there is a lack of ways to grasp the specific amount of savings or weight loss that can be achieved by not purchasing unnecessary items. For this reason, there is a need for a system that allows consumers to make healthy choices and realize the savings they are making.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for customers to enter their current height and weight and goals, means for customers to submit purchase receipts after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and the amount of weight lost, and means for displaying the detection results in a graph. This allows consumers to easily identify unnecessary items when shopping and specifically understand both health management and savings.

"Customer" refers to a consumer who uses the system to make purchases.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

"Purchase receipt" refers to the paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unnecessary items" refer to items that should be removed from the shopping list based on the customer's height, weight, and goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Visualization methods" refers to ways to display the money saved and weight lost in visual form, such as graphs or charts.

"Means of detection" refers to a method for identifying unwanted products based on the customer's height, weight, and goals.

"Means of calculation" refers to a method for calculating the amount of money saved and weight loss that would be achieved by not purchasing unnecessary items.

"Means of displaying in a graph" refers to a method of displaying detection results in a visually easy-to-understand format.

A system for implementing this invention includes means for a customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and weight lost, and means for displaying the detection results in a graph.

System Configuration

The system is implemented as a smartphone application. The application is built using Python and Flask, and performs data processing on the server side. The smartphone sends the user's height, weight, and goals, as well as receipt information, to the server.

Data processing and calculation

The server receives the data sent by the user and performs the following processing:

1. Detecting unwanted products:

The server identifies unnecessary items on the receipt based on the user's height, weight, and goals. For example, if the user's goal is to "limit carbohydrates," it will detect "bread" and "sweets" on the receipt as unnecessary items.

2. Calculate savings and weight loss:

The server calculates the amount of money saved and the amount of weight lost if the unnecessary items were not purchased. For example, it adds up the prices of the unnecessary items and calculates the amount of weight lost based on the calories in those items.

3. Visualizing the results:

The server generates the calculation results in the form of graphs and charts and sends them to a smartphone application. Users can visually check the results through the application.

Hardware and software used

Hardware: Smartphone

Software: Python, Flask

Specific examples

If a user's goal is to "limit carbohydrates" and the receipt includes "bread" and "sweets," the system will detect these as unnecessary items. The savings will be calculated as 350 yen (bread 150 yen + sweets 200 yen), and the weight loss will be 0.064 kg (500 kcal / 7700 kcal). These results are displayed as a graph on the smartphone application.

Example prompt

If a user is aiming to limit carbohydrates, detect unnecessary items in their receipts and calculate savings and weight loss. Use the following data:

Height: 170cm

Weight: 70kg

Goal: Carbohydrate restriction

Receipt items: ["Bread", "Sweets", "Vegetables"]

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goal.

Input: height, weight, goal

Output: Height, weight, and goal data

Specific action: The user enters their height, weight, and goal into the application's input form and presses the submit button.

Step 2:

After shopping, users scan their purchase receipt with their smartphone and upload it to the application.

Input: Purchase receipt image

Output: Receipt text data

Specific operation: The user takes a photo of the receipt using their smartphone camera and uploads it to the application. The application uses OCR technology to extract text data from the image.

Step 3:

The server receives the height, weight, goal, and receipt text data sent by the user.

Input: Height, weight, goal, and receipt text data

Output: Save received data

Specific operation: The server receives the HTTP request and stores the submitted data in the database.

Step 4:

The server detects unwanted items on the receipt based on the user's goals.

Input: Height, weight, goal, and receipt text data

Output: List of unwanted items

Specific operation: The server references the list of unwanted items related to the goal and identifies the unwanted items by matching them with the text data on the receipt.

Step 5:

The server calculates the amount of savings and weight loss you would have achieved if you had not purchased unnecessary items.

Input: List of unwanted items

Output: Savings, weight loss

Specific operation: The server adds up the prices of the unwanted items and calculates the amount of weight loss based on the calories of the unwanted items. For example, if the total price of the unwanted items is 350 yen and the total calories are 500 kcal, the amount of weight loss will be calculated as 0.064 kg (500 kcal / 7700 kcal).

Step 6:

The server generates the calculation results in the form of graphs and charts and sends them to the smartphone application.

Input: Amount saved, weight loss

Output: Image data for graphs and charts

Specific operation: The server generates graphs and charts based on the calculation results and sends them as image data to the smartphone application.

Step 7:

Users can visually check the savings and weight loss results through a smartphone application.

Input: Image data for graphs and charts

Output: Visual results

What it does: The user opens the app and looks at the graphs and charts that are displayed, allowing them to understand the specific savings and weight loss they could have achieved by not purchasing unnecessary items.

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

This embodiment of the present invention provides a dedicated input form as a means for consumers to enter their height, weight, and diet goals. This input form is provided on a website or smartphone application, and allows consumers to directly input their height, weight, diet goals, etc.

"Example 2"

Next, as a means for consumers to submit their purchase receipts after shopping, we provide a feature that allows them to take an image of the receipt using their smartphone camera and upload it to the application. This feature allows consumers to communicate purchase information to the system without having to directly submit a physical receipt.

"Example 3"

Then, as a means of correcting items that are deemed unnecessary for this shopping trip, the system automatically detects unnecessary items based on the height, weight, and goal entered by the consumer, as well as the information on the submitted receipt. For example, if the diet goal is to "limit carbohydrates," the system will detect items on the receipt that are high in carbohydrates, such as "bread" and "sweets," as unnecessary.

"Example 4"

Finally, as a means of visualizing the money saved and weight loss that would result from not purchasing unnecessary products, the system calculates the total cost of the detected unnecessary products and the estimated amount of weight loss that would result from not consuming those products, and displays this information to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their own health management and savings.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Consumers enter their height, weight, diet goals, etc. into a dedicated input form. For example, they might enter a height of 170 cm, a weight of 65 kg, and a diet goal of "restricting carbohydrates."

Step 2: Once the information is complete, the consumer presses the "Submit" button to send the information to the system.

"Example 2"

Step 1: The consumer makes a purchase and obtains a receipt for the items purchased.

Step 2: The consumer takes a picture of the receipt using their smartphone camera.

Step 3: The captured image is uploaded to the application and the system reads the receipt information.

"Example 3"

Step 1: The system automatically detects unwanted items based on the consumer's height, weight, and goals entered, as well as the submitted receipt information.

Step 2: For example, if your diet goal is "restrict carbohydrates," the system will

Products with high sugar content, such as "bread" and "sweets," are detected as unnecessary items.

"Example 4"

Step 1: The system calculates the total value of the detected unwanted products and the estimated weight loss that would occur if the product were not consumed.

Step 2: The calculation results are displayed to consumers in the form of graphs, charts, etc. This allows consumers to concretely understand both their health management and savings.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional diet support systems required users to input their height, weight, and diet goals, as well as submit purchase receipts after shopping and identify unwanted items. However, these systems were time-consuming for users, making them difficult to use on an ongoing basis. Furthermore, they lacked the ability to individually suggest appropriate diet plans based on the data entered by the user, making them unable to provide effective diet support.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for receiving and saving the data entered by the customer, means for generating prompt sentences for the generative AI model based on the saved data, and means for the generative AI model to suggest a diet plan. This allows users to use the system continuously without any hassle, and makes it possible to suggest appropriate diet plans individually.

"Customer" refers to the user of the system.

"Height and Weight" refers to the customer's current height and weight data.

"Goals" refers to the diet and health goals set by the customer.

"Input form" refers to the interface where customers enter data such as height, weight, and goals.

"Purchase receipt" refers to the detailed list of purchases that a customer receives after making a purchase.

"Means for correcting" refers to the function that determines unnecessary products based on the customer's height, weight, and goals, and removes them from the list.

"Visualization means" refers to a function that visually displays the money you can save and the weight you will lose if you don't buy unnecessary products.

"Means for receiving and storing data" refers to the function by which the server receives data entered by the customer and stores it in a database, etc.

"Generative AI model" refers to an artificial intelligence model that generates an appropriate diet plan based on input data.

"Prompt sentence" refers to an instruction sentence generated for a generative AI model based on input data.

"Diet Plan" refers to a specific action plan proposed by the generative AI model to help customers achieve their goals.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is installed on a website or smartphone application, and a generative AI model proposes an appropriate diet plan based on the data entered by the customer.

Hardware and software used

Hardware: Servers (e.g., AWS EC2), user devices (e.g., smartphones, PCs)

Software: Web servers (e.g., Apache, Nginx), databases (e.g., MySQL, PostgreSQL), smartphone applications (e.g., iOS apps, Android apps)

Specific system operation

Server

The server provides a dedicated input form on a website or smartphone application. This input form allows customers to enter their height, weight, diet goals, etc. The server receives the input data and stores it in a database. The stored data is used as prompts for the generative AI model.

Device

The device displays the websites and smartphone applications that the user accesses. When the user enters their height, weight, and diet goals into an input form, the device sends the data to a server. The device then displays the data the user has entered in real time, allowing them to check it.

User

Users access a website or smartphone application and enter their height, weight, and diet goals into a dedicated input form. Once the user has completed the input, the data is sent to the server. The user can then review the data they have entered and make any necessary corrections.

Specific examples

Example 1: Website use

A user visits a website and enters the following information into a form:

Height: 170 cm

Weight: 70 kg

Diet goal: Lose 5 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Please suggest an appropriate diet plan based on this information.

Example 2: Use in a smartphone application

A user opens a smartphone application and enters the following information into the input form:

Height: 160 cm

Weight: 60 kg

Diet goal: Lose 3 kg

Once the input is complete, the device sends this data to the server. The server receives the data and stores it in a database. It then generates the following prompt for the generative AI model:

Example prompt:

The user's height is 160 cm, weight is 60 kg, and their diet goal is to lose 3 kg. Please suggest an appropriate diet plan based on this information.

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application and accesses an input form. The input form is an interface where the user can enter their height, weight, and diet goals. Once the input form is displayed, the user is ready to enter the required information.

Step 2:

The user enters the data.

The user enters their height, weight, and diet goal into the input form. For example, the user might enter "Height: 170 cm", "Weight: 70 kg", and "Diet goal: Lose 5 kg". Once the input is complete, the user taps the "Submit" button. The input data is saved on the device in JSON format.

Step 3:

The device sends the data to the server.

The device sends the data entered by the user to the server. Specifically, the device uses an HTTP POST request to send the input data to the server. At this time, the data is sent in JSON format. The input data (height, weight, diet goal) is sent to the server.

Step 4:

The server receives and stores the data.

The server receives the data sent from the device. The received data is stored in a database. For example, the server connects to a MySQL database and executes the following SQL query:

sql

INSERT INTO user_data (height, weight, goal) VALUES (170, 70, '5 kg weight loss');

This process saves the user's input data in the database.

Step 5:

The server generates a prompt for the generative AI model.

The server generates prompts for the generative AI model based on the stored data. For example, it generates prompts like the following:

"The user's height is 170 cm, weight is 70 kg, and their diet goal is to lose 5 kg. Based on this information, please suggest an appropriate diet plan."

This prompt is sent to the generative AI model.

Step 6:

Generative AI models suggest diet plans.

The generative AI model generates a diet plan based on the prompt it receives. For example, the generative AI model might suggest the following diet plan:

1. Do 30 minutes of aerobic exercise every day.

2. Limit your daily calorie intake to 1,500 kcal

3. Strength training three times a week

This diet plan is sent to the user's device via the server and displayed to the user.

In this way, a system is created in which a generative AI model suggests an appropriate diet plan based on data entered by the user.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional food delivery systems struggled to provide meal plans based on customers' health status and diet goals, making it impossible to provide services tailored to individual needs. Furthermore, problems arose, such as customers buying unnecessary items and making meal choices that hindered the achievement of diet goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height, weight, and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for generating an appropriate meal plan based on the customer's height, weight, and goals, and means for presenting the generated meal plan to the customer. This makes it possible to provide an appropriate meal plan based on the customer's health condition and diet goals, and support the achievement of diet goals while reducing unnecessary spending.

"Customer" refers to the consumer who uses the service.

"Height and Weight" refers to the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that a customer wishes to achieve.

"Purchase receipt" refers to the detailed list of purchases received by a customer when making a purchase.

"Correction means" refers to a method or system for evaluating products purchased by customers and identifying unwanted products.

"Visualization means" refers to a method or system that visually displays the money saved or weight lost.

"Meal Plan" refers to the meal content and schedule suggested based on the client's health and diet goals.

"Means for generating" refers to a method or system for creating a meal plan based on customer input data.

"Presentation means" refers to the method or system for showing the generated meal plan to the customer.

The following system configuration is proposed as an embodiment of this invention.

System Configuration

The system includes a smartphone application used by customers, a server, and a generative AI model. The smartphone application provides an interface for customers to input their height, weight, and diet goals. The server receives the input data and sends prompts to the generative AI model to generate an appropriate meal plan. The generated meal plan is then presented to the customer again via the smartphone application.

Hardware and software used

Hardware: Smartphones, servers

Software: Smartphone applications, generative AI models (e.g., OpenAI GPT-3)

Data processing and calculation

1. Data entry: Customers use a smartphone application to enter their height, weight, and diet goals.

2. Data transmission: The smartphone application sends the entered data to the server.

3. Prompt generation: Based on the data received by the server, a prompt sentence is generated to be sent to the generative AI model.

4. Meal plan generation: The generative AI model receives the prompt and generates an appropriate meal plan.

5. Data reception and presentation: The server receives the generated meal plan and presents it to the customer via the smartphone application.

Specific examples

For example, if a customer enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg," the server will send the following prompt text to the generative AI model:

Example prompt

"Please suggest an appropriate meal plan based on the following: height: 170 cm, weight: 70 kg, goal: to lose 5 kg."

Based on this prompt, the generative AI model generates a balanced meal plan. For example, it suggests specific meal options such as "Breakfast - oatmeal, Lunch - salad, Dinner - chicken breast and vegetables." This meal plan is presented to the customer via a smartphone application.

In this way, we can provide customers with appropriate meal plans based on their health condition and diet goals, helping them achieve their diet goals while reducing unnecessary spending.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goal. The entered data is temporarily saved within the smartphone application. An example of input data is: height 170cm, weight 70kg, diet goal "lose 5kg."

Step 2:

The smartphone application sends the entered data to the server. The data sent includes the user's height, weight, and diet goals. The server analyzes the received data and proceeds to the next processing step.

Step 3:

The server generates a prompt based on the received data. Specifically, it combines the user's height, weight, and diet goal to create a prompt to send to the generative AI model. For example, a prompt might be generated that reads, "Based on height: 170 cm, weight: 70 kg, goal: to lose 5 kg, please suggest an appropriate meal plan."

Step 4:

The server sends the generated prompt text to the generative AI model. The generative AI model performs data calculations based on the received prompt text and generates an appropriate meal plan. The generative AI model uses, for example, OpenAI GPT-3.

Step 5:

The generative AI model generates a meal plan based on the prompt and sends the results back to the server. An example of a generated meal plan is "Breakfast - Oatmeal, Lunch - Salad, Dinner - Chicken Breast and Vegetables."

Step 6:

The server analyzes the meal plan received from the generative AI model and sends it to the smartphone application. The transmitted data includes the generated meal plan.

Step 7:

The smartphone application presents the meal plan received from the server to the user. The user can review and implement the proposed meal plan through the application.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Under the previous system, customers had to physically submit receipts for their purchases, which was time-consuming and risked loss or damage. Furthermore, there were insufficient ways to identify unnecessary purchases and visualize savings and weight loss. This made it difficult for customers to effectively manage their shopping to achieve their goals.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to take an image of their purchase receipt after shopping and upload it to the application so that they can submit it, means for correcting items that are determined to be unnecessary for this purchase, and means for visualizing the amount of money saved and weight loss that would be achieved if the unnecessary items were not purchased. This saves the customer the trouble of submitting a physical receipt and allows them to easily communicate their purchase information to the system. Furthermore, by identifying unnecessary items and visualizing the amount of money saved and weight loss that would be achieved, customers can effectively manage their shopping toward their goals.

"Customer" refers to an individual who uses the system to shop and manages their purchase information.

"Height and Weight" refers to the customer's current physical height and weight.

"Goal" refers to a specific physical or financial goal that the client wishes to achieve.

"Purchase receipt" means a paper or electronic record issued to a customer when they make a purchase that contains details of the items purchased.

"Taking an image" refers to the act of taking a photo of a physical receipt using a device such as a smartphone or camera.

"Application" refers to a software program that runs on a device such as a smartphone or tablet.

"Uploading" refers to the act of sending captured images to a server via the Internet.

"Server" refers to a computer system that receives, processes, and stores data over a network.

"Analyzing an image" refers to the act of using image recognition technology to extract the contents of a photographed receipt as digital data.

"Unnecessary products" refer to products that are determined not to be necessary for the customer to achieve their goals.

"Savings" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss you would expect if you had not purchased unnecessary items.

"Visualizing" refers to the act of displaying data visually so that customers can easily understand it.

"Displaying graphically" refers to the act of displaying data in a visual format such as a graph or chart.

This invention provides a system that allows customers to take an image of their purchase receipt after shopping and upload it to an application in order to submit it. A specific embodiment of this system is described below.

The user takes a picture of the purchase receipt using the smartphone camera. The image is then uploaded using a dedicated application (e.g., "ReceiptUploader"). The application has the function of sending the image taken by the user to the server.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted.

The analyzed data is stored in a database by the server. The stored data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database.

Users can check their purchase history using the application. By opening the "Purchase History" section within the application, data retrieved from the server is displayed. Users can view a list of past purchases.

As a concrete example, consider a scenario after a user has shopped at a supermarket. The user takes a picture of the receipt using their smartphone camera and opens the "ReceiptUploader" application. Within the application, the user taps the "Upload" button, selects the image of the receipt, and uploads it. The application sends the image to a server, which uses the Google Cloud Vision API to analyze the contents of the receipt. The analyzed data is stored in a database, and the user can open the "Purchase History" section within the application to view information about recent purchases.

Examples of prompts to input into a generative AI model include:

"Please describe the functionality that allows users to take a picture of a receipt using their smartphone camera and upload it to the application. Please include the names of any specific hardware or software."

Using this prompt, the generative AI model can generate sentences that explain in detail how the system will operate.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user takes a picture of the receipt.

The user uses their smartphone camera to take an image of the receipt for the item they purchased. When taking the photo, they are careful to capture the entire receipt clearly. The input is the physical receipt, and the output is a digital image.

Step 2:

The device uploads the image to the application.

The device (smartphone) uploads the captured image of the receipt to the "ReceiptUploader" application. The user opens the application, taps the "Upload" button, and selects the captured image. The input is a digital image, and the output is the transmission of image data to the server.

Step 3:

The server receives and analyzes the image.

The server receives the image of the receipt sent from the device. After receiving it, the server uses image recognition software such as Google Cloud Vision API to analyze the contents of the receipt. Optical character recognition (OCR) technology is used for the analysis, and information such as the product name, price, and purchase date and time written on the receipt is extracted. The input is image data, and the output is analyzed text data.

Step 4:

The server stores the analysis results in a database.

The server saves the analyzed data in a database. The saved data includes the product name, price, purchase date and time, store name, etc. This allows the user's purchase history to be accumulated in the database. The input is the analyzed text data, and the output is saved in the database.

Step 5:

The user checks their purchase history.

Users can use the "ReceiptUploader" application to check their purchase history. When they open the "Purchase History" section in the application, data retrieved from the server is displayed. Users can view a list of past purchases. The input is data retrieved from the database, and the output is displayed in the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional shopping systems required customers to physically submit receipts for their purchases, which was time-consuming and made managing purchase information cumbersome. Furthermore, there was a lack of systems that could identify unnecessary products based on customers' health goals and visualize their savings and health benefits.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application; a means for correcting items determined to be unnecessary for the current purchase; a means for visualizing the money saved and weight lost if the unnecessary items were not purchased; a means for analyzing the contents of the purchase receipt and automatically extracting purchase information; a means for rewarding points to the customer based on the extracted purchase information; and a means for allowing the rewarded points to be used for the next purchase. This saves the customer the trouble of submitting a physical receipt, simplifies management of purchase information, and enables visualization of point rewards and savings. It also identifies unnecessary items based on the customer's health goals and visualizes savings and health benefits.

"Customer" refers to a consumer who purchases goods or services.

"Height and weight" refers to basic physical information that refers to the customer's physical height and weight.

A "goal" refers to a specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record detailing a transaction issued to a customer when they purchase goods.

A "smartphone" is a type of mobile phone that can connect to the Internet and use applications.

A "camera" is a device used to take images and videos.

An "application" is a software program that runs on a smartphone or computer.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Unnecessary products" are products that are deemed not necessary for achieving the customer's goals.

"Correction" is the act of checking the content and making corrections or improvements.

"Money saved" is the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" refers to the visual display of data or information.

"Analysis" is the act of examining data or information in detail to clarify its structure and meaning.

"Purchase information" refers to detailed information about the products purchased by the customer.

"Automatic extraction" means that the system extracts data without human intervention.

"Point redemption" is the act of awarding points to customers based on the purchase amount.

"Can be used on next purchase" means that the customer can use the points the next time they purchase a product.

The following describes in detail the mode for implementing this invention.

First, a smartphone application is used by the customer to enter their current height, weight, and goals. The customer launches the application and enters their height, weight, and goals. This registers the customer's health goals in the system.

Next, we provide a way for customers to take a photo of their purchase receipt with their smartphone camera after shopping and upload it to the application. Customers take a photo of their receipt and use the upload function within the application to send the image to the server.

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. Next, it uses pytesseract to extract text from the preprocessed image. It then analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text.

Based on the analyzed purchase information, the application provides a means for rewarding customers with points. The server calculates points based on the purchase amount and adds them to the customer's account. Rewarded points are managed within the application so they can be used for the next purchase.

Furthermore, as a means of filtering out items that may be deemed unnecessary for this shopping trip, an algorithm is implemented to determine unnecessary items based on the customer's height, weight, and goals. The server compares purchase information with the customer's health goals to identify unnecessary items.

To visualize the money saved and weight lost by not buying unnecessary products, the server calculates the amount saved and weight lost and displays them graphically. This allows customers to visually see the savings and health benefits.

As a concrete example, consider the case where a user purchases a drink at a convenience store, takes a photo of the receipt, and uploads it to the app. The app extracts "drink" and "150 yen" from the receipt and rewards 15 points. Furthermore, if the drink is deemed an unnecessary item based on the user's health goals, the amount of money saved and the weight lost will be displayed.

Examples of prompts for a generative AI model are as follows:

"A user has uploaded a receipt image taken with their smartphone camera. Please create a program that extracts purchase information from this image and rewards points. 10% of the purchase amount will be rewarded as points."

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goals. The entered data is temporarily saved within the application and sent to the server. The entered data is used as the basis for setting the user's health goals.

Step 2:

After shopping, the user takes a photo of the purchase receipt with their smartphone camera and uploads it to the application. The captured receipt image is temporarily saved within the application and then sent to the server. The input data is the receipt image.

Step 3:

The server receives the uploaded receipt image and preprocesses it using OpenCV. Specifically, it converts the image to grayscale and performs binarization. The input data is the receipt image, and the output data is the preprocessed image.

Step 4:

The server extracts text from the preprocessed image using pytesseract. The input data is the preprocessed image, and the output data is the extracted text. Specifically, it uses OCR (optical character recognition) technology to read the text information in the image.

Step 5:

The server analyzes and automatically extracts purchase information (product name, price, etc.) from the extracted text. The input data is the extracted text, and the output data is the analyzed purchase information. Specifically, it uses a text analysis algorithm to extract the required information.

Step 6:

The server rewards the customer with points based on the analyzed purchase information. The input data is the analyzed purchase information, and the output data is the rewarded points. Specifically, it calculates points based on the purchase amount and adds them to the customer's account.

Step 7:

The server determines unnecessary products based on the customer's height, weight, and goals. The input data is the customer's health goals and purchase information, and the output data is a list of unnecessary products. Specifically, it runs an algorithm that compares the health goals with the purchase information and identifies unnecessary products.

Step 8:

The server calculates the amount of money saved and weight lost if unnecessary items are not purchased, and displays this graphically. The input data is a list of unnecessary items and purchase information, and the output data is the amount saved and weight lost. Specifically, it calculates the savings and health benefits and displays them visually to the user.

Step 9:

The user uses the points they have earned on their next purchase. The input data is the points earned, and the output data is the balance after the points have been used. Specific operations include using the points to discount the purchase amount and updating the balance.

(Example 3)

Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Today's consumers find it difficult to balance health management and saving money. Consumers with diet or health goals in particular have difficulty determining which products are unnecessary when shopping. Furthermore, there are few ways to specifically assess the savings and health benefits of not purchasing unnecessary products. For this reason, a system that helps consumers effectively manage their health and save money is needed.

The identification processing by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for converting the submitted receipt information into text data using optical character recognition technology, means for determining unnecessary products based on the customer's height, weight, and goals, means for calculating the amount of money saved and the weight lost if the unnecessary products are not purchased, and means for graphically displaying the calculation results. This allows consumers to easily identify unnecessary products when shopping and specifically understand the amount of money saved and the health benefits.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and Weight" refers to the customer's current physical measurements and is used as basic data for health management.

"Goals" are specific health and diet objectives set by customers, and include, for example, "carbohydrate restriction" or "calorie restriction."

A "receipt" is a detailed list of purchased items that a customer receives after shopping, listing the product names and prices.

"Optical character recognition technology" is a technology that extracts character information from image data and is used to convert receipt images into text data.

"Text data" refers to text information converted using optical character recognition technology, and includes information such as product names and prices.

"Unnecessary products" are products that are determined not to be necessary to purchase from the perspective of health management or dieting, based on the customer's height, weight, and goals.

"Amount saved" is the total amount of money you can save by not purchasing unnecessary items.

"Weight loss" is the amount of weight loss predicted if the unnecessary product is not consumed.

"Displaying graphically" means displaying the calculation results in the form of graphs, charts, etc. to make them easier to understand visually.

Form for implementing the invention

This invention is a system that enables consumers to effectively manage their health and save money. Specific embodiments of this system are described below.

System Configuration

The system consists of the following main components:

1. A device for users to enter their height, weight, and goals.

2. A terminal where users can submit receipts after shopping.

3. Hardware and software for the server to receive and process data

Hardware and software used

Device: Mobile devices such as smartphones and tablets

Server: High-performance data processing server

OCR technology: Optical character recognition technology such as Google Cloud Vision API

Database: Relational database such as MySQL or PostgreSQL

Data visualization libraries: D3.js, Chart.js, etc.

Program processing

The server receives the height, weight, and goal information entered by the user, as well as the submitted receipt. Using this data, the server processes and calculates the data to detect unwanted products. Specifically, the process is carried out as follows:

1. Data reception and analysis:

The server receives the user's input height, weight, and goal (e.g., carbohydrate restriction).

The server converts the information on the submitted receipt into text data using OCR technology.

2. Detecting unwanted products:

The server detects unnecessary products based on the goal. For example, if the goal is "restricting carbohydrates," the server will list high-carbohydrate products on the receipt, such as "bread" and "sweets," as unnecessary products.

3. Calculate savings and weight loss:

The server calculates the total price of the unwanted items detected.

The server calculates the estimated weight loss that would occur if the unnecessary products were not consumed.

4. Visualizing the results:

The server displays the calculation results to the consumer in the form of graphs, charts, etc.

Specific examples

Let's say a user is 170cm tall, weighs 70kg, sets their goal to "restrict carbohydrates," and submits a receipt. The receipt includes "bread (300 yen)," "sweets (200 yen)," and "vegetables (100 yen)." The system detects "bread" and "sweets" as unnecessary items and calculates that a total of 500 yen can be saved. The system also calculates that the estimated weight loss would be 0.5kg if these items were not consumed, and displays this in a graph.

Example prompt

"I'm 170cm tall and weigh 70kg. My goal is to limit my carbohydrate intake. Use the following receipt information to identify unnecessary items and calculate the savings and weight loss. Receipt information: Bread (300 yen), sweets (200 yen), vegetables (100 yen)."

The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

Users enter their height, weight, and goal, then submit a receipt.

Input: The user opens the app on their smartphone and enters their height (170cm), weight (70kg), and goal of "restricting carbohydrates." After shopping, they also take a photo of the receipt with their smartphone camera and upload it to the server via the app.

Output: User input data and receipt image are sent to the server.

Step 2:

The server receives the input data and receipt information.

Input: Height, weight, goal, and receipt image submitted by the user.

Output: User input data and receipt image stored in a database on the server.

Step 3:

The server converts the receipt information into text data using OCR technology.

Input: Receipt image stored on the server.

Data processing: The server calls the Google Cloud Vision API, analyzes the receipt image, and extracts the product name and price.

Output: Converted text data (e.g., "Bread (300 yen)", "Sweets (200 yen)", "Vegetables (100 yen)").

Step 4:

The server detects unwanted items based on your goals.

Input: User's goal (e.g., "Limit carbohydrates") and converted text data.

Data calculation: The server performs rule-based filtering based on predefined food categories and goals to list unwanted products.

Output: List of unwanted products (e.g., "bread", "sweets").

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

Input: List of unwanted items and their respective prices.

Data calculation: The server calculates the total price of the unnecessary products and also calculates the estimated weight loss if the unnecessary products were not consumed. The model used here is a simulation model based on the relationship between calories consumed and weight loss.

Output: Amount saved (e.g., 500 yen) and estimated weight loss (e.g., 0.5 kg).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them to the user.

Enter: Amount saved and estimated weight loss.

Data processing: The server uses data visualization libraries such as D3.js and Chart.js to visualize the calculation results in graphs and charts.

Output: Graphs and charts displayed on the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Today's consumers find it difficult to simultaneously manage their health and save money. In particular, consumers with dietary goals need to identify and avoid unnecessary products from their purchases in order to achieve health and financial benefits. However, doing this manually is time-consuming and inaccurate. Therefore, a system is needed that allows consumers to easily identify unnecessary products and efficiently manage their savings and health.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for analyzing receipt information and automatically detecting unnecessary items based on diet goals, means for calculating the amount of money saved and estimated weight loss if the unnecessary items are not purchased and displaying them in graphs and charts, and an application installed on a smartphone. This allows consumers to easily identify unnecessary items and efficiently save money and manage their health.

"Customer" refers to consumers who use the system.

"Height and Weight" is data indicating the customer's physical height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

A "purchase receipt" is paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unwanted products" refer to products that are deemed inappropriate or unnecessary for the customer's goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Receipt information" refers to data such as the product name, price, and quantity listed on the purchase receipt.

"Diet Goals" refers to health goals set by a customer, such as weight loss or specific nutritional restrictions.

"Automatically detect" means that the system identifies unwanted products programmatically without human intervention.

"Graphs and charts" refer to diagrams used to visually display data.

"Applications installed on smartphones" refers to software programs that run on smartphones.

This invention is a system that allows consumers to efficiently manage their health and save money. Specific embodiments for implementing this system are described below.

System Configuration

The system consists of the following main components:

1. User device: A mobile device such as a smartphone.

2. Server: A central server that processes and analyzes data.

3. Application: A dedicated application installed on a smartphone.

Program processing

User device

The user device has the following functions:

User information input: Users enter their height, weight, and diet goals through the application.

Receipt scanning: The user scans the receipt for the item they purchased with their smartphone camera and obtains the image data.

Server

The server performs the following process:

Analyzing receipt information: The acquired receipt image is converted into text data using OCR (optical character recognition) technology. This process uses the pytesseract library.

Detection of unnecessary products: Automatically detect unnecessary products from receipt text data based on the user's diet goals. For example, for a user aiming to limit carbohydrate intake, high-carbohydrate products such as "bread" and "sweets" will be detected.

Calculation of savings and weight loss: Calculates the total cost of the detected unnecessary products and the estimated weight loss if you did not consume those products. Weight loss is calculated based on the calories required to lose 1 kg of weight (approximately 7700 kcal).

application

The application has the following features:

Visualization of results: Calculated savings and estimated weight loss are displayed in graphs and charts, allowing users to visually see the effects of their health management and savings.

Specific examples

For example, if a user sets a goal of "restricting carbohydrates" and the receipt includes "bread" and "sweets," these items will be detected as unnecessary products. Suppose the total amount of the detected unnecessary products is 500 yen, and the estimated weight loss is 0.2 kg.

Example prompt

Please enter your user information:

Height: 170 cm
Weight: 70 kg
Diet goal: Carbohydrate restriction

Scan your receipt:

(Upload receipt image)

Unwanted items detected...

Unwanted items detected:

Bread: 200 yen

Sweets: 300 yen

Savings: ¥500

Estimated weight loss: 0.2kg

The results are displayed in a graph.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

The user launches the smartphone application and enters their height, weight, and diet goals.

Input: User's height, weight, and diet goal

Output: User information data

Specific operation: The user enters their height, weight, and diet goal (e.g., carbohydrate restriction) into the application's input form and presses the submit button. This generates user information data and sends it to the server.

Step 2:

After shopping, the user scans the receipt for the purchased items using their smartphone camera.

Input: Receipt image

Output: Receipt image data

Specific operation: The user uses the application's camera function to take a photo of the purchase receipt. The captured receipt image is saved within the application and sent to the server.

Step 3:

The server receives the receipt image and converts it into text data using OCR technology.

Input: Receipt image data

Output: Receipt text data

Specific operation: The server uses the pytesseract library to extract text from the received receipt image data. The extracted text data is saved as receipt text data.

Step 4:

The server automatically detects unnecessary items from receipt text data based on the user's diet goals.

Input: User information data, receipt text data

Output: List of unwanted items

Specific operation: The server analyzes receipt text data based on the user's diet goal (e.g., carbohydrate restriction). Products containing specific keywords (e.g., "bread," "sweets") are listed as unwanted items.

Step 5:

The server calculates the amount of money you could save and the estimated weight loss if you didn't buy unnecessary items.

Input: List of unwanted items

Output: Money saved, estimated weight loss

Specific operation: The server calculates the savings amount by adding up the prices of each item based on the list of unwanted items. It also calculates the estimated weight loss amount based on the calorie information of each item. The weight loss amount is calculated based on the calories required to lose 1 kg (approximately 7700 kcal).

Step 6:

The server visualizes the calculation results in graphs and charts and displays them on the user's smartphone.

Input: Amount saved, estimated weight loss

Output: Image data for graphs and charts

Specific operation: Based on the calculation results, the server generates graphs and charts showing the amount of savings and estimated weight loss. Image data of the generated graphs and charts is sent to the user's smartphone and displayed within the application.

In this way, users can easily identify unnecessary products and efficiently save money and manage their health.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention provides a system that incorporates an emotion engine. This system recognizes the user's emotions and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products. When the user is relaxed, it recommends healthy foods.

"Example 2"

Furthermore, the emotion engine graphically displays the money saved and weight lost based on the user's emotions. Specifically, it displays the amount of money saved and the amount of weight lost if the user refrains from purchasing unnecessary products when feeling stressed. This allows users to visually understand the relationship between their emotions, health management, and saving money.

"Example 3"

For example, if a user buys chocolate when feeling stressed, the emotion engine will detect the chocolate as an unnecessary item and display a graph of the amount of money saved and weight loss they would have experienced if they had not bought the chocolate. This helps users recognize their tendency to buy unhealthy foods when they are stressed and understand the resulting financial and health impacts.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine determines which products are unnecessary based on the user's emotions and notifies the user of the results.

"Example 2"

Step 1: The user enters their current height, weight, and goal into the system.

Step 2: The user makes a purchase and submits the receipt to the system.

Step 3: The emotion engine recognizes the user's emotions.

Step 4: The emotion engine graphically displays the money saved and weight lost based on the user's emotions.

"Example 3"

Step 1: Buy chocolate when the user is feeling stressed.

Step 2: The emotion engine recognizes the user's emotions and detects chocolate as an unwanted item.

Step 3: The emotion engine displays a graph of the amount of money saved and weight loss you would have experienced if you had not bought the chocolate.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

The previous system allowed customers to enter their height, weight, and goals, and to submit receipts after shopping, but it lacked the ability to identify unnecessary products based on the customer's emotions and recommend healthy products. This made it difficult for customers to select appropriate products when they were stressed or relaxed. It also lacked a way to visualize the money saved and weight lost by not purchasing unnecessary items. This made it difficult to raise customers' awareness of health management and saving money.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for the customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if the unnecessary items are not purchased, means for recognizing the customer's emotions and determining unnecessary items based on those emotions, and means for recommending healthy products based on the customer's emotions. This enables the customer to select appropriate products according to their emotional state, thereby increasing their awareness of health management and saving money.

"Customer" refers to an individual who uses the system to enter their height, weight, and diet goals, and then submits a purchase receipt after shopping.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to specific numerical values or conditions related to dieting or health that a customer wishes to achieve.

"Input means" refers to the interface that allows customers to input their height, weight, and goals into the system.

"Purchase receipt" refers to the paper or electronic data containing the purchase details of a product that a customer receives when making a purchase.

"Submission means" refers to the interface through which customers submit purchase receipts to the system.

"Correction method" refers to the function by which the system determines unnecessary products based on customer input data and emotional data and notifies the customer.

"Visualization means" refers to the system's ability to graphically display the money saved and weight lost if unnecessary items are not purchased.

"Emotion recognition means" refers to the system's ability to analyze customer emotions and determine which products are unnecessary based on those emotions.

"Healthy product recommendation means" refers to the system's ability to recommend healthy products based on customer sentiment.

This invention is a system that provides a dedicated input form for customers to enter their height, weight, and diet goals. The system is set up on a website or smartphone application, and allows customers to directly input their height, weight, diet goals, etc. In addition, by combining it with an emotion engine, it has the function of determining unnecessary products based on the customer's emotions and recommending healthy products.

The server places an input form on a website or smartphone application. This input form is created using HTML and CSS, and JavaScript is used to validate the input data in real time. When a user enters data into the input form, the server receives the data and stores it in a database (for example, MySQL or PostgreSQL).

Next, the server uses an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) to analyze the user's emotions. The emotion engine recognizes the user's current emotions based on the user's input data and past behavioral data. For example, if the user is feeling stressed, the server will use that information to determine that unhealthy foods such as sweets are unnecessary. On the other hand, if the user is relaxed, the server will recommend healthy foods.

As a concrete example, suppose a user uses a smartphone application to enter their height as 170cm, weight as 70kg, and diet goal as "lose 5kg." This data is sent to the server and stored in a database. The emotion engine then analyzes the user's emotions, and if it determines that the user is feeling stressed, the server displays a message such as "Avoid sweet snacks such as chocolate and cookies."

Example of a prompt to input to a generative AI model:

A user enters their height as 170cm, weight as 70kg, and their diet goal as "lose 5kg." The emotion engine analyzes the user's emotions and determines that the user is feeling stressed. In this case, what products would be deemed unnecessary and what products would be recommended?

By inputting this prompt into a generative AI model, specific product recommendations and a determination of unnecessary products can be obtained. In this way, the system can make appropriate product recommendations based on the user's input data and emotional data.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user accesses the input form.

A user opens a website or smartphone application. The device sends a request to the server through the browser or application to retrieve an input form. The server generates the input form using HTML and CSS and sends it to the device. The device renders the received HTML and CSS and displays the input form to the user.

Input: User access request

Output: Display input form

Step 2:

Users enter their height, weight, and diet goals.

The user enters their height, weight, and diet goals into the displayed input form. The device uses JavaScript to validate the format of the input data in real time to ensure there are no errors.

Input: User's height, weight, and diet goal

Output: Validated input data

Step 3:

The server receives the input data and stores it in the database.

When the user presses the send button, the device sends the input data in JSON format to the server. The server parses the received data and saves it in a MySQL database. Specifically, it adds the data to the database using the INSERT statement.

Input: JSON format input data

Output: Data stored in the database

Step 4:

The server calls the emotion engine and analyzes the user's emotions.

The server calls an emotion engine (for example, Microsoft Azure's Emotion API or Google Cloud's Natural Language API) based on the stored data. The data is sent to the API, which analyzes the user's emotions. The API determines that the user is feeling stressed and returns the result to the server.

Input: Data stored in the database

Output: Sentiment analysis results from the emotion engine

Step 5:

The server determines which products are unnecessary based on emotions and decides which products to recommend.

The server determines which products are unnecessary and which are recommended based on emotional data obtained from the emotion engine. For example, if the user is feeling stressed, the server will determine that "sweets such as chocolate and cookies should be avoided" and decide to "recommend healthy foods such as fruits and nuts."

Input: Sentiment analysis results from the emotion engine

Output: List of unwanted and recommended products

Step 6:

The server displays the results to the user.

The server generates HTML to display the results of the judgment to the user and sends it to the device. The device renders the received HTML and displays unwanted products and recommended products to the user. The user checks the displayed results and decides on their next course of action.

Input: List of unwanted and recommended products

Output: The result displayed to the user

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional food delivery systems don't recommend foods based on the user's health status or emotions, which can lead to users choosing unhealthy foods when they're stressed. Furthermore, there's a lack of support for users to choose foods that align with their diet goals, making it difficult to diet effectively.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for the customer to enter their current height and weight and goals, a means for the customer to submit a purchase receipt after shopping, a means for correcting items determined to be unnecessary in this purchase, a means for visualizing the money saved and weight lost if the unnecessary items are not purchased, and a means for recognizing the customer's emotions and determining unnecessary items based on those emotions. This makes it possible to recommend foods based on the user's health condition and emotions, allowing the user to avoid unhealthy foods when feeling stressed and to choose foods that are in line with their diet goals.

"Customer" refers to a consumer who uses a service or product.

"Height and weight" are numbers that indicate an individual's physical height and weight.

"Goal" refers to the specific objective or state that the customer wants to achieve.

A "purchase receipt" is a paper or electronic record of a transaction issued when a customer purchases a product.

"Unnecessary products" refer to products that are deemed inappropriate for the customer's health condition or goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight Loss" is a numerical value that represents the weight loss a customer can achieve by avoiding unnecessary products.

"Means for recognizing emotions" refers to technologies or devices that detect a customer's emotional state and make decisions based on that.

"Visualization means" refers to technology or devices that visually display the money saved or weight lost.

A system for implementing this invention is configured as follows: First, a dedicated form is provided for users to enter their height, weight, and diet goals. This form is provided on a website or smartphone application, allowing users to easily enter information.

Next, we present a system that combines an emotion engine. This system recognizes the user's emotions in real time and determines which products are unnecessary based on those emotions. Specifically, when the user is feeling stressed, it detects unhealthy foods such as sweets as unnecessary products, and when the user is relaxed, it recommends healthy foods.

The following hardware and software are used to implement this system:

Hardware: Smartphone, tablet, or PC

Software: Python, EmotionRecognizer library, FoodRecommender library

The server receives the user's input data such as height, weight, and diet goal, and uses the EmotionRecognizer library to recognize the user's emotions in real time. It then uses the FoodRecommender library to recommend appropriate foods based on the user's input data and the recognized emotions.

For example, if a user enters their height as 170cm, weight as 65kg, and diet goal as "lose weight," and the emotion engine recognizes that the user is feeling stressed, the system will avoid unhealthy foods (e.g., sweet snacks) and instead recommend healthy foods (e.g., salads and fruit).

Examples of prompt statements include:

"If a user is 170cm tall, weighs 65kg, and their diet goal is to lose weight, and the emotion engine determines that the user is feeling stressed, what foods would you recommend?"

In this way, food recommendations can be made based on the user's health status and emotions, helping users avoid unhealthy foods when they are stressed and choosing foods that align with their diet goals.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The user uses the terminal to input their height, weight, and diet goal. The input data is sent to the server. The input data includes the user's height (e.g., 170 cm), weight (e.g., 65 kg), and diet goal (e.g., "lose weight").

Step 2:

The server stores the received user input data and uses the EmotionRecognizer library to recognize the user's emotions in real time. For emotion recognition, the user's facial expressions and voice data are used as input. The output is the user's emotional state (e.g., stressed, relaxed).

Step 3:

The server uses the FoodRecommender library to recommend appropriate foods based on the recognized emotional state and user input data. Input data includes the user's height, weight, diet goals, and emotional state. The output is a list of recommended foods (e.g., salads, fruits).

Step 4:

The server sends a list of recommended foods to the user's device, where the user can check the list of recommended foods on their device. Specifically, if the user is feeling stressed, unhealthy foods will be avoided and healthy foods will be recommended.

Step 5:

A user makes a purchase and submits the purchase receipt to the server using a terminal. The server analyzes the submitted receipt data and generates a list of the purchased items. Input data includes receipt images or electronic data. The output is a list of the purchased items.

Step 6:

The server compares the list of purchased items with the user's diet goals and emotional state to determine which items are unnecessary. Specific operations include detecting items that are inappropriate for the user's goals (e.g., sweet snacks). Input data includes the list of purchased items, the user's diet goals, and their emotional state. The output is a list of unnecessary items.

Step 7:

The server calculates and visualizes the money saved and weight lost if unnecessary items are not purchased. Specific operations include graphically displaying the amount saved and weight lost. The input data includes a list of unnecessary items. The output is a graphical representation of the amount saved and weight lost.

Step 8:

The server sends the visualized data to the user's device, where the user can check the amount of money saved and the weight lost on the device. Specifically, the user can visually understand the benefits they can gain by avoiding unnecessary products.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

With conventional health management systems, customers had to submit physical proof of purchase after shopping, which was time-consuming. Furthermore, the amount of savings and weight loss were not visualized taking into account the customer's emotional state, making it difficult for customers to understand the relationship between their emotions, health management, and savings.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for the customer to enter their current physical information and goals, means for the customer to submit a proof of purchase after shopping, means for the customer to capture an image of the proof of purchase using the camera function of their mobile device and upload it to the application, means for the server to convert the contents of the proof of purchase into text data using image processing technology, means for the server to save the converted text data, means for the customer to input emotional data, means for the server to analyze the customer's emotional data using emotion analysis technology, means for the server to determine unnecessary products based on the analysis results and calculate the amount of savings and weight loss, and means for the server to graphically display the calculation results. This allows customers to communicate their purchase information to the system without having to directly submit a physical proof of purchase, and further enables them to visually understand the relationship between their emotions, health management, and savings.

"Customer" refers to an individual who uses the system to manage their own health and save money.

"Physical information" refers to physical data such as the customer's height, weight, and body fat percentage.

"Goals" refers to the health or savings goals the customer wants to achieve.

"Proof of purchase" refers to a document that proves a purchase, such as a receipt or invoice that a customer receives after shopping.

"Mobile device" refers to portable electronic devices such as smartphones and tablets.

"Photography function" refers to the camera function built into the mobile device.

"Application" refers to a software program that runs on a mobile device.

"Image processing technology" refers to the technology of analyzing image data and extracting necessary information.

"Text data" refers to text information extracted using image processing technology.

"Emotional data" refers to data that indicates a customer's emotional state.

"Emotion analysis technology" refers to technology that analyzes emotional data and evaluates a customer's emotional state.

"Unnecessary products" refer to products that are determined not to be necessary for the customer's health or savings goals.

"Amount saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight you would lose if you had not purchased unnecessary items.

"Displaying graphically" means displaying the calculation results in a visually easy-to-understand manner.

Form for implementing the invention

This invention is a system in which customers submit a purchase certificate after shopping and visualize the amount of savings and weight loss based on emotional data. A specific embodiment of this system is described below.

System Configuration

This system primarily uses the following hardware and software:

Mobile devices (smartphones and tablets)

Camera function

application

Server

Image processing software (e.g., OpenCV, Tesseract)

Emotion analysis technology

Program processing

After shopping, the user uses the camera function on their mobile device to take a picture of the purchase certificate. The image is then uploaded to the server via the application. The server then uses image processing software (e.g., OpenCV or Tesseract) to extract text from the image and convert information such as the name, price, and quantity of the purchased item into text data. The server then stores this text data in a database.

Next, the user enters their emotional data through the application. The emotional data may be entered manually by the user or automatically obtained from the wearable device. The server then analyzes the emotional data using emotion analysis technology to assess whether the user is feeling stressed and to what extent.

Based on the results of the emotion analysis, the server calculates the amount of money saved and the amount of weight lost if the user refrains from buying unnecessary items when feeling stressed. For example, if the user refrains from buying chocolate when feeling stressed, the calculation is based on the price (200 yen) and calories (300 kcal). The calculation results are displayed graphically by the server, and the user can check them through the application.

Specific examples

As a concrete example, consider the case where a user decides not to buy chocolate when they are feeling stressed. The user takes a photo of the receipt with their smartphone camera and uploads it to the application. The server uses image processing software to convert the contents of the receipt into text data and stores it in a database. Next, the user enters emotional data into the application, which the server analyzes using emotion analysis technology. Based on the analysis results, the server calculates the amount of savings and weight loss based on the price of the chocolate (200 yen) and calories (300 kcal). Finally, the server displays the calculation results graphically, and the user can check them through the application.

Example prompt

Example of a prompt to input to a generative AI model:

"Generate a program that calculates and graphically displays the amount of money saved and weight loss a user would achieve if they avoided buying unnecessary items when they were feeling stressed."

In this way, users can visually understand the relationship between their emotions, health management, and saving money.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user takes a photo of the purchase certificate using their smartphone camera.

Enter: Proof of purchase

Output: Image of proof of purchase

Specific operation: The user activates the smartphone camera, frames the purchase certificate, and presses the capture button. The captured image is saved to the smartphone's gallery.

Step 2:

Upload the image of the purchase certificate taken by the device to the application.

Input: Image of proof of purchase

Output: Image data sent to the server

Specific operation: The user launches the application, selects an image of the purchase certificate, and presses the upload button. The device then sends the selected image to the server.

Step 3:

The server uses image processing technology to convert the contents of the purchase certificate into text data.

Input: Image data of purchase certificate

Output: Text data

Specific operation: The server analyzes the received image data using image processing software (e.g., OpenCV, Tesseract) and extracts text information from the image. The extracted text information is converted into text data such as the name, price, and quantity of the purchased item.

Step 4:

The server stores the converted text data in a database.

Input: Text data

Output: Purchase information stored in the database

Specific operation: The server saves the converted text data in a database. The saved data includes the purchase date and time, store name, product information, etc.

Step 5:

The user inputs emotional data into the application.

Input: Emotion data

Output: Emotion data sent to the server

Specific operation: The user inputs their emotional state through the application. Emotional data includes information such as stress level and mood. It may also be obtained automatically from the wearable device.

Step 6:

The server analyzes the user's emotional data using emotion analysis technology.

Input: Emotion data

Output: Analysis results

Specific operation: The server uses emotion analysis technology to analyze emotional data and evaluate whether the user is feeling stressed and to what extent. The analysis results are displayed as numbers and graphs.

Step 7:

The server determines which items are unnecessary based on the analysis results and calculates the amount of money saved and the weight lost.

Input: Analysis results, text data

Output: Savings and weight loss data

Specific operation: Based on the results of emotion analysis and purchase information, the server calculates the amount of money saved and the amount of weight lost if the user does not buy unnecessary items when feeling stressed. For example, the calculation is based on the price (200 yen) and calories (300 kcal) of chocolate.

Step 8:

The server displays the calculation results graphically.

Input: Savings and weight loss data

Output: Graphical display

Specific operation: The server generates data for graphically displaying the calculation results and provides it to the user through the application. The user can visually check the amount of savings and weight loss through the application.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional shopping support systems required customers to physically submit receipts for purchased items, which was a time-consuming process. Furthermore, the shopping support did not take into account the customer's emotions or health status, leading to unnecessary purchases when they were feeling stressed. This led to wasteful spending and negative health effects, which was an issue.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: a means for the customer to enter their current height and weight and goals; a means for the customer to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to the application; a means for correcting items that are determined to be unnecessary in this purchase; a means for visualizing the money saved and weight lost if the customer does not buy unnecessary items; and a means for analyzing the customer's emotions and graphically displaying the amount of money saved and weight lost if the customer does not buy unnecessary items when they are feeling stressed. This saves the customer the trouble of submitting physical receipts, and enables shopping support that takes their emotions and health state into consideration.

"Customer" refers to a consumer who purchases goods or services.

"Height and Weight" refers to the customer's physical height and weight.

"Goal" refers to the specific numerical value or state that the customer wants to achieve.

A "purchase receipt" is a proof of purchase issued when a customer purchases a product.

A "smartphone camera" is a photographic device built into a smartphone.

An "application" is a software program that runs on a smartphone or other device.

"Uploading" is the act of transferring data from a local device to a server or cloud.

"Correction" is the act of checking the content and correcting or deleting unnecessary parts.

"Unnecessary items" are items that customers have determined they do not need to purchase.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight lost" refers to the weight lost as a result of not purchasing unnecessary items.

"Visualization" is the act of visually displaying data or information.

"Analyzing emotions" is the process of assessing and understanding a customer's emotional state.

"Stress" refers to mental or physical tension or strain.

"Graphically displayed" means displaying information in a visual format such as a graph or chart.

This invention provides a system that allows customers to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to an application. It also includes a function that analyzes the customer's emotions and graphically displays the amount of money saved and the amount of weight lost if they avoid buying unnecessary items when they are feeling stressed.

System Configuration

1. Hardware:

Smartphone: Use the camera function to take a photo of your purchase receipt.

Server: Serves as the backend for the application, storing and processing data.

2. Software:

Application: Installed on a smartphone, it takes photos of receipts, uploads them, performs sentiment analysis, and displays them graphically.

OpenCV: A library for loading and preprocessing receipt images.

Tesseract OCR: Optical character recognition software for extracting text from receipt images.

EmotionRecognizer: Emotion recognition software for analyzing customer emotions.

Matplotlib: A library for graphically displaying savings and weight loss.

System Operation

1. Take a photo of your receipt and upload it:

Users take a photo of their purchase receipt using their smartphone camera.

The captured images are uploaded to the server via the application.

2. Text extraction:

The server preprocesses the image using OpenCV and extracts text from the receipt using Tesseract OCR.

3. Sentiment analysis:

The server analyzes the user's emotions using EmotionRecognizer.

When you're feeling stressed, calculate how much money you'll save and how much weight you'll lose if you don't buy unnecessary items.

4. Graphical display:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

The application will display this visually to the user.

Specific examples

For example, if a user avoids buying unnecessary items when feeling stressed, the application will graphically display a savings of 1,000 yen and a weight loss of 0.5 kg.

Example prompt

Create a program that calculates and graphically displays the savings and weight loss that a user could achieve by not purchasing unnecessary items when they are feeling stressed. Take a photo of the receipt with a smartphone camera and extract the text using Tesseract OCR. Analyze the user's emotions using EmotionRecognizer and display the results using Matplotlib.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

Users take a photo of their purchase receipt using their smartphone camera.

Input: Physical image of purchase receipt

Output: Receipt image saved on your smartphone

Specific operation: The user launches the smartphone's camera app and takes a photo of the purchase receipt. The captured image is saved to the smartphone's storage.

Step 2:

The device will upload the captured receipt image to the application.

Input: Receipt image saved on your smartphone

Output: Receipt image uploaded to the server

Specific operation: The user launches the application, selects the captured receipt image, and presses the upload button. The image is sent to the server via the Internet.

Step 3:

The server preprocesses the receipt image using OpenCV and extracts text using Tesseract OCR.

Input: Receipt image uploaded to the server

Output: Extracted text data

Specific operation: The server uses OpenCV to remove noise from the image and highlight the text. Then, it uses Tesseract OCR to extract the text from the image.

Step 4:

The server analyzes the user's emotions using EmotionRecognizer.

Input: User emotional data (e.g., the emotional state the user enters into the application)

Output: Analyzed emotional state

Specific operation: The server uses EmotionRecognizer to analyze the user's emotional data and determine whether they are under stress.

Step 5:

When you feel stressed, the server calculates the amount of money you would save and the amount of weight you would lose if you didn't buy unnecessary items.

Input: Analyzed emotional state, extracted text data

Output: Savings and weight loss data

Specific operation: If the emotional state is stressed, the server identifies unnecessary items from the extracted text data and calculates the amount of savings and weight loss that would result if the item were not purchased.

Step 6:

The server uses Matplotlib to generate data that graphically displays the savings and weight loss, and sends it to the application.

Input: Savings and weight loss data

Output: Image data for graphical display

Specific operation: The server uses Matplotlib to generate images that visually display the amount of savings and weight loss in the form of bar graphs, etc., and sends the image data to the application.

Step 7:

The terminal displays the graphical data received from the server to the user.

Input: Graphical image data sent from the server

Output: Graphical information displayed on a smartphone screen

Specific operation: The application displays the image data received from the server on the smartphone screen, allowing the user to visually check the amount of savings and weight loss.

(Example 3)

Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Today's consumers aim to balance health management with saving money, but they often end up buying unnecessary products in their daily shopping. These unnecessary products not only go against their health goals but also lead to wasteful spending. However, it is not easy for consumers to identify and avoid these unnecessary products. Therefore, there is a need for a system that automatically detects unnecessary products based on consumers' health goals and visualizes the savings and health impact.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for customers to input their current height and weight and goals, means for customers to upload an image of their purchase receipt after shopping, means for converting the receipt image into digital data using optical character recognition technology, means for analyzing product information from the digital data and detecting unnecessary products, means for calculating the total price of unnecessary products and estimated weight loss, and means for visualizing the calculation results in graphs and charts. This allows consumers to automatically detect unnecessary products based on their own health goals and specifically understand the amount of money they can save and the impact on their health.

"Customers" are consumers who use the system to manage their own health and save money.

"Height and weight" refers to the customer's basic physical data and is fundamental information for setting health goals.

"Goals" refer to the health and weight management goals that a customer wishes to achieve, such as carbohydrate restriction or calorie restriction.

"Receipt image" refers to photo data of the purchase details that a customer obtains after shopping.

"Optical character recognition technology" is a technology that converts text information in an image into digital data, and is also known as OCR (Optical Character Recognition).

"Digital data" means text information converted using optical character recognition technology.

"Product information" refers to data such as the product name, price, and nutritional information listed on the receipt.

"Unnecessary products" are products that are contrary to the customer's health goals and should be avoided.

"Total amount" is the total price of the unwanted items detected.

"Estimated weight loss" is the amount of weight loss predicted if the unnecessary product was not consumed.

"Graphs and charts" are diagrams that visually display calculation results.

This invention is a system that allows consumers to efficiently manage their own health and save money. Specific embodiments of this system are described below.

First, the user accesses the system's web page using a terminal. The user enters their height, weight, and health goals (e.g., carbohydrate restriction). This data is sent to the server and stored in a database.

Next, the user uploads an image of the receipt obtained after shopping from their device to the system. The uploaded receipt image is sent to the server, which then uses optical character recognition technology such as Google Cloud Vision API to convert the receipt image into digital data. This digital data includes the text information on the receipt.

The server analyzes the converted digital data and extracts product names and their nutritional information. For example, if the receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

The server then calculates the total cost of the unnecessary items, as well as the estimated weight loss that would occur if these items were not consumed. A standard calorie expenditure model is used to calculate weight loss. The results are visualized as graphs and charts using a data visualization library such as D3.js.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

As a concrete example, consider the case where a user purchases chocolate when feeling stressed. The emotion engine analyzes the user's emotional state and determines that stress is high. In this case, it detects chocolate as an unnecessary item and displays a graph of the amount of money saved and weight loss that would have been achieved if the user had not purchased the chocolate. This allows the user to recognize their tendency to purchase unhealthy foods when they are stressed and understand the resulting financial and health impacts.

Example prompt:

Detect the foods the user purchases when they are stressed, calculate the amount of money saved and the weight loss they would have lost if they had not purchased those foods, and display the results in a graph.

This system allows users to specifically understand both their own health management and savings. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

Users enter their height, weight, and goals.

Using a terminal, a user accesses the system's web page and enters their height (e.g., 170 cm), weight (e.g., 70 kg), and health goal (e.g., carbohydrate restriction) into the form. The entered data is sent to the server and stored in a database.

Input: height, weight, health goals

Output: User information stored in the database

Step 2:

The user uploads an image of the receipt.

Users use their device to upload images of recent purchase receipts to the system. The uploaded images are then sent to the server.

Input: Receipt image

Output: Receipt image saved on the server

Step 3:

The server performs OCR processing on the receipt image.

The server uses optical character recognition (OCR) technology to convert the receipt image into digital data. Specifically, it calls the Google Cloud Vision API to perform the OCR process.

Input: Receipt image

Output: Digital data (text information)

Step 4:

The server analyzes product information and detects unnecessary products.

The server analyzes the digital data obtained through OCR processing and extracts product names and their nutritional information. For example, if a receipt includes "bread" or "sweets," these items are listed as unnecessary because they go against the goal of carbohydrate restriction.

Input: Digital data (text information), user's health goals

Output: List of unwanted items

Step 5:

The server calculates the total price of unwanted items and the estimated weight loss.

The server calculates the total value of the unwanted products detected, as well as the estimated weight loss that would occur if the user did not consume these products. A standard calorie expenditure model is used to calculate weight loss.

Input: List of unwanted items, product price, calorie information

Output: Total amount, estimated weight loss

Step 6:

The server visualizes the calculation results in graphs and charts.

The server uses a data visualization library such as D3.js to generate graphs and charts showing the total price of unwanted items and estimated weight loss.

Input: Total amount, estimated weight loss

Output: Graphs and charts

Step 7:

The user checks the results.

The user uses their device to view the graphs and charts sent from the server, allowing them to visually understand how much money they could save and how much weight they could lose by not buying unnecessary items.

Input: Graphs and charts

Output: User understanding and recognition

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Today's consumers are expected to manage their health and save money at the same time, but it can be difficult to determine which products are unnecessary when shopping. Furthermore, there is a lack of ways to grasp the specific amount of savings or weight loss that can be achieved by not purchasing unnecessary items. For this reason, there is a need for a system that allows consumers to make healthy choices and realize the savings they are making.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for customers to enter their current height and weight and goals, means for customers to submit purchase receipts after shopping, means for correcting items determined to be unnecessary for this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and the amount of weight lost, and means for displaying the detection results in a graph. This allows consumers to easily identify unnecessary items when shopping and specifically understand both health management and savings.

"Customer" refers to a consumer who uses the system to make purchases.

"Height and Weight" refers to the customer's current height and weight.

"Goals" refer to the specific health or diet objectives that the customer wants to achieve.

"Purchase receipt" refers to the paper or electronic data showing the purchase details of a product that a customer receives when making a purchase.

"Unnecessary items" refer to items that should be removed from the shopping list based on the customer's height, weight, and goals.

"Money saved" refers to the amount of money you can save by not purchasing unnecessary items.

"Weight loss" refers to the amount of weight loss predicted if the unnecessary product is not consumed.

"Visualization methods" refers to ways to display the money saved and weight lost in visual form, such as graphs or charts.

"Means of detection" refers to a method for identifying unwanted products based on the customer's height, weight, and goals.

"Means of calculation" refers to a method for calculating the amount of money saved and weight loss that would be achieved by not purchasing unnecessary items.

"Graphical display" refers to a method of displaying detection results in a visually easy-to-understand format.

A system for implementing this invention includes means for a customer to enter their current height and weight and goals, means for the customer to submit a purchase receipt after shopping, means for correcting items that are determined to be unnecessary in this purchase, means for visualizing the money saved and weight lost if unnecessary items are not purchased, means for detecting unnecessary items and calculating the amount saved and weight lost, and means for displaying the detection results in a graph.

System Configuration

The system is implemented as a smartphone application. The application is built using Python and Flask, and performs data processing on the server side. The smartphone sends the user's height, weight, and goals, as well as receipt information, to the server.

Data processing and calculation

The server receives the data sent by the user and performs the following processing:

1. Detecting unwanted products:

The server identifies unnecessary items on the receipt based on the user's height, weight, and goals. For example, if the user's goal is to "limit carbohydrates," it will detect "bread" and "sweets" on the receipt as unnecessary items.

2. Calculate savings and weight loss:

The server calculates the amount of money saved and the amount of weight lost if the unnecessary items were not purchased. For example, it adds up the prices of the unnecessary items and calculates the amount of weight lost based on the calories in those items.

3. Visualizing the results:

The server generates the calculation results in the form of graphs and charts and sends them to a smartphone application. Users can visually check the results through the application.

Hardware and software used

Hardware: Smartphone

Software: Python, Flask

Specific examples

If a user's goal is to "limit carbohydrates" and the receipt includes "bread" and "sweets," the system will detect these as unnecessary items. The savings will be calculated as 350 yen (bread 150 yen + sweets 200 yen), and the weight loss will be 0.064 kg (500 kcal / 7700 kcal). These results are displayed as a graph on the smartphone application.

Example prompt

If a user is aiming to limit carbohydrates, detect unnecessary items in their receipts and calculate savings and weight loss. Use the following data:

Height: 170cm
Weight: 70kg
Goal: Low-carb Receipt items: ["bread", "sweets", "vegetables"]

The flow of the specific processing in Application Example 3 is explained using Figure 22.

Step 1:

The user launches the smartphone application and enters their current height, weight, and goal.

Input: height, weight, goal

Output: Height, weight, and goal data

Specific action: The user enters their height, weight, and goal into the application's input form and presses the submit button.

Step 2:

After shopping, users scan their purchase receipt with their smartphone and upload it to the application.

Input: Purchase receipt image

Output: Receipt text data

Specific operation: The user takes a photo of the receipt using their smartphone camera and uploads it to the application. The application uses OCR technology to extract text data from the image.

Step 3:

The server receives the height, weight, goal, and receipt text data sent by the user.

Input: Height, weight, goal, and receipt text data

Output: Save received data

Specific operation: The server receives the HTTP request and stores the submitted data in the database.

Step 4:

The server detects unwanted items on the receipt based on the user's goals.

Input: Height, weight, goal, and receipt text data

Output: List of unwanted items

Specific operation: The server references the list of unwanted items related to the goal and matches them with the text data on the receipt to identify the unwanted items.

Step 5:

The server calculates the amount of savings and weight loss you would have achieved if you had not purchased unnecessary items.

Input: List of unwanted items

Output: Savings, weight loss

Specific operation: The server adds up the prices of the unwanted items and calculates the amount of weight loss based on the calories of the unwanted items. For example, if the total price of the unwanted items is 350 yen and the total calories are 500 kcal, the amount of weight loss will be calculated as 0.064 kg (500 kcal / 7700 kcal).

Step 6:

The server generates the calculation results in the form of graphs and charts and sends them to the smartphone application.

Input: Amount saved, weight loss

Output: Image data for graphs and charts

Specific operation: The server generates graphs and charts based on the calculation results and sends them as image data to the smartphone application.

Step 7:

Users can visually check the savings and weight loss results through a smartphone application.

Input: Image data for graphs and charts

Output: Visual results

What it does: The user opens the app and looks at the graphs and charts that are displayed, allowing them to understand the specific savings and weight loss they could have achieved by not purchasing unnecessary items.

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged above and below the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the top side of the concentric circles, and the emotion of "discomfort" is arranged on the bottom side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a calm impression.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, discomfort is felt, and when they approach the ideal, comfort is felt. Robots, cars, motorcycles, etc. also use various balances, such as posture and battery level, as their basis. When these balances deviate from the ideal, discomfort is felt, and when they approach the ideal, comfort is felt.

Emotions can be created in this way. An emotion map may be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and emotional brain physiological signal analysis systems, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map lists emotions that belong to an area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions that belong to an area called "situation," where situational awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may also be performed by multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The following types of processors can be used as hardware resources for executing specific processes. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically for executing specific processes. All processors have built-in or connected memory, and all use the memory to execute specific processes.

The hardware resource that executes the specific processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific processing may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiments.

(Claim 1)

The system includes a means for customers to enter their current height and weight and goals, a means for customers to submit purchase receipts after shopping, a means for customers to correct any items that are determined to be unnecessary in this purchase, and a means for customers to visualize the money they could save and the weight they would lose if they did not buy the unnecessary items.

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

(Claim 4)
The system of claim 1 , wherein the system further comprises an emotion engine that recognizes an emotion of a user.

(Claim 5)
The system of claim 4 , wherein the emotion engine determines unwanted products based on the user's emotions.

(Claim 6)
The system of claim 4 , wherein the emotion engine graphically displays money saved and weight lost based on the user's emotions.

"Example 1"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
means for receiving and storing data entered by the customer;
A means for generating a prompt sentence for the generative AI model based on the stored data;
A system including a means for a generative AI model to suggest a diet plan.

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Application Example 1"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
A means for generating an appropriate meal plan based on the customer's height, weight, and goals;
a means for presenting the generated meal plan to the customer;
A system including:

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Example 2"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to take an image of the purchase and upload it to the application to provide a receipt for the purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A system that includes a means to visualize the amount of money saved and the weight lost by not buying unnecessary items.

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the amount of money saved and the weight lost.

"Application Example 2"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
After shopping, customers can take a photo of their purchase receipt with their smartphone camera and upload it to the application.
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
means for analyzing the contents of a purchase receipt and automatically extracting purchase information;
A means for redeeming points to customers based on the extracted purchase information;
A system that includes a means to use earned points for your next purchase.

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Example 3"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A means for converting the submitted receipt information into text data using optical character recognition technology;
A means to determine unnecessary products based on the customer's height, weight, and goals;
A way to calculate how much money you could save and how much weight you could lose if you didn't buy unnecessary items,
A system including means for graphically displaying the results of the calculations.

(Claim 2)
2. The system according to claim 1, wherein the means for converting into text data using optical character recognition technology analyzes a receipt image to extract the product name and price.

(Claim 3)
The system of claim 1 , wherein the means for determining unwanted items performs rule-based filtering based on predefined food categories and goals.

"Application Example 3"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
A means for analyzing receipt information and automatically detecting unnecessary products based on diet goals;
A way to calculate the amount of money you can save and the estimated amount of weight you can lose by not buying unnecessary items, and display them in graphs and charts.
A system that includes applications that are installed on smartphones.

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Example 1: Combining Emotion Engines"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
A means of recognizing customer emotions and determining unnecessary products based on those emotions;
A means of recommending healthy products based on customer sentiment;
A system including:

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Application example 1: Combining emotion engines"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
A means of recognizing customer emotions and determining unnecessary products based on those emotions;
A system including:

(Claim 2)
The system according to claim 1 , wherein the correction means determines unnecessary products based on the customer's height, weight, goals, and emotions.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Example 2: Combining Emotion Engines"

(Claim 1)
a means for the customer to enter their current physical information and goals;
A means for customers to submit proof of purchase after making a purchase;
A means for the customer to take an image of the purchase certificate using the photographing function of the mobile device and upload it to the application;
A means for the server to convert the contents of the purchase certificate into text data using image processing technology;
A means for the server to store the converted text data;
a means for the customer to input emotion data;
A means for the server to analyze customer emotion data using emotion analysis technology;
A means for the server to determine unnecessary products based on the analysis results and calculate the amount of money saved and the weight lost;
The system includes a means for the server to graphically display the results of the calculations.

(Claim 2)
The system of claim 1 , wherein the determining means determines unnecessary products based on physical information and goals of the customer.

(Claim 3)
10. The system of claim 1, wherein said displaying means graphically displays the amount of money saved and the weight lost.

"Application Example 2: Combining Emotion Engines"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
A means for customers to take a photo of their purchase receipt using their smartphone camera after shopping and upload it to the application;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
The system includes a means for analyzing customer emotions and graphically displaying savings and weight loss if unnecessary items are not purchased when the customer is under stress.

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

"Example 3: Combining Emotion Engines"

(Claim 1)
a means for the customer to input their current height and weight and goals;
A means for customers to upload an image of their purchase receipt after making a purchase;
A means for converting receipt images into digital data using optical character recognition technology;
A means for analyzing product information from digital data and detecting unnecessary products;
a means for calculating the total value of unwanted items and the estimated weight loss;
A system that includes a means for visualizing the results of calculations in graphs and charts.

(Claim 2)
The system of claim 1 , wherein the means for detecting unwanted products determines unwanted products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the means for visualizing the calculation results graphically displays the money saved and weight lost.

"Application Example 3: Combining Emotion Engines"

(Claim 1)
A means for the customer to enter their current height and weight and goals;
a means for the customer to submit a purchase receipt after making a purchase;
A way to correct items that you think are unnecessary for this shopping trip,
A way to visualize the money you can save and the weight you can lose if you don't buy unnecessary products,
A means for detecting unnecessary items and calculating savings and weight loss;
a means for displaying the detection results in a graph;
A system including:

(Claim 2)
The system according to claim 1 , wherein the correcting means determines unnecessary products based on the customer's height, weight, and goals.

(Claim 3)
The system of claim 1 , wherein the visualization means graphically displays the money saved and weight lost.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (1)

A means for the customer to enter their current height, weight and diet goals;
a means for the customer to submit a purchase receipt after making a purchase;
means for analyzing the emotional state of a customer using an emotion engine;
A means for identifying and correcting products that are determined to be unnecessary for the current purchase based on the customer's height, weight, and diet goals entered by the customer and the customer's emotional state analyzed using an emotion engine, by referring to the information on the submitted purchase receipt;
A way to visualize the amount of money you could save by not buying unnecessary products and the estimated weight loss you would experience if you didn't consume those products.
means for generating a prompt sentence instructing the customer to propose a diet plan based on the customer's current height and weight and diet goal;
A means for suggesting a diet plan using the generated prompt sentence and the generative AI model;
means for transmitting the proposed diet plan and the visualized savings and estimated weight loss information to the customer's terminal;
The correction means determines unnecessary products by inputting a prompt sentence including the height and weight of the customer , a diet goal, and an analysis result of the emotional state of the customer into a generation AI model.
system.