JP7571970B2 - Cooking assistance system, cooking assistance method, and cooking assistance program - Google Patents

Description

The present invention relates to a cooking assistance system, a cooking assistance method, and a cooking assistance program that assist in deep frying using a fryer.

Labor shortages have become a serious issue in the food and beverage industry. There is known technology that automates cooking in establishments such as restaurants and convenience stores, allowing businesses to maintain the quality of cooked food even in situations where there is a labor shortage.

Patent Document 1 discloses a technology that enhances the convenience of cooking using a cooking appliance in which ingredients are added or removed during cooking by using a camera to detect the addition or removal of ingredients, and automatically adjusting the heating time according to the type of ingredients added and the number of ingredients added or removed.

Patent Document 2 also discloses a technology that uses image processing of images captured under irradiation with ultraviolet light, purple, and blue visible light to identify foreign objects in food or cooked foods, peeling coating in fried foods, or frozen areas.

JP 2020-103284 A JP 2020-159581 A

However, although the technology described in Patent Document 1 can detect the insertion and removal of food, it cannot grasp the actual state of the food being cooked, and there is room for improvement in terms of improving the quality of the food through more optimal cooking.

In addition, although the technology described in Patent Document 2 can detect fried foods that have failed to be cooked, there is room for improvement in terms of preventing cooking failures in the actual cooking process.

The present invention was made in consideration of the above-mentioned circumstances, and aims to solve the problem of estimating the actual state of the food being cooked and supporting deep frying using a fryer.

To solve the above-mentioned problems, the present invention provides a cooking assistance system that assists in deep frying using a fryer, and includes an image data processing means for acquiring image data captured by a camera of the cooking process using the fryer, an identification means for identifying the type of food contained in the image data, an estimation means for estimating the cooking progress of the food using a machine-learned model that is trained using the image data of the food whose type has been identified as input, and an assistance means for outputting cooking assistance instructions to the fryer based on the estimated cooking progress.

This configuration makes it possible to estimate the cooking progress based on image data of the actual food being cooked, and provide appropriate cooking assistance.

In a preferred form of the present invention, the identification means identifies element values for the identified type of cooked food, and one or more element values selected from the color of the batter, the position in the cooking oil, and the shape of the bubbles, and the estimation means estimates the cooking progress of the cooked food using a trained model that has been machine-learned using each of the element values of the identified type of cooked food as input.
With this configuration, the progress of cooking can be estimated with greater accuracy.

In a preferred form of the present invention, the image data includes temperature image data, the identification means identifies the temperature of the food item and the temperature of the cooking oil as element values, and the estimation means inputs the temperature of the food item whose type has been identified and the temperature of the cooking oil, and estimates the cooking progress of the food item using a machine-learned trained model.
With this configuration, the progress of cooking can be estimated more accurately based on the actual temperature of the food being cooked.

In a preferred embodiment of the present invention, the method further includes: a sound data processing means for acquiring sound data recorded by a microphone of the cooking process using the fryer; wherein the identification means identifies at least the popping sounds of bubbles generated in the cooking oil as an element value; and the estimation means estimates the cooking progress of the food using a trained model trained by machine learning using the popping sounds of the food whose type has been identified as an input.
This configuration allows for more accurate estimation of cooking progress based on actual cooking sounds. In addition, the use of sound data can reduce the amount of data required for machine learning to achieve a certain level of accuracy for the trained model.

In a preferred embodiment of the present invention, when the cooking progress is determined to be above a certain level, the assistance means identifies the element value used to estimate the cooking progress, and outputs a cooking assistance instruction to notify the fryer of the element value.
With this configuration, it is possible to provide the chef with feedback as to which element of the actual food being cooked is responsible for estimating the cooking progress.

In a preferred embodiment of the present invention, the cooking progress of the food item is weighted according to each of the input element values, and the estimation means calculates an overall cooking progress based on each cooking progress and weight.
With this configuration, the progress of cooking can be estimated comprehensively from data acquired from the actual food being cooked.

In a preferred form of the invention, the cooking progress includes an estimated time for cooking to be completed.
With this configuration, the cook can be notified of the time when cooking will be completed.

In a preferred embodiment of the present invention, the assistance means determines the estimated time and outputs a cooking assistance instruction to the fryer to remove the food.
With this configuration, a command to remove the food item is automatically output when cooking is complete, and cooking can be completed.

In a preferred embodiment of the invention, the assistance means determines a first time before the estimated time and outputs a cooking assistance instruction to notify the fryer to prompt the user to remove the food.
With this configuration, the cook can be notified in advance that cooking is complete.

In a preferred form of the present invention, the identification means identifies whether or not a chef is present in the image data, and if it is identified that no chef is present, the assistance means determines a second time which is before the first time, and outputs a cooking assistance instruction to notify the fryer to prompt the user to remove the cooked food.
With this configuration, if the cook is not near the fryer, the cook can be notified that cooking will be completed sooner.

The present invention is a cooking assistance method for assisting deep frying using a fryer, in which a computer executes the following steps: an image data processing step for acquiring image data captured by a camera showing the cooking process using the fryer; a specification step for identifying the type of food contained in the image data; an estimation step for estimating the cooking progress of the food using a machine-learned model that uses the image data of the food whose type has been identified as input; and a support step for outputting cooking assistance instructions to the fryer based on the estimated cooking progress.

The present invention is a cooking assistance program that assists in deep frying using a fryer, and causes a computer to function as an image data processing means for acquiring image data captured by a camera of the cooking process using the fryer, an identification means for identifying the type of food contained in the image data, an estimation means for estimating the cooking progress of the food using a trained model that is machine-learned using the image data of the food whose type has been identified as input, and an assistance means for outputting cooking assistance instructions to the fryer based on the estimated cooking progress.

The present invention provides a cooking assistance system, cooking assistance method, and cooking assistance program that assists deep-frying using a fryer and contributes to automating the cooking process, shortening cooking time, and homogenizing cooked foods.

1 shows a system configuration diagram of a cooking assistance system according to an embodiment of the present invention. FIG. 1 shows a configuration diagram of a fryer according to an embodiment of the present invention. 1 shows a hardware configuration diagram of a cooking assistance server according to an embodiment of the present invention. 4 illustrates an example of a data structure of training data in an embodiment of the present invention. 13 illustrates an example of a data structure of result data in the embodiment of the present invention. 1 shows a flowchart of a learning process in an embodiment of the present invention. 1 shows a flowchart of an estimation process using a trained model in an embodiment of the present invention.

Below, a cooking assistance system according to an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment shown below is an example of the present invention, and the present invention is not limited to the following embodiment, and various configurations can be adopted.

In this embodiment, the configuration, operation, etc. of a cooking assistance system including a fryer and a cooking assistance server are described, but a similarly configured method, a computer program, and a program recording medium on which the program is recorded also provide similar effects. By using a program recording medium, for example, the program can be installed on a computer. The series of processes according to this embodiment described below is provided as a computer-executable program, and can be provided via a non-transitory computer-readable recording medium such as a CD-ROM or a flexible disk, or even via a communication line.

The means of the cooking assistance system and the steps of the cooking assistance method achieve the same effects. The means in the cooking assistance system, cooking assistance program, and cooking assistance program recording medium are each realized by a calculation device such as a CPU. Similarly, the steps of the cooking assistance method are also realized by a calculation device.

Figure 1 shows a functional block diagram of the cooking assistance system according to this embodiment. In Figure 1, the cooking assistance system is embodied as cooking assistance system 1. Cooking assistance system 1 includes a fryer 2 used for deep-frying food, and a cooking assistance server 3 that acquires data on the cooking status from the fryer 2 and processes information related to cooking assistance. The fryer 2 and cooking assistance server 3 are configured to be able to communicate via a communication network NW. The communication network NW is composed of an IP (Internet Protocol) network such as the Internet, a dedicated line, or the like. In the following explanation, the inclusion of the communication network NW will be omitted unless it is unclear.

In this embodiment, the cooking assistance system 1 is installed in each store where cooking is performed, such as a restaurant or convenience store. The cooking assistance server 3 may be installed by the operator of the cooking assistance system 1 and configured to communicate data with the fryers 2 installed in each store. Furthermore, multiple fryers 2 may be installed in one store.

The cooking assistance server 3 is equipped with an internal or external memory unit 4 that stores various data acquired by the fryer 2, and is configured to be capable of communication. It is equipped with the memory unit 4, a teacher data memory unit 41 that stores teacher data, a model memory unit 42 that stores a trained model generated by machine learning using the teacher data, a judgment target data memory unit 43 that stores image data and sound data acquired from the fryer 2, and a result data memory unit 44 that stores the judgment results of the judgment target data.

2 shows a configuration diagram of the fryer 2. The fryer 2 includes an oil tank 21 for storing cooking oil, a temperature control unit 22 for heating the cooking oil in the oil tank 21, an imaging module 23 for photographing the state of the food being fried, a microphone 24 for collecting cooking sounds during frying, a frying basket 25 for putting the food F in, a lift control unit 26 for controlling the lifting and lowering operation of the frying basket 25, a display 27 for displaying information on cooking assistance and functioning as an operation panel for the fryer 2, a control unit 28 such as a CPU for executing operation control for each component, a communication unit 29 for communicating with the cooking assistance server 3, a display unit including a liquid crystal display or the like for visually notifying in accordance with the operation control for each component, the general information acquired, and the result of the determination of information on cooking assistance, and an audio alert unit including a speaker or the like for visually notifying in accordance with the operation control for each component, the general information acquired, and the result of the determination of information on cooking assistance. Here, the control unit 28 and the communication unit 29 appropriately cooperate with an AI function realized by the estimation means 304 or the like. In addition, the "AI function" here may take the form of edge AI or cloud AI.

The imaging module 23 includes a visible light camera 231 having an image sensor with light receiving sensitivity in the visible light spectrum, and capturing visible light image data showing a visible light image of the target cooking item and its cooking status, and a thermal camera 232 having a bolometer that detects infrared energy emitted by the cooking item or cooking oil being imaged, and capturing temperature image data showing a temperature distribution obtained by converting the detected amount of infrared energy into temperature. In this embodiment, the optical axes of the visible light camera 231 and thermal camera 232 mounted on the imaging module 23 are configured to be approximately parallel. This makes it easy to overlay the image data captured by each camera, since they include a common image area.

There are no particular limitations on the placement of the imaging module 23, so long as it is positioned so that the entire oil vat 21 can be included in the imaging range. It is preferable to place the imaging module 23 so that the chef who uses the fryer 2 to cook the food is included in the imaging range. It may also be placed so that multiple fryers 2 are included in the imaging range.

The imaging module 23 acquires image data capturing the cooking process. Each camera may be configured as a video camera that acquires video data, and may acquire scenes at predetermined time intervals as image data. The acquired image data is transmitted to the cooking assistance server 3 via the communication unit 29. Each imaging module 23 has unique ID information, and the ID information is associated with the transmitted image data.

The microphone 24 acquires sound data including the popping sounds of bubbles in the cooking oil during cooking. The microphone 24 may be provided integrally with the imaging module 23. The acquired sound data is associated with ID information unique to the microphone 24 and is transmitted to the cooking assistance server 3 via the communication unit 29.

Figure 3 shows the hardware configuration of the cooking assistance server 3. The cooking assistance server 3 includes, as hardware components, a calculation unit (CPU) 31, a main memory device (RAM) 32 as a working memory, an auxiliary memory device 33 such as a HDD, SSD, or flash memory, a communication unit 34 for communicating with external devices, and a bus 35 connecting each of the components. The auxiliary memory device 33 also stores an operating system (OS) 36, a cooking assistance program 37 that works with the OS 36 to perform its functions, and various information (including data).

The cooking assistance server 3 includes, as functional components, an image data processing means 301 that acquires and processes image data captured by the imaging module 23, an audio data processing means 302 that acquires and processes audio data collected by the microphone 24, an identification means 303 that identifies element values for determining the type of food to be evaluated and the cooking progress, an estimation means 304 that inputs the data to be evaluated into a trained model and estimates the cooking progress, an assistance means 305 that outputs cooking assistance instructions, and a learning means 306 that generates a trained model by machine learning using teacher data.

In this embodiment, the data stored in the cooking assistance server 3 will be described. The cooking assistance server 3 includes a teacher data storage unit 41 that stores teacher data for performing image recognition processing and sound recognition processing.

Figure 4 shows an example of the structure of teacher data stored in the teacher data storage unit 41. The structure of the teacher data has, for example, a label attached to the image data and sound data as a key, and includes ID information of the imaging module 23 or microphone 24 from which the data was acquired, the data type, features of the image data and sound data, a file name, etc. The ID information of the imaging module 23 and microphone 24 in the common fryer 2 is stored in the respective storage units 4 in a corresponding relationship. Note that the ID information is used, for example, to modify the teacher data to be suitable for each store in which the fryer 2 is installed, but does not need to be included in the structure of the teacher data in order to standardize the accuracy of the image recognition process for each store. It is preferable that the image data and sound data used as teacher data are acquired for individual cooking items.

The features are stored as a data set such as the brightness and color of each pixel obtained from image data that has been subjected to image recognition processing by the learning means 306. The learning means 306 automatically obtains the features contained in the image data, for example, by a machine learning technique using deep learning, but the machine learning technique is not limited to this and conventional techniques can be adopted. Deep learning does not require the user to set the features, and the computer can automatically extract the features of objects in image data or sound data and classify and identify the objects.

In this embodiment, the label has multiple element values according to the data type. The element values of the label in the visible light image data include the type of food, the color of the batter, whether the food is on the surface or in the oil, the size and shape of the bubbles around the food, whether a cook is present, and the timing of adding the food to the cooking oil. The element values of the label in the temperature image data include the type of food, the temperature of the food, and the temperature of the cooking oil. The element values of the label in the sound data include the type of food, the popping sound of the bubbles when the food is deep-frying, and the like. The popping sound may be assigned with element values such as high and low pitch and loudness. Popping sounds are generally high-pitched and loud at the beginning of deep-frying. In addition, it is not necessary for all element values to be input in the labels of the training data, and some element values may be missing.

In this embodiment, the learning means 306 performs machine learning using image data and sound data as training data to generate a specific model that identifies the element values of the target cooking ingredient, and stores the model in the model storage unit 42.

The learning means 306 also performs machine learning using element values identified in the image data and sound data serving as training data to generate an estimation model for estimating the cooking progress of the target food, and stores the model in the model storage unit 42. In this embodiment, the cooking progress includes, for example, the estimated time required to complete cooking or the time when cooking will be completed, and the current value when completion is set to a value of 10.

The image data processing means 301 acquires image data (visible light image, temperature image) from the imaging module 23 and stores it in the judgment target data storage unit 43 as judgment target data. The sound data processing means 302 acquires sound data from the microphone 24 and stores it in the judgment target data storage unit 43 as judgment target data. The judgment target data storage unit 43 uses the ID information of the imaging module 23 or microphone 24 from which the data was acquired as a key and stores the judgment target data including the target data type, file name, acquisition date and time of the data, etc.

Figure 5 shows an example of the data structure stored in the result data storage unit 44. The result data storage unit 44 stores result data determined by inputting the data stored in the determination target data storage unit 43 into the specific model and the estimation model stored in the model storage unit 42.

The result data storage unit 44 uses the ID information as a key to store the data type used in the judgment, the file name, the judgment result judged using the specific model, and the cooking status and estimated time further judged by inputting the judgment result into the estimation model as result data. Here, the judgment result corresponds to each element value of the label in the training data. The identification means 303 receives the judgment target data as input, identifies each element value using a specific model trained by machine learning, and stores them as the judgment result.

In this embodiment, each element value of the label is assigned a weight. The estimation means 304 outputs an estimate of the cooking progress based on the element values stored in the determination result and their weights. Furthermore, the cooking progress is further weighted for each data type, and the estimation means 304 outputs an estimate of the overall cooking progress based on the cooking progress for each data type and its weight. Furthermore, the estimation means 304 outputs an estimated time for a desirable cooking progress based on the cooking progress.

The various storage units and means described above can be configured to be included in the fryer 2, eliminating the need for communication with the cooking assistance server 3.

Figure 6 shows a flowchart of the machine learning process in this embodiment. In this embodiment, an example in which the machine learning process is performed in the cooking assistance server 3 is described, but the machine learning process may also be performed in the fryer 2. In addition, the machine learning process may be performed by a server or terminal device outside the cooking assistance system 1, and a trained model may be provided to the cooking assistance system 1.

First, the cooking assistance server 3 acquires visible light image data, temperature image data, and sound data from the fryer 2 as training data, assigns labels to them, and stores them in the training data storage unit 41 (step S61). The learning means 306 extracts features contained in the image data and sound data by machine learning using the labeled training data (step S62). The learning means 306 updates the features by repeating the machine learning processes of S61 and S62 (step S63). As a result, a specific model and an estimated model are generated, and are stored in the model storage unit 42, respectively (step S64).

Next, the processing procedure for the data to be judged will be described. FIG. 7 shows a processing flowchart for the data to be judged in this embodiment. Note that steps S82 to S84, steps S85 to S86, and step S87 in FIG. 7 may be performed in parallel, and there is no restriction on the order in which they are performed.

First, the image data processing means 301 acquires image data of the cooking process using the fryer 2 from the imaging module 23 as data to be determined. The sound data processing means 302 acquires sound data from the microphone 24 as data to be determined. The acquired data to be determined is stored in the data to be determined storage unit 43 (step S81).

The identification means 303 inputs the visible light image data stored in the determination target data storage unit 43 into a specific model to identify the type of food (step S82). The identification means 303 also identifies a specific image area that includes the individual food whose type has been identified (step S83). The image area has data related to coordinates and size. The identification means 303 uses the specific model to identify each element value (color of batter, position in cooking oil, shape of bubbles, etc.) for each type of food included in the image area (step S84).

The identification means 303 identifies the image area of the temperature image data based on the image area of the visible light image data (step S85). The identification means 303 identifies the temperature of the food and the temperature of the cooking oil included in the image area of the temperature image data as element values using a specific model (step S86). At this time, the identification means 303 may be configured to determine the specific model for inputting the temperature image data according to the type of food identified based on the visible light image data.

The identification means 303 inputs the sound data stored in the judgment target data storage unit 43 into a specific model, thereby identifying at least the popping sound of bubbles occurring in the cooking oil as an element value (step S87). At this time, the identification means 303 may be configured to determine the specific model into which the sound data is input according to the type of cooking food previously identified based on the visible light image data.

The estimation means 304 receives as input each of the element values identified by the identification means 303, estimates result data including the cooking progress and estimated time of the food using a machine-learned estimation model, and stores the result data in the result data storage unit 44 (step S88). In this embodiment, the result data is stored according to the data type. The estimation means 304 can estimate the overall cooking progress and estimated time by calculating and processing the cooking progress and estimated time estimated according to the data type (visible light image data, temperature image data, sound data) based on their respective weights.

The support means 305 outputs cooking support instructions to the fryer 2 for which the determination target data was acquired in step S81 based on the cooking progress status included in the result data (step S89), and completes the process. Here, the support means 305 may output the cooking support instructions using at least one notification from the display unit 38 and the audio alert unit 39.

The cooking assistance instructions include a notification instruction and a lift control instruction. The notification instruction is an instruction to notify the cook of the cooking progress. Here, the cooking assistance instruction may be output via at least one of the display unit 38 and the audio alert unit 39, or may be output via a terminal capable of communicating with the cooking assistance system via a network. The fryer 2 can display the cooking progress and the estimated time on the display 27 based on the result data received as the notification instruction. Specifically, the support means 305 determines a first time before the estimated time and outputs a notification instruction notifying the cook that the cooking is complete. For example, by receiving the notification instruction, the fryer 2 can notify the cook that the cooking is complete by outputting an alert sound or an estimated time. In addition, when the identification means 303 identifies that there is no cook in the image data, the support means 305 determines a second time before the first time and outputs a notification instruction notifying the cook that the cooking is complete. In addition, when the cooking assistance instructions are output visually, for example via the display unit 38, the cooking assistance instructions for each of the one or more cooking ingredients may be displayed by being superimposed on the visible light image data, for example.

The notification instruction also includes an instruction to identify the element value used to estimate the cooking progress and to notify the identified element value. When the support means 305 determines that the cooking progress is above a certain level, it outputs a notification instruction to the fryer 2 to identify the element value and notify the identified element value. This makes it possible to notify via the display 27 which element value is used to determine the progress of cooking, and allows appropriate training to be provided to employees/cooks who are unfamiliar with cooking tasks.

The lift control instruction is a command to the fryer 2 to remove the cooked food depending on the cooking progress. When the fryer 2 receives the lift control instruction, it drives the lift control unit 26 to raise the frying basket 25 from the oil tank 21. The support means 305 outputs a lift control instruction, for example, when the estimated time becomes 0. Here, the lift control instruction may be configured not as an instruction to the cook, but as a control command to a device that can control the lifting of the lift. With such a configuration, the present invention can also contribute to the realization of unmanned operations related to the cooking of fried foods.

In one embodiment of the present invention, when the food to be cooked is fried chicken, the cooking assistance instructions include, for example:
(A): A state in which deep-frying is nearing completion and it is recommended to quickly remove the food from the cooking oil (B): A state in which deep-frying is progressing and it is recommended to turn the food in the cooking oil over (C): A state in which deep-frying has just started and it is recommended to keep an eye on the food in the cooking oil and continue the deep-frying Here, in each of the above (A), (B), and (C), the visual output may be in the form of superimposing a corresponding predetermined display object on the corresponding food. Also, here, in the case of a predetermined cooking support instruction (for example, the above (A)), the above display object may be made to blink, etc., so as to be distinguished from other display objects in appearance.

REFERENCE SIGNS LIST 1 Cooking assistance system 2 Fryer 21 Oil tank 22 Temperature control unit 23 Imaging module 231 Visible camera 232 Thermal camera 24 Microphone 25 Frying basket 26 Lift control unit 27 Display 28 Control unit 29 Communication unit 3 Cooking assistance server 31 Arithmetic unit (CPU)
32 Main memory (RAM)
33 Auxiliary storage device 34 Communication device 35 Communication bus 36 Operating system (OS)
37 Cooking assistance program 301 Image data processing means 302 Sound data processing means 303 Identification means 304 Estimation means 305 Assistance means 306 Learning means 4 Storage unit 41 Teacher data storage unit 42 Model storage unit 43 Judgment target data storage unit 44 Result data storage unit NW Communication network

Claims (12)

A cooking assistance system that assists in deep frying using a fryer,
An image data processing means for acquiring image data of the cooking state by the fryer captured by a camera;
A means for identifying a type of food contained in the image data;
An estimation means for estimating a cooking progress of the food using an input of image data of the food whose type has been identified and a machine-learned model;
A cooking assistance system comprising: an assistance means for outputting cooking assistance instructions to the fryer based on the estimated cooking progress. The identification means identifies one or more element values selected from a color of a batter, a position in the cooking oil, and a shape of a bubble, which are element values for each type of the food contained in the image data;
The cooking assistance system according to claim 1 , wherein the estimation means inputs each of the element values of the food item whose type has been identified, and estimates the cooking progress of the food item using a machine-learned trained model. the image data includes temperature image data;
The identification means identifies the temperature of the food and the temperature of the cooking oil included in the temperature image data as element values;
The cooking assistance system according to claim 2 , wherein the estimation means inputs the temperature of the food whose type has been identified and the temperature of the cooking oil, and estimates the cooking progress of the food using a machine-learned model. and a sound data processing means for acquiring sound data recorded by a microphone of the cooking state by the fryer,
The identification means identifies at least a popping sound of bubbles occurring in cooking oil included in the sound data as an element value,
The cooking assistance system according to any one of claims 1 to 3, wherein the estimation means inputs the popping sound of the food whose type has been identified, and estimates the cooking progress of the food using a machine-learned trained model.
The cooking assistance system according to any one of claims 2 to 4, wherein the assistance means, when it is determined that the cooking progress is equal to or greater than a certain level, identifies the element value used to estimate the cooking progress and outputs a cooking assistance instruction to notify the fryer of the element value. The cooking progress of the food is weighted according to each of the input element values,
6. The cooking assistance system according to claim 2, wherein the estimation means calculates an overall cooking progress status based on each cooking progress status and weight. The cooking assistance system according to any one of claims 1 to 6, wherein the cooking progress includes an estimated time for cooking to be completed. The cooking assistance system according to claim 7, wherein the assistance means determines the estimated time and outputs a cooking assistance instruction to the fryer to remove the food. The cooking assistance system according to claim 7 or 8, wherein the assistance means determines a first time before the estimated time and outputs a cooking assistance instruction to notify the fryer to prompt the user to remove the food. The identification means identifies whether or not a chef is included in the image data,
The cooking support system of claim 9, wherein when it is determined that the cook is not present, the support means determines a second time that is even earlier than the first time, and outputs a cooking support instruction to notify the fryer to prompt the cook to remove the cooked food. A cooking assistance method for assisting deep-frying using a fryer, comprising:
An image data processing step of acquiring image data of the cooking state by the fryer captured by a camera;
A step of identifying a type of food included in the image data;
an estimation step of estimating a cooking progress of the food using a trained model that is machine-learned using image data of the food whose type has been identified;
A cooking assistance method executed by a computer, comprising: an assistance step of outputting cooking assistance instructions to the fryer based on the estimated cooking progress status. A cooking assistance program that assists in deep frying using a fryer,
An image data processing means for acquiring image data of the cooking state in the fryer captured by a camera,
A means for identifying a type of food contained in the image data;
An estimation means for estimating a cooking progress of the food using a machine-learned model that is trained by inputting image data of the food whose type has been identified;
and a cooking assistance program that causes the cooking assistance program to function as an assistance means for outputting cooking assistance instructions to the fryer based on the estimated cooking progress.