JP7779732B2 - Evaluation system, information processing device, evaluation method, and control program - Google Patents

Description

The present invention relates to an evaluation system, an information processing device, an evaluation method, and a control program.

In the past, attempts have been made to gain useful knowledge by analyzing the human act of eating. For example, Patent Document 1 below discloses a technology that measures the myoelectric potentials of a subject when they chew a sample, and then analyzes the measured myoelectric potentials to quantitatively estimate the texture of the sample.

Japanese Patent Application Laid-Open No. 2016-52516

However, the above-mentioned conventional technologies were aimed at adults, and there was no technology to analyze and objectively evaluate how infants eat and feed their baby food. As a result, even if there was room for improvement in feeding methods, there was no opportunity to notice this. Furthermore, while baby food is introduced in stages as the infant's physical growth and feeding function improves, many people were unsure of whether or not to move on to the next stage.

One aspect of the present invention was made in consideration of the above-mentioned problems, and one of its objectives is to provide technology that can objectively evaluate how a baby eats and feeds baby food.

In order to solve the above problems, an evaluation system according to one aspect of the present invention includes a monitoring device that monitors an infant eating baby food, an information processing device that uses data obtained from the monitoring to evaluate or judge how the infant eats or is being fed, and an output device that outputs feedback information generated based on the results of the evaluation or judgment.

In order to solve the above problem, an information processing device according to one aspect of the present invention includes a monitoring device that monitors an infant eating baby food, a control device that uses data obtained from the monitoring to evaluate or judge how the infant is eating or being fed, and an output device that outputs feedback information generated based on the results of the evaluation or judgment.

In order to solve the above problems, one aspect of the present invention provides an evaluation method executed by an evaluation system including a monitoring device, an information processing device, and an output device, and includes a monitoring step in which the monitoring device monitors an infant eating baby food, an evaluation and judgment step in which the information processing device evaluates or judges the infant's eating or feeding habits using data obtained from the monitoring, and an output step in which the output device outputs feedback information generated based on the results of the evaluation or judgment.

The information processing device according to each aspect of the present invention may be realized by a computer. In this case, the control program for the information processing device, which causes the computer to operate as each unit (software element) of the information processing device, thereby realizing the information processing device on the computer, and the computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention.

One aspect of the present invention makes it possible to objectively evaluate how a baby eats and feeds baby food.

FIG. 1 is a diagram illustrating an example of a configuration of a main part of an information processing device according to the present disclosure. FIG. 1 is a diagram illustrating an overview of an evaluation system according to the present disclosure. 10A and 10B are diagrams showing a specific example of an image and pattern information obtained when feeding is not appropriate. 10A and 10B are diagrams showing a specific example of an image and pattern information obtained when the feeding method is appropriate. 5A and 5B are diagrams showing a specific example of feedback information presented to a user on the display device according to the first embodiment. 4 is a flowchart illustrating an example of an evaluation method executed by the evaluation system according to the first embodiment. 10 is a diagram showing a specific example of detected feature points and analysis result information of mastication movements obtained based on the feature points. FIG. 10A and 10B are diagrams showing a specific example of feedback information presented to a user on the display device according to the second embodiment. 10 is a flowchart illustrating an example of an evaluation method executed by an evaluation system according to a second embodiment.

[Embodiment 1]
<Overview>
2 is a diagram showing an overview of the evaluation system according to the present embodiment. The evaluation system 100 according to the first embodiment is a system for evaluating at least one of how an infant eats baby food and how a waiter who provides the baby food feeds the infant.

The evaluation system 100 disclosed herein includes a monitoring device that monitors infants and young children eating baby food, an information processing device that uses data obtained from the monitoring to evaluate or judge how the infants and young children eat or are fed, and an output device that outputs feedback information generated based on the results of the evaluation or judgment.

In the illustrated example, the above-mentioned information processing device is shown as information processing device 1, which may be realized as a smartphone, for example. Also, in the illustrated example, the above-mentioned monitoring device is shown as a photographing device 3 that photographs an infant being fed and eating baby food, which may be realized as a camera provided in the information processing device 1 as a smartphone, for example. In the illustrated example, the photographing device 3 is provided on the back of the information processing device 1. Also, in the illustrated example, the above-mentioned output device is shown as a display device 2 that displays and outputs the feedback information as visualized information, which may be realized as a liquid crystal display panel or an organic EL display panel integrated with a touch panel, which may be provided in the information processing device 1 as a smartphone, for example. In other words, in the illustrated example, one information processing device 1 functions as the evaluation system 100 of the present disclosure.

While Figure 2 shows an example in which the information processing device 1 is a smartphone, any one or more computers can function as the information processing device 1. For example, the information processing device 1 may be realized as a tablet terminal, personal computer, laptop PC, server device, cloud computing device, etc.

While Figure 2 shows an example in which the evaluation system 100 is realized by a single information processing device 1 (smartphone), the evaluation system 100 can be configured by combining any number of devices. For example, the evaluation system 100 may be configured to include an imaging device 3 as a monitoring device, an information processing device 1, and a display device 2 as an output device. In this case, the imaging device 3 may be a video camera for capturing moving images that is separate from the information processing device 1. The display device 2 may be a liquid crystal display device or an organic EL display device that is separate from the information processing device 1.

Hereinafter, the subject being observed to evaluate how an infant eats baby food or how an infant is fed baby food will be referred to as the subject. In this embodiment, the subject is an infant eating baby food.

In this embodiment, for example, the evaluation system 100 evaluates how an infant or toddler eats or is fed as follows:

A server (for example, a caregiver such as a mother, a nursery teacher, or a nurse) feeds the subject baby food, and the subject eats the baby food. A user uses the camera device 3 of the information processing device 1 to capture images of the server feeding the baby food and the subject eating the baby food (a monitoring step for monitoring the infant eating the baby food). The user refers to a person who uses the evaluation system 100 to obtain an evaluation of how an infant eats or is fed, and the user may be the server himself or herself, or someone other than the server. The other person may be, for example, an instructor who instructs the server on how to feed the baby food. The server may take the images by, for example, mounting his or her own information processing device 1 on a tripod, or someone other than the server may take the images.

The information processing device 1 obtains time-series images taken by photography as data obtained by monitoring. The information processing device 1 evaluates or judges the infant's eating or feeding habits using the time-series images showing the subject's eating behavior (evaluation and judgment step).

The information processing device 1 outputs the feedback information generated based on the results of the evaluation or judgment to the output device (output step). For example, the feedback information may be displayed on the display device 2.

The above-described configuration and method monitor infants eating baby food and use the data obtained from the monitoring to evaluate how the infant eats or is fed the baby food, making it possible to objectively evaluate how the infant eats or is fed the baby food. Furthermore, the objective evaluation results can be presented to the server or a user other than the server. Therefore, if there is room for improvement in the server's feeding method, the server or a user other than the server can be made aware of this, motivating the server to improve the feeding method.

For example, the display device 2 of the information processing device 1 shown in Figure 2 displays, as feedback information (hereinafter referred to as FB (Feedback) information, as appropriate), an image 161 symbolizing the results of the evaluation performed by the information processing device 1 and a message 162 for the user regarding how to eat or feed infants.

By displaying image 161, the user (e.g., the server) can intuitively recognize from the image whether the way the subject is eating the baby food or the way the server is feeding the baby is appropriate. This gives the user an opportunity to notice if there is room for improvement in the way the baby is being fed, and motivates them to improve their feeding methods.

Message 162 may also include text explaining the current state of the subject's eating habits and text explaining areas for improvement in the way the server feeds the baby. Message 162 can specifically communicate to the user measures to improve the subject's eating of baby food or the server's feeding habits. As a result, the server is motivated to improve the way they feed the baby, just like image 161. Furthermore, message 162 allows the user to understand specifically how to make improvements, making message 162 even more useful information for the user.

Note that the feedback information shown in Figure 2 is merely an example, and its content can be arbitrary as long as it makes the user aware that there is room for improvement in the way they feed their pet. Furthermore, feedback to the user is not limited to visual feedback; feedback can be provided in the form of sound, vibration, or a combination of these. Feedback can also be provided via another device, such as by displaying the feedback information on a display device external to the information processing device 1.

<Configuration of information processing device>
FIG. 1 is a diagram illustrating an example of the main configuration of an information processing device 1 according to the present disclosure. As illustrated, the information processing device 1 includes a control unit 10 (control device) that controls each unit of the information processing device 1, a storage unit 11 that stores various data used by the information processing device 1, a photographing device 3 (monitoring device), and a display device 2 (output device). The photographing device 3 is a monitoring device that monitors an infant eating baby food. In this embodiment, the photographing device 3 is, for example, a device that photographs an infant eating baby food. The control unit 10 is a control device that uses data obtained through monitoring to evaluate or judge the infant's eating or feeding habits. The display device 2 is an output device that outputs feedback information generated based on the results of the evaluation or judgment by the control unit 10. In this embodiment, the display device 2 is, for example, a device that presents the feedback information to a user as visualized information.

In this embodiment, the image capture device 3 may be a monocular camera that captures images, or may capture multiple images of the same subject at the same time from multiple viewpoints. Multiple image capture devices 3 may be provided to capture multiple images of the same subject at the same time from multiple viewpoints. Alternatively, various well-known types of 3D cameras may be used as the image capture device 3, such as infrared (IR) sensor type, stereo type, ToF (Time Of Flight) type, and structured lighting type.

Although not shown, the information processing device 1 may also include a communication unit that enables the information processing device 1 to communicate with other devices, and a sound output unit such as a speaker that outputs sound. Furthermore, the information processing device 1 may also include an input unit that accepts input of various data into the information processing device 1. The functions of the input unit and the display device 2 may be realized by a touch panel.

The control unit 10 also includes an image acquisition unit 101, a detection unit 102, an evaluation/determination unit 103, and an output control unit 105. These processing units within the control unit 10 can also be realized, for example, by installing a control program into the information processing device 1 that causes the information processing device 1 to function as these processing units. The memory unit 11 also stores an image 111, operation information 112, result information 113, and FB information 114.

The image acquisition unit 101 acquires images captured in chronological order of scenes in which the subject eats. Scenes in which the subject eats include the action of the subject putting baby food in their mouth (hereinafter referred to as eating action) and the action of moving various oral organs from the time the baby food is put in their mouth until they swallow it (hereinafter referred to as chewing action). Note that in the early stages of baby food, it is expected that infants will swallow the baby food they put in their mouths by moving various oral organs such as the upper lip, lower lip, tongue, jaw, and muscles of mastication, without the action of "crunching" the food. Hereinafter, the action of moving various oral organs from the time the food is put in their mouth until they swallow it will be referred to collectively as chewing action, regardless of whether or not such a "crunching" action is involved.

In this embodiment, the image acquisition unit 101 acquires images captured by the imaging device 3 built into the information processing device 1. In another example, the image acquisition unit 101 may receive images from a separate imaging device 3 by communicating with the imaging device 3 via a communication unit (not shown). Alternatively, the image acquisition unit 101 may acquire images stored in a removable external memory or the like in the information processing device 1 via a terminal (not shown).

The image acquisition unit 101 also stores the acquired images as images 111 in the storage unit 11. The images 111 may be moving images or still images captured continuously over time. The image acquisition unit 101 may also acquire moving images and store frame images extracted from the moving images as images 111.

The detection unit 102 analyzes images 111 taken in chronological order of scenes in which the subject is eating, and detects specific movements made by the subject while eating. In this embodiment, the detection unit 102 analyzes the images to detect eating movements made by the subject when putting baby food in their mouths.

The detection unit 102 stores the detection results of the movement in the memory unit 11 as movement information 112. In this embodiment, the detection unit 102 stores the detection results of the eating movement in the memory unit 11 as movement information 112.

For example, the detection unit 102 may analyze the image 111 to identify a pattern of change over time in the movements of the head and mouth of an infant eating baby food. More specifically, the detection unit 102 may quantify movements related to the subject's eating behavior, detect the subject's eating behavior, and generate pattern information indicating a pattern of the eating behavior, which may be stored in the storage unit 11 as movement information 112. The pattern information may include, for example, movements related to the eating behavior;
(1) The amount of forward and backward movement of the subject's face,
(2) The pitch of the subject's head (an angle for expressing the forward or backward tilt of the head, which may be expressed, for example, as Euler angles), and (3) the numerical information of the opening and closing of the subject's mouth.

The evaluation/determination unit 103 evaluates or determines how the infant eats or is being fed based on the behavior information 112 generated by the detection unit 102. In this embodiment, the evaluation/determination unit 103 may evaluate how the infant is being fed baby food, for example, based on the difference between the pattern of the subject's eating behavior identified by the detection unit 102 and a standard pattern of changes over time in the eating behavior when the infant is being fed in an appropriate manner.

For example, the evaluation and judgment unit 103 compares the subject's pattern indicated by the pattern information generated by the detection unit 102 with a standard pattern. If the comparison shows that the difference between the two patterns is smaller than a predetermined value, the evaluation and judgment unit 103 evaluates that the way the baby eats or is being fed is appropriate, and if the difference is larger than a predetermined value, it can evaluate that there is room for improvement in the way the baby eats or is being fed. The evaluation and judgment unit 103 stores the evaluation result, which indicates whether the way the baby eats or is being fed is appropriate, in the memory unit 11 as result information 113.

The method for generating the standard pattern and the method for determining which pattern to adopt as the standard pattern are arbitrary. For example, the memory unit 11 may store a database that stores, as standard patterns, typical patterns of changes over time in eating behavior when the feeding method is performed appropriately, which have been prepared in advance. The evaluation and determination unit 103 may evaluate the appropriateness of the feeding method by comparing it with the standard patterns in the database.

In another example, the pattern of changes over time in feeding behavior when a subject infant or toddler is able to eat appropriately may be stored in a database as a standard pattern for that subject. When evaluating the subject's feeding method, the evaluation and determination unit 103 may evaluate the appropriateness of the subject's feeding method by comparing it with the standard pattern associated with that subject.

In yet another example, multiple standard patterns may be used when evaluating a subject. For example, by observing the subject's eating habits multiple times, several patterns of changes in the subject's eating behavior over time are accumulated. Therefore, during the first to several evaluations, the evaluation and determination unit 103 may use a standard pattern generated based on a typical pattern as a comparison target. Next, the information processing device 1 may learn the standard pattern for the subject from the multiple patterns of the subject obtained through these several observations, and generate a standard pattern for the subject. Then, during evaluations after a predetermined number of times, the evaluation and determination unit 103 may use the learned standard pattern for the subject as a comparison target.

The output control unit 105 outputs feedback information generated based on the result information 113. In this embodiment, the output control unit 105 generates FB information 114 to provide feedback to the user on how to eat or feed baby food in accordance with the result information 113, as visualized information to be displayed on the display device 2, and outputs it to the display device 2.

In this embodiment, the Facebook information 114 includes an evaluation result indicating whether the way of eating or feeding baby food is appropriate. The Facebook information 114 may also include advice information that provides guidance on the appropriate way of feeding baby food. The output control unit 105 may generate the evaluation result and advice information as an image 161, or as text information such as a message 162.

For example, the memory unit 11 may store in advance feedback information 114 to be presented when the way of eating or feeding baby food is appropriate, and feedback information 114 to be presented when the way of eating or feeding baby food is inappropriate. If the result information 113 output by the evaluation determination unit 103 is an evaluation result indicating that the way of eating or feeding baby food is appropriate, the output control unit 105 can select the feedback information 114 corresponding to the evaluation result and output it to the display device 2. On the other hand, if the result information 113 output by the evaluation determination unit 103 is an evaluation result indicating that the way of eating or feeding baby food is inappropriate, the output control unit 105 can select the feedback information 114 corresponding to the evaluation result and output it to the display device 2.

<Detecting eating behavior>
An example of the process for detecting eating behavior will be described with reference to Figures 3 and 4. Figure 3 shows a specific example of an image 111 of a subject eating baby food, obtained when the baby food is eaten or fed in an inappropriate manner. Figure 4 shows a specific example of an image 111 of a subject eating baby food, obtained when the baby food is eaten or fed in an appropriate manner.

Each figure shows one frame image extracted from a video, but before and after that frame image, successive frame images capturing a series of eating movements of the subject are acquired by the image acquisition unit 101, and these successive frame images are stored in the memory unit 11 as a video.

The process of detecting eating behavior is the same whether or not the baby is eating or feeding the baby food appropriately. Therefore, below, the process of detecting eating behavior will be explained using the example of a video including the frame image shown in Figure 3 input to the detection unit 102. The detection unit 102 can also perform the process of detecting eating behavior when a video including the frame image shown in Figure 4 is input.

When a moving image including the frame images shown in Fig. 3 is input to the detection unit 102, the detection unit 102 analyzes the moving image, digitizes the movement related to the eating behavior, and generates pattern information.
(1) A graph 301 showing the pattern of change over time in the amount of forward and backward movement of the subject's face;
(2) a graph 302 showing the pattern of change over time in the roll, pitch, and yaw of the subject's head; and (3) a graph 303 showing the pattern of change over time in the size of the subject's mouth opening.
and output it as the motion information 112.

In another example, the detection unit 102 may further generate (4) a graph 304 showing the pattern of color change over time in the inner lip area as one piece of pattern information.

More specifically, the detection unit 102 may track facial feature points (e.g., the tip of the nose) in the video image and generate graph 301 based on the time-series depth position information of the feature points. The detection unit 102 may also generate graph 301 by determining the distance between the image capture device 3 and the subject's face based on the size of the subject's face or head area.

The detection unit 102 may observe the angle of the center of gravity of the head (e.g., Euler angles) in the video image and generate the graph 302 based on the changes in the angle over time.

The detection unit 102 may track two or more feature points around the mouth in the video and generate graph 303 based on the time-series vertical and horizontal (up, down, left, right) position information of the feature points. Referring to FIG. 7, the feature points around the mouth may be, for example, point 313 corresponding to the tip of the nose, point 314 corresponding to the center of the upper lip, point 315 corresponding to the center of the lower lip, point 317 corresponding to the right corner of the mouth, and point 318 corresponding to the left corner of the mouth.

Furthermore, if necessary, the detection unit 102 may observe color changes in the area inside the subject's lips (hereinafter referred to as the inner lip area) included in the video, and generate graph 304 based on the color changes observed in the inner lip area. For example, the detection unit 102 may identify the area between the upper lip and lower lip and between the left corner and the right corner of the mouth as the inner lip area. The detection unit 102 may then identify the opening and closing of the mouth during a period in which a predetermined or greater color change is observed in the inner lip area as the opening and closing of the mouth when taking in baby food, i.e., the opening and closing of the mouth related to feeding.

The horizontal axis of each graph showing pattern information in Figure 3 represents the passage of time. The detection unit 102 can synchronize and arrange each graph, and by understanding the time series of events observed from the waveforms of each graph, it can detect the occurrence of eating behavior within the bold frame shown in the figure and output pattern information for the eating behavior period as behavior information 112. Specifically, when the detection unit 102 detects event 311 indicating the opening and closing of the mouth based on graph 303, it may detect the several seconds before and after the timing of the occurrence of event 311 as eating behavior period 313 (bold frame in Figure 3).

The detection unit 102 detects one or more eating movements from the video image and stores pattern information for the period of the eating movement in the memory unit 11 as movement information 112.

The detection unit 102 may observe color changes around the subject's mouth (for example, the inner lip area described above) and detect an event 312 in which a significant color change is observed. In another example, the detection unit 102 may perform image recognition and track a color area (such as a spoon) of a specific color that differs from the color that makes up the subject, and detect an event 312 in which the tracked object enters the frame, approaches the area around the mouth, and then exits the frame again.

When the detection unit 102 detects event 312 at the same time as event 311, it may determine that event 311 is a mouth opening and closing action associated with eating. On the other hand, the detection unit 102 may determine that event 311 that is not accompanied by event 312 is a mouth opening and closing action unrelated to eating, and exclude the period before and after event 311 from the period of eating action. With the above-described configuration, it is possible to identify the timing when a spoon or the like crosses, and accurately detect eating action. For example, mouth opening and closing actions unrelated to eating action can be excluded from the evaluation as noise.

<Evaluation of eating behavior>
The evaluation/determination unit 103 evaluates how the subject eats or feeds the baby food based on the operation information 112 output by the detection unit 102 (for example, each graph included in the pattern information shown in Figures 3 and 4).

As one example, the evaluation and determination unit 103 may evaluate the feeding method as appropriate if the identified pattern information indicates a change in which the head moves forward and then rotates downward as the mouth opens and closes during the feeding action of taking in baby food. Alternatively, the evaluation and determination unit 103 may evaluate the feeding method as inappropriate if the pattern information does not indicate such a change. This allows a feeding method that does not encourage the infant to eat spontaneously to be correctly evaluated as "inappropriate." As one example, a feeding method in which the provider brings food to the infant's mouth with a spoon or the like, resulting in the infant's head not moving forward spontaneously, such as a feeding method similar to that used when assisting with feeding in a caregiver, may be evaluated as "inappropriate."

More specifically, the proper way to feed a baby solid food is thought to involve the following series of steps:
Step 1: Scoop some baby food onto the tip of the spoon, lightly place the spoon on the baby's lower lip, and wait for the baby to take it in with their upper lip.
Step 2: The infant opens his/her mouth, moves his/her head, places the baby food at the tip of the spoon into his/her mouth, closes his/her mouth, and then pulls the spoon out horizontally.

The inventors noticed that when the actions in steps 1 and 2 above are performed correctly, the subject (infant)'s feeding action when taking baby food into their mouth involves moving their face forward and tilting their head forward to bring their mouth closer to the spoon.

The inventors then considered that if the following events are detected in the video in the following order, it can be determined that the subject's eating behavior is accompanied by the above-mentioned movements and that the baby food has been given according to the above-mentioned procedure.
Event 1: Move your face forward as if approaching the spoon in front of you (forward and backward movement of the subject's face)
Event 2: The subject tilts his head forward and looks down to try to take the baby food from the tip of the spoon into his mouth (change in head pitch).
Event 3: The lower lip moves downward and the mouth opens (change in the distance between the upper and lower lips)
Event 4: The spoon crosses the inner lip area between the upper and lower lips and the left and right corners of the mouth (color change in the inner lip area)
Event 5: The lower lip moves upward and the mouth closes (change in the distance between the upper and lower lips)
Event 6: Returning the head to its original forward tilt position (change in head pitch)
Event 7: The face returns to its original position, i.e., the face moves backward (forward and backward movement of the subject's face)
After the above events 1 to 7, the subject moves on to the chewing behavior. The eating behavior, which is the stage preceding the chewing behavior, can be evaluated as appropriate by detecting the above events 1 to 7. The order in which events 1 to 7 occur may also be taken into consideration when evaluating the appropriateness of the feeding method.

Therefore, the evaluation and determination unit 103 refers to graph 301 during the eating movement period 313 detected by the detection unit 102 and determines whether event 1, in which the face moves forward, has been detected. The evaluation and determination unit 103 also refers to graph 302 during the eating movement period 313 and determines whether event 2, in which the head tilts forward, has been detected. The evaluation and determination unit 103 also refers to graph 303 during the eating movement period 313 and determines whether event 3, in which the mouth opens, has been detected. The evaluation and determination unit 103 may also refer to graph 304 during the eating movement period 313 and determine whether event 4, in which the spoon crosses the inner lip region, has been detected. The evaluation and determination unit 103 also refers to graph 303 during the eating movement period 313 and determines whether event 5, in which the mouth closes, has been detected. The evaluation and determination unit 103 also refers to graph 302 during the eating movement period 313 and determines whether event 6, in which the head returns to its original forward tilted position, has been detected. The evaluation and determination unit 103 also references the graph 301 during the eating activity period 313 to determine whether event 7, which moves the face backward, has been detected.

If the evaluation and determination unit 103 determines that events 1 to 7 have been detected in order during the eating behavior period 313, it evaluates that the subject's way of eating the baby food or the waiter's way of feeding the baby food is appropriate.

The frame image shown in Figure 3 depicts a scene of the subject's eating behavior when the subject's way of eating or feeding baby food is inappropriate. An example of an inappropriate case would be when, during the eating behavior period 313, in addition to the opening and closing of the mouth, the subject does not move their face forward and backward, or tilt their head forward and backward. Therefore, the evaluation determination unit 103 may evaluate that the way the subject is eating or feeding baby food is inappropriate if, during the eating behavior period 313, at least one of the forward and backward movement of the face and the forward and backward tilt of the head does not occur.

A more detailed explanation will be given with reference to Figure 3. In the illustrated example, during feeding behavior period 313, which includes event 311 of the mouth opening and closing behavior and event 312 of the spoon passing by, no events indicating forward and backward tilting of the head are shown in graph 302. Furthermore, during feeding behavior period 313, no events indicating forward and backward movement of the face are shown in graph 301. This means that, as shown in the frame image in the same figure, when feeding baby food, the subject is simply opening their mouth and waiting for the baby food to be brought into their mouth, maintaining their original sitting position without moving their head. Alternatively, when the subject has their mouth open and is waiting for the baby food to be brought into their mouth, only a backward tilt of the head is observed when opening their mouth, without any forward tilting of the head being observed.

The above means that the server was not feeding the subject in a way that would encourage them to actively participate in the meal. More specifically, the server brought the baby food on a spoon to the subject, who was looking up with his mouth open, and then, without waiting for the subject to take the food into his mouth, the server rubbed the spoon against the subject's upper palate before removing it and feeding the baby food. Feeding a baby by putting the food in their mouth in this way does not foster the awareness or attitude of eating food on their own, and may have an impact on the underdevelopment of oral function. Furthermore, if the subject puts the baby food in their mouth while looking up, they may develop the habit of swallowing it whole, which may lead to aspiration, and they may not use their jaw muscles sufficiently, which may also have an impact on the underdevelopment of oral function.

As described above, if eating behavior that is unfavorable for the development of an infant is observed during the eating behavior period 313, the evaluation and judgment unit 103 evaluates that the subject's way of eating or feeding the baby food is inappropriate. The evaluation and judgment unit 103 stores the evaluation result indicating inappropriateness as result information 113 in the memory unit 11.

The frame image shown in Figure 4 depicts a scene of the subject's eating behavior when the subject's way of eating or feeding baby food is appropriate. An example of an appropriate case would be when, in addition to opening and closing the mouth, the subject moves their face back and forth and tilts their head back and forth during the eating behavior period 413. Therefore, the evaluation determination unit 103 may evaluate the way the subject eats or feeds baby food as appropriate if the face moves back and forth and the head tilts back and forth during the eating behavior period 413.

A more detailed explanation will be given with reference to Figure 4. The evaluation unit 103 focuses on graphs 401 and 402 included in the pattern information for eating behavior period 413, which includes event 411 of mouth opening and closing behavior in graph 403 and event 412 of spoon crossing in graph 404. According to graph 401, event 414 of moving the face forward is observed immediately before event 411 of mouth opening behavior. Next, according to graph 402, event 415 of tilting the head forward is observed. This means that, as shown in the frame image in the figure, when eating baby food, the subject put their face forward, looked down, and moved as if they were grasping the tip of the spoon with their mouth. In other words, this means that the server was able to feed the subject in a way that encouraged them to actively participate in the meal.

As described above, if eating behavior favorable for the development of an infant is observed during the eating behavior period 413, the evaluation and judgment unit 103 evaluates that the subject's way of eating or feeding baby food is appropriate. The evaluation and judgment unit 103 stores the evaluation result indicating appropriateness as result information 113 in the memory unit 11.

<Feedback>
5 is a diagram showing a specific example of the Facebook information 114 presented to the user on the display device 2 according to the first embodiment. A first display screen example 501 shows an example of the Facebook information 114 displayed when a negative evaluation is given on how to eat or feed baby food. A second display screen example 502 shows an example of the Facebook information 114 displayed when a positive evaluation is given on how to eat or feed baby food.

The output control unit 105 selects or generates the FB information 114 corresponding to the evaluation result output by the evaluation determination unit 103, configures a display screen, and displays it on the display device 2. For example, the output control unit 105
(1) Images (images 511, 521) that symbolize the evaluation results output by the evaluation determination unit 103, and (2) Messages (messages 512, 522) that provide guidance on the evaluation results and appropriate ways to feed or feed infants.
The display screen may be configured based on the FB information 114 including the above.

Furthermore, the output control unit 105 may (3) include link information (link information 513, 523) in the Facebook information 114 to guide the user to a screen that provides more detailed information on how to properly feed or feed infants. The output control unit 105 may also place link information 523 on the display screen when a positive evaluation is given.

As described above, by displaying Facebook information 114 on display device 2, servers and other users can be made aware of any room for improvement in the way servers serve food, motivating the servers to improve their serving methods. Furthermore, messages 512 and link information 513 allow servers to understand how to improve their serving methods and actually make improvements. Messages 522 and link information 523 can also make servers who have received positive feedback aware of proper serving methods. This encourages servers to consciously continue serving food in an appropriate manner.

<Processing flow>
6 is a flowchart showing an example of an evaluation method executed by the evaluation system 100 according to embodiment 1. A series of processes of the evaluation method is started, for example, when a moving image captured by the image capturing device 3 is input to the control unit 10.

In S11, the image acquisition unit 101 acquires an image of the subject eating, and stores it in the memory unit 11 as image 111. For example, if a video of the subject eating is captured by the imaging device 3, the image acquisition unit 101 may acquire a frame image extracted from the video as image 111.

In S12, the detection unit 102 detects the subject's eating behavior. As a specific example, the detection unit 102 may detect feature points indicating the subject's mouth movement from the group of images 111 and generate pattern information (e.g., graph 303) in which the up, down, left, and right position information of the detected feature points is arranged in time series. The detection unit 102 may also detect facial feature points, such as the tip of the nose, and generate pattern information (e.g., graph 301) in which the depth position information of the detected feature points is arranged in time series. The detection unit 102 may also observe the angle of the center of gravity of the head (particularly the pitch) and generate graph 302 based on changes in the angle over time. If necessary, the detection unit 102 may further detect the inner lip region, observe color changes in the detected inner lip region, and generate graph 304 representing the color changes over time. The inner lip region may be defined, for example, by detecting feature points at the center of the upper lip, the center of the lower lip, the left corner of the mouth, and the right corner of the mouth.

In S13, the detection unit 102 stores pattern information of various movements related to the detected eating behavior in the memory unit 11 as movement information 112.

In S14, the evaluation and determination unit 103 determines whether a movement of moving the face forward and backward and a movement of tilting the head forward and backward were observed during the period in which eating behavior was detected (e.g., eating behavior period 313). If the two movements were observed, the evaluation and determination unit 103 proceeds to YES in S14, and if at least one of the two movements was not observed, the evaluation and determination unit 103 proceeds to NO in S14.

In S15, the evaluation and determination unit 103 evaluates that the subject's way of eating the baby food or the waiter's way of feeding the baby food is appropriate.

In S16, the evaluation and determination unit 103 evaluates that the subject's way of eating the baby food or the waiter's way of feeding the baby food is inappropriate.

In S17, the evaluation determination unit 103 stores the evaluation result, indicating whether the evaluation is appropriate or inappropriate, in the memory unit 11 as result information 113.

In S18, the output control unit 105 selects or generates feedback information 114 corresponding to the evaluation result indicated by the result information 113, and causes the output device to output it. For example, the output control unit 105 creates a display screen including the feedback information 114 and causes the display device 2 to display it.

<Effects>
According to the above-described configuration and method, the camera 3 monitors an infant eating baby food, and the information processing device 1 uses the data obtained through the monitoring to identify a pattern of change over time in the head and mouth movements of the infant while eating baby food. The information processing device 1 evaluates the baby food feeding method for the infant based on the difference between the identified pattern and the pattern of change over time in the movements when the infant is fed properly. This allows for an objective evaluation of the baby food eating and feeding method. Furthermore, the objective evaluation results can be presented to the server or a user other than the server. This allows the server or a user other than the server to be made aware of any room for improvement in the server's feeding method, motivating the server to improve the feeding method.

[Embodiment 2]
Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

<Overview>
The evaluation system according to this embodiment may be, for example, as shown in Fig. 2, similar to embodiment 1. The evaluation system 100 according to embodiment 2 is a system for evaluating how an infant eats baby food and determining the progress stage of the infant's baby food.

In this embodiment, for example, the evaluation system 100 determines the progress stage of an infant's weaning food as follows:

The subject is photographed eating the baby food using the photographing device 3 (monitoring step).

The information processing device 1 obtains time-series images taken by photography as data obtained by monitoring. The information processing device 1 determines the progress stage of the subject's weaning food using the time-series images showing the subject's eating behavior (evaluation and determination step). In this embodiment, the information processing device 1 particularly detects, within the eating behavior, chewing movements that move various oral organs from the time the subject puts the baby food in their mouth until they swallow it, and determines the progress stage of the subject's weaning food based on the detected chewing movements.

The display device 2, which serves as an output device, outputs the FB information generated based on the results of the evaluation or judgment (output step). For example, the display device 2 displays, as the FB information, an image 161 showing the results of the judgment made by the information processing device 1 and a message 162 for the user regarding the judged progress stage of the baby food.

This allows the user to recognize the progress of the baby's weaning food, which is determined according to the developmental stage related to feeding function. In other words, the FB information is useful information that helps the user determine whether or not to advance the baby's weaning food to the next stage. The term "user" refers to a person who uses the evaluation system 100 to obtain the results of the assessment of the baby's weaning food progress, and the user may be the server themselves or someone other than a server.

Note that the FB information shown in Figure 2 is merely an example, and its content can be any as long as it allows the user to understand the progress of the baby's weaning diet. Furthermore, feedback to the user is not limited to visual feedback; it can also be provided by sound, vibration, or a combination of these. Furthermore, feedback can also be provided via another device, such as by displaying the FB information on a display device external to the information processing device 1.

<Configuration of information processing device>
The information processing device 1 according to this embodiment may be, for example, as shown in FIG. 1, similarly to the first embodiment.

In this embodiment, the detection unit 102 analyzes the image to detect the chewing action that occurs following the eating action of putting baby food into the mouth. The detection unit 102 stores the detection result of the chewing action as action information 112 in the memory unit 11.

Specifically, the detection unit 102 detects the magnitude of the subject's mouth movement while chewing, based on images 111 captured in chronological order of the subject's chewing behavior. As an example, the detection unit 102 may detect the magnitude of the mouth's vertical (up-down) and horizontal (left-right) movement. More specifically, the detection unit 102 may be configured to function as a feature point detection unit that detects feature points, and as a movement analysis unit that analyzes mouth movement based on position information of the feature points.

The detection unit 102 detects the magnitude of mouth movement during chewing of the subject based on the time-series images obtained by photographing. The detection unit 102 may detect the vertical and horizontal sizes of the mouth as distances, or may detect the magnitude of mouth movement as an area. Specifically, the distance may be the vertical and horizontal movement distances of the mouth or feature points around it. The area may be the area of the region surrounded by the trajectory drawn by the mouth or feature points around it.

It is also possible to use images captured by an imaging device 3 such as a 3D camera that is capable of capturing mouth movement in the depth direction. In this case, the detection unit 102 may detect the vertical, horizontal, and depth dimensions of the mouth as distances, or may detect the size of the mouth movement as a volume. Specifically, the volume may be the volume of the space enclosed by the trajectory of the mouth or its surrounding feature points.

First, the detection unit 102 detects predetermined feature points on the subject's face from the image 111. The detection unit 102 detects chewing movements based on time-series position information of the detected feature points. The detection unit 102 may store the time-series position information of the feature points corresponding to the detected chewing movements as movement information 112 in the storage unit 11.

Furthermore, the detection unit 102 may monitor the distance between the apex of the infant's upper lip and the apex of the infant's lower lip observed in each frame image of the video (for example, the distance between points 314 and 315 in Figure 7). A series of mouth opening and closing movements, including a mouth opening movement where the distance is equal to or greater than a predetermined threshold, may then be excluded from the chewing movements to be detected. In this way, the mouth opening and closing movements made when taking in baby food can be distinguished and excluded from the chewing movements necessary to determine the baby's stage of progress in baby food. This prevents the opening movement made when putting a spoon in the mouth, rather than chewing, from being mistaken for the opening movement during chewing, making it possible to accurately determine the baby's stage of progress in baby food based on the nature of the chewing movements.

Next, the detection unit 102 analyzes the mouth movement during chewing using the above-mentioned time-series position information. The detection unit 102 outputs information indicating the magnitude of the mouth movement identified as a result of the analysis. The detection unit 102 may store the information indicating the magnitude of the mouth movement in the memory unit 11 as movement information 112. Specifically, the detection unit 102 may calculate an index value indicating the magnitude of the mouth movement as information indicating the magnitude of the mouth movement. The detection unit 102 may generate visualization information that visualizes the magnitude of the mouth movement as information indicating the magnitude of the mouth movement. The visualization information may be a graph, figure, illustration, etc. that represents the magnitude of the mouth movement. The detection unit 102 generates the above-mentioned graph, figure, illustration, etc. based on the time-series position information and index value.

The detection unit 102 may store at least one of an index value indicating the magnitude of mouth movement and visualization information as movement information 112 in the memory unit 11.

The evaluation and determination unit 103 evaluates the chewing movement based on the detected chewing movement information 112. Specifically, the evaluation and determination unit 103 identifies the chewing movement pattern. Then, the evaluation and determination unit 103 determines the baby's progress in weaning food from the identified chewing movement pattern. The evaluation and determination unit 103 may store the determination result indicating the determined progress pattern in the memory unit 11 as result information 113.

The output control unit 105 selects or generates FB information 114 corresponding to the result information 113 output from the evaluation and judgment unit 103, and outputs it to the output device. For example, the output control unit 105 may create a display screen including FB information 114 corresponding to the judgment result indicated by the result information 113, and display it on the display device 2.

<Detection of chewing movements>
(Detection of feature points)
The method for detecting feature points by the detection unit 102 will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example of feature points detected by the detection unit 102 and analysis result information (motion information 112) of the mastication movement obtained based on the detected feature points.

The schematic diagram 31 of the face in Figure 7 shows examples of feature points to be detected. The feature points detected by the detection unit 102 are used to analyze the movement of the mouth during chewing, and so may be parts that move during chewing in a way that reflects the chewing pattern. For example, the detection unit 102 may detect parts around the mouth on the center line L of the face as feature points, such as point 314 (the apex of the upper lip), point 315 (the apex of the lower lip), point 316 (the center position of the jaw) in the schematic diagram 31, or the tip of the jaw (not shown). Detecting parts on the center line L as feature points makes it easier to identify the magnitude of lateral movement of the mouth during chewing.

The detection unit 102 may also detect points 317 and 318 (corners of the mouth) in the schematic diagram 31 as feature points. In this case, the detection unit 102 may analyze the movement of the mouth by regarding the movement of the midpoint between point 317 (right corner of the mouth) and point 318 (left corner of the mouth) as the movement of the mouth. In addition, for example, the detection unit 102 may detect the area of the cheek near the masseter muscle as a feature point. In this case, the detection unit 102 may analyze the movement of the mouth from the change in the area of that area.

The detection unit 102 may also detect areas that do not move due to chewing. For example, the detection unit 102 may detect point 313 (tip of the nose). In this case, the detection unit 102 can represent the positions of feature points in the image 111 using the position of point 313 (tip of the nose) as a reference. This makes it possible to accurately identify the movement of each feature point over time, even if the position at which the subject appears shifts between the time-series images 111.

Furthermore, the detection unit 102 may detect multiple areas that do not move due to chewing in order to determine the distance on the image 111 that serves as a reference for evaluating the magnitude of mouth movement. For example, the detection unit 102 may detect points 311 and 312 (inner corners of the eyes). This allows the evaluation determination unit 103 to evaluate the magnitude of mouth movement based on the distance between point 311 (inner corner of the right eye) and point 312 (inner corner of the left eye). The detection unit 102 may use points 311, 312, 313, 317, 318, etc. to determine the center line L of the face.

In this way, the detection unit 102 may detect the magnitude of vertical and horizontal movement of the subject's mouth while chewing based on the distance between predetermined feature points on the subject's face that are detected from the image 111 and whose positions do not change due to chewing. This makes it possible to accurately detect the magnitude of mouth movement whether the subject appears large or small in the image 111.

The reference distance for evaluating the magnitude of mouth movement may be input by the subject. Furthermore, if the distance to the subject is known (for example, when photographing with a 3D camera that outputs distance information), the magnitude of mouth movement can be determined according to that distance, making it possible to omit body part detection for calculating the reference distance.

The method for detecting feature points as described above is not particularly limited; for example, an image analysis method may be applied, a machine learning method may be applied, or a combination of these may be applied. Furthermore, when detecting multiple feature points, a detection unit 102 may be provided for each feature point to be detected. Furthermore, detection markers, two-dimensional codes, or three-dimensional codes may be attached to the detection target parts of the subject's face, and these may be detected from the image.

(Movement analysis)
The detection unit 102 analyzes the movement of the mouth during mastication from the time-series positional changes of the feature points detected as described above, and calculates an index value indicating the movement. This will be explained with reference to the graph 32 shown in FIG. 7.

Graph 32 is a graph showing the time-series positional changes of feature points on image 111. Graph 32 can be generated from the positional information of feature points detected in each of multiple images 111 in the time series.

For example, the positions of the apex of the lower lip (point 315 in schematic diagram 31) and the jaw (point 316 in schematic diagram 31) change during chewing as shown in graph 32. In graph 32, initial position 321 indicates the position of the feature point at the start of chewing (just before starting to open the mouth). As shown in graph 32, after chewing begins, the feature point moves downward and left, then upward and right, and then toward the left. It should be noted that the left and right described here refer to the left and right as seen from the side photographing (observing) the subject, and that the left and right are reversed from the subject's perspective.

As such, graph 32 has the characteristic that the feature points move from the initial position 321 at the start of chewing and then return to the vicinity of the initial position 321. Therefore, by utilizing this characteristic, the detection unit 102 can identify one chewing action from graph 32 (more precisely, the position information of the feature points detected from each of the multiple images 111 in time series). This also makes it possible to calculate an index value that indicates the number of chews from the start of chewing the object to the time it is swallowed.

Furthermore, from graph 32, it is possible to identify the position when the feature point moves furthest to the left and the position when it moves furthest to the right during one chewing action. Therefore, from the position information of the feature point detected from each of the multiple images 111 in the time series, the detection unit 102 can also determine that the magnitude of the horizontal movement of the feature point is Mox. Similarly, from the position information of the feature point detected from each of the multiple images 111 in the time series, the detection unit 102 can also determine the position when the feature point is at the top (for example, the initial position 321 at the start of chewing) and the position when the feature point moves furthest to the bottom, and determine that the magnitude of the vertical movement of the feature point is Moy. Here, the detection unit 102 may take the movement distance when the movement of the feature point is represented on a two-dimensional plane (on the captured image) as Mox and Moy, or may take the three-dimensional movement distance of the feature point as Mox and Moy.

Specifically, the detection unit 102 extracts position information contained in one chewing movement from the position information of feature points detected from each of the multiple time-series images 111. Note that one chewing movement can also be detected based on the movement of the corners of the mouth, the thickness of the lips, the change in the area of the cheek near the masseter muscles, etc. The detection unit 102 then calculates the distance Mox between the leftmost and rightmost positions of that position information, and the distance Moy between the uppermost and lowermost positions of that position information, as index values indicating the size of the mouth during one chewing movement.

In this case, the detection unit 102 may calculate index values indicating the magnitude of vertical and horizontal movement of the subject's mouth based on the distance between predetermined feature points on the subject's face detected from the image 111, whose position does not change due to chewing. For example, the detection unit 102 may calculate the distance Ex between point 311 (the inner corner of the right eye) and point 312 (the inner corner of the left eye) in the schematic diagram 31, calculate the value Mox/Ex as an index value indicating the magnitude of horizontal movement of the mouth, and calculate the value Moy/Ex as an index value indicating the magnitude of vertical movement of the mouth. This makes it possible to non-dimensionalize the values Mox and Moy and cancel out differences in the size of the subject appearing in the image 111.

Note that the index value indicating the movement of the mouth during chewing is not limited to the above example. For example, the area of the portion enclosed by graph 32 (hereinafter referred to as Mos) can be said to indicate the magnitude of the movement of the mouth, and therefore the area of this portion can be used as the index value indicating the magnitude of the movement of the mouth during chewing. Mos can also be made dimensionless by dividing it by the distance between predetermined feature points whose position does not change due to chewing (for example, the above-mentioned Ex).

The detection unit 102 may also calculate the length of the diagonal distance found from Mox and Moy using Pythagoras' theorem as an index value representing the magnitude of the vertical and horizontal movements of the mouth. To explain this more specifically with reference to graph 32 shown in FIG. 7, the detection unit 102 may calculate the length of the diagonal of a rectangle (indicated by the dashed line) whose horizontal length is Mox and whose vertical length is Moy as an index value representing the magnitude of the vertical and horizontal movements of the mouth.

The detection unit 102 may also calculate index values other than those indicating the magnitude of mouth movement. For example, the detection unit 102 may use the number of chews per unit time, the number of chews from placing a chewing object in the mouth until swallowing, the chewing rhythm, etc. as index values. In addition, the detection unit 102 may determine whether chewing is being performed on the left or right side based on, for example, the movement of the corners of the mouth, and calculate the number of chews on the left and right sides or the ratio thereof as index values. Furthermore, for example, the detection unit 102 may determine whether the movement of the feature points is within a normal range and output an index value indicating the determination result.

When obtaining the index value, the detection unit 102 may use the average value of Mox and Moy, the sum of Mox and Moy, or either Mox or Moy.

(Output of operation information)
As described above, the detection unit 102 can output analysis result information of the chewing movement as the movement information 112. The analysis result information of the chewing movement specifically refers to various index values and visualized information (graphs) related to the movement of the subject's mouth. The detection unit 102 may select, from the various analysis results related to the chewing movement, information required by the evaluation determination unit 103 to determine the progress stage of the weaning food of the subject, who is an infant, and output the selected information as the movement information 112.

In this embodiment, for example, the detection unit 102 detects one chewing motion or multiple chewing motions.
(1) A graph like graph 32 (hereinafter referred to as a lower lip trajectory graph during mastication) showing time-series positional changes of the feature point 315 corresponding to the apex of the lower lip,
(2) A graph showing the time-series positional change of the feature point 318 corresponding to the left corner of the mouth (hereinafter referred to as the left corner of the mouth trajectory graph during mastication);
(3) A graph showing the time-series positional change of the feature point 316 corresponding to the center of the lower jaw (hereinafter referred to as a lower jaw trajectory graph during chewing) may be output as analysis result information of the chewing movement (movement information 112).

The lower lip trajectory graph during mastication may be referenced by the evaluation and determination unit 103, for example, to evaluate the magnitude of vertical and horizontal movement of the subject's mouth while masticating. The left mouth corner trajectory graph during mastication may be referenced by the evaluation and determination unit 103, for example, to evaluate the magnitude of vertical and horizontal movement of the corners of the mouth while masticating. The lower jaw trajectory graph during mastication may be referenced by the evaluation and determination unit 103, for example, to evaluate the masticatory cycle (chewing cycle) that represents the path of movement of the subject's lower jaw while masticating.

Instead of a lower lip trajectory graph during chewing, the detection unit 102 may output Mox and Moy for one chewing motion, or the average values of Mox and Moy for multiple chewing motions, for the feature point 315 corresponding to the apex of the lower lip.

Instead of the left mouth corner trajectory graph during chewing, the detection unit 102 may output the Mox and Moy values for one chewing motion or the average values of Mox and Moy values for multiple chewing motions for the feature point 318 corresponding to the left corner of the mouth.

If necessary, the detection unit 102 may also output as operation information 112 (4) a graph showing the change in the distance between the apex 314 of the upper lip and the apex 315 of the lower lip (hereinafter referred to as a chewing mouth width graph), corresponding to a single detected chewing action or multiple chewing actions.

In another example, the detection unit 102 may output, as the motion information 112, a graphic showing the diagonal length of a rectangle (shown by a dashed line) whose horizontal length is Mox and whose vertical length is Moy for the graph 32 shown in FIG. 7.

<Assessing the progress of weaning food>
The period from the start of weaning (eating) to the completion of weaning (eating) is generally thought of as being divided into four stages according to the developmental stage of the infant's feeding function, and the progress stages of weaning food are also set out as guidelines for how to proceed according to these four stages. Specifically, the progress stages of weaning food can be divided into the following four stages.
Stage 1 (early weaning): In terms of age, this corresponds to infants aged 5 to 6 months. In terms of feeding function, infants at this stage are able to close their mouths and take in and swallow. However, after taking in food, the chewing movement leading up to swallowing may not be sufficient, and the mouth may remain slightly open. This is commonly referred to as the "gulping stage."
Stage 2 (mid-weaning): In terms of age, this corresponds to infants aged 7 to 8 months. In terms of feeding function, infants at this stage are able to crush food with their tongue and upper palate. This is commonly referred to as the "chewing stage."
Stage 3 (late weaning): In terms of age, this corresponds to infants aged 9 to 11 months. In terms of feeding function, infants at this stage are able to crush food with their gums. This is commonly referred to as the "chewing stage."
Stage 4 (Complete weaning): In terms of age, this corresponds to infants aged 12 to 18 months. In terms of feeding function, infants at this stage are able to chew food using their teeth. This is commonly referred to as the "munching stage."

The inventors noticed that infants in the early weaning stage exhibit vertical movement at the corners of their mouths as they open and close their mouths during chewing. Therefore, the evaluation and determination unit 103 can be configured to determine the subject's stage of progress in weaning food as early weaning when it identifies a chewing pattern in which the corners of the mouth exhibit vertical movement. As an example, the evaluation and determination unit 103 may determine the stage of progress as early weaning when the index value Moy indicating the magnitude of vertical movement of the apex of the lower lip (point 315) is equal to or greater than a predetermined threshold, and the index value Moy of the left corner of the mouth (point 318) is equal to or greater than a predetermined threshold.

Furthermore, the evaluation and determination unit 103 may identify a chewing pattern in which the insides of the upper and lower lips remain open during chewing if the shortest distance between the apex of the upper lip (point 314) and the apex of the lower lip (point 315) of the infant during the chewing action is equal to or greater than a predetermined threshold. If such a chewing pattern is identified, the evaluation and determination unit 103 may determine that the progress stage of weaning food is in the early weaning stage. This makes it possible to more accurately determine whether the infant is in the early weaning stage. It also contributes to the early detection and improvement of tongue thrusting habits.

The inventors noticed that infants in the middle stage of weaning exhibit horizontal movement at the corners of their mouths in addition to vertical movement during chewing. Therefore, the evaluation and determination unit 103 can be configured to determine the progress stage of the subject's weaning food as being in the middle stage of weaning when it identifies a chewing pattern in which the corners of the mouth exhibit horizontal movement in addition to vertical movement. As an example, the evaluation and determination unit 103 may determine the progress stage as being in the middle stage of weaning when the index value Moy of the apex of the lower lip (point 315) is equal to or greater than a predetermined threshold, and the index value Mox indicating the magnitude of horizontal movement of the left corner of the mouth (point 318) is equal to or greater than a predetermined threshold.

The inventors noticed that infants in the later weaning stage also show lateral movements of the lips and jaw during chewing movements. Therefore, the evaluation and determination unit 103 can be configured to determine the progress stage of the subject's weaning food as the later weaning stage when it identifies a chewing pattern in which the lips and jaw also show lateral movements. As an example, the evaluation and determination unit 103 may determine the progress stage as the later weaning stage when the index value Mox of the apex of the lower lip (point 315) is equal to or greater than a predetermined threshold.

The inventors noticed that in infants who have completed the weaning stage, the chewing cycle observed during chewing movements changes from a long, vertical rod shape to a pointed teardrop shape, becoming rounder and closer to a circle. Therefore, the evaluation and determination unit 103 can be configured to determine that the subject's progress in weaning food is in the weaning completion stage when a chewing pattern in which a chewing cycle that is more rounded than a predetermined value is identified in the mandibular trajectory graph during chewing is identified. Alternatively, the evaluation and determination unit 103 may determine that the subject is in the weaning completion stage when a chewing pattern in which the ratio of the index value Mox of the mandibular center (point 316) to the index value Moy is equal to or greater than a predetermined threshold value is identified.

The evaluation and determination unit 103 stores the determination result, which indicates one of the above-mentioned baby food progress stages, as result information 113 in the memory unit 11.

The information processing device 1 may receive information related to the subject in advance from a user (e.g., the subject's caregiver) via an input unit (not shown) and store it in the memory unit 11. Possible information related to the subject includes, for example, the subject's age in months and the state of tooth eruption. The information related to the subject may also include information that can be used as a reference when determining the progress stage of weaning food, such as height, weight, duration of each feeding, number of feedings per day, survey results regarding the subject's behavior, and survey results regarding the hardness of the weaning food the subject is currently eating.

The evaluation and determination unit 103 may determine the subject's progress in weaning food by taking into account at least one of the subject's age in months and the state of tooth eruption. This allows the subject's progress in weaning food to be determined with even greater accuracy.

<Feedback>
8 is a diagram showing a specific example of the FB information 114 presented to the user on the display device 2 according to embodiment 2. The example display screen shown in the figure shows an example of the FB information 114 displayed when the progress stage of the weaning food of the subject is determined to be the "mid-weaning stage."

The output control unit 105 selects or generates FB information 114 corresponding to the judgment result output by the evaluation/judgment unit 103, configures a display screen, and displays it on the display device 2. For example, the output control unit 105
(1) an image 801 that symbolizes the judgment result output by the evaluation/judgment unit 103; and (2) a message 803 that provides guidance on the hardness and size of ingredients to be fed to infants at the judgment result and the progress stage corresponding to the judgment result.
The display screen may be configured based on the FB information 114 including the above.

Furthermore, the output control unit 105 may (3) include in the FB information 114 the name 802 of the weaning stage, which succinctly expresses the progress stage corresponding to the judgment result.

Furthermore, the output control unit 105 may (4) include link information (not shown) in the FB information 114 to guide the user to a screen that provides more detailed guidelines for the hardness and size of the baby food to be given at the determined progress stage.

When presenting the image 801 or name 802, the output control unit 105 may arrange the images and names of all four stages of weaning food on the display screen in chronological order, and then place a cursor 804 to clearly indicate the image and name of the weaning food stage that applies to the subject. This display allows the user to not only understand the subject's current situation, but also to understand which stage the subject is at in the weaning period from the start to the end of weaning. For example, if the user has been feeding the subject weaning food equivalent to the early weaning stage, but is presented with the subject's weaning food stage as being in the middle weaning stage, the user can realize that the subject can move on to the next weaning food stage. The user can then be motivated to step up the weaning food by changing the hardness and size of the ingredients, etc.

Furthermore, the information processing device 1 may receive in advance from a user via an input unit (not shown) the current baby food progress stage of the subject predicted by the user and store it in the memory unit 11. The evaluation and judgment unit 103 can compare the baby food progress stage of the infant predicted in advance by the user with the baby food progress stage determined from the identified chewing behavior, and determine whether there are any differences or similarities between them. The evaluation and judgment unit 103 may then store the judgment result indicating the differences or similarities in the memory unit 11 as result information 113.

This allows the output control unit 105 to present the user with Facebook information 114 informing the user whether the progress stage the user believes to be consistent with the progress stage determined by the evaluation determination unit 103 matches. For example, if the two do not match, the output control unit 105 can display in the Facebook information 114 a message recommending that the user feed the baby food appropriate for the determined progress stage.

As a result, users can introduce solid foods at the appropriate time to their infant's physical growth and improving feeding function without having to worry about their own judgment.

<Processing flow>
9 is a flowchart showing an example of an evaluation method executed by the evaluation system 100 according to embodiment 2. A series of processes of the evaluation method is started, for example, when a moving image captured by the image capturing device 3 is input to the control unit 10.

In S31, the image acquisition unit 101 acquires an image of the subject eating, and stores it in the memory unit 11 as image 111. For example, if a video of the subject eating is captured by the imaging device 3, the image acquisition unit 101 may acquire a frame image extracted from the video as image 111.

In S32, the detection unit 102 detects the subject's chewing movements and analyzes the chewing movements. As a specific example, the detection unit 102 may output Mox and Moy of the lower lip trajectory graph during chewing or point 315 at the apex of the lower lip. The detection unit 102 may output an index value calculated from Mox and Moy of the left mouth corner trajectory graph during chewing or point 318 at the left mouth corner. Note that, in obtaining the index value, the detection unit 102 may use the values of Mox and Moy for one chewing movement, the average values of Mox and Moy for any number of chewing movements, or the total values of Mox and Moy for any number of chewing movements. Here, the detection unit 102 may use Mox and Moy as the movement distance when the movement of the feature points is represented on a two-dimensional plane (on a captured image), or may use Mox and Moy as the three-dimensional movement distance of the feature points. The detection unit 102 may output a mandibular trajectory graph during mastication or a chewing cycle of the mandibular center point 316.

In S33, the detection unit 102 stores the above-mentioned various trajectory graphs obtained as a result of analyzing the detected chewing movements in the memory unit 11 as analysis result information (movement information 112).

In S34, the evaluation and determination unit 103 attempts to identify, for the detected chewing movements, a chewing pattern in which the corners of the mouth move horizontally in addition to a chewing pattern in which the corners of the mouth move vertically. If the evaluation and determination unit 103 identifies a chewing pattern in which the corners of the mouth move vertically but is unable to identify a chewing pattern in which the corners of the mouth move horizontally, the process proceeds from NO in S34 to S35. On the other hand, if the evaluation and determination unit 103 is able to identify a chewing pattern in which the corners of the mouth move horizontally in addition to a chewing pattern in which the corners of the mouth move vertically, the process proceeds from YES in S34 to S36.

In S35, the evaluation and determination unit 103 determines that the subject's baby food progress stage is in the early weaning stage based on the fact that a chewing pattern in which the corners of the mouth move vertically has been identified.

In S36, the evaluation and determination unit 103 further attempts to identify a chewing pattern in which the lips or jaw move sideways from the detected chewing movement. If the evaluation and determination unit 103 is unable to identify a chewing pattern in which the lips or jaw move sideways, the process proceeds from NO in S36 to S37. On the other hand, if the evaluation and determination unit 103 is able to identify a chewing pattern in which the lips or jaw move sideways, the process proceeds from YES in S36 to S38.

In S37, the evaluation and determination unit 103 determines that the subject's baby food progress stage is the mid-weaning stage based on the fact that a chewing pattern in which the corners of the mouth move sideways has been identified.

In S38, the evaluation and determination unit 103 attempts to identify a chewing pattern in which the chewing cycle of the lower jaw is rounded for the detected chewing movement. If the evaluation and determination unit 103 is unable to identify a chewing pattern in which the chewing cycle of the lower jaw is rounded, the process proceeds from NO in S38 to S39. On the other hand, if the evaluation and determination unit 103 is able to identify a chewing pattern in which the chewing cycle of the lower jaw is rounded, the process proceeds from YES in S38 to S40.

In S39, the evaluation and determination unit 103 determines the subject's stage of progress in weaning food as being in the late weaning stage based on the chewing pattern identified, in which the lips and jaw move sideways.

In S40, the evaluation and determination unit 103 determines that the subject's progress in weaning food is at the weaning completion stage based on the fact that the chewing pattern is characterized by a rounded chewing cycle of the lower jaw.

In S41, the evaluation and assessment unit 103 stores the assessment result indicating one of the progress stages assessed for the subject as result information 113 in the memory unit 11.

In S42, the output control unit 105 selects or generates FB information 114 corresponding to the judgment result indicated by the result information 113, and outputs it to the output device. For example, the output control unit 105 creates a display screen including the FB information 114 and displays it on the display device 2.

<Effects>
According to the above-described configuration and method, the image capture device 3 monitors an infant eating baby food, and the information processing device 1 uses the data obtained from the monitoring to identify the infant's chewing behavior and determine the infant's progress in baby food from the identified behavior. Therefore, the infant's progress in baby food can be accurately determined based on the object of the infant's chewing. Furthermore, the determination results can be presented to the server or a user other than the server.

In the past, the only option was to refer to the weaning stage, which was indicated as a guideline based on the baby's age and developmental stage related to feeding function. However, the evaluation system 100 and evaluation method according to this embodiment make it possible for servers and other users to recognize the progress stage of an infant's weaning food, determined based on the infant's chewing movements. The presented FB information indicating the progress stage is extremely useful information that helps the user decide whether or not to advance the baby's weaning food to the next stage.

[Embodiment 3]
The evaluation system 100 of the present disclosure can be configured to evaluate at least one of how an infant eats baby food and how a server who provides the baby food feeds the infant, and to determine the infant's progress in baby food.

In this case, the detection unit 102 of the information processing device 1 is configured to detect eating behavior and generate pattern information of the eating behavior (e.g., the graphs shown in Figures 3 and 4) as described in embodiment 1, and to detect chewing behavior and analyze the chewing behavior as described in embodiment 2. The detection unit 102 generates, for example, the index values and graph shown in Figure 7 as analysis result information of the chewing behavior. The detection unit 102 stores the pattern information of the eating behavior and the analysis result information of the chewing behavior in the memory unit 11 as behavior information 112.

The evaluation and judgment unit 103 executes the evaluation and judgment steps shown in Figure 6 based on the pattern information of eating behavior, and evaluates the appropriateness of the way the infant eats the baby food or the way the waiter feeds the infant the baby food. Furthermore, the evaluation and judgment unit 103 executes the evaluation and judgment steps shown in Figure 9 based on the analysis result information of chewing behavior, and judges the progress stage of the infant's baby food.

The evaluation and judgment unit 103 may use information on the analysis results of chewing movements to improve the accuracy of evaluating the appropriateness of the eating or feeding method. The evaluation and judgment unit 103 may also use information on the pattern of eating movements to improve the accuracy of judging the progress stage of baby food. For example, the evaluation and judgment unit 103 may identify the subject's chewing pattern after excluding the opening and closing of the mouth during the eating movement period identified based on the pattern of color change in the inner lip area from the chewing movements used to judge the progress stage of baby food.

The output control unit 105 causes the display device 2 to display the FB information 114 shown in FIG. 5 and the FB information 114 shown in FIG. 8. The output control unit 105 may configure the display screen so that each piece of FB information 114 is displayed on a single screen, or may configure the display screen so that two display screens displaying each piece of FB information 114 can be switched between.

[Embodiment 4]
The evaluation or judgment process performed by the evaluation and judgment unit 103 of the information processing device 1 may be performed by AI (Artificial Intelligence).

In this case, the evaluation and judgment unit 103 is configured to use data obtained by monitoring an infant eating baby food as explanatory variables, and the evaluation results of how the infant eats or is fed baby food, or the judgment results of the infant's baby food progress stage as objective variables, and to perform evaluation or judgment using an evaluation model that has learned the relationship between the explanatory variables and the objective variables.

As an example, a first evaluation model may be constructed that learns the relationship between the explanatory variables and the objective variables, with data obtained by monitoring an infant eating baby food as the explanatory variables and the evaluation results of how the infant eats or is fed the baby food as the objective variables. The evaluation determination unit 103 is configured to input the data obtained by monitoring into the first evaluation model and evaluate how the infant eats or is fed the baby food by obtaining output data output from the first evaluation model.

The data obtained by monitoring an infant eating baby food may be, for example, video or still images obtained by capturing images of the infant eating baby food using the imaging device 3. Alternatively, the data may be pattern information of feeding behavior (behavior information 112) obtained by the detection unit 102 analyzing the video or still images. Furthermore, the output data output from the first evaluation model may be an evaluation result indicating whether the feeding method is appropriate or inappropriate.

In another example, a second evaluation model may be constructed that learns the relationship between the explanatory variables and the objective variables, with data obtained by monitoring an infant eating baby food as the explanatory variables and the outcome of the assessment of the infant's baby food progress stage as the objective variable. The evaluation and assessment unit 103 is configured to input the data obtained by monitoring into the second evaluation model and determine the baby food progress stage by obtaining output data output from the second evaluation model.

The data obtained by monitoring an infant eating baby food may be, for example, video or still images obtained by capturing images of the infant eating baby food using the imaging device 3. Alternatively, the data may be analysis result information (movement information 112) of chewing movements obtained by the detection unit 102 analyzing the video or still images. Furthermore, the output data output from the second evaluation model may be a judgment result indicating one of the baby food progress stages applicable to the infant.

[Modification]
(Another example of a monitoring device)
The monitoring device for monitoring an infant eating baby food is not limited to the imaging device 3. The monitoring device may be a position sensor for determining the position of the infant's head, an electromyograph for measuring the amount of muscle activity of the infant's masticatory muscles, or a combination of these.

(Another example of Facebook information)
If there is an action that was not observed in S14 of Figure 6, the evaluation judgment unit 103 of the information processing device 1 of embodiment 1 may generate an evaluation result in S16 that includes information indicating the action that was not observed.

The output control unit 105 may cause the display device 2 to display FB information 114 indicating what movements were not observed. For example, if a movement of moving the face forward or a movement of tilting the head forward was not observed, the output control unit 105 may cause the display device 2 to display, as FB information 114, a sentence explaining that these movements were not observed.

Furthermore, the evaluation determination unit 103 may classify the way of eating or feeding baby food into several types depending on the type of observed or unobserved behavior. For example, the evaluation determination unit 103 may classify into types such as "pressing the spoon against the upper lip or upper jaw," "putting the spoon too far back in the mouth," and "tilting the head back."

The output control unit 105 may select a message indicating points for improvement in the way of eating or feeding, depending on the type classified by the evaluation determination unit 103, and display this on the display device 2 as Facebook information 114. For example, if the type is classified as "pressing the spoon against the upper lip or upper jaw," the output control unit 105 may select a message such as "place the spoon lightly on the lower lip and wait until the upper lip takes the food in," and display this on the display device 2 as Facebook information 114.

Alternatively, instead of or in addition to a message, the output control unit 105 may cause the display device 2 to display a video or animation showing points for improvement as Facebook information 114.

In embodiment 1, if the detection unit 102 does not detect from the video image the subject's eating behavior of putting baby food into their mouth, the evaluation determination unit 103 may determine that the appropriateness of the eating or feeding method cannot be evaluated. In this case, the output control unit 105 may cause the output device to output, as FB information 114, an error message to the effect that no eating behavior was detected to evaluate the appropriateness of the eating or feeding method.

In embodiments 1 to 3, the detection unit 102 may analyze the time-series positions of both the left and right corners of the subject's mouth or the path of jaw movement in moving images. If the analysis result information indicates a bias in the amount of movement of the motor organs on the left and right sides of the face during chewing, the evaluation and determination unit 103 may identify a chewing pattern that indicates an oral bad habit of biased chewing. In this case, the output control unit 105 may cause the output device to output a warning message indicating suspected biased chewing as FB information 114.

[Additional Notes]
Some or all of the above-described embodiments can also be described as follows, although the present disclosure is not limited to the aspects described below.

(Appendix 1)
An information processing device (1) according to one aspect of the present disclosure includes an image acquisition unit (101) that acquires video images of an infant eating baby food, a detection unit (102) that detects, from the video images, the feeding behavior of the infant when taking in baby food or the chewing behavior of the infant until the baby food taken in the mouth is swallowed, an evaluation and judgment unit (103) that evaluates or judges the way the infant eats or is being fed based on the analysis results of the detected eating behavior or chewing behavior, and an output control unit (105) that outputs feedback information generated based on the results of the evaluation or judgment.

With the above configuration, infants and young children eating baby food are monitored, and the data obtained from the monitoring is used to evaluate how the infant eats or is fed the baby food, making it possible to objectively evaluate how the infant eats or is fed the baby food.

(Appendix 2)
In the information processing device of Appendix 1, the detection unit detects the feeding behavior based on changes over time in the head and mouth movements of an infant eating baby food, and generates pattern information indicating a pattern of the feeding behavior by quantifying the changes over time in the head and mouth movements of the infant during the feeding behavior, and the evaluation and determination unit may evaluate the way the infant eats or is fed based on the pattern information.

The method used to feed baby food is reflected in the pattern of changes over time in the head and mouth movements of the infant eating the food. In other words, when baby food is fed using the appropriate method, the head and mouth movements will change over time in a predetermined pattern, and if the feeding method is not appropriate, the changes over time in the head and mouth movements will differ from the predetermined pattern. Therefore, with the above configuration, it is possible to appropriately evaluate the method of feeding baby food.

(Appendix 3)
In the information processing device of Supplementary Note 1 or Supplementary Note 2, the detection unit may detect the chewing action based on changes over time in the positions of each feature point on the face of an infant eating baby food, and analyze the magnitude of vertical and horizontal movement of the infant's mouth during the chewing action, as well as the chewing cycle, and the evaluation and determination unit may determine the progress stage of the infant's baby food based on at least one of the magnitude of the movement and the chewing cycle.

Baby food is introduced in stages as the infant's physical growth and feeding function improves, and this progress is reflected in the manner of chewing movements. Therefore, the above configuration makes it possible to appropriately determine the progress of the baby's baby food.

(Appendix 4)
A control method for an information processing device (1) according to one aspect of the present disclosure includes an image acquisition step (S11, S31) of acquiring a video image of an infant eating baby food, a detection step (S12, S32) of detecting from the video image the feeding action of the infant when taking in the baby food or the chewing action of the infant until swallowing the baby food that has been taken into the mouth, an evaluation and judgment step (S14 to S17, S34 to S41) of evaluating or judging the way the infant eats or is being fed based on the analysis results of the detected feeding action or chewing action, and an output control step (S18, S42) of outputting feedback information generated based on the results of the evaluation or judgment.

(Appendix 5)
A control program for causing a computer to function as the information processing device described in Appendix 1, which causes a computer to function as the image acquisition unit, the detection unit, the evaluation and judgment unit, and the output control unit, is also included in the technical scope of the present disclosure.

〔effect〕
The configuration of the present disclosure can promote the development of motor organs related to oral function in growing infants and enable early detection of bad oral habits. Chewing food, swallowing behavior, saliva secretion, and other factors have a significant impact on the brain and the entire body, and are factors that significantly affect physical and mental health and healthy lifespan. Therefore, the configuration of the present disclosure is expected to contribute to maintaining the physical and mental health of infants and young children, promoting their development, and preventing oral dysfunction caused by bad oral habits. Ultimately, this will contribute to achieving Goal 3 of the Sustainable Development Goals (SDGs), "Good Health and Well-Being."

[Software implementation example]
The functions of the evaluation system 100 or the information processing device 1 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 10).

In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in each of the above embodiments are realized by executing the program using this control device and storage device.

The above program may be stored non-transitory on one or more computer-readable storage media. These storage media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

Furthermore, some or all of the functions of each of the above control blocks can be realized by logic circuits. For example, integrated circuits incorporating logic circuits that function as each of the above control blocks are also included in the scope of the present invention. In addition, the functions of each of the above control blocks can also be realized by, for example, a quantum computer.

Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI may run on the control device, or on another device (for example, an edge computer or a cloud server).

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

1 Information processing device 2 Display device (output device)
3. Imaging equipment (monitoring equipment)
10 control unit (control device)
11 Storage unit 100 Evaluation system 101 Image acquisition unit 102 Detection unit 103 Evaluation determination unit 105 Output control unit

Claims (12)

a monitoring device for monitoring an infant eating baby food;
an information processing device that uses the data obtained by the monitoring to evaluate or judge how the infant eats or is fed;
an output device that outputs feedback information generated based on the result of the evaluation or judgment ,
The information processing device identifies the pattern of change over time in the head and mouth movements of the infant while eating baby food, and evaluates the way in which the infant is fed baby food based on the difference between the identified pattern and the pattern of change over time in the movements when the infant is fed in an appropriate manner . The information processing device is configured to:
When the baby opens and closes its mouth to take in baby food, if the head moves forward and then rotates downward, the feeding method is evaluated as appropriate.
The evaluation system according to claim 1 , wherein if the change is not observed, the feeding method is evaluated as being inappropriate. the monitoring device is an imaging device that images the infant eating baby food,
the information processing device analyzes the captured video and evaluates or judges how the infant eats or is being fed,
The information processing device includes:
The evaluation system according to claim 2 , wherein the opening and closing movements of the mouth are identified based on a color change in an inner region of the baby's lips included in the moving image. The evaluation system according to claim 1 , wherein the information processing device identifies a chewing behavior pattern of the infant and determines a progress stage of the infant's weaning food from the identified behavior pattern. a monitoring device for monitoring an infant eating baby food;
an information processing device that uses the data obtained by the monitoring to evaluate or judge how the infant eats or is fed;
an output device that outputs feedback information generated based on the result of the evaluation or judgment ,
the information processing device identifies a chewing behavior of the infant and determines a progress stage of the infant's weaning food from the identified behavior;
the monitoring device is an imaging device that images the infant eating baby food,
the information processing device analyzes the captured video and evaluates or judges how the infant eats or is being fed,
The information processing device includes:
An evaluation system that monitors the distance between the apex of the infant's upper lip and the apex of the infant's lower lip contained in the video, and excludes from the chewing action a series of mouth opening and closing actions that includes a mouth opening action in which the distance is equal to or greater than a predetermined threshold. The information processing device includes:
monitoring the distance of the infant during the chewing activity;
The evaluation system according to claim 5 , wherein, when the shortest distance among the distances is equal to or greater than a predetermined threshold, a state in which the inner sides of the upper and lower lips remain open during mastication is identified. The evaluation system according to claim 4 , wherein the information processing device determines the progress stage of the baby's weaning food by taking into account at least one of the baby's age in months and the state of tooth eruption. The information processing device includes:
8. The evaluation system according to claim 4 , wherein the feedback information indicates whether or not the baby's weaning food progress stage predicted in advance by a user is different from the baby's weaning food progress stage determined from the identified chewing behavior. The information processing device includes:
9. The evaluation system according to claim 1, wherein data obtained by monitoring an infant eating baby food is used as an explanatory variable, and the evaluation results of how the infant eats or is fed the baby food, or the judgment results of the infant's baby food progress stage, are used as a dependent variable, and the evaluation or judgment is performed using an evaluation model that has learned the relationship between the explanatory variables and the dependent variable. a monitoring device for monitoring an infant eating baby food;
a control device that uses the data obtained by the monitoring to evaluate or judge how the infant eats or is fed;
an output device that outputs feedback information generated based on the result of the evaluation or judgment ,
The control device is an information processing device that identifies patterns of change over time in the head and mouth movements of the infant while it is eating baby food, and evaluates how the infant is being fed baby food based on the difference between the identified patterns and the patterns of change over time in the movements when the infant is being fed in an appropriate manner . An evaluation method executed by an evaluation system including a monitoring device, an information processing device, and an output device,
a monitoring step in which the monitoring device monitors an infant eating baby food;
an evaluation and judgment step in which the information processing device evaluates or judges how the infant eats or is fed using the data obtained by the monitoring;
an output step in which the output device outputs feedback information generated based on a result of the evaluation or judgment ,
In the evaluation and judgment step, the information processing device identifies a pattern of change over time in the head and mouth movements of the infant while eating baby food, and evaluates the way in which the infant is fed baby food based on the difference between the identified pattern and the pattern of change over time in the movements when the infant is fed in an appropriate manner . A control program for causing a computer to function as the information processing device according to claim 1 .