JP7650874B2 - Developmental stage determination device, program and terminal device - Google Patents

Description

The present invention relates to a development stage determination device , a program , and a terminal device.
This application claims priority based on Japanese Patent Application No. 2020-085752, filed on May 15, 2020, the contents of which are incorporated herein by reference.

Oral functions such as eating, speaking, and breathing decline with age. As a decline in oral function can lead to aspiration pneumonia, for example, preventing the decline in oral function is a major challenge in extending healthy life expectancy.
It is known that the degree of decline in oral function is greatly influenced by the state of oral function acquisition during infancy and school-age. In particular, for the development of oral function in infants and young children, it is important to pay attention to the preparation and feeding of baby food according to the level of development.
Therefore, it is necessary for parents to correctly recognize the degree of oral function development of infants and young children, and a method for objectively judging the degree of oral function development is required. In order to objectively judge the degree of oral function development, it is preferable that the judgment be automated.
Conventionally, techniques for automatically determining oral cavity function have been disclosed, for example, in Patent Document 1.

JP 2004-242902 A

However, the technology shown in Patent Document 1 can only obtain limited information, such as the number of chewing sessions, and is not necessarily able to objectively assess the degree of development of oral function during infancy.

One embodiment of the present invention is a developmental stage determination device that includes a motion information acquisition unit that acquires motion information indicating an infant's oral intake of food, a developmental stage determination unit that determines, based on the motion information acquired by the motion information acquisition unit and predetermined determination criteria, which of multiple developmental stages of oral function in infancy the motion indicated by the motion information corresponds to, and a display control unit that causes a display device to display information indicating the developmental stage determined by the developmental stage determination unit.

In one embodiment of the developmental stage determination device, the specified determination criteria include reference movements of at least one part of the infant's lips, corners of the mouth, tongue, and jaw according to the developmental stage.

In one embodiment of the development stage determination device, the movement information includes information on the movement of at least one of the lips, corners of the mouth, tongue, and jaw during at least a portion of the period from when food is consumed to when it is swallowed.

In one embodiment of the development stage determination device, the movement information includes information on the movement of at least one of the lips, corners of the mouth, tongue, and jaw during the period before food is consumed.

In one embodiment of the developmental stage determination device, the display control unit further causes a display device to display information corresponding to the developmental stage determined by the developmental stage determination unit.

In one embodiment of the developmental stage assessment device, the display control unit graphs information corresponding to the developmental stage assessed by the developmental stage assessment unit together with a history of past assessment results, and displays the graph on a display device.

In one embodiment of the developmental stage determination device of the present invention, the display control unit causes the display device to display information selected from a plurality of pieces of information corresponding to the developmental stage as updated information corresponding to the developmental stage until the developmental stage determined by the developmental stage determination unit is updated.

One embodiment of the developmental stage determination device of the present invention further includes an operation receiving unit that receives an update operation to update the information to be displayed, and the display control unit updates the information corresponding to the developmental stage in response to the operation receiving unit receiving the update operation, and displays the updated information on the display device.

In one embodiment of the development stage determination device, the display control unit displays information on a display device encouraging the provision of the operation information based on the frequency with which the operation information is provided to the operation information acquisition unit and the frequency with which the update operation is received by the operation receiving unit.

One embodiment of the developmental stage assessment device of the present invention further comprises a food information acquisition unit that acquires food information indicating the type of food given to an infant, and a type determination unit that determines whether the type of food indicated by the food information acquired by the food information acquisition unit is suitable as a food to be ingested at the developmental stage assessed by the developmental stage assessment unit, and the display control unit further causes the result of the assessment made by the type determination unit to be displayed on a display device.

One embodiment of the developmental stage assessment device further includes a portion determination unit that determines the amount of food given to the infant based on the movement information acquired by the movement information acquisition unit, and the display control unit further causes the display device to display information based on the determination result of the portion determination unit.

One embodiment of the present invention is a developmental stage determination method having a motion information acquisition step in which a computer acquires motion information indicating an infant's oral intake of food, a developmental stage determination step in which a motion indicated by the motion information corresponds to one of a number of developmental stages of oral function in infancy based on the motion information acquired in the motion information acquisition step and a predetermined determination criterion, and a display control step in which a display device displays information indicating the developmental stage determined in the developmental stage determination step.

One embodiment of the present invention is a program for causing a computer to execute a motion information acquisition procedure for acquiring motion information indicating an infant's oral intake of food, a developmental stage determination procedure for determining which of multiple developmental stages of oral function in infancy the motion indicated by the motion information corresponds to based on the motion information acquired in the motion information acquisition procedure and predetermined determination criteria, and a display control procedure for displaying on a display device information indicating the developmental stage determined in the developmental stage determination procedure.

One embodiment of the present invention is a terminal device that includes an imaging unit that images an infant's oral intake of food, an output unit that outputs movement information to a developmental stage determination device that determines, based on movement information based on an image captured by the imaging unit and predetermined determination criteria, which of multiple developmental stages of oral function in infancy the movement indicated by the movement information corresponds to, an acquisition unit that acquires information indicating the developmental stage determined by the developmental stage determination device based on the movement information output by the output unit, and a display control unit that causes a display device to display the information indicating the developmental stage acquired by the acquisition unit.

According to the present invention, it is possible to provide a development stage determination device , a program , and a terminal device that can objectively determine the degree of development of oral function during infancy.

FIG. 2 is a diagram illustrating an example of a functional configuration of the development stage determination system according to the present embodiment. FIG. 4 is a diagram showing a first example of criteria information according to the present embodiment. FIG. 11 is a diagram illustrating a second example of the criteria information according to the present embodiment. FIG. 11 is a diagram illustrating a third example of the criteria information of the present embodiment. FIG. 11 is a diagram showing a fourth example of the criteria information of the present embodiment. FIG. 13 is a diagram showing a fifth example of the criteria information of the present embodiment. FIG. 2 is a diagram showing an example of the operation of the development stage determination system of the present embodiment. FIG. 2 is a diagram showing an example of baby food information according to the present embodiment. FIG. 2 is a diagram showing an example of baby food feeding method information according to the present embodiment. 13 is an example of information displayed on a terminal device according to the present embodiment. 13 is another example of information displayed on the terminal device of the present embodiment.

Hereinafter, the development stage determination system 1 of the present embodiment will be described with reference to the drawings.
FIG. 1 is a diagram showing an example of the functional configuration of a development stage determination system 1 according to the present embodiment.
The developmental stage assessment system 1 includes a developmental stage assessment device 10 and a terminal device 20. The developmental stage assessment device 10 and the terminal device 20 include a network 30, the developmental stage assessment device 10, and the terminal device 20. The developmental stage assessment device 10 and the terminal device 20 are connected to each other via the network 30.
The developmental stage determination device 10 is configured, for example, by a server device. The terminal device 20 is configured, for example, by a smartphone or a tablet.

[Functional configuration of terminal device 20]
The terminal device 20 includes an imaging unit 21, a display unit 22, and an operation detection unit 23. The imaging unit 21 captures an image based on an operation by a user of the terminal device 20. The display unit 22 includes, for example, a liquid crystal display, and displays an image captured by the imaging unit 21 and information transmitted from the development stage assessment device 10 via the network 30. The operation detection unit 23 includes, for example, a touch panel, and detects an operation by the user of the terminal device 20.
The terminal device 20 transmits information such as images captured by the imaging unit 21 to the development stage determination device 10 via the network 30 .
As described above, information such as images and text is communicated between the development stage assessment device 10 and the terminal device 20 via the network 30, but in the following explanation, the phrase "via the network 30" will be omitted.

In one example of this embodiment, the terminal device 20 transmits motion information of the subject 40 (e.g., an infant) to the developmental stage determination device 10. One example of the motion information is an image of the mouth of the subject 40 while eating.
The user of the terminal device 20 performs an imaging operation by adjusting the position and direction of the terminal device 20 so that the mouth of the baby during a meal is included within the imaging angle of view of the imaging unit 21. As a result, the movement of the mouth of the baby during a meal is included in the video image captured by the imaging unit 21. That is, in this example, the movement information is information indicating the movement of the baby orally taking food.

The movement information preferably includes movements of the baby's lips, corners of the mouth, tongue and jaw while eating.
In addition, the motion information preferably includes information on the movement of at least one of the lips, corners of the mouth, tongue, and jaw during at least a portion of the period from when food is eaten to when it is swallowed.
The motion information preferably also includes information on the movement of at least one of the lips, corners of the mouth, tongue, and jaw during the period before the food is eaten.
That is, it is preferable that the movement information includes more detailed information about the mouth movements of the infant while eating. If the movement information includes more detailed information about the mouth movements of the infant while eating, the accuracy of determining the development stage can be further improved.

Note that the movement information does not necessarily have to be a video (e.g., a series of images in which multiple frame images captured at different times are arranged in chronological order) as long as it includes information that can determine the movement of the lips, corners of the mouth, tongue, and jaw. For example, the movement information may be vector information extracted from the movement of a subject in a video.

[Functional configuration of the development stage determination device 10]
The development stage determination device 10 includes a motion information acquisition unit 110, a memory unit 120, a development stage determination unit 130, a display control unit 140, an operation reception unit 150, a food information acquisition unit 160, a type determination unit 170, and a portion determination unit 180.
Here, we will explain the operation information acquisition unit 110, the memory unit 120, the development stage determination unit 130, the display control unit 140, and the operation reception unit 150, and the food information acquisition unit 160, the type determination unit 170, and the portion determination unit 180 will be explained later.

The motion information acquiring unit 110 acquires the motion information transmitted by the terminal device 20. As an example, the terminal device 20 includes an output unit (not shown) that transmits the motion information. In this case, the motion information acquiring unit 110 acquires the motion information transmitted from the output unit.
The motion information includes information indicating the motion of the infant taking in food orally, that is, the motion information acquiring unit 110 acquires the motion information indicating the motion of the infant taking in food orally.
The motion information acquisition section 110 outputs the acquired motion information to the development stage determination section 130 .

The memory unit 120 stores judgment criteria information 121. An example of the judgment criteria information 121 will be described with reference to Figures 2 to 6.

2 to 6 are diagrams showing an example of the criterion information 121 of this embodiment. The criterion information 121 is information in which the developmental stage of an infant is associated with a criterion for determining the developmental stage. In this example, the developmental stages of an infant are five stages, from stage 1 to stage 5.
For example, the developmental stage "Stage 1" is associated with the following criteria: "frequent protrusion of tongue, sucking of finger when brought close to tongue (primitive reflex), thumb sucking, licking toys, and beginning to hold up the head."
That is, the storage unit 120 stores predetermined criteria for determining the developmental stage of an infant.

It is preferable that the judgment criteria information 121 includes reference movements of at least one of the infant's lips, corners of the mouth, tongue, and jaw according to the developmental stage.

Here, an example of the judgment criteria stored as the judgment criteria information 121 will be described.
(1) Throughout each development stage, the important parts in the reference movements of the judgment criteria are "tongue, lips, corners of the mouth, and chin." In this case, the predetermined judgment criteria includes the reference movement of at least one of the above parts according to the development stage.

(2) At each developmental stage, the above areas are involved as follows:
- All stages (stages 1 to 5): They all have in common that the important part in the evaluation criteria is the "tongue."
- Stages 2 to 4: The important areas in the evaluation criteria are the lips, corners of the mouth, and chin.
-Stage 4 only: Evaluation criteria include the use of the teeth and the movement of the cheeks.
- Stages 1 and 5: The criteria for assessment include how the hands are used.
- Stages 2 to 5: The criteria for assessment include "neck and head" movement.
In light of the above, the following can be considered:
In the judgment of the first stage, it is preferable to use "tongue" information as an essential judgment criterion. More preferably, it is desirable to use "tongue" information as an essential judgment criterion in the judgment of each development stage.
In the second to fourth stages, it is preferable to further add information on the "lips and chin." It is even more preferable to further add information on the "corners of the mouth." In this case, it is particularly preferable to add it in the fourth stage.
In the fourth stage, it is preferable to further add auxiliary information on "teeth and cheeks."
In the first and fifth stages, it is preferable to further add auxiliary information about the "hands."
In the second to fifth stages, it is preferable to further add supplementary information on the "neck and head."

(3) The specific changes in movement of each part across each developmental stage are as follows:
-Tongue (Stage 1 to Stage 5): ``When food is brought close to the mouth, the tongue frequently protrudes from the tip of the mouth'' → ``The above movement is suppressed'' → ``The tongue moves up and down'' → ``The tongue moves up and down and side to side'' → ``The tongue moves freely.''
-Lips (stages 2 to 4): "The lower lip goes under the upper lip" → "This movement is suppressed, and the upper and lower lips move symmetrically" → "The upper and lower lips move asymmetrically."
- Corners of the mouth (stage 2 → stage 4): "The corners of the mouth do not move" → "They expand and contract symmetrically on the left and right sides at the same time" → "They move asymmetrically."
・Jaw (stages 2 to 4): "The jaw moves slightly up and down" → "Moves up and down" → "Moves up and down and side to side."
- How to use teeth (Stage 4 to Stage 5): "Taking in food by biting it with the front teeth" → "Being able to bite off large foods with the front teeth."
・Cheeks (stage 4 to stage 5): "Puff out your cheeks and chew" → "Puff out your cheeks and chew rhythmically."
- Hands: "Thumb sucking (Stage 1)", "Able to eat with fingers, able to use utensils to eat by oneself (Stage 5)".
- Neck and head (Stages 2 to 5): "Neck is strong and neck and head do not move frequently" → "Head juts out slightly from neck toward food" → "Head juts out significantly from neck toward food" → "Neck and head can move freely and maintain correct posture."

In other words, the judgment criteria stored as judgment criteria information 121 may include additional items according to the developmental stage of the subject to be judged, and relative priorities between each item.

Furthermore, the judgment criteria information 121 may include information on the "flow in which the judgment criteria action occurs" and the "actions that have a high priority in determining the developmental stage." Specific examples of the above-mentioned "judgment criteria," "flow in which the judgment criteria action occurs," and "actions that have a high priority in determining the developmental stage" will be described below.

(1) First stage (suckling stage)
A. Judgment Criteria:
Frequent protrusion of the tongue.
- When you bring your finger close to the tongue, it will stick to it (primitive reflex).
・Thumb sucking.
・Licking toys.
・The neck begins to stiffen.
B. The sequence in which the action of the criterion occurs:
1. When a spoon or other object is brought close to the mouth, the tongue frequently sticks out from the mouth.
When you bring your finger close to your tongue it sticks to it.
Sucking fingers and licking toys.
C. Priority actions for determining developmental stage:
- When bringing an object such as a spoon close to the mouth, the tongue frequently protrudes from the mouth.

(2) Stage 2 (Early weaning (gulping stage))
A. Judgment Criteria:
The lower lip tucks under the upper lip.
The corners of the mouth barely move.
- The tongue moves back and forth, sending food to the back of the throat.
The jaw moves slightly up and down.
-Close your lips and swallow.
-Taking food with the upper lip
- Moving their mouth when they see food (showing interest in food).
-Opens mouth when food is brought close to mouth.
・The neck is stable.
-Neck and head do not move frequently.
- The frequency of the behavior of "sticking out the tongue when bringing an object such as a spoon close to the mouth" decreases.
- Even if you bring a spoon or other object close to your mouth, you will no longer push it away with your tongue.
B. The sequence in which the action of the criterion occurs:
1. The frequency of the behavior of "sticking out your tongue when bringing a spoon or other object close to your mouth" decreases.
Mouth movements when seeing food (indicating interest in food).
The head is stable.
The neck and head do not move frequently.
2. Open your mouth when food is brought close to it.
The tongue moves back and forth, pushing food to the back of the throat with the tongue, and the corners of the mouth barely move.
3. Even if you bring a spoon or other object close to your mouth, you will no longer push it away with your tongue.
The upper lip takes in food, the lower lip goes under the upper lip, and the jaw moves slightly up and down.
Close your lips and swallow.
C. Priority actions for determining developmental stage:
The lower lip tucks under the upper lip.
-Opens mouth when food is brought close to mouth.
-Close your lips and swallow.
The jaw moves slightly up and down.

(3) Third stage (mid-weaning (munching stage))
A. Judgment Criteria:
-Keep the upper and lower lips tightly closed while crushing and swallowing.
- The corners of the mouth expand and contract horizontally at the same time.
-The tongue moves up and down and presses against the roof of the mouth.
- Move your jaw up and down to crush food.
-Close your lips and chew for 2-3 seconds before swallowing.
-Positioning head slightly towards food.
B. The sequence in which the action of the criterion occurs:
1. Close your lips and chew for 2-3 seconds before swallowing.
The tongue moves up and down, pressing against the roof of the mouth.
The corners of the mouth expand and contract horizontally at the same time.
2. The head is slightly jutting out from the neck towards the food.
Move your jaw up and down to crush food.
The upper and lower lips are kept tightly closed while crushing and swallowing.
C. Priority actions for determining developmental stage:
-Keep the upper and lower lips tightly closed while crushing and swallowing.
-Close your lips and chew for 2-3 seconds before swallowing.
- Move your jaw up and down to crush food.

(4) Stage 4 (late weaning (chewing stage))
A. Judgment Criteria:
-The upper and lower lips move with an asymmetrical twist.
- The corners of the mouth move asymmetrically (pulled to one side).
・The tongue moves up, down and side to side.
The jaw moves up and down and side to side.
- Food is taken in by biting it with the front teeth.
-Puff out your cheeks and chew.
-Posing their head far out from their neck toward the food.
- You can take food in firmly with your lips.
- Grinding hard foods with the gums.
B. The sequence in which the action of the criterion occurs:
1. They stick their head out from their neck toward the food.
The lips can hold food firmly.
The tongue moves up and down and left and right, and the cheeks are puffed out while chewing.
2. The upper and lower lips move with asymmetrical twisting, and the corners of the mouth move asymmetrically.
The jaw moves up and down and side to side.
3. Take in food by biting it off with your front teeth.
The gums make a grinding motion to crush hard foods.
C. Priority actions for determining developmental stage:
- The corners of the mouth move asymmetrically.
-The upper and lower lips move with an asymmetrical twist.
-Puff out your cheeks and chew.
- Food is taken in by biting it with the front teeth.
The jaw moves up and down and side to side.

(5) 5th stage (weaning completion stage)
A. Judgment Criteria:
The tongue moves freely.
- You can chew and crush rhythmically with your teeth and gums.
- Your neck and head can move freely, allowing you to maintain proper posture.
- Able to grasp and release food with fingers.
・Be able to hold toys and other items in your hands and bring them to your mouth.
・You can eat it with your hands.
-Can bite off large foods with front teeth.
-Can bring hand to mouth without moving head.
・Can eat using tableware by themselves.
B. The sequence in which the action of the criterion occurs:
1. The tongue can move freely.
You can chew and crush rhythmically with your teeth and gums.
2. The neck and head can move freely and maintain correct posture.
Able to grasp and release food with fingers.
They will be able to hold toys and other items in their hands and bring them to their mouths.
3. You can eat it with your fingers.
It can bite off large pieces of food with its front teeth.
Can bring hand to mouth without moving head.
4. Able to eat using tableware by themselves.
C. Priority actions for determining developmental stage:
・You can eat it with your hands.
・Can eat using tableware by themselves.
- You can chew and crush rhythmically with your teeth and gums.

Returning to Figure 1, the developmental stage determination unit 130 determines which of the multiple developmental stages of oral function in infancy the movement indicated by the movement information corresponds to, based on one or more of the movement information acquired by the movement information acquisition unit 110 or the determination criteria information 121 (predetermined determination criteria).
A known image processing algorithm is applied to the determination procedure performed by the development stage determination unit 130. The image algorithm referred to here broadly includes algorithms for processing still images and algorithms for processing moving images.
The developmental stage determination unit 130 may determine the developmental stage by referring to the results of machine learning of the movements of the baby's lips, corners of the mouth, tongue, and jaw.
The developmental stage determination section 130 outputs the result of the developmental stage determination to the display control section 140 .

The developmental stage determination unit 130 may also determine the developmental stage while quantifying the oral function of the infant from the motion indicated by the motion information based on one or more of the motion information or the determination criteria information acquired by the motion information acquisition unit 110. In that case, the developmental stage may be determined while indicating the desired numerical value for oral function for each month of age together with the average value and standard deviation. A graph may also be created from the numerical value of oral function and output to the display control unit 140. This graph may also indicate information such as the average value and standard deviation of the numerical value of oral function together with the history. This configuration allows the developmental process to be grasped in more detail, making it easier for the user to get a sense of the growth of the infant, who is the subject of the image. Furthermore, it is possible to motivate the user to continue using the system.

The developmental stage determination unit 130 may also determine the developmental stage based on a still image showing a characteristic moment among the infant's oral intake of food. In this case, the terminal device 20 may include a characteristic motion determination unit (not shown) that determines a moment that reflects a characteristic of a developmental stage among the infant's oral intake of food. When a moment that reflects a characteristic of a developmental stage appears during the imaging of the infant's oral intake of food by the imaging unit 21, this characteristic motion determination unit determines the moment as a characteristic motion. When the characteristic motion determination unit determines that a characteristic motion has appeared, it may cause the imaging unit 21 to capture a still image or present a display (or sound) that prompts the infant to capture a still image.
Furthermore, the terminal device 20 may display explanatory information instructing the timing for capturing a still image (for example, when the spoon is brought close, or 3 seconds after it is placed in the mouth, etc.).
By configuring in this manner, the developmental stage assessment system 1 can reduce the amount of information compared to transmitting a video (i.e., a long-term video) of an infant or young child orally ingesting food from the terminal device 20 to the assessment device 10.

The terminal device 20 may also transmit to the determination device 10 as movement information an image captured by the imaging unit 21 from which parts of the image not used in determining the developmental stage (for example, an image other than the baby's mouth, such as a background image) have been deleted. The terminal device 20 configured in this manner can reduce the amount of image information transmitted as movement information.

The display control unit 140 outputs information based on the determination result of the development stage determination unit 130 to the terminal device 20. As a result, the information based on the determination result of the development stage determination unit 130 is displayed on the display unit 22 of the terminal device 20.
That is, the display control unit 140 causes the terminal device 20 (i.e., the display device) to display information indicating the development stage determined by the development stage determination unit 130 .

The operation reception unit 150 receives operations performed on the terminal device 20. Specifically, when the operation detection unit 23 (e.g., a touch panel) of the terminal device 20 detects an operation on the display surface of the display unit 22, it outputs operation information corresponding to the detected operation to the development stage assessment device 10. The operation reception unit 150 receives the operation information output from the terminal device 20. The operation reception unit 150 outputs operation information to each part of the development stage assessment device 10 according to the content of the received operation. As a result, each part of the development stage assessment device 10 operates according to the operation performed on the terminal device 20.

[Operation of development stage determination system 1]
Next, an example of the operation of the development stage determination system 1 will be described with reference to FIG.
FIG. 7 is a diagram showing an example of the operation of the development stage determination system 1 of this embodiment.

(Step S10) The movement information acquisition unit 110 of the developmental stage assessment device 10 acquires movement information from the terminal device 20. As described above, in this example, the movement information is a video of the mouth of an infant eating a meal.

In order to avoid reducing the accuracy of determining the developmental stage, it is preferable for the user operating the terminal device 20 to capture an image of the infant's face from under the nose to the tip of the chin within the angle of view of the imaging unit 21. The display control unit 140 may cause the terminal device 20 to display on the display unit 22 an instruction such as "capture an image from under the nose to the tip of the chin", or may cause the display unit 22 to display guidelines indicating the external shape of the face and the contours of the nose and chin superimposed on the image captured by the imaging unit 21.

The display control unit 140 may also cause the display unit 22 to display an instruction regarding the timing of image capture.
For example, an instruction such as "take a picture from the time you bring the spoon to your mouth until you swallow it" may be displayed on the display unit 22.

In addition, in order to prevent a decrease in the accuracy of judging the developmental stage, it is preferable to make the length of the motion information longer. The display control unit 140 may cause the display unit 22 to display the image capturing time, the time during which the image capturing unit 21 captures the front of the baby's face, and the like.
The display control unit 140 may also present information (for example, a blinking flash or a sound) to attract the baby's attention so that the baby turns its face toward the imaging unit 21 during imaging.

Furthermore, the display control unit 140 may cause an image for inputting information on the growth state of the baby, such as age in months and weight, to be displayed on the display unit 22. In this case, the motion information acquiring unit 110 acquires information on the growth state together with the motion information.
Furthermore, the motion information acquiring section 110 may store the acquired information relating to the growth state in the storage section 120 as history information.

Furthermore, the display control unit 140 may change or add motion information to be acquired depending on the developmental stage of the infant.
Furthermore, the display control unit 140 may display information indicating the imaging timing and the part of the subject to be imaged (i.e., imaging guide information). When the imaging guide information is displayed, the display control unit 140 may display imaging guide information with different contents according to the developmental stage of the infant.

Furthermore, for example, when the important part in the judgment criteria is "hands", the display control unit 140 may display a display for prompting the user to input the hand movement (for example, a question display of "Will you use your hands?" and a response box for this question) instead of capturing an image of the hand movement. In other words, the display control unit 140 may display information other than the image captured by the imaging unit 21 as information to be included in the movement information.
The answer field displayed by the display control unit 140 may be in a text input format or in a selection format such as Yes/No.

(Step S20) The development stage determination unit 130 acquires the determination criteria information 121 stored in the storage unit 120.
(Step S30) The development stage determination unit 130 determines which stage the baby's eating movement imaged by the terminal device 20 corresponds to among the development stages of oral function during infancy.
Specifically, the developmental stage determination unit 130 searches the determination criteria information 121 obtained in step S20 using the movement information obtained in step S10 as a search key. The developmental stage determination unit 130 obtains the developmental stage corresponding to the movement information as a result of the search. The developmental stage determination unit 130 outputs the determination result of the developmental stage to the display control unit 140.

The developmental stage determination unit 130 may determine the developmental stage by performing statistical calculations on the motion information based on a plurality of motion information captured at different times. The developmental stage determination unit 130 configured in this manner can improve the accuracy of determining the developmental stage.

In addition, the accuracy of determining the developmental stage may decrease due to changes in the facial expression of an infant. In this case, when the movement information includes an expression in which the shape of the mouth differs from that of a normal expression (e.g., a crying face or a laughing face), the developmental stage determination unit 130 may determine the developmental stage by excluding the part of the expression in which the shape of the mouth differs from that of a normal expression.

In addition, depending on the content of the operation information, the developmental stage determination unit 130 may not be able to determine the developmental stage. The developmental stage determination unit 130 may calculate the accuracy of the developmental stage determination (e.g., stage 3 with a 60% probability, etc.). In this case, the display control unit 140 may cause the display unit 22 to display the calculated accuracy of the developmental stage determination.

The developmental stage assessment unit 130 may also set the number to 100 when the criteria for each developmental stage are met, and calculate the numerical value as a percentage (1 to 100, etc.). Vector information obtained by extracting the subject's movement from the video may also be quantified. The display control unit 140 may graph the calculated numerical value together with past assessment results (numerical values) as a numerical trend and display the graph on the display unit 22.

In addition, the developmental stage determination unit 130 may correct the determination result based on information such as the age (age in months) and number of teeth of the subject 40, i.e., the infant.

In addition, the developmental stage determination unit 130 may determine the developmental stage based on a history of past determination results. The past determination results may be stored in the storage unit 120 as history information. In this case, the developmental stage determination unit 130 determines the developmental stage further based on the history information stored in the storage unit 120.

(Step S40) The display control unit 140 acquires information corresponding to the developmental stage determined in step S30. Here, an example of "information corresponding to the developmental stage" is provided.

[Examples of information corresponding to developmental stages]
(1) Information on the Form of Baby Food The display control unit 140 acquires information on the form of baby food as information corresponding to a developmental stage. An example of the information on the form of baby food will be described with reference to FIG. 8.

8 is a diagram showing an example of the baby food information 122 of the present embodiment. In this example, the baby food information 122 is stored in the storage unit 120 of the development stage determination device 10.
The baby food information 122 associates a developmental stage with information on the type of baby food corresponding to the developmental stage. The display control unit 140 searches the baby food information 122 using the developmental stage determined in step S30 as a search key to obtain information on the type of baby food corresponding to the developmental stage (for example, "thick potage-like/yogurt-like" for "stage 2").

(2) Information on How to Feed Baby Food The display control unit 140 acquires information on how to feed baby food as information corresponding to a developmental stage. An example of the information on how to feed baby food will be described with reference to FIG. 9 .

9 is a diagram showing an example of the baby food feeding method information 123 of this embodiment. In this example, the baby food feeding method information 123 is stored in the storage unit 120 of the development stage determination device 10.
The baby food feeding method information 123 associates a developmental stage with information on how to feed baby food according to the developmental stage. The display control unit 140 searches the baby food feeding method information 123 using the developmental stage determined in step S30 as a search key to obtain information on how to feed baby food corresponding to the developmental stage (for example, in the "third stage,""Place the spoon on the lower lip and make sure that the baby takes in the food, munches, and swallows it before bringing in the next spoonful").

(Step S50) Returning to FIG. 7, the display control unit 140 outputs to the terminal device 20 information on the developmental stage determined in step S30 and information corresponding to the developmental stage obtained in step S40.

Figure 10 is an example of information displayed on the terminal device 20 of this embodiment. A judgment result image P11 is displayed on the display unit 22 of the terminal device 20. This judgment result image P11 is an image showing the developmental stage judged in step S30. If it is judged to be the "third stage" in step S30, the display unit 22 displays "third stage" (or the "munching stage", which is a simpler expression of the developmental stage) as the judgment result image P11.

Furthermore, on the display unit 22, operation images B11 to B14 may be displayed.
Operation image B11 is an operation image for displaying a "dietary advice screen."
When the operation image B11 is operated by a user of the terminal device 20, the terminal device 20 outputs operation information indicating that the operation image B11 has been operated to the development stage assessment device 10. The operation receiving unit 150 receives the operation information output by the terminal device 20 and outputs it to the display control unit 140. When the display control unit 140 determines that the operation image B11 "dietary advice screen" has been operated, it outputs the dietary advice screen to the terminal device 20 as information corresponding to the development stage. As a result, the "dietary advice screen" is displayed on the display unit 22 of the terminal device 20.
Here, the "dietary advice screen" is, for example, an image generated based on the information stored in the baby food feeding method information 123 described above.

Examples of information displayed on the dietary advice screen include:
1. How to feed a child (how to bring the spoon to the mouth, the infant's posture, the timing of each bite, etc.)
2. Food type and nutrition (the type of weaning food appropriate for that stage, and the amount of nutrition needed) 3. Advice on weaning food steps (if the weaning food you are giving does not match the developmental stage, we will advise you on the correct step. For example, "Take one step back,""Take one step forward,""Let's wait a while before resuming weaning food.")
etc.

In addition, if the subject 40's infant or toddler reaches a certain age but has not made any progress, the display control unit 140 may cause the terminal device 20 to display information on advice for promoting oral function development (e.g., advice on the type of baby food, how to feed it, training, etc.).

The operation image B12 is an operation image for displaying a "baby food recipe screen." When the operation image B12 is operated by a user of the terminal device 20, the display control unit 140 outputs the baby food recipe screen to the terminal device 20 as information corresponding to the developmental stage, in the same manner as described above. As a result, the "baby food recipe screen" is displayed on the display unit 22 of the terminal device 20.
Here, the "baby food recipe screen" is, for example, an image generated based on the information stored in the above-mentioned baby food information 122. Note that the "baby food recipe screen" may include, for example, videos showing each stage of cooking and still images of the completed baby food.

The operation image B13 is an operation image for displaying a "childcare diary (step-up calendar) screen." When the operation image B13 is operated by the user of the terminal device 20, the display control unit 140 outputs the step-up calendar screen to the terminal device 20 as information corresponding to the developmental stage, in the same manner as described above. As a result, the "step-up calendar screen" is displayed on the display unit 22 of the terminal device 20.
Here, the "step-up calendar screen" is a screen that displays the change in developmental stage over time, for example, by date, week, month, etc., and is a screen that allows the user to easily keep a record of growth, such as by displaying step-up anniversaries. Also, "step-up" refers to an increase in the developmental stage (for example, a change from the first stage to the second stage).

An example of the information displayed on the Step Up Calendar screen is:
Check the baby food step-up history:
- Videos of meals on each date - The actual baby food menu that was fed - Whether or not there were any allergic reactions to each ingredient

Furthermore, if a developmental stage declines (steps down, for example, from stage 3 to stage 2) after it has been determined that a step has been made up, the user may feel uneasy. When making a developmental stage determination multiple times, the developmental stage determination unit 130 may be configured not to cause a step down even if the developmental stage is determined to be low after a step up.
Furthermore, the developmental stage determination unit 130 may determine the developmental stage by dividing the developmental stage into smaller sections. For example, when the developmental stage determination unit 130 determines the detailed stages by dividing the developmental stage into 100 sections, the developmental stage determination unit 130 may determine the developmental stage as "60% of the third stage."

In addition, if the development stage stagnates (for example, there is no change from the first stage even after repeated assessments), the user may become bored. The development stage assessment unit 130 may divide the development stage into smaller sections and assess the development stage by assigning a numerical value to each section.

The operation image B14 is an operation image for displaying detailed information on the developmental stage. When the operation image B12 is operated by the user of the terminal device 20, the display control unit 140 outputs a detailed information screen on the developmental stage to the terminal device 20 as information corresponding to the developmental stage, in the same manner as described above. As a result, a "detailed information screen on the developmental stage" is displayed on the display unit 22 of the terminal device 20. An example of the "detailed information screen on the developmental stage" is shown in Figure 11.

11 shows another example of information displayed on the terminal device 20 of this embodiment. The display unit 22 of the terminal device 20 displays determination result images P21 to P25, respectively.
The determination result image P21 is an image showing the developmental stage determined in step S30, and is the same as the determination result image P11 described with reference to Fig. 10. If the "third stage" is determined in step S30, the display unit 22 displays "third stage" (or "munching stage", which is a simpler expression of the developmental stage) as the determination result image P21.
The judgment result image P22 is a photographic image showing an example of a typical baby at the developmental stage judged in step S30.
The determination result image P23 is an illustrated image that emphasizes the movement of the baby's mouth at the developmental stage determined in step S30.
The judgment result image P24 is an illustrated image that emphasizes the movement in the oral cavity of an infant at the developmental stage judged in step S30.
The judgment result image P25 may be a numerical value (eg, 1 to 100) representing the development stage, along with past judgment results (numerical values), which may be graphed as a progression of numerical values and displayed on the display unit 22.
The display control unit 140 causes the display unit 22 to display the above-mentioned various types of image information and text information as information corresponding to the development stage.

As described above, the display control unit 140 causes the display unit 22 to display information indicating the developmental stage (e.g., “Stage 3”, “Chewing stage”), as well as information corresponding to the developmental stage (e.g., weaning food recipes and how to feed weaning food, training to promote good dentition, and oral care methods such as brushing the teeth).
That is, the display control unit 140 further causes the terminal device 20 (display device) to display information corresponding to the development stage determined by the development stage determination unit 130 .

[Summary of the embodiment]
As described above, in the developmental stage determination system 1, the developmental stage determination unit 130 determines the developmental stage based on the movement of the mouth of the infant while eating captured by the imaging unit 21 and on the predetermined determination criteria. Therefore, the developmental stage determination system 1 can objectively determine the degree of development of oral function during infancy.

Furthermore, according to the developmental stage determination system 1, the developmental stage of oral function is displayed as a history along with the growth of the infant, so that the user can be given an objective indicator of the type of baby food and how to feed it. This allows the user (e.g., the parent of an infant) to select the appropriate type and method of feeding according to the developmental stage of oral function. Therefore, the developmental stage determination system 1 can reduce the anxiety of parents raising children.

[Modification]
Next, a modified example of the embodiment will be described. Note that the same functions and operations as those in the above-described embodiment will be denoted by the same reference numerals and the description thereof will be omitted.

[Updating display information when the development stage has not changed (1)]
As described above, the developmental stage determination unit 130 determines the developmental stage each time it obtains motion information from the terminal device 20. It takes a certain amount of time for the developmental stage of an infant to be updated (for example, for the developmental stage to increase by one stage). For this reason, even if the motion information of an infant is output from the terminal device 20 to the developmental stage determination device 10, the developmental stage may not be updated for a long period of time. In such a case, if the information displayed on the terminal device 20 is not updated, the motivation to continue using the developmental stage determination system 1 may be weakened.

In this modified example, when the developmental stage has not changed despite having acquired operation information from the terminal device 20, the display control unit 140 outputs to the terminal device 20 information corresponding to the developmental stage that is different from the information previously transmitted.
In other words, until the developmental stage determined by the developmental stage determination unit 130 is updated, the display control unit 140 displays information selected from among multiple pieces of information corresponding to the developmental stage on the terminal device 20 (display device) as updated information corresponding to the developmental stage.
According to the developmental stage determination system 1 configured in this manner, even if the developmental stage of an infant has not changed, it is possible to change the information displayed on the terminal device 20. Therefore, according to the developmental stage determination system 1, even if the developmental stage of an infant has not changed, it is possible to motivate the user of the developmental stage determination system 1 to continue using it.

[Updating display information when development stage has not changed (2)]
The operation receiving unit 150 may be configured to receive an update operation for updating the information to be displayed by the user of the terminal device 20. In this case, the display control unit 140 updates the information corresponding to the development stage in response to the operation receiving unit 150 receiving the update operation, and displays the updated information on the terminal device 20 (display device).
According to the developmental stage determination system 1 configured in this manner, even if the developmental stage of an infant has not changed, it is possible to change the information displayed on the terminal device 20. Therefore, according to the developmental stage determination system 1, even if the developmental stage of an infant has not changed, it is possible to motivate the user to continue using the system.

[Use reminders based on operation frequency]
If the developmental stage assessment system 1 determines that the user has not used the developmental stage assessment system 1 for a certain period of time, it may display a message encouraging the user to use the system (a usage reminder).
That is, the display control unit 140 displays information on the terminal device 20 (display device) prompting the user to provide operation information based on the frequency with which operation information is provided to the operation information acquisition unit 110 and the frequency with which update operations are received by the operation receiving unit 150.

For example, when the frequency of receiving update operations and the frequency of providing operation information are both low, the display control unit 140 displays information prompting the user to provide operation information on the terminal device 20 (display device). In this case, a usage reminder can be sent to a user who does not use the development stage determination system 1 often.
Furthermore, when the frequency of receiving update operations is high but the frequency of providing operation information is low, the display control unit 140 displays information prompting the user to provide operation information on the terminal device 20 (display device). In this case, a usage reminder can be sent to a user who frequently uses the information provision function of the development stage determination system 1 but does not often use the development stage determination function.

[Determining whether the content of baby food is appropriate for the developmental stage]
The developmental stage determination system 1 may have a function of determining whether the baby food given to an infant is suitable for the developmental stage of the infant. In this case, the developmental stage determination device 10 includes a food information acquisition unit 160 and a type determination unit 170.
The food information acquisition unit 160 acquires food information indicating the types of food given to the infant.
Specifically, a user uses the imaging unit 21 of the terminal device 20 to capture an image of the baby food being given to an infant. The terminal device 20 outputs the image of the baby food captured by the imaging unit 21 to the developmental stage assessment device 10. The food information acquisition unit 160 of the developmental stage assessment device 10 acquires the image of the baby food from the terminal device 20. The food information acquisition unit 160 applies a known image matching method to determine the type of baby food captured in the image of the baby food.
The food information acquisition unit 160 may acquire, as food information, information obtained as a result of a character input operation or a menu selection operation performed by the user on the terminal device 20 to input the type of baby food.
Furthermore, the type of food indicated by the above-mentioned food information broadly includes food attributes such as ingredients, cooking method, softness, size, etc.

The type determining section 170 determines whether the type of food indicated by the food information acquired by the food information acquiring section 160 is suitable as food to be ingested at the development stage determined by the development stage determining section 130 .
Specifically, the type determination unit 170 determines the type of food to be given to an infant (the type of food appropriate for the infant's developmental stage) based on the type of food predetermined according to the infant's developmental stage and the developmental stage determined by the developmental stage determination unit 130. The type determination unit 170 compares the determined type of food (i.e., the ideal type of food appropriate for the developmental stage) with the type of food indicated by the food information acquired by the food information acquisition unit 160 (i.e., the type of food actually given to the infant) to determine whether the food is appropriate for the developmental stage determined by the developmental stage determination unit 130.
In this case, the display control unit 140 further causes the terminal device 20 (display device) to display the determination result by the type determination unit 170 .

In addition, if an infant has likes and dislikes depending on the type of food given to the infant, the facial expression of the infant may differ when given a type of food that the infant likes and when given a type of food that the infant dislikes. If a difference occurs in the facial expression of the infant, the developmental stage determination unit 130 may erroneously determine the developmental stage, and the accuracy of the determination of the developmental stage may decrease.
The developmental stage determination unit 130 may be configured to improve the accuracy of determining the developmental stage by performing statistical calculations on multiple pieces of motion information based on the type of food determined by the type determination unit 170 (i.e., determining whether the animal is given a favorite type of food or a disliked type of food).

In addition, there may be cases where the type of food given to an infant is suitable for determining the developmental stage. In this case, the developmental stage determination unit 130 may be configured to improve the accuracy of determining the developmental stage by using the behavior information obtained when the infant is given a type of food suitable for determining the developmental stage, based on the type of food determined by the type determination unit 170.

[Determining the amount of baby food]
The developmental stage determination system 1 may have a function of determining the portion of baby food given to an infant. In this case, the developmental stage determination device 10 includes a portion determination unit 180.
The portion determination unit 180 determines the portion of food given to the infant based on the motion information acquired by the motion information acquisition unit 110. In this case, the display control unit 140 further causes the terminal device 20 (display device) to display information based on the determination result of the portion determination unit 180.
According to the developmental stage determination system 1 configured in this manner, it is possible to provide the user with information when the amount of baby food is too much or too little.

The portion determination unit 180 may determine the portion of baby food based on the type of food given to the baby or the history of foods given to the baby in the past.
For example, the portion determining unit 180 determines the portion of food given to the infant based further on the type of food determined by the type determining unit 170 .
For example, in the case where the memory unit 120 stores a history of foods that have been fed to an infant in the past, the portion determination unit 180 may determine the portion of baby food based further on the food history stored in the memory unit 120.
According to the developmental stage determination system 1 configured in this manner, it is possible to present to the user whether the amount of baby food is appropriate depending on the situation, such as whether the food given to an infant is a type of food given to the infant for the first time, a type of food that the infant has been given multiple times and to which the infant is accustomed, or whether the infant has just progressed through a developmental stage.

In each of the above-described embodiments, an example of the division of functions between the development stage assessment device 10 and the terminal device 20 has been described, but each device may be configured to divide functions differently from that described above.
For example, the developmental stage assessment system 1 may be realized in such a manner that all or part of the functions of the developmental stage assessment device 10 are realized as software that runs on the terminal device 20 .

Above, an embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and appropriate modifications can be made without departing from the spirit of the present invention.

Each of the above-mentioned devices has a computer inside. The process steps of each of the above-mentioned devices are stored in the form of a program on a computer-readable recording medium, and the above-mentioned processes are performed by the computer reading and executing this program. Here, computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Also, this computer program may be distributed to a computer via a communication line, and the computer that receives this distribution may execute the program.

The program may be for realizing part of the functions described above.
Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

1... developmental stage determination system, 10... developmental stage determination device, 110... operation information acquisition unit, 120... storage unit, 130... developmental stage determination unit, 140... display control unit, 150... operation reception unit, 160... food information acquisition unit, 170... type determination unit, 20... terminal device

Claims (9)

a motion information acquisition unit that acquires motion information indicating an infant's oral intake of food;
A developmental stage determination unit that determines which of a plurality of developmental stages of oral function in infants the movement indicated by the movement information is in accordance with the movement information acquired by the movement information acquisition unit and a predetermined determination criterion;
a display control unit that displays information indicating the developmental stage determined by the developmental stage determination unit on a display device, graphs information corresponding to the developmental stage determined by the developmental stage determination unit together with a history of past determination results, and displays information selected from a plurality of pieces of information corresponding to the developmental stage on a display device as updated information corresponding to the developmental stage until the developmental stage determined by the developmental stage determination unit is updated;
A developmental stage determination device comprising: an operation receiving unit that receives an update operation to update the information to be displayed,
The display control unit is
The developmental stage determination device according to claim 1 , further comprising: a controller configured to update the information corresponding to the developmental stage in response to the operation receiving unit receiving the update operation, and to display the updated information on a display device. The display control unit is
The developmental stage determination device according to claim 2 , further comprising: displaying, on a display device, information prompting the user to provide the motion information based on a frequency with which the motion information is provided to the motion information acquisition unit and a frequency with which the update operation is received by the operation receiving unit. On the computer,
a motion information acquisition step of acquiring motion information indicating an infant's oral intake of food;
a developmental stage determination step of determining which of a plurality of developmental stages of oral function in infants corresponds to the movement indicated by the movement information based on the movement information acquired in the movement information acquisition step and a predetermined determination criterion;
a display control procedure for displaying information indicating the developmental stage determined in the developmental stage determination procedure on a display device, graphing information corresponding to the developmental stage determined in the developmental stage determination procedure together with a history of past determination results, and displaying information selected from a plurality of pieces of information corresponding to the developmental stage on a display device as updated information corresponding to the developmental stage until the developmental stage determined in the developmental stage determination procedure is updated;
A program for executing. an imaging unit that images an infant's oral intake of food;
An output unit that outputs the motion information to a development stage determination device that determines which of a plurality of development stages of oral function in infants the motion indicated by the motion information corresponds to based on the motion information based on the image captured by the imaging unit and a predetermined determination criterion;
an acquisition unit that acquires information indicating the developmental stage determined by the developmental stage determination device based on the motion information output by the output unit;
a display control unit that displays, on a display device, information indicating the developmental stage acquired by the acquisition unit;
A terminal device comprising: The developmental stage determination device determines which of the developmental stages the infant belongs to based on the movement information based on the image captured by the imaging unit and on a predetermined determination criterion including a reference movement corresponding to a plurality of developmental stages of oral cavity function in infancy for at least one part of the infant's lips, corners of the mouth, tongue, and jaw.
The terminal device according to claim 5. The motion information is information indicating the motion of an infant taking food orally, and includes information on the motion of at least one part of the lips, corners of the mouth, tongue, and jaw during at least a part of the period from when the food is ingested to when it is swallowed.
The terminal device according to claim 5. The developmental stage determination device is realized as software that runs on the terminal device.
The terminal device according to claim 5. The terminal device includes a computer.
capturing an image of an infant's oral intake of food by an imaging unit;
Outputting the motion information to a development stage determination device that determines which of a plurality of development stages of oral function in infants the motion indicated by the motion information corresponds to based on the motion information based on the image captured by the imaging unit and a predetermined determination criterion;
acquiring information indicating the developmental stage determined by the developmental stage determination device based on the outputted motion information;
displaying the acquired information indicating the developmental stage on a display device;
A program for executing the above.

Images