JP7787616B2 - Dementia and/or depression estimation AI system and training data creation method - Google Patents

Description

The present invention relates to AI for assessing medical conditions, and more specifically to technology for estimating dementia and/or depression using facial photographs.

Early detection and treatment of dementia and depression are important. One method for detecting symptoms is to use AI to identify the subject's facial photograph and estimate the degree of dementia or depression.
However, building such a system generally requires information managed by the medical institution, and it is not easy for other businesses to do so.
Therefore, there was a need for a method that uses AI to estimate the degree of a subject's dementia or depression from information obtained outside of a hospital.

Various technologies have been proposed to address these problems. For example, a technique for assisting doctors in diagnosis through image analysis using AI has been proposed and is now known (see Patent Document 1). More specifically, this is a device that uses training data consisting of image data of an image of a subject, observation data, and information on the subject's lifestyle and dietary habits to input image data and subject information, performs machine learning to output observation data, and outputs the observation data using this AI.
However, since data acquisition in this prior art is performed at the hospital, other businesses cannot handle it, and the above problem has not been resolved.

Japanese Patent Application Laid-Open No. 2021-108800

In consideration of the above-mentioned problems, the present invention aims to develop AI that can estimate the degree of dementia and/or depression of a subject based on information obtained outside of a hospital.

To solve the above problem, the present invention provides a method for creating training data for use in AI that estimates the level of dementia and/or depression from a user's facial photograph. In this system for estimating the level of dementia and/or depression, information that serves as training data for the AI can be obtained from the location where the user picks up prescription medication or from an application on the user's mobile device. The training data includes at least a facial photograph of the user and information that can be used to estimate the user's dementia and/or depression, obtained from the prescription, instructions, or questions asked of the user. The facial photograph of the user is either an expressionless facial photograph or a smiling facial photograph, or both. Facial features are extracted from the facial photograph to generate example data for training data. The features include at least the degree of drooping of the corners of the mouth, the degree of drooping of the eyelids, the degree of drooping of the cheeks, the degree of drooping of the eyebrows, and the degree of mouth opening. The level of dementia and depression is estimated from the type and amount of medication listed on the prescription or instructions, and used as correct answer data for training data, thereby creating the training data for machine learning.

The present invention also employs a method for estimating the feature values expected at specific dementia and depression levels, using the estimated feature values as the example data, and then performing machine learning using the predicted dementia and depression levels as the correct answer data.

Furthermore, the present invention provides an AI system for predicting dementia and/or depression that uses training data created by the training data creation method, and employs a means for estimating the degree of dementia and depression of a subject based on a facial photograph of the subject.

Furthermore, the present invention employs a means for taking a facial photograph of the subject at the pharmacy and providing advice based on the results of the AI system.

Furthermore, the present invention employs a means for taking a photograph of the subject's face and directing the subject to video content based on the results of the AI system's analysis.

Furthermore, the present invention employs a means for taking a facial photograph of the subject and, depending on the results of the AI system, referring the subject to a specialized medical institution.

Furthermore, the present invention employs a means for taking a photograph of the subject's face and providing advice to the family based on the results of the AI system's analysis.

Furthermore, the present invention employs a means for taking a facial photograph of the subject at an OTC medical supply store and providing advice based on the results of the AI system's analysis.

Furthermore, the present invention employs a means for acquiring a facial photograph of the subject taken with a mobile device used by the subject via a network, and providing advice based on the results of the AI system's analysis of the acquired facial photograph.

According to the dementia and/or depression estimation AI system and teacher data creation method of the present invention, information for constructing AI that estimates the degree of dementia or depression of a subject can be obtained outside of a hospital, so that AI that estimates dementia and/or depression can be constructed by anyone, not just hospital personnel.
In addition, by utilizing applications on the mobile devices used by users, it is possible to easily obtain a large amount of information needed to build AI that predicts dementia and/or depression.

FIG. 1 is a flow chart showing an embodiment of a teacher data creation method according to the present invention. 1 is a schematic diagram showing an embodiment of a teacher data creation method according to the present invention; 10 is a calculation table showing an example of generating correct answer data in the teacher data generating method according to the present invention. FIG. 1 is a table showing an example of training data in a training data creation method according to the present invention. FIG. 1 is a schematic diagram showing an example of the operation of the AI in the dementia and/or depression prediction AI system according to the present invention. FIG. 10 is a flow chart showing another embodiment of the teaching data creation method according to the present invention. FIG. 1 is a schematic diagram showing an example of extracting facial features in the dementia and/or depression prediction AI system according to the present invention. FIG. 10 is a table showing another example of the training data in the training data creation method according to the present invention. FIG. 1 is a flow chart showing an example of teacher data in a teacher data creation method according to the present invention. 1 is an explanatory diagram showing an example of teacher data in a teacher data creation method according to the present invention; FIG. 1 is a flow chart showing an example of the dementia and/or depression prediction AI system of the present invention at a pharmacy. FIG. 1 is a flow chart showing an example of monitoring support in the dementia and/or depression prediction AI system according to the present invention. FIG. 10 is a system diagram of another embodiment of the present invention. FIG. 10 is a diagram of a screen of a mobile terminal according to another embodiment of the present invention. 10 is a table showing information for generating training data according to another embodiment of the present invention. FIG. 10 is a flowchart showing a procedure for acquiring teacher data according to another embodiment of the present invention. FIG. 10 is a flow chart showing an example of a mobile terminal implementation of the dementia and/or depression prediction AI system of the present invention.

The greatest feature of the dementia and/or depression prediction AI system and teacher data creation method of the present invention is that it can obtain information for estimating the degree of dementia or depression of a subject in a location other than a hospital, thereby making it possible to build AI that can predict dementia and/or depression not only for hospital personnel but also for anyone.
Hereinafter, embodiments of the dementia and/or depression prediction AI system and training data creation method according to the present invention will be described with reference to the drawings.

Note that the overall configuration and the configuration of each part of the dementia and/or depression estimation AI system and teacher data creation method shown below are not limited to the examples described below, but can be modified as appropriate within the scope of the technical idea of the present invention, i.e., within the scope of configurations that can achieve the same functional effects.
Furthermore, since the present invention is a dementia and/or depression estimation AI system and a method for creating training data, it naturally includes a dementia estimation AI system, a depression estimation AI system, and a dementia and depression estimation AI system, but the examples will focus on examples of estimating dementia and depression.
In the case of a dementia prediction AI system, the system can be easily constructed by excluding the depression factor, which is another factor. The same is true for a depression prediction AI system.
In addition, in the embodiments, a user refers to a person who cooperates in providing training data, and a subject refers to a person whose condition is estimated using AI.

The present invention will be described with reference to FIGS.
FIG. 1 is a flow diagram showing an embodiment of a training data creation method according to the present invention. FIG. 2 is a schematic diagram showing an embodiment of a training data creation method according to the present invention, where (a) is a schematic diagram of data acquisition work performed in a pharmacy, (b) is a schematic diagram of the work of creating sample data from a user's facial photograph, and (c) is a schematic diagram of the work of creating correct answer data from a prescription or other information. FIG. 3 is a calculation table showing an embodiment of generating correct answer data in the training data creation method according to the present invention, where (a) is a table for estimating and calculating dementia level and depression level, and (b) is a calculation table obtained by estimating and calculating dementia level and depression level based on actual medication type and dosage. FIG. 4 is a table showing an embodiment of training data in the training data creation method according to the present invention, where (a) is an example in which correct answer data is calculated in 10% frequency increments, (b) is an example in which the frequency is 1 or 0, and (c) is an example in which age, etc., is included in the sample data. Figure 5 is a schematic diagram showing an example of the operation of AI in the dementia and/or depression prediction AI system of the present invention, where (a) shows an example of predicting dementia, etc. using AI, and (b) shows the process of machine learning using AI.

The dementia and/or depression estimation AI1 is an AI that estimates the degree of dementia and depression, and its estimation capabilities are improved through machine learning.
One method for constructing a dementia and/or depression estimation AI1 is to generate training data and have the AI learn a large amount of it to make highly accurate estimations.
Training data is data used for machine learning in AI. Training data is often generated as pairs of example data and correct answer data.
Example data is data that represents the event that you want the AI to estimate, and correct answer data is data that you want the AI to answer.

When asking AI to estimate whether or not a person is ill, for example, the subject's symptoms are used as example data, and the subject's disease name is used as correct answer data. By having the AI learn a large amount of such training data, an AI that can estimate illness is constructed.

When examining illnesses such as dementia and depression, doctors often look at facial expressions as part of their diagnosis.
When considering a system that uses AI to predict dementia or depression, the subject's facial expressions can be obtained by taking a photo with a camera, but the subject's illness name is managed by a doctor in a medical record, etc., and it is generally difficult to take this data outside the hospital.

Therefore, in this embodiment, the name and severity of the subject's illness are estimated from the prescription, etc. at the pharmacy, and training data is obtained.
In other words, this is a method for creating training data in which information that serves as training data for AI is obtained at the location where the user receives the medicine according to the prescription, and the training data includes at least a photograph of the user's face and the prescription or instructions.
Information that serves as training data includes information that serves as the basis for the training data and information that is used as training data.
Prescriptions, etc. include at least prescriptions, medicine notebooks, and instructions. Instructions, also known as medication instructions, are written documents in which a doctor instructs guardians, facilities, etc. on the time and amount of medication to be administered.
The procedure for acquiring training data will be described with reference to FIGS.
Fig. 1 shows a flow for generating teacher data 40. Fig. 2 shows a procedure for generating teacher data 40.
The information is collected by the pharmacy 10. Here, the term "pharmacy" refers to both pharmacies that are independent of the hospital and pharmacies that are located within the hospital. The data handled by both pharmacies is the same.
First, the contents of the prescription 31 are obtained from the user P at the pharmacy 10. The contents of the medicine notebook 30 and the instruction sheet may also be obtained along with the prescription 31. Since the user P has come to the pharmacy 10 to obtain the medicine listed on the prescription 31 and the medicine notebook 30, obtaining the prescription 31 is easy.
Apart from dispensing medicine, permission is obtained to use information from the prescription 31, medicine notebook 30, and instructions to build the dementia and/or depression prediction AI1, and the information is acquired (S101, Figure 2(a)).
Next, in order to construct the dementia and/or depression estimation AI1, permission is obtained from the user to obtain a facial photograph 12, and the user P is photographed with the photographing device 11 to obtain the facial photograph 12 of the user (S102, Figure 2(a)).
As the user's facial photograph 12, a facial photograph 13 of the user with a neutral expression and a facial photograph 14 of the user with a smiling expression are acquired. The facial photograph 12 of the user with a neutral expression can be said to be a natural, ordinary face.
It is said that in cases of dementia or depression, facial expressions tend to be dull, with drooping eyelids and a lack of overall expression. Smiles may also be unnatural or may be no different from a blank smile.
Therefore, by using facial photos of users with a neutral expression and a smiling expression, the difference between normal people and those with dementia or depression can be emphasized.
When taking a photo with no expression, if it is difficult to say, "Please make sure you have no expression," you can use a phrase such as, "Please let me take a photo of your face in a natural, normal way."
By steps S101 and S102, the acquisition of information from the user is completed.

Next, the presence and severity of dementia or depression are estimated from the type and amount of medication written on the prescription 31, etc. (S103, FIG. 2(b)). While a pharmacist or the like can make a certain degree of estimation, a calculation table 43 is used to reduce variability. An example of the calculation table is shown in FIG. 3(a). Correct answer data is generated using the calculation table 43.
Next, facial photograph data 20 is generated from the user's facial photograph 12. Generally, facial photographs taken outside a studio or the like have different angles of view, resolutions, backgrounds, etc., so by correcting the images so that the angles of view, resolutions, and backgrounds are similar, appropriate facial photograph data 20 can be generated as example data 41 (S104, FIG. 2(c)).
The facial photograph data 20 consists of facial photograph data 21 with a neutral expression and facial photograph data 22 with a smiling expression.
A series of operations completes one piece of training data (S105).
In other words, the process involves using a photograph of the user's face as example data for the training data, and using the level of dementia and depression estimated from the type and amount of medication listed on the prescription or instructions as correct answer data for the training data, to generate training data for machine learning.
Increasing the amount of training data can improve the accuracy of machine learning.

A calculation table 43 is used to estimate the degree of dementia and depression from the prescription 31, medicine notebook 30, and instructions.
The calculation table 43 has a column for recording the type and amount of medication, which is filled in according to the user's information. There are columns for dementia and depression, and coefficients are listed for each type of medication. Medications that are closely related to each disease have high coefficients, and medications that are less related have low coefficients.
An evaluation score is determined by entering the type and amount of medication. The severity of each illness can be determined by adding up the evaluation scores. To obtain correct data, normalization is required, such as 0 to 100%, so normalization is performed using a predetermined number. There are two ways to calculate correct data: by expressing it as a percentage, or by expressing it as 1 or 0.
Figure 3(b) shows a calculation example using specific numbers. In this example, since there are many medications related to dementia, the total evaluation score for dementia is 23 points, and the total evaluation score for depression is 4 points. As an example, normalization is performed at 30, and the evaluation for dementia is 76.7% and the evaluation for depression is 13.3%. As correct answer data, correct answer data 1 with 10 levels and correct answer data 2 with two levels (1 and 0) are generated.
Correct answer data 1 has a dementia level of 70% and a depression level of 10%, and correct answer data 2 has a dementia level of 1 and a depression level of 0.
This process generates correct data on the dementia level and depression level.

4A shows examples of training data. In FIG. 4A, facial photograph data 20 is used as example data of the training data, and as the facial photograph data 20, a facial photograph 21 of a neutral face and a facial photograph 22 of a smiling face are used.
The correct answer data for both dementia and depression is based on a 10-point scale. The correct answer data has 100 correct answer elements (10 points x 10 points), so a correspondingly large amount of training data is required for machine learning.
Figure 4(b) shows the case where two levels of data are used as correct answer data for both dementia and depression as training data. The correct answer data consists of four correct answer elements (two levels x two levels), so the amount of training data required for machine learning may be smaller than in the case of Figure 4(a).
Figure 4(c) shows an example data set in which the user's age, gender, and medical history are added in addition to facial photograph data. Family medical history can also be added. Using a large amount of data can sometimes make estimation easier. However, when using AI, it is necessary to input the same data as during learning, so in this case, the user's age, gender, and medical history must be input in addition to facial photograph data, which reduces versatility.

An overview of the operations during AI learning and utilization will be described with reference to Fig. 5. A machine learning system 50 shows an example of deep learning using a neural network.
The dementia and/or depression estimation AI1 consists of an input section 51, an intermediate section 52, and an output section 53. During machine learning, a correct answer section 54 is added. The input section 51 has elements X1 to Xn corresponding to the information to be input to the AI. The intermediate section 52 is the section that inherits information from the input layer and performs various calculations. The more layers the intermediate section 52 has, the more complex the analysis that can be performed. The calculation results are sent to the output section 53. The output section 53 is the section that outputs the answer and has elements Z1 to Zm corresponding to the answer. The value of each element is, for example, between 0 and 1, and the element with the highest value is estimated to be the answer (Figure 5(a)).
The correct answer unit 54 is a part that sets the correct answer data 42 to be given to the dementia and/or depression estimation AI1 during machine learning, and has elements A1 to Am. The number of elements is the same as that of the output unit 53. Of A1 to Am, the numerical value of the correct answer element is set to 1, and the numerical values of the other elements are set to 0. The differences between the numerical values Z1 to Zm and A1 to Am are calculated, and the sum of these values becomes the error D. Minimizing the error D improves the accuracy of the AI ( FIG. 5( b) ).

The operation of the dementia and/or depression estimation AI1 will be simplified for the sake of explanation, and an example of an AI that estimates whether or not a person has dementia will be described.
Regarding the output unit 53, the AI's answer is either dementia or no dementia. Therefore, the output unit 53 has two elements, for example, Z1 "has dementia" and Z2 "does not have dementia."
When face photograph data 20 including expressionless face photograph data 21 and smiling face photograph data 22 is input, the number of elements X in input unit 51 is an element according to the resolution.
When facial photo data 20 is input to the AI input unit 51, elements Z1 and Z2 of the output unit 53 each output a numerical value between 0 and 1. The higher the numerical value, the more accurate the answer estimated by the AI. If the difference between the numerical values is large, the accuracy of the estimation is high, and if the difference between the numerical values is small, the accuracy of the estimation is low.
For example, if Z1 is 0.1 and Z2 is 0.9, Z2 is the higher value, so the AI's answer will be "not dementia," which is Z2. Because the difference between the values of Z1 and Z2 is large, the accuracy can be said to be high.

4(b), the output unit 53 of the dementia and/or depression estimation AI 1 will have four levels: dementia level 0 and depression level 0, dementia level 0 and depression level 1, dementia level 1 and depression level 0, and dementia level 1 and depression level 1. Therefore, Z1 to Z4 can be assigned, respectively.
After the AI is constructed, facial photo data 20, which are expressionless facial photo data 21 and smiling facial photo data 22, are input to the dementia and/or depression estimation AI 1, and processed by an intermediate unit 52 consisting of a neural network, and the results are output as numbers between 0 and 1 for Z1 to Z4 of the output unit 53.
For example, if Z1 is 0.1, Z2 is 0.6, Z3 is 0.2, and Z4 is 0.1, the highest value, Z2, is adopted. Z2 has a dementia level of 0 and a depression level of 1, so the AI's response is "There is a possibility of depression."
Alternatively, the highest value may be used as the probability. In this case, since the value of Z2 is 0.6, it may be possible to say "There is a 60% chance of depression."

4(a), the output unit 53 of the dementia and/or depression estimation AI 1 can be divided into 100 categories in 10% increments, from "dementia level 0% and depression level 0%" to "dementia level 90% and depression level 90%." In this case, the output unit 53 has 100 elements, Z1 to Z100.
The values Z1 to Z100 of the output unit 53 for the input are checked, and the highest value is the answer. If a certain element of Z is the highest, the corresponding dementia level of XX% and depression level of XX% are the answers.

Another method for simplifying the output unit 53 is to separate it into a dementia estimation AI and a depression estimation AI. By doing so, the number of data items in the input unit 51, intermediate unit 52, and output unit 53 can be reduced.
In this case, the final output of the dementia and/or depression estimation AI1 is the combination of the answers of the two AIs.

Next, the machine learning method of the dementia and/or depression estimation AI 1 will be explained with reference to Figure 5(b). Facial photo data 20, consisting of expressionless facial photo data 21 and smiling facial photo data 22, is input to the input unit 51 of the dementia and/or depression estimation AI 1, and the output unit 53 has 100 elements ranging from a dementia level of 0% and a depression level of 0% to a dementia level of 90% and a depression level of 90%.
First, one of the training data 40 is used for the dementia and/or depression estimation AI1. Example data 41 is input to the input unit 51. Correct answer data 42 is set in the correct answer section. If the correct answer data 42 of the training data 40 is a dementia level of 70% and a depression level of 10%, Aj (j is the element number corresponding to a dementia level of 70% and a depression level of 10%) corresponding to a dementia level of 70% and a depression level of 10% is set to 1, and A1 to A100 excluding Aj are set to 0.
When example data 41 is input to dementia and/or depression estimation AI1, estimated values are output as numbers between 0 and 1 to Z1 to Z100 of output unit 53. The sum of the differences between Z1 to Z100 and the corresponding A1 to A100 becomes error value D. The neural network of intermediate unit 52 is adjusted to lower this error value D.
By performing this process on all of the large amounts of training data, the error value D is reduced and the accuracy of the AI is improved.

In this way, according to this embodiment, information for constructing an AI that estimates the degree of dementia and/or depression of a subject can be obtained in a location other than a hospital, making it possible to construct an AI for estimating dementia and/or depression not only for hospital personnel.
In addition, by using a facial photograph of the person being diagnosed, it becomes possible to operate and utilize AI to estimate the degree of dementia and/or depression of the person.

In addition, the dementia and/or depression estimation AI1 can be used at the pharmacy 10 to provide advice to pharmacy customers about dementia and depression.
A photograph of the face of a person who requests AI advice is taken, and advice on dementia and depression is provided by the dementia and/or depression estimation AI 1. This will be explained along the flow of Figure 11.
1) A photograph of the subject's face is taken (S401). The photographs should include one with a neutral expression and one with a smiling expression. If this is difficult, only one of them will suffice. This may be done in parallel with the acquisition of training data 40 to improve the accuracy of the dementia and/or depression estimation AI1.
2) Image processing is performed from the subject's expressionless facial photo 13 and the subject's smiling facial photo 14 to generate expressionless facial photo data 21 and smiling facial photo data 22, which are facial photo data 20 to be input into the dementia and/or depression estimation AI 1 (S402).
3) Input to AI (S403). If only one photo is acquired, input two of the same photo.
4) Check the value of the output section 53 of the dementia and/or depression estimation AI1 and determine the estimated values of the dementia level and depression level (S404).
5) The degree of dementia and depression of the subject are estimated from the prescription 31 and used as the estimated prescription value (S405).
6) If the degree of dementia or depression estimated by AI is worse than the prescription estimate, the subject is notified of advice (S406).
The same applies if the AI estimates a certain degree of dementia or depression even though Prescription 31 does not contain any prescription for dementia or depression.
In addition, in the case of hospital pharmacies that are not related to dementia or depression, such as dermatology or surgery, advice may be given without checking prescriptions, etc.

By performing this type of work, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and depression.

The dementia and/or depression estimation AI1 can also be used in a health check system, which estimates the user's level of dementia and depression during a health check.
1) Take a photograph of the face of the person undergoing the health checkup. Take two photographs: one with a blank expression and one with a smile. This may be easy if you explain that the photographs will be used for the health checkup, but if this is difficult, just one of the photographs will suffice.
2) Generate facial photo data 20 for AI.
3) Input the data into the AI. If only one photo is needed, input two of the same photo.
4) From the AI output results, estimates of the degree of dementia and depression are determined.
5) The target person will be notified of the video content that corresponds to the estimated level of dementia and depression, and will be referred to a specialized medical institution.
The notification may be provided on paper with a QR code (registered trademark) along with the health check results, or a URL may be sent to the subject by email.

In this way, by adding the dementia and/or depression prediction AI1 to a health checkup, subjects can obtain diagnostic results for dementia and depression without any special burden.

The dementia and/or depression estimation AI1 can also be used in a monitoring support system. When the monitoring support system is used, it estimates the degree of dementia and depression of the person being monitored. This will be explained with reference to the flow in Figure 12.
A monitoring support system is a system that uses cameras and sensors to check on elderly family members and relatives who live far away, and provides support or contacts family members if an emergency occurs.
1) In the monitoring support system, a facial photograph of the subject is acquired from video to confirm the safety of the subject (S501). Because the monitoring support system requires constant monitoring of the subject's condition, the subject is often constantly photographed with a camera, and it is easy to acquire facial photographs of the subject with a blank expression and a smiling expression. For example, a facial photograph 13 of the user with a blank expression and a facial photograph 14 of the user with a smiling expression may be acquired based on the subject's behavior, voice, etc.
2) Generate facial photo data 20 for AI from the subject's facial photo (S502).
3) Input to AI (S503).
4) From the output results of the AI, estimates of the degree of dementia and depression are determined (S504).
5) If the estimated dementia or depression level is worse than expected, the AI estimates of the subject's dementia or depression level and corresponding advice will be notified to the subject's relatives or family members (S505).
Specifically, if a subject who is not diagnosed with dementia is suspected to have a high level of dementia, the system will notify pre-registered relatives and family members of the subject's possible dementia via notifications, etc. At that time, the system will also refer the subject to a medical institution to confirm dementia and provide advice on how to respond to the subject.

In this way, by adding the dementia and/or depression estimation AI1 to the monitoring support system, it is possible to estimate the possibility of a subject having dementia or depression without placing any special burden on the subject, and to take early action by contacting relatives or family members.

The dementia and/or depression estimation AI1 can also be used for over-the-counter (OTC) products. When dealing with customers who visit an OTC medical supply store, the AI estimates the degree of dementia and depression of the customer.
1) Take a facial photograph of the subject over the counter. The subject can be encouraged to take a photograph as it allows for a simple check of dementia and depression. Take a facial photograph 13 of the user with a neutral expression and a facial photograph 14 of the subject with a smiling face. If this is difficult, just one of them will suffice.
2) Image processing is performed on the subject's facial photograph 12 so that it becomes facial photograph data to be input into the dementia and/or depression estimation AI 1.
3) Input the data into the AI. If only one photo is needed, input two of the same photo.
4) Check the value of the output section 53 of the dementia and/or depression estimation AI1 and determine the estimated values of the dementia level and depression level.
5) The subject will be informed that this is merely an estimate, and will be notified of the estimated level of dementia and depression, along with advice based on those estimates.
Specifically, if a subject who is not aware of dementia or depression is suspected to have a high level of dementia or depression, the system will refer them to a medical institution to confirm dementia or depression and provide advice such as guidelines on how to respond.

In this way, by utilizing the dementia and/or depression prediction AI1 over-the-counter, subjects can receive a simple diagnosis and advice about dementia and depression without any special burden.

It can also be used when renewing your license.
When renewing a driver's license, a cognitive test may be conducted in the form of questions to check the onset and severity of dementia, etc. However, as this takes a certain amount of time, it is often considered a roundabout way of doing things by those who are eligible.
By using the dementia and/or depression prediction AI1 instead of or as a complement to cognitive testing, the accuracy of cognitive testing can be improved, or the amount of cognitive testing can be reduced or eliminated.
Specifically, this is done as follows.
1) When a driver's license photo is taken at a license renewal venue, the photo data is acquired as the subject's facial photo 12. As in the past, only the driver's license photo may be taken, or a smiling facial photo 14 may be acquired with the explanation that it is for dementia confirmation.
2) Image processing is performed on the subject's facial photograph 12 to create facial photograph data 20 to be input into the dementia and/or depression estimation AI 1.
3) Input the data into the AI. If you only have a driver's license photo, input two copies of the photo data.
4) Check the value of the output section 53 of the dementia and/or depression estimation AI1 and determine the estimated values of the dementia level and depression level.
5) Subjects will be notified of the test results, including AI-based estimates of the degree of dementia and depression, as a complement to the question-based cognitive test.

In this way, using AI as a means to complement cognitive tests will contribute to improving the accuracy of tests. Furthermore, using AI in place of cognitive tests will reduce the burden on users.

Although an example has been described in which the expressionless face photograph 13 and the smiling face photograph 14 are used as sample data, training data can also be generated from only one of the photographs.
Generally, many identification photos, such as driver's licenses, are expressionless, and when assessing dementia or depression from such photos, it may be necessary to use only expressionless photos.
The training data is constructed in the same way as the other data, except that the two facial photographs, one with a blank expression and one with a smiling face, are reduced to one blank facial photograph 13. The advantage of using only the blank facial photograph 13 is that it can be easily acquired without requiring the user to "please smile."

It is also possible to use only one smiling face photograph 14. This is because when using photographs of the subject, there may be many smiling faces.
Dementia and depression can sometimes show distinctive characteristics when smiling, so photos of smiling faces only are also effective. By creating training data of smiling faces only and performing machine learning, it is possible to infer illness from smiling face photos 14 alone.

Regarding how to utilize the acquired data, a facial photo 13 of the user with a blank expression and a facial photo 14 of the user with a smiling expression can be acquired as facial photo 12 of the user, and facial photo data 20 can be generated as facial photo data 21 of the blank expression and facial photo data 22 of the smiling expression. After that, AI can be constructed by deliberately using only one of the photographic data.
In this case, from one data group, it is possible to construct AI that distinguishes between expressionless and smiling faces, AI that distinguishes from expressionless faces, and AI that distinguishes from smiling faces.When using the system, if a photo of the user with a expressionless face and a smiling face can be obtained, the AI that distinguishes between expressionless and smiling faces can be used, and if only one photo of an expressionless face or the like can be obtained, the AI that distinguishes from expressionless faces can be used, allowing the AI to be used according to the situation.

Another embodiment will be described with reference to Figures 6 to 8. The same parts as in the first embodiment will be omitted.
Fig. 6 is a flow chart showing a method for creating training data for a dementia and/or depression diagnostic AI according to this embodiment. Fig. 7 is a schematic diagram showing how facial features are extracted for a dementia and/or depression diagnostic AI, where (a) is a schematic diagram showing the area from which facial features are extracted, and (b) is a diagram explaining the rising and falling of the corners of the mouth. Fig. 8 is a table showing an example of training data in the method for creating training data for a dementia and/or depression diagnostic AI according to this embodiment.

Example 1 makes it possible to build AI for dementia and/or depression without obtaining information from medical institutions.
However, in order to build AI from facial photograph data of a subject, a relatively large number of samples and training data are required.
Therefore, there is a need for a method to reduce the workload of building AI by using features closely related to dementia and depression obtained from facial photograph data.

Specialists have pointed out that facial expressions of patients with dementia or depression have specific characteristics. Fig. 7 shows a schematic diagram of a user's face 60.
As an example for discriminating patients with dementia and depression, the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, the degree of mouth opening 65, etc. may be considered.
By quantifying each feature, they can be used as input data for AI.
For example, with regard to the degree of downward movement of the corners of the mouth 61, when the corners of the mouth are raised to the maximum, as shown in Fig. 7(b), the degree of downward movement of the corners of the mouth is set to 0. When the corners of the mouth are lowered to the maximum, as shown in Fig. 7(d), the degree of downward movement of the corners of the mouth is set to 9. When the corners of the mouth are lowered to a certain extent, as shown in Fig. 7(c), the degree of downward movement of the corners of the mouth is set to 5.
In this way, by defining and quantifying the degree of decline for each feature 70, it becomes possible to input this information into the dementia and/or depression estimation AI1.
In addition, by quantifying data from expressionless and smiling faces, it is possible to build AI with higher accuracy.

An example of the training data in this embodiment will be described with reference to the table in FIG.
In Example 1, facial photo data 20, which consisted of expressionless facial photo data 21 and smiling facial photo data 22, were the example data, and the recognition and depression levels were the correct data. The correct data was the same as in Example 1, but the example data was feature quantities 70 extracted from the user's facial photo. Extracted from the facial photo data 20, the degree of drooping of the mouth corners 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 were determined as numerical values. In this example, there are 10 levels ranging from 0 to 9.
As the feature amounts 70, a group of feature amounts 71 of neutral expressions and a group of feature amounts 72 of smiling expressions are used.
The feature amount 70 is extracted only from the facial photograph data 20, so the burden on the user P is not increased.

The present embodiment will be described along the flow of FIG.
The process of obtaining the contents of the prescription 31 etc. from the user (S201), obtaining a facial photograph 12 of the user (S202), and estimating the degree of dementia and depression from the type and amount of medication listed on the prescription 31 (S203) is the same as in Example 1.
Facial feature amounts 70 are extracted from the facial photograph and generated as example data (S204). In this step, similar to the first embodiment, facial photograph data 20 is generated from the user's facial photograph 12. Thereafter, facial feature amounts 70 are extracted from expressionless facial photograph data 21 and smiling facial photograph data 22, which are the facial photograph data 20, and generated as example data.
For example, in the example of training data using feature quantity 70 in Figure 8, in the case of number 1, first, image processing is performed on expressionless face photo data 21, which is the source of the example data, to obtain a mouth corner drooping degree of 8, eyelid drooping degree of 5, cheek drooping degree of 3, eyebrow drooping degree of 5, and mouth opening degree of 2. Next, image processing is performed on smiling face photo data 22 to obtain a mouth corner drooping degree of 7, eyelid drooping degree of 5, cheek drooping degree of 2, eyebrow drooping degree of 5, and mouth opening degree of 2. The method of generating correct answer data is the same as in Example 1.
In this way, one piece of training data 40 is completed (S205). The same process is performed on the other data to generate training data 40.

In this way, by using facial features 70 as example data, the example data becomes specialized for dementia and/or depression, and the amount of input data is reduced, making it possible to improve the accuracy of the AI with a small amount of training data.

Furthermore, by using the facial feature amount 70 as example data, the generation of the training data 40 can be partially simplified.
Dementia and depression experts have extensive knowledge and experience regarding facial feature values 70 and the degree of dementia and depression. Therefore, with the support of experts, it is possible to create a certain amount of training data without using photos of the user or prescriptions 31.
In other words, by estimating the feature values 70 expected at a specific level of dementia and depression, using the estimated feature values 70 as example data, and then using the predicted level of dementia and depression as correct answer data and performing machine learning, it becomes easier to obtain data.

In other words, by asking experts to list the characteristics that are likely to be present in people with dementia or depression when they have a blank expression, it is possible to build up a certain amount of data without collecting photographs.
Similarly, in the case of smiles, by asking people to list the characteristics that distinguish them from a smile and a neutral expression, it is possible to build up some data.

In AI machine learning, there can be a large discrepancy between the AI output and the correct answer in the early stages of learning. By performing machine learning using training data created in advance with the assistance of experts, it is possible to reduce the discrepancy between the AI output and the correct answer when performing machine learning using user data, allowing the AI to converge more quickly.

The explanation will be given along the flow of FIG.
The facial feature quantities 70 and the degree of dementia/depression are quantified by an expert (S301). For example, the facial feature quantities 70 include a combination of the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is expressionless, and the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is smiling, and the degree of dementia/depression is quantified between 0 and 100%.
An example is shown in Figure 10(a). This example is a table showing the estimated dementia level when the change in the degree of drooping of the corners of the mouth and the degree of drooping of the eyelids from a neutral expression to a smiling expression is changed on a 10-point scale, assuming that the cheeks are drooped to a degree of 5, the eyebrows are drooped to a degree of 4, the mouth opening is open to a degree of 2, the eyelids are drooped to a degree of 4, and the corners of the mouth are drooped to a degree of 5 when the face is expressionless.
The range of change in the degree of downward movement of the mouth corners is from −5 to 4, since the degree of downward movement of the mouth corners when the face is expressionless is 5. The range of change in the degree of downward movement of the eyelids is from −4 to 5, since the degree of downward movement of the eyelids when the face is expressionless is 4.
For example, if the change in the degree of eyelid drooping when smiling is 0 and the change in the degree of mouth corner drooping is -3, the smile is appropriately expressed and the recognition rate is considered to be low. As an example, the maximum and minimum values of the change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 34%.
Similarly, if the change in the degree of downward movement of the corners of the mouth when smiling is +2, the corners of the mouth are further downwards despite the smile, and it is considered that the smile is not being properly expressed, so the recognition rate is considered to be high. As an example, the maximum and minimum values of the change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 59%.

Such a table is created for all the feature quantities 70 .
Using these, the feature levels are set as example data, and the estimated dementia and depression levels are set as correct data (S302). Figure 10(b) shows an example of a training data group. When all facial feature values 70 are changed from 0 to 9, the correct data is listed as dementia level XX% and depression level XX%. The example in Figure 10(b) is an example of a single feature value group. If a neutral feature value 71 and a smiling feature value 72 are used, the number of classifications will be even larger.
The training data is completed through expert guidance (S303).
Next, experts are asked to generate dementia and depression levels, focusing on combinations of feature quantities 70 that are likely to actually exist (S304). This process increases the amount of training data that is closer to reality. This completes the initial training data for machine learning (S305).

By generating training data in this way, the amount of data to be collected can be reduced and the convergence of AI machine learning can be accelerated.

Another embodiment will be described with reference to Figures 13 to 17. Portions that are the same as those in the first embodiment will be omitted.
Fig. 13 is a system diagram of Example 3 according to the present invention. Fig. 14 is a diagram of screens of a mobile terminal according to Example 3 according to the present invention, in which (a) is a screen for asking questions to confirm the level of energy, (b) is a screen for requesting permission to acquire teacher data, (c) is a screen for displaying guidance for acquiring a facial photo of the user by the mobile terminal, and (d) is a screen for displaying guidance for confirming the medications taken by the user by the mobile terminal.
Fig. 15 is a table showing information for generating teacher data according to Example 3 of the present invention. Fig. 16 is a flow diagram showing a teacher data acquisition procedure according to Example 3 of the present invention. Fig. 17 is a flow diagram showing an example of a mobile terminal in the dementia and/or depression prediction AI system according to the present invention.

According to Example 1, it is possible to build AI for dementia and/or depression by generating training data from information at pharmacies without obtaining information from medical institutions.
However, in order to build AI from facial photograph data of a subject, a relatively large number of samples and training data are required.
Therefore, there is a need for a method to obtain training data on dementia and depression, not just for pharmacies.

In recent years, apps for mobile devices such as smartphones have been developed that provide advice on diet and exercise to prevent dementia.
Therefore, in this embodiment, it is possible to obtain information that serves as training data for AI from an application on a mobile terminal used by a user.
The training data includes at least a facial photograph of the user, a prescription, instructions, or information that can be used to infer the user's dementia and/or depression. The information that can be used to infer the user's dementia and/or depression is obtained by asking the user questions.
By obtaining information related to training data from a mobile device app in this way, it is possible to obtain sample data from many people, not just pharmacies.

FIG. 13 shows a system diagram of this embodiment.
The dementia and/or depression estimation AI1 in this embodiment is a system that obtains training data from the user's mobile terminal via a network.
The dementia and/or depression prediction AI1 consists of a machine learning system 50, a management unit 80, a server 81, and a mobile terminal 90.

The machine learning system 50 is the core part of the AI, as in the first embodiment.
It has a learning phase where it learns using training data, and a utilization phase where it uses the intermediate part constructed through learning to provide AI answers.
During learning, training data sent from the management unit 80 is used to learn, intermediate adjustments are made, and the accuracy of the AI is improved.
When in use, the recognition level and depression level are estimated from the photograph data sent from the management unit 80, and the estimated values are returned to the management unit 80.
The management unit 80 is a part that acts as a bridge between the mobile terminal 90 and the machine learning system 50. The management unit 80 writes data related to the generation of teacher data to and reads data from the server 81.
The information necessary for generating teacher data is received from the mobile terminal 90 via the Internet N, and some of it is processed and adjusted, and then stored in the server 81, including what has been converted into teacher data.
The information to be processed may be, for example, a photograph of a medicine taken by the user. If the received photograph of a medicine is a medicine notebook, the management unit 80 reads the name of the medicine written in the notebook and sets it as the medicine content. If the received photograph of a medicine is a medicine bag or the medicine itself, the name of the medicine is identified from the name on the surface of the medicine bag or the name on the medicine sheet.
Thereafter, similarly to Example 1, the recognition level and the depression level are estimated and calculated using the recognition level and depression level calculation table for the drug name as shown in Figure 3. The calculated values are stored in the server 81.
Facial photographs are also processed and adjusted in the same way. Since facial photographs taken by users may not be suitable as training data depending on the shooting environment and the user's shooting method, the management unit 80 performs the same operations as in Example 1, such as simplifying the size of the face and the background. The processed photographs are stored in the server 81.
The management unit 80 reads out the training data stored in the server 81, which includes the facial photograph, recognition level, and depression level, and sends the training data to the machine learning system 50 that is currently learning.

The server 81 mainly processes user information 83, information acquired from the user 84, and information related to the teacher data, and stores the information that has been used as example data 41 of the teacher data and information that has been used as correct answer data 42. These are collectively referred to as server management data 82.
The server management data 82 will be described with reference to FIG.
A user ID is assigned to each user, and information is managed by the user ID. For example, the user with user ID 1001 is Yamada Taro, and the user with user ID 1056 is Suzuki Hanako.
Each person's date of birth, gender, and medical history are recorded, and this data is generally not changed after registration.
A data number and a data acquisition date are assigned to each collection of information 84 acquired from the user by the management unit 80. The data numbers are unique and there are no duplicates. Age, a photo of a face with a neutral expression, a photo of a face with a smiling expression, a photo related to medication, and awareness and depression levels estimated from questions posed by the app are sent to the management unit 80 and recorded on the server 81.
Medications may be photographs of a medicine notebook, a medicine packet, or a medicine sheet.
The management unit 80 adjusts and processes the information from the mobile terminal 90, records the adjusted facial photo as example data 41, and records the contents of the medicine and the recognition and depression values estimated from the medicine as correct answer data on the server 81.
Information managed by one data number corresponds to one piece of training data. As information related to training data, multiple data numbers may be stored for one user. This is because the user's status may change depending on the time of acquisition.

The server management data 82 records the awareness and depression level information generated from the drug information, and the awareness and depression level estimated from the questions posed by the app.
The awareness and depression level information generated from the drug information may be used as the correct answer data 42, or the awareness and depression level estimated from the questions posed by the app may be used as the correct answer data.
Alternatively, the correct answer data may be the average of the data generated from the drug information and the data estimated from the questions posed by the app.
In addition, if the photo of the medication taken by the user is unclear, the awareness and depression levels estimated from the questions posed by the app may be used as an alternative.

The mobile terminal 90 connects each user with the management unit 80 .
The mobile terminal 90 mainly comprises a display unit 91, a control unit, a storage unit, an operation unit, a communication unit, and a camera unit, all of which are not shown.
The control unit is a part that controls the entire portable terminal 90. While the application 92 is running, the control unit also performs control according to the content of the application.
The control unit acquires the operation contents of the user from the operation unit, displays characters, symbols, images, etc. on the display unit 91, takes pictures with the camera, and instructs the communication unit to send and receive data to and from the network. It also writes and reads temporary data to and from the memory unit, and reads out applications 92.
The storage unit stores a program for operating the control unit and temporarily stores variables, data, etc. used for control. The application 92 of the embodiment is also stored as one of the programs.

The camera is a camera that takes pictures of the surroundings of the mobile terminal 90. In this embodiment, the camera takes pictures of the user's face and the medicine notebook.
The communication unit is a communication unit that communicates with the Internet N. The communication unit connects to the Internet N using a mobile communication system such as Wi-Fi or 4G.
The operation unit is the mechanism that allows the user to operate the terminal. It is operated using buttons and a touch panel.
The display unit 91 is a part that displays text and images such as guidance and messages, and serves as a screen display for an application and a monitor for captured images.

In this embodiment, the application 92 is assumed to be an exercise recommendation application for preventing dementia in the elderly, or a health check application.
Some of these apps are configured to have a section that acquires information about training data.
Users who install dementia prevention apps are people who are concerned about their risk of developing dementia. In other words, they are people who are at a high risk of developing dementia or depression.
Therefore, the training data obtained from a mobile device on which such an app is installed is highly likely to be training data that indicates the early stages of dementia and depression, and by using it for AI training, the accuracy of AI in estimating dementia and depression can be improved.

The flow of acquiring information related to the teacher data will be described with reference to the flow of FIG.
In this embodiment, the application 92 functions as a part of the application 92 in the mobile terminal 90 .
Examples of the application 92 include an exercise recommendation application for preventing dementia in the elderly and a health check application.
When a user launches application 92, it is checked using in-app data, etc., whether this is the first time the app has been launched or whether three months have passed since the last request to obtain teacher data-related information (S601).
If not, the result is NO and the application proceeds to normal operation (S608).
If so, an information provision request screen 94 for AI learning is displayed on the screen (FIG. 14(b)) to request cooperation in AI learning in S602.

If the user selects "Yes" in S603, information acquisition will proceed. If the user selects "No," the application will proceed to normal operation (S608).

As a guidance for taking a face photo, a face photo guidance screen 95 is displayed on the display unit 91, and a face photo with a neutral expression and a face photo with a smiling expression are taken (S604, FIG. 14(c)). If it is difficult to specify a face photo with a neutral expression, a face photo with a normal, natural face may be specified.
As guidance for obtaining information about medicine, a medicine information photographing guidance screen 96 is displayed on the screen, and a photograph of the page of the medicine notebook showing the medicine information, the medicine envelope, etc. is taken (S605, FIG. 14(d)).

Questions related to dementia and depression (FIG. 14(a)) are asked, and the awareness level and depression level estimated from the user's answers are acquired as awareness level and depression level according to the questions (S606). The awareness level and depression level are estimated simply using an app on the mobile device.
Alternatively, the content of the question may be notified to the management unit 80 via the Internet, and an appropriate estimation result may be obtained from the management unit 80.
The user's ID number, a facial photo, a photo related to medication, and the degree of awareness and depression estimated from answers to questions, all managed under a single data number, are transmitted to the management unit 80 via the Internet N (S607).
Once the information has been acquired and transmitted, the application proceeds to normal operation (S608).
Through this series of operations, information related to the teacher data can be obtained extremely easily via the mobile terminal 90.

In addition, the dementia and/or depression estimation AI1 can be used on the mobile terminal 90 to provide advice to the person being diagnosed.
The system is designed so that a photograph of the subject's face taken with the mobile device used by the subject can be obtained via the network, and advice is provided based on the results of the AI system's analysis of the obtained photograph.
For example, a photo of the subject's face is taken in response to a call from an application 92 on a mobile terminal 90, and advice on dementia and depression is provided based on the estimated dementia and depression scores by AI. This will be explained along the flow in FIG. 17 .

1) The application 92 in the smartphone, which is the mobile terminal 90, proposes a simple diagnosis of dementia and depression to the subject (S701).
The suggestions will come from an exercise recommendation app to prevent dementia in the elderly and a health check app, so subjects can receive a simple diagnosis without feeling uncomfortable.

2) The subject takes a photograph of their own face according to the guidance of the application 92 (S702). It is preferable to take two photographs of the face, one with a neutral expression and one with a smiling face, as this can improve the accuracy of estimating dementia and depression.
By taking a picture while viewing the image of the user on the mobile terminal 90, the user can take a picture while being conscious of his or her expressionless face and smiling face, and thus can take a more appropriate facial picture.

3) The application 92 sends the facial photograph to the management unit 80 via the Internet N.
The management unit 80 processes the facial photograph of the subject by removing the background, adjusting the size of the face, etc., so that the facial photograph data becomes the same as the sample data (S703).

4) The management unit 80 inputs the photo data into the input unit 51 of the machine learning system 50, which is the core of the AI (S704).
5) The values of the output section 53 of the machine learning system 50 are checked, and the estimated values of the dementia level and depression level are determined (S705).
6) The estimated value is transmitted to the mobile terminal 90 via the Internet N (S706).

7) The subject will be notified of the AI-estimated level of dementia and depression, and advice based on the AI estimate will be displayed on the screen. For example, the advice may read, "The level of dementia is estimated to be 70%. We recommend that you seek medical advice as soon as possible."
The content of the advice may be generated by the management unit 80, or the content may be displayed on the mobile terminal 90 in advance according to the estimated value.
Furthermore, in the case of the second or subsequent simple diagnosis, the previous estimated values may also be displayed, and advice may be given according to the increase or decrease in the value (S707). For example, the advice may be, "The dementia level is estimated to be 30%, which is better than the previous estimated value of 40%. Please continue to exercise to prevent dementia."

By making such suggestions, the subject's potential level of dementia and depression can be confirmed without burdening the subject, thereby contributing to the early detection of dementia and/or depression.

The dementia and/or depression prediction AI system and training data creation method of the present invention can naturally be used as technology for building dementia and/or depression prediction AI outside of hospitals, and can also be applied to other cases in which the illness manifests as a distinctive facial expression. Therefore, it is believed that the present invention has great industrial applicability.

1. Dementia and/or depression prediction AI
10 Pharmacy 11 Photography equipment 12 User's face photo 13 Expressionless face photo 14 Smiling face photo 20 Face photo data 21 Expressionless face photo data 22 Smiling face photo data 30 Medicine notebook 31 Prescription 32 Medicine data 40 Teacher data 41 Example data 42 Correct answer data 43 Calculation table 50 Machine learning system 51 Input unit 52 Intermediate unit 53 Output unit 54 Correct answer unit 60 User's face 61 Degree of drooping of mouth corners 62 Degree of drooping of eyelids 63 Degree of drooping of cheeks 64 Degree of drooping of eyebrows 65 Mouth opening degree 70 Feature 71 Expressionless feature 72 Smile feature 80 Management unit 81 Server 82 Server management data 83 User information 84 Information acquired from the user 90 Mobile terminal 91 Display unit 92 Application 93 Energy level check question column 94 Information provision request screen for AI learning 95 Face photo guidance screen 96 Drug information photography guidance screen P User D Error N Internet

Claims (9)

A method for creating training data to be used in AI that estimates the degree of dementia and/or depression from a user's facial photograph, comprising:
1. A system for estimating the degree of dementia and/or depression,
The information that serves as training data for the AI can be obtained from the location where the user picks up the prescription medicine or from an application on the mobile device used by the user.
The training data includes at least a facial photograph of the user, and information that can be used to estimate the user's dementia and/or depression, obtained from a prescription, an instruction sheet, or questions to the user,
Using either a facial photograph of the user with a neutral expression or a facial photograph of a smiling expression, or both, and extracting facial features from the facial photograph to generate example data of training data;
The feature amount has at least one of the degree of drooping of the corners of the mouth, the degree of drooping of the eyelids, the degree of drooping of the cheeks, the degree of drooping of the eyebrows, and the degree of mouth opening,
The degree of dementia and the degree of depression are estimated from the type and amount of the medicine written in the prescription or instruction sheet and used as correct answer data for training data.
A teacher data creation method for creating teacher data for machine learning. Estimating the feature amounts expected at a specific dementia level and depression level;
The teacher data creation method described in claim 1, characterized in that the estimated features are used as the example data, and the predicted dementia level and depression level are used as the correct answer data, and machine learning is performed. An AI system using teacher data created by the teacher data creation method according to claim 1 or 2,
A dementia and/or depression estimation AI system that estimates the degree of dementia and depression of a subject based on a facial photograph of the subject. The dementia and/or depression prediction AI system described in claim 3, characterized in that a facial photograph of the subject is taken at the pharmacy, and advice is provided based on the results of the AI system's analysis of the photograph. The dementia and/or depression prediction AI system described in claim 3, characterized in that a photograph of the subject's face is taken and, depending on the results of the AI system's analysis of the photograph, the subject is guided to video content. The dementia and/or depression prediction AI system described in claim 3, characterized in that it takes a facial photo of the subject and, depending on the results of the AI system's analysis, refers the subject to a specialized medical institution. The dementia and/or depression prediction AI system described in claim 3, characterized in that a photograph of the subject's face is taken and advice is provided to the family based on the results of the AI system's analysis of the photograph. The dementia and/or depression prediction AI system described in claim 3, characterized in that a facial photograph of the subject is taken at an OTC medical supply store, and advice is provided based on the results of the AI system's analysis of the photograph. The dementia and/or depression prediction AI system described in claim 3, characterized in that a facial photograph of the subject taken with a mobile device used by the subject can be obtained via a network, and advice is provided depending on the results of the AI system's analysis of the obtained facial photograph.