JP7626487B2 - Dementia and/or depression prediction AI system and teacher data creation method - Google Patents

Description

The present invention relates to AI for assessing medical conditions, and more specifically, to technology for estimating dementia and/or depression using facial photographs.

Early detection and treatment of dementia and depression are important. One method for detecting symptoms is to use AI to identify the subject's facial photograph and estimate the degree of dementia or depression.
However, building such a system generally requires information managed by the medical institution, and it is not easy for other businesses to do so.
There was therefore a need for a method that could use AI to estimate the degree of a subject's dementia or depression from information obtained outside of a hospital.

To address such problems, various techniques have been proposed. For example, a technique for assisting doctors in diagnosis by image analysis using AI (see Patent Literature 1) has been proposed and is a known technique. More specifically, this is a device that uses training data consisting of image data of an image of a subject, findings data, and lifestyle and dietary habits information of the subject, inputs image data and subject information, performs machine learning to output findings data, and outputs the findings data using this AI.
However, since data acquisition in this prior art is carried out at the hospital, other businesses cannot handle it, and the above problem is not resolved.

JP 2021-108800 A

In consideration of the above problems, the present invention aims to develop an AI that can estimate the degree of dementia and/or depression of a subject based on information obtained in a place other than a hospital.

To solve the above problems, the present invention provides a method for creating training data for a system that estimates the degree of dementia and/or depression, in which information serving as training data for AI can be obtained from a location where a user receives prescription medication or from an application on a mobile device used by the user, and the training data includes at least a facial photograph of the user and a prescription or instruction sheet, or information that can estimate the user's dementia and/or depression.

Information that can be used to infer the user's dementia and/or depression is obtained by asking questions to the user.

The present invention also employs a means for creating training data for machine learning using two facial photographs of the user, one with a blank expression and one with a smiling expression, as example data for the training data, and using the degree of dementia and depression estimated from the type and amount of medication listed on the prescription or instructions as correct answer data for the training data.

Furthermore, the present invention employs a means for creating training data for machine learning by using either a facial photograph of the user with a neutral expression or a facial photograph of a smiling expression as example data for the training data, and using the degree of dementia and depression estimated from the type and amount of medication listed on the prescription or instructions as correct answer data for the training data.

Furthermore, the present invention employs a means for including age, sex, medical history, and family medical history as the example data of the training data.

Furthermore, the present invention employs a means for using features extracted from a facial photograph of the user as the example data of the training data.

Furthermore, the present invention employs a means for having, as the feature amount, at least one of the degree of drooping of the corners of the mouth, the degree of drooping of the eyelids, the degree of drooping of the cheeks, the degree of drooping of the eyebrows, and the degree of mouth opening.

Furthermore, the present invention estimates the feature values predicted for a specific dementia level and depression level, sets the estimated feature values as the example data, and employs a means for performing machine learning by setting the predicted dementia level and depression level as the correct answer data.

Furthermore, the present invention provides an AI system for predicting dementia and/or depression that uses any of the above-mentioned training data and employs a means for predicting the degree of dementia and depression of a subject based on a facial photograph of the subject.

Furthermore, the present invention employs a means for taking a photograph of the user's face at the pharmacy and providing advice based on the results of the AI system.

Furthermore, the present invention employs a means for taking a photograph of the subject's face and directing the subject to video content depending on the results of the AI system.

Furthermore, the present invention employs a method for taking a photograph of the subject's face and referring them to a specialized medical institution depending on the results of the AI system.

Furthermore, the present invention employs a means for taking a photograph of the subject's face and notifying the family of advice based on the results of the AI system.

The present invention also employs a method for taking a facial photo of the user at an OTC medical store and providing advice based on the results of the AI system.

Furthermore, the present invention employs a means for acquiring a facial photograph of the user taken with a mobile device used by the user via a network, and for notifying advice based on the results of the AI system for the acquired facial photograph.

According to the dementia and/or depression prediction AI system and teacher data creation method of the present invention, information for constructing an AI that predicts the degree of dementia or depression of a subject can be obtained outside of a hospital, so that AI that predicts dementia and/or depression can be constructed by anyone, not just hospital personnel.
In addition, by utilizing applications on the mobile devices used by users, a large amount of information can be easily obtained to build AI that predicts dementia and/or depression.

FIG. 2 is a flow chart showing an embodiment of a teacher data creation method according to the present invention. FIG. 1 is a schematic diagram showing an embodiment of a teacher data creation method according to the present invention. 1 is a calculation table showing an example of generating correct answer data in the teaching data generating method according to the present invention. 1 is a table showing an example of teacher data in a teacher data creation method according to the present invention; FIG. 2 is a schematic diagram showing an example of AI operation in the dementia and/or depression prediction AI system according to the present invention. FIG. 11 is a flow chart showing another embodiment of the teaching data creation method according to the present invention. FIG. 1 is a schematic diagram showing an example of extracting facial features in the dementia and/or depression prediction AI system according to the present invention. FIG. 11 is a table showing another embodiment of the teacher data in the teacher data creation method according to the present invention. FIG. 2 is a flow diagram showing an example of teacher data in the teacher data creation method according to the present invention. FIG. 2 is an explanatory diagram showing an example of teacher data in the teacher data creation method according to the present invention; FIG. 11 is a flow chart showing an example of an embodiment of the dementia and/or depression prediction AI system of the present invention at a pharmacy. FIG. 1 is a flow chart showing an example of monitoring support in the dementia and/or depression prediction AI system according to the present invention. FIG. 11 is a system diagram of another embodiment of the present invention. FIG. 11 is a diagram of a screen of a mobile terminal according to another embodiment of the present invention. 13 is a table showing information for generating training data in another embodiment of the present invention. FIG. 11 is a flowchart showing a procedure for acquiring teacher data according to another embodiment of the present invention. FIG. 11 is a flow chart showing an example of a dementia and/or depression prediction AI system according to the present invention on a mobile terminal.

The greatest feature of the dementia and/or depression prediction AI system and teacher data creation method of the present invention is that it can obtain information for estimating the degree of dementia or depression of a subject in a location other than a hospital, thereby making it possible to construct AI that can predict dementia and/or depression not only for hospital personnel but also for anyone other than hospital personnel.
Hereinafter, embodiments of the dementia and/or depression prediction AI system and teacher data creation method according to the present invention will be described with reference to the drawings.

Furthermore, the overall configuration and the configuration of each part of the dementia and/or depression prediction AI system and teacher data creation method described below are not limited to the examples described below, and can be modified as appropriate within the scope of the technical idea of the present invention, i.e., within the scope of configurations that can achieve the same functional effects.
In addition, since the present invention is a dementia and/or depression estimation AI system and a teacher data creation method, it naturally includes a dementia estimation AI system, a depression estimation AI system, and a dementia and depression estimation AI system, but the examples will focus on examples of estimating dementia and depression.
In the case of a dementia prediction AI system, the system can be easily constructed by excluding the depression factor, which is another factor. The same is true for a depression prediction AI system.
In addition, in the embodiments, a user refers to a person who cooperates in providing training data, and a subject refers to a person whose medical condition is estimated using AI.

The present invention will be described with reference to FIGS.
FIG. 1 is a flow diagram showing an embodiment of the teacher data creation method according to the present invention. FIG. 2 is a schematic diagram showing an embodiment of the teacher data creation method according to the present invention, where (a) is a schematic diagram of data acquisition work performed in a pharmacy, (b) is a schematic diagram of the work of creating example data from a user's face photo, and (c) is a schematic diagram of the work of creating correct data from a prescription or the like. FIG. 3 is a calculation table showing an embodiment of generating correct answer data in the teacher data creation method according to the present invention, where (a) is a table for estimating and calculating the dementia degree and depression degree, and (b) is a calculation table for estimating and calculating the dementia degree and depression degree by actually entering the type and amount of medicine. FIG. 4 is a table diagram showing an embodiment of teacher data in the teacher data creation method according to the present invention, where (a) is an example in which the correct answer data is in 10% frequency increments, (b) is a case where the frequency is 1 or 0, and (c) is a case where age, etc. are included in the example data. FIG. 5 is a schematic diagram showing an example of AI operation in the dementia and/or depression prediction AI system of the present invention, where (a) shows an example of dementia, etc. being predicted by AI, and (b) shows the process of machine learning being performed by AI.

The dementia and/or depression estimation AI1 is an AI that estimates the degree of dementia and depression, and its estimation capabilities are improved through machine learning.
One method for constructing a dementia and/or depression prediction AI1 is to generate training data and have the AI learn a large amount of it to make highly accurate predictions.
Teacher data is data used to train AI in machine learning. Teacher data is often generated by pairing example data with correct answer data.
Example data is data that represents the phenomenon that you want the AI to estimate, and answer data is data that you want the AI to answer.

When asking an AI to estimate whether or not a person has an illness, for example, the subject's symptoms are used as example data, and the name of the subject's illness is used as correct answer data. By having the AI learn a large amount of such training data, an AI that can estimate illness is constructed.

When examining illnesses such as dementia or depression, doctors often look at facial expressions as part of their diagnosis.
When considering a system that uses AI to predict dementia or depression, the subject's facial expressions can be obtained by photographing them with a camera, but the subject's illness is managed by a doctor in a medical record, etc., and it is generally difficult to take that data outside the hospital.

Therefore, in this embodiment, the name and severity of the subject's illness is estimated from the prescription, etc. at the pharmacy, and training data is obtained.
In other words, this is a method for creating teacher data in which information that serves as teacher data for AI is obtained at the location where the user receives the prescription medication, and the teacher data includes at least a photograph of the user's face and the prescription or instructions.
The information that serves as the teacher data includes information that serves as the basis of the teacher data and information that is used as the teacher data.
Prescriptions, etc. include at least prescriptions, medicine notebooks, and instructions. Instructions, also called medication instructions, are documents in which a doctor instructs a guardian or facility on the time and amount of medication to be administered.
The procedure for acquiring teacher data will be described with reference to FIG. 1 and FIG.
Fig. 1 shows a flow for generating teacher data 40. Fig. 2 shows a procedure for generating the teacher data 40.
The information is collected by the pharmacy 10. The pharmacy here includes both pharmacies that are independent of the hospital and pharmacies that are installed within the hospital. The data handled is the same in both cases.
First, the contents of the prescription 31 are obtained from the user P at the pharmacy 10. The contents of the medicine notebook 30 and the instruction sheet may be obtained together with the prescription 31. Since the user P has come to the pharmacy 10 to obtain the medicine listed on the prescription 31 and the medicine notebook 30, obtaining the prescription 31 can be done easily.
Apart from dispensing medication, permission is obtained to use information from the prescription 31, medication notebook 30, and instructions to build the dementia and/or depression prediction AI1, and the information is acquired (S101, Figure 2 (a)).
Next, in order to construct the dementia and/or depression prediction AI1, permission is obtained to obtain a facial photograph 12 of the user P, a photograph of the user P is taken with the photographing device 11, and the facial photograph 12 of the user is obtained (S102, FIG. 2(a)).
As the user's facial photograph 12, a facial photograph 13 of the user with a neutral expression and a facial photograph 14 of the user with a smiling expression are acquired. An expressionless face can be said to be a natural, ordinary face.
It is said that in cases of dementia or depression, the eyes tend to look lifeless, the eyelids droop, and the overall expression tends to be lacking. In addition, the smile may be unnatural or may be no different from an expressionless face.
Therefore, by using facial photos of the user with both a blank expression and a smiling expression, the difference between normal people and those with dementia or depression can be emphasized.
When taking a photo with a blank expression, if it is difficult to say, "Please make sure you have a blank expression," you can use a phrase such as, "Please let me take a photo of your face in a natural, normal way."
By steps S101 and S102, the acquisition of information from the user is completed.

Next, the presence or absence and the degree of dementia or depression are estimated from the type and amount of medicine written on the prescription 31, etc. (S103, FIG. 2(b)). A pharmacist or the like can estimate to some extent, but in order to reduce variability, a calculation table 43 is used. An example of the calculation table is shown in FIG. 3(a). Correct answer data is generated by the calculation table 43.
Next, facial photograph data 20 is generated from the user's facial photograph 12. Generally, facial photographs taken outside a studio or the like have different angles of view, resolutions, backgrounds, etc., so by performing image correction so that the angles of view, resolutions, and backgrounds are similar, appropriate facial photograph data 20 can be generated as example data 41 (S104, FIG. 2(c)).
The face photograph data 20 consists of expressionless face photograph data 21 and smiling face photograph data 22.
Through this series of operations, one piece of teacher data is completed (S105).
In other words, the process involves using a photo of the user's face as example data for the training data, and using the level of dementia and depression estimated from the type and amount of medication listed on the prescription or instructions as correct answer data for the training data, to generate training data for machine learning.
The accuracy of machine learning can be improved by increasing the amount of training data.

A calculation table 43 is used to estimate the degree of dementia and depression from the prescription 31, medicine book 30, and instructions.
The calculation table 43 has a column for recording the type and amount of medication, which is to be filled in according to the user's information. There are columns for dementia and depression, and coefficients are recorded according to the type of medication. Drugs closely related to each disease have high coefficients, and drugs less related have low coefficients.
An evaluation score is determined by entering the type and amount of medication. The severity of each illness can be determined by adding up the evaluation scores. To make the data correct, normalization is required, such as from 0 to 100%, so normalization is performed using a predetermined number. There are two ways to calculate correct data: by expressing it as a percentage, or by expressing it as 1 or 0.
3B shows a calculation example using specific numbers. In this example, since there are many medications related to dementia, the total evaluation score for dementia is 23 points, and the total evaluation score for depression is 4 points. As an example, normalization is performed by 30, and the evaluation of dementia is 76.7% and the evaluation of depression is 13.3%. As correct answer data, correct answer data 1 with 10 levels and correct answer data 2 with two levels of 1 and 0 are generated.
Correct answer data 1 has a dementia level of 70% and a depression level of 10%, while correct answer data 2 has a dementia level of 1 and a depression level of 0.
Through this process, correct data on the dementia level and depression level are generated.

4A shows an example of training data. In FIG. 4A, face photo data 20 is used as example data of the training data. As the face photo data 20, a face photo 21 of a neutral face and a face photo 22 of a smiling face are used.
The correct answer data for both dementia and depression is 10-level data. The correct answer data has 100 correct answer elements, 10 levels x 10 levels, so a correspondingly large amount of training data is required for machine learning.
Fig. 4(b) shows a case where the correct answer data for the training data is two-level data for both dementia and depression. The correct answer data is two levels x two levels, which is four correct answer elements, so the amount of training data for machine learning may be less than that in the case of Fig. 4(a).
FIG. 4(c) shows an example data set in which the user's age, sex, and medical history are added in addition to the face photo data. Family medical history can also be added. Using a lot of data can make estimation easier. However, when using AI, it is necessary to input the same data as when learning, so in this case, it is necessary to input the user's age, sex, and medical history in addition to the face photo data, which reduces versatility.

An overview of the operation during learning and utilization of AI will be described with reference to Fig. 5. A machine learning system 50 shows an example of deep learning using a neural network.
The dementia and/or depression estimation AI1 consists of an input section 51, an intermediate section 52, and an output section 53. During machine learning, a correct answer section 54 is added. The input section 51 has elements X1 to Xn for the information to be input to the AI. The intermediate section 52 is a section that inherits information from the input layer and performs various calculations. The more layers the intermediate section 52 has, the more complex the analysis that can be performed. The calculation results are sent to the output section 53. The output section 53 is a section that outputs the answer, and has elements Z1 to Zm for the answer. The value of each element is, for example, between 0 and 1, and the element with the highest value is estimated to be the answer (FIG. 5(a)).
The correct answer unit 54 is a part that sets the correct answer data 42 to be given to the dementia and/or depression estimation AI1 during machine learning, and has elements A1 to Am. The number of elements is the same as that of the output unit 53. Of A1 to Am, the numerical value of the correct answer element is set to 1, and the numerical values of the other elements are set to 0. The differences between the numerical values of Z1 to Zm and A1 to Am are calculated, and the sum of these values becomes the error D. Minimizing the error D increases the accuracy of the AI ( FIG. 5B ).

The operation of the dementia and/or depression prediction AI1 will be simplified for the purposes of explanation, and an example of an AI that predicts whether or not a person has dementia will be described.
Regarding the output unit 53, the AI's answer is either dementia or no dementia. Therefore, the elements of the output unit 53 are, for example, Z1 "has dementia" and Z2 "does not have dementia".
When face photo data 20 including expressionless face photo data 21 and smiling face photo data 22 is input, the number of elements X in input unit 51 is an element according to the resolution.
When face photo data 20 is input to the input unit 51 of the AI, elements Z1 and Z2 of the output unit 53 each output a numerical value between 0 and 1. The higher the numerical value, the more likely the answer is to be estimated by the AI. If the difference between the numerical values is large, the accuracy of the estimation is high, and if the difference between the numerical values is small, the accuracy of the estimation is low.
For example, if Z1 is 0.1 and Z2 is 0.9, Z2 is the higher value, so the AI's answer will be "not dementia," which is Z2. The difference between the values of Z1 and Z2 is large, so the accuracy is high.

4(b) is used, the output unit 53 of the dementia and/or depression estimation AI 1 has four levels: dementia level 0 and depression level 0, dementia level 0 and depression level 1, dementia level 1 and depression level 0, and dementia level 1 and depression level 1. Therefore, Z1 to Z4 can be assigned, respectively.
After the AI is constructed, facial photo data 20, which are expressionless facial photo data 21 and smiling facial photo data 22, are input to the dementia and/or depression prediction AI 1 and processed in an intermediate unit 52 consisting of a neural network, and the results are output as numbers between 0 and 1 for Z1 to Z4 of output unit 53.
For example, if Z1 is 0.1, Z2 is 0.6, Z3 is 0.2, and Z4 is 0.1, the highest value, Z2, is adopted. Since Z2 has a dementia level of 0 and a depression level of 1, the AI's answer is "There is a possibility of depression."
In addition, the highest value may be used as the probability. In this case, since the value of Z2 is 0.6, it may be possible to say that there is a 60% possibility of depression.

4A, the output unit 53 of the dementia and/or depression estimation AI 1 can be divided into 100 categories in increments of 10%, from "dementia level 0% and depression level 0%" to "dementia level 90% and depression level 90%." In this case, the output unit 53 has 100 elements Z1 to Z100.
The Z1 to Z100 of the output unit 53 for the input are checked, and the highest one becomes the answer. If a certain element of Z is the highest, the corresponding dementia level XX% and depression level XX% become the answer.

Another method for simplifying the output unit 53 is to separate it into a dementia estimation AI and a depression estimation AI. By doing so, the number of pieces of data in the input unit 51, the intermediate unit 52, and the output unit 53 can be reduced.
In this case, the final output of the dementia and/or depression prediction AI1 will be the combination of the answers from the two AIs.

Next, the machine learning method of the dementia and/or depression estimation AI 1 will be described with reference to Fig. 5(b). An input unit 51 to the dementia and/or depression estimation AI 1 receives facial photo data 20, which are expressionless facial photo data 21 and smiling facial photo data 22, and an output unit 53 has 100 elements ranging from a dementia degree of 0% and a depression degree of 0% to a dementia degree of 90% and a depression degree of 90%.
First, one of the teacher data 40 is used for the dementia and/or depression estimation AI1. Example data 41 is input to the input unit 51. Correct answer data 42 is set in the correct answer section. If the correct answer data 42 of the teacher data 40 is a dementia level of 70% and a depression level of 10%, Aj (j is the element number corresponding to a dementia level of 70% and a depression level of 10%) corresponding to a dementia level of 70% and a depression level of 10% is set to 1, and A1 to A100 except for Aj are set to 0.
As a result of inputting example data 41 into dementia and/or depression estimation AI1, estimated values are output as numbers between 0 and 1 to Z1 to Z100 of output unit 53. The sum of the differences between Z1 to Z100 and the corresponding A1 to A100 becomes error value D. The neural network of intermediate unit 52 is adjusted to lower this error value D.
By performing this task on all of the large amounts of training data, the error value D is reduced and the accuracy of the AI is improved.

Thus, according to this embodiment, information for constructing an AI that estimates the degree of dementia and/or depression in a subject can be obtained in a location other than a hospital, making it possible to construct an AI for estimating dementia and/or depression that is not limited to hospital personnel.
In addition, by using a facial photo of the person being diagnosed, it will be possible to operate and utilize AI to estimate the subject's level of dementia and/or depression.

In addition, the dementia and/or depression prediction AI1 can be used at the pharmacy 10 to provide advice about dementia and depression to pharmacy customers.
A facial photograph of a subject who desires AI advice is taken, and advice on dementia and depression is provided by the dementia and/or depression estimation AI 1. The process will be described with reference to the flow of FIG.
1) Take a photograph of the subject's face (S401). Take a photograph of the face with both a blank expression and a smiling expression. If this is difficult, only one of them will suffice. This may be done in parallel with the acquisition of teacher data 40 for improving the accuracy of the dementia and/or depression estimation AI1.
2) Image processing is performed from the subject's expressionless facial photo 13 and the subject's smiling facial photo 14 to generate expressionless facial photo data 21 and smiling facial photo data 22, which are facial photo data 20 to be input into the dementia and/or depression prediction AI 1 (S402).
3) Input to AI (S403). If only one photo is acquired, input two of the same photo.
4) Check the value of the output section 53 of the dementia and/or depression estimation AI1, and determine the estimated values of the dementia degree and depression degree (S404).
5) The degree of dementia and depression of the subject are estimated from the prescription 31 and used as the estimated prescription value (S405).
6) If the degree of dementia or depression estimated by the AI is worse than the prescription estimate, the subject is notified of advice (S406).
The same applies if the AI estimates a certain degree of dementia or depression even though Prescription 31 does not contain any prescription for dementia or depression.
In addition, in the case of hospital pharmacies in departments not related to dementia or depression, such as dermatology or surgery, advice may be given without checking prescriptions, etc.

By carrying out such tasks, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and depression.

The dementia and/or depression estimation AI1 can also be used in a health check system, which estimates the degree of dementia and depression of a user during a health check.
1) Take a photograph of the face of the person undergoing the health check. Take two photographs, one with a blank expression and one with a smile. This may be easy if you explain that the photographs will be used for a health check, but if this is difficult, it is fine to take just one of the photographs.
2) Generate facial photo data 20 for AI.
3) Input the image into the AI. If only one image is required, input two identical images.
4) From the AI output results, estimates of the degree of dementia and depression are determined.
5) The target person will be notified of the video content that corresponds to the estimated level of dementia and depression, and will be referred to a specialized medical institution.
The notification may be provided on paper with a QR code (registered trademark) along with the health check results, or a URL may be sent to the subject by email.

In this way, by adding the dementia and/or depression prediction AI1 to a health check, subjects can obtain diagnosis results for dementia and depression without any special burden.

In addition, the dementia and/or depression estimation AI1 can be used in a monitoring support system. When the monitoring support system is used, it estimates the dementia level and depression level of the person being monitored. This will be described along the flow of FIG. 12.
The monitoring support system is a system that uses cameras and sensors to check on the condition of elderly family members and relatives who live far away, and provides support or contacts family members if an emergency occurs.
1) In the monitoring support system, a facial photograph of the subject is obtained from a video image for confirming the safety of the subject (S501). In the monitoring support system, since it is necessary to constantly check the condition of the subject, the subject is often photographed by a camera at all times, and it is easy to obtain facial photographs of the subject with a blank expression and a smiling face. For example, a facial photograph 13 of the user with a blank expression and a facial photograph 14 of the user with a smiling face may be obtained based on the behavior, voice, etc. of the subject.
2) Generate facial photo data 20 for AI from a facial photo of the subject (S502).
3) Input to AI (S503).
4) From the output results of the AI, estimates of the degree of dementia and depression are determined (S504).
5) If the estimated level of dementia or depression is worse than expected, the AI-estimated level of dementia or depression of the subject and corresponding advice will be provided to the relatives or family members (S505).
Specifically, if a subject who is not diagnosed with dementia is suspected to have a high level of dementia, the system will notify pre-registered relatives and family members of the subject's possible dementia. At that time, the system will also introduce medical institutions to confirm dementia and provide advice on how to treat the subject.

In this way, by adding the dementia and/or depression prediction AI1 to the monitoring support system, it is possible to predict the possibility of dementia or depression in a subject without placing any special burden on the subject, and to take early action by contacting relatives or family members.

The dementia and/or depression estimation AI1 can also be used for OTC products. When dealing with a user who visits an OTC medical product store, the AI1 estimates the degree of dementia and depression of the user.
1) Take a facial photograph of the subject over the counter. The subject can be encouraged to take a photograph as it allows for simple confirmation of dementia and depression. Take a facial photograph 13 of the user with no facial expression and a facial photograph 14 of the subject with a smile. If this is difficult, one of them will suffice.
2) Image processing is performed on the subject's facial photo 12 so that it becomes facial photo data to be input into the dementia and/or depression prediction AI 1.
3) Input the image into the AI. If only one image is required, input two identical images.
4) Check the value of the output section 53 of the dementia and/or depression estimation AI1 and determine the estimated values of the dementia level and depression level.
5) The subjects will be informed that these are merely estimates and will be informed of the estimated level of dementia and depression, as well as advice based on those estimates.
Specifically, if a subject who is unaware of dementia or depression is suspected to have a high level of dementia or depression, the system will provide advice such as a referral to a medical institution to confirm dementia or depression and guidelines on how to respond.

In this way, by using the dementia and/or depression prediction AI1 over-the-counter, subjects can receive simple diagnoses and advice about dementia and depression without any special burden.

It can also be used when renewing a driver's license.
When renewing a driver's license, a cognitive test may be conducted in the form of questions to check the onset and severity of dementia, etc. However, as this takes a certain amount of time, it is often considered a roundabout way of doing things by those who are eligible.
By using the dementia and/or depression prediction AI1 instead of or as a complement to cognitive testing, the accuracy of cognitive testing can be improved, or the amount of cognitive testing can be reduced or eliminated.
Specifically, the procedure is as follows.
1) When a driver's license photograph is taken at a license renewal venue, the photographic data is acquired as the subject's facial photograph 12. As in the past, only the driver's license photograph may be taken, or a smiling facial photograph 14 may be acquired with the explanation that this is for dementia confirmation.
2) Image processing is performed on the subject's facial photo 12 to generate facial photo data 20 to be input into the dementia and/or depression prediction AI 1.
3) Input the data into the AI. If you only have a driver's license photo, input two copies of the photo data.
4) Check the value of the output section 53 of the dementia and/or depression estimation AI1 and determine the estimated values of the dementia level and depression level.
5) Subjects will be notified of the test results, which will include AI-based estimates of the level of dementia and depression, as a complement to the question-based cognitive test.

In this way, using AI as a means to complement cognitive tests will contribute to improving the accuracy of tests. In addition, using AI instead of cognitive tests will reduce the burden on users.

Although an example has been described in which the expressionless face photograph 13 and the smiling face photograph 14 are used as sample data, training data can also be generated from only one of the photographs.
Generally, photos taken for identification purposes such as driver's licenses often have a blank expression on the face, and when trying to infer dementia or depression from such photos, it may be possible to use only photos with a blank expression.
The training data is constructed in the same way as the other data, except that the two facial photographs, one with a blank expression and one with a smile, are reduced to one blank facial photograph 13. The advantage of using only blank facial photograph 13 is that it can be easily acquired without requiring the user to "smile", etc.

It is also possible to use only one smiling face photograph 14. This is because, when photographs of the subject are used, there may be many photographs in which the subject is smiling.
Dementia and depression may show distinctive features when smiling, so photos of smiling faces only are also effective. By creating training data of smiling faces only and performing machine learning, it is possible to infer illness from only smiling face photos 14.

Regarding the method of utilizing the acquired data, a facial photo 12 of the user is acquired as a facial photo 13 of the user with a blank expression and a facial photo 14 of the user with a smiling expression. As facial photo data 20, facial photo data 21 of the user with a blank expression and facial photo data 22 of the user with a smiling expression are generated. Then, AI can be constructed by deliberately using only one of the photo data.
In that case, from one data group, it is possible to construct AI that distinguishes between a blank expression and a smiling face, an AI that distinguishes from a blank expression, and an AI that distinguishes from a smiling face. When using the AI, if a photo of the user with a blank expression and a smiling face can be obtained, the AI that distinguishes between a blank expression and a smiling face can be used, and if only one photo of a blank expression can be obtained, the AI that distinguishes from a blank expression can be used, etc., allowing the AI to be used according to the situation.

Another embodiment will be described with reference to Figures 6 to 8. The same parts as in the first embodiment will be omitted.
Fig. 6 is a flow diagram showing a method for creating training data for dementia and/or depression diagnostic AI according to the present embodiment. Fig. 7 is a schematic diagram showing how facial features are extracted for dementia and/or depression diagnostic AI, where (a) is a schematic diagram showing the part where facial features are extracted, and (b) is a diagram explaining the rise and fall of the corners of the mouth. Fig. 8 is a table showing an example of training data in the method for creating training data for dementia and/or depression diagnostic AI according to the present embodiment.

Example 1 makes it possible to build AI for dementia and/or depression without obtaining information from medical institutions.
However, in order to build an AI from a subject's facial photo data, a relatively large amount of sample and training data is required.
Therefore, there is a need for a method to reduce the workload of building AI by using features closely related to dementia and depression obtained from facial photo data.

Specialists have pointed out that facial expressions of patients with dementia or depression have distinctive features. Fig. 7 shows a schematic diagram of a user's face 60.
As an example for discriminating patients with dementia and depression, the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, the degree of mouth opening 65, etc. may be considered.
By converting each feature into a numerical value, they can be used as input data for AI.
For example, with regard to the degree of downward movement of the corners of the mouth 61, when the corners of the mouth are raised to the maximum, as shown in Fig. 7(b), the degree of downward movement of the corners of the mouth is set to 0. When the corners of the mouth are lowered to the maximum, as shown in Fig. 7(d), the degree of downward movement of the corners of the mouth is set to 9. When the corners of the mouth are lowered to a certain degree, as shown in Fig. 7(c), the degree of downward movement of the corners of the mouth is set to 5.
In this way, by defining and quantifying the degree of decline for each feature 70, it is possible to input this to the dementia and/or depression estimation AI1.
In addition, by quantifying data from expressionless and smiling faces, it is possible to build a more accurate AI.

An example of the training data in this embodiment will be described with reference to the table in FIG.
In the first embodiment, facial photo data 20, which consisted of expressionless facial photo data 21 and smiling facial photo data 22, were the example data, and the recognition and depression levels were the correct data. The correct data was the same as in the first embodiment, but the example data was feature quantity 70 extracted from the facial photo of the user. Extracted from the facial photo data 20, the degree of descent of the mouth corners 61, the degree of descent of the eyelids 62, the degree of descent of the cheeks 63, the degree of descent of the eyebrows 64, and the degree of mouth opening 65 were determined as numerical values. In this example, there are 10 levels from 0 to 9.
As the feature amounts 70, a group of feature amounts 71 of neutral expressions and a group of feature amounts 72 of smiling expressions are used.
Since the feature amount 70 is extracted only from the facial photograph data 20, the burden on the user P is not increased.

The present embodiment will be described with reference to the flow chart of FIG.
The parts of obtaining the contents of the prescription 31 etc. from the user (S201), obtaining a facial photo 12 of the user (S202), and estimating the degree of dementia and depression from the type and amount of medication listed on the prescription 31 (S203) are the same as in Example 1.
Facial feature values 70 are extracted from the facial photograph and generated as example data (S204). In this step, similar to the first embodiment, facial photograph data 20 is generated from the user's facial photograph 12. Then, facial feature values 70 are extracted from expressionless facial photograph data 21 and smiling facial photograph data 22, which are the facial photograph data 20, and generated as example data.
For example, in the case of number 1 in the example of training data using feature amount 70 in Fig. 8, first, expressionless face photo data 21 that is the source of the example data is image-processed to obtain a mouth corner drooping degree of 8, eyelid drooping degree of 5, cheek drooping degree of 3, eyebrow drooping degree of 5, and mouth opening degree of 2. Next, smiling face photo data 22 is image-processed to obtain a mouth corner drooping degree of 7, eyelid drooping degree of 5, cheek drooping degree of 2, eyebrow drooping degree of 5, and mouth opening degree of 2. The method of generating the correct answer data is the same as in Example 1.
In this manner, one piece of teacher data 40 is completed (S205). The same process is carried out for the other data to generate teacher data 40.

In this way, by using facial features 70 as example data, the example data becomes specialized for dementia and/or depression, and the amount of input data is reduced, making it possible to improve the accuracy of the AI with a small amount of training data.

Furthermore, by using facial features 70 as example data, the generation of the teacher data 40 can be partially simplified.
Dementia and depression experts have ample knowledge and experience regarding the facial feature quantities 70 and the degree of dementia and depression. Therefore, with the support of experts, it is possible to create a certain amount of training data without using the user's photograph or prescription 31.
In other words, by estimating the feature value 70 expected at the time of a specific dementia level and depression level, and using the estimated feature value 70 as example data, and at that time, using the predicted dementia level and depression level as correct answer data, machine learning can be performed, thereby making it easier to obtain data.

In other words, by asking experts to list characteristics that are likely to be present in dementia or depression when the face is expressionless, it is possible to build up a certain amount of data without collecting photographs.
Similarly, in the case of smiling, by asking subjects to list the characteristics that distinguish them from a smile and a neutral expression, it is possible to build up some data.

In AI machine learning, in the early stages of learning, there may be a large discrepancy between the AI output and the correct answer. By performing machine learning using training data created in advance with the assistance of experts, it is possible to reduce the discrepancy between the AI output and the correct answer when performing machine learning using user data, allowing the AI to converge more quickly.

The explanation will be given along the flow of FIG.
The facial feature amount 70 and the degree of dementia/depression are quantified by an expert (S301). For example, the facial feature amount 70 is a combination of the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is expressionless, and the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is smiling, and the degree of dementia/depression is quantified between 0 and 100%.
An example is shown in Fig. 10(a). This example is a table showing the estimated dementia level when the change in the degree of drooping of the corners of the mouth and the change in the degree of drooping of the eyelids from a neutral expression to a smiling expression is changed on a 10-point scale, for a face with a neutral cheek drooping degree of 5, a face with a drooping eyebrow degree of 4, an opening degree of 2, a face with a drooping eyelid degree of 4, and a face with a drooping corner of the mouth degree of 5.
The range of change in the degree of descent of the mouth corners when the face is expressionless is 5, so the range of change is −5 to 4. The range of change in the degree of descent of the eyelids when the face is expressionless is 4, so the range of change in the degree of descent of the eyelids when the face is expressionless is −4 to 5.
For example, if the change in the degree of eyelid drooping when smiling is 0 and the change in the degree of mouth corner drooping is -3, the smile is appropriately expressed and the recognition rate is considered to be low. As an example, the maximum and minimum values of the change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 34%.
Similarly, if the change in the degree of downward movement of the corners of the mouth when smiling is +2, the corners of the mouth are further downward despite the smile, and it is considered that the smile is not being expressed appropriately, so the recognition rate is considered to be high. As an example, the maximum and minimum values of the amount of change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 59%.

Such a table is created for all the feature quantities 70 .
Using these, the feature levels are set as example data, and the estimated dementia and depression levels are set as correct answer data (S302). An example of a training data group is shown in FIG. 10(b). As correct answer data when all face feature amounts 70 are changed from 0 to 9, dementia level XX% and depression level XX% are listed. The example in FIG. 10(b) is an example of the case of one feature group. When a neutral feature amount 71 and a smiling feature amount 72 are used, the number of divisions becomes larger.
Teacher data is completed through expert guidance (S303).
Next, we ask an expert to generate dementia and depression levels with a focus on combinations of feature quantities 70 that are likely to actually exist (S304). This process increases the amount of teacher data that is closer to reality. Through the above steps, the initial teacher data for machine learning is completed (S305).

By generating training data in this way, the amount of data to be collected can be reduced and the convergence of AI machine learning can be accelerated.

Another embodiment will be described with reference to Figures 13 to 17. The same parts as in the first embodiment will be omitted.
Fig. 13 is a system diagram of the third embodiment of the present invention. Fig. 14 is a diagram of a screen of a mobile terminal of the third embodiment of the present invention, (a) is a screen for a question to confirm the level of vitality by a question, (b) is a screen for requesting permission to obtain teacher data, (c) is a screen for displaying guidance for obtaining a face photo of the user by the mobile terminal, and (d) is a screen for displaying guidance for confirming the medicine taken by the user by the mobile terminal.
Fig. 15 is a table showing information for generating teacher data according to the third embodiment of the present invention. Fig. 16 is a flow diagram showing a teacher data acquisition procedure according to the third embodiment of the present invention. Fig. 17 is a flow diagram showing an embodiment of the dementia and/or depression estimation AI system according to the present invention on a mobile terminal.

According to Example 1, it is possible to build an AI for dementia and/or depression by generating training data from information at a pharmacy without obtaining information from a medical institution.
However, in order to build an AI from a subject's facial photo data, a relatively large amount of sample and training data is required.
Therefore, there is a need for a method to obtain supervisory data on dementia and depression, not limited to pharmacies.

In recent years, apps for mobile devices such as smartphones have been developed that provide advice on diet and exercise to prevent dementia.
Therefore, in this embodiment, it is made possible to obtain information that serves as training data for AI from an application on a mobile device used by a user.
The training data includes at least a facial photograph of the user, a prescription, an instruction sheet, or information that can be used to infer the user's dementia and/or depression. The information that can be used to infer the user's dementia and/or depression is obtained by asking the user questions.
In this way, by obtaining information related to teacher data from an app on a mobile device, it is possible to obtain sample data from many people, not just pharmacies.

FIG. 13 shows a system diagram of this embodiment.
The dementia and/or depression prediction AI1 in this embodiment is a system that obtains teacher data from a user's mobile terminal via a network.
The dementia and/or depression prediction AI1 consists of a machine learning system 50, a management unit 80, a server 81, and a mobile terminal 90.

The machine learning system 50 is the core part of the AI, similar to the first embodiment.
It has a learning phase in which it learns from teacher data, and a utilization phase in which the intermediate part constructed through learning is used to provide AI answers.
During learning, the learning is carried out using the teacher data sent from the management unit 80, adjustments are made to the intermediate parts, and the accuracy of the AI is improved.
When in use, the recognition level and depression level are estimated from the photographic data sent from the management unit 80, and the estimated values are returned to the management unit 80.
The management unit 80 is a part that acts as an intermediary between the mobile terminal 90 and the machine learning system 50. The management unit 80 writes data related to the generation of teacher data to the server 81 and reads it out.
The information necessary for generating teacher data is received from the mobile terminal 90 via the Internet N, and some of it is processed and adjusted, and then stored in the server 81, including the information that has been converted into teacher data.
The information to be processed may be, for example, a photograph of a medicine taken by a user. If the received medicine photograph is a medicine notebook, the management unit 80 reads the name of the medicine written in the notebook and sets it as the medicine contents. If the received medicine photograph is a medicine bag or the medicine itself, the medicine name is identified from the name on the surface of the medicine bag or the name on the medicine sheet.
Thereafter, similarly to the first embodiment, the recognition degree and the depression degree are estimated and calculated using the recognition degree and depression degree calculation table for the drug name as shown in FIG.
Facial photographs are also processed and adjusted in the same manner. Since facial photographs taken by users may not be suitable as training data depending on the shooting environment and the user's shooting method, the management unit 80 performs the same operations as in the first embodiment, such as simplifying the size of the face and simplifying the background. The processed photographs are stored in the server 81.
The management unit 80 reads out the facial photograph, recognition level, and depression level values, which are the training data stored in the server 81, and sends the training data to the machine learning system 50 that is currently learning.

The server 81 mainly processes user information 83, information acquired from the user 84, and information related to the teacher data, and holds the information as example data 41 of the teacher data and information as correct answer data 42. These are collectively referred to as server management data 82.
The server management data 82 will be described with reference to FIG.
A user ID is assigned to each user, and information is managed by the user ID. For example, a user with a user ID of 1001 is Yamada Taro, and a user with a user ID of 1056 is Suzuki Hanako.
Each person's date of birth, gender, and medical history are recorded. This data is generally not changed after registration.
A data number and a data acquisition date are assigned to each collection of information 84 acquired from the user by the management unit 80. The data numbers are unique and do not overlap. Age, a face photo with a neutral expression, a face photo with a smile, a photo related to medication, and awareness and depression levels estimated from questions posed by the app are transmitted to the management unit 80 and recorded in the server 81.
Photographs of medications may be of a medicine book, a medicine packet, or a medicine sheet.
The management unit 80 records on the server 81 the adjusted and processed data from the mobile terminal 90, the adjusted facial photo as example data 41, the contents of the medication, and the awareness and depression values estimated from the medication as correct answer data.
Information managed under one data number corresponds to one piece of teacher data. As information related to teacher data, multiple data numbers may be stored for one user. This is because the user's status may change depending on the time of acquisition.

The server management data 82 records awareness and depression level information generated from drug information, and awareness and depression level estimated from questions posed by the app.
The awareness and depression level information generated from the drug information may be used as the correct answer data 42, or the awareness and depression level estimated from the questions posed by the app may be used as the correct answer data.
In addition, the correct answer data may be an average of the data generated from the drug information and the data estimated from the questions posed by the app.
In addition, if the photo of the medication taken by the user is unclear, the awareness and depression levels estimated from the questions posed by the app may be used as an alternative.

The mobile terminal 90 connects each user with the management unit 80 .
The mobile terminal 90 mainly comprises a display unit 91, a control unit, a storage unit, an operation unit, a communication unit, and a camera unit, all of which are not shown.
The control unit is a part that controls the entire mobile terminal 90. When the application 92 is running, the control unit also performs control according to the content of the application.
The control unit acquires the operation contents of the user from the operation unit, displays characters, symbols, images, etc. on the display unit 91, takes pictures with the camera, and instructs the communication unit to transmit and receive data to and from the network. It also writes and reads temporary data to and from the memory unit, and reads out applications 92.
The storage unit stores a program for operating the control unit and temporarily stores variables, data, etc. used for control. The application 92 of the embodiment is also stored as one of the programs.

The camera is a camera that captures images of the surroundings of the mobile terminal 90. In this embodiment, the camera captures a photograph of the user's face and a photograph of the user's medicine notebook.
The communication unit is a communication unit that communicates with the Internet N. The communication unit connects to the Internet N using a mobile communication system such as Wi-Fi or 4G.
The operation unit is a mechanism that allows the user to operate the terminal using buttons and a touch panel.
The display unit 91 is a part that displays text and images such as guidance and messages, and serves as a display screen for applications and a monitor for captured images.

In this embodiment, the application 92 is assumed to be an exercise recommendation application for preventing dementia in the elderly and a health check application.
Some of these apps are configured to have a section that acquires information about teacher data.
Users who install dementia prevention apps are people who are concerned about their risk of developing dementia. In other words, they are people who are at high risk of developing dementia or depression.
Therefore, the training data obtained from a mobile device on which such an app is installed is highly likely to be training data that indicates the early stages of dementia or depression, and by using it for AI training, the accuracy of the AI in estimating dementia and depression can be improved.

The flow of acquiring information related to the teacher data will be described with reference to the flow of FIG.
In this embodiment, the function is an example of a part of an application 92 in a mobile terminal 90 .
Examples of the application 92 include an exercise recommendation application for preventing dementia in the elderly and a health check application.
When a user launches application 92, it is confirmed by data within the app, etc. whether this is the first time the app has been launched or whether three months have passed since the last request to obtain teacher data-related information (S601).
If not, the result is NO and the application proceeds to normal operation (S608).
If so, an information provision request screen 94 for AI learning is displayed on the screen as a request for cooperation in the AI learning of S602 (Figure 14 (b)).

If the user selects "Yes" in S603, the information acquisition proceeds. If the user selects "No", the application proceeds to normal operation (S608).

As a guidance for taking a face photo, a face photo guidance screen 95 is displayed on the display unit 91, and a face photo with no expression and a face photo with a smile are taken (S604, FIG. 14(c)). If it is difficult to specify a face photo with no expression, a natural, normal face photo may be specified.
As guidance for obtaining information about medicine, a medicine information photographing guidance screen 96 is displayed on the screen, and a photograph of the page of the medicine notebook showing the medicine information, the medicine envelope, etc. is taken (S605, FIG. 14(d)).

Questions related to dementia and depression (FIG. 14(a)) are asked, and the awareness level and depression level estimated from the user's answers are acquired as awareness level and depression level according to the questions (S606). The awareness level and depression level are estimated simply by an app in the mobile terminal.
Also, the content of the question may be notified to the management unit 80 via the Internet, and an appropriate estimation result may be obtained from the management unit 80.
The user's ID number, a face photo, a photo related to medication, and the degree of awareness and depression estimated from answers to questions, all of which are managed under a single data number, are transmitted to the management unit 80 via the Internet N (S607).
Since the acquisition and transmission of information is completed, the application proceeds to normal operation (S608).
Through this series of operations, information related to the teacher data can be obtained extremely easily via the mobile terminal 90.

In addition, the dementia and/or depression prediction AI1 can be used on the mobile terminal 90 to provide advice to the person being diagnosed.
The system is designed to enable the subject's facial photograph taken with a mobile device used by the subject to be obtained via the network, and advice is provided based on the results of the AI system's application of the obtained facial photograph.
For example, a facial photograph of a target person is taken by calling from an application 92 of a mobile terminal 90, and advice on dementia and depression is provided based on the estimated dementia and depression score by AI. The flow of FIG. 17 will be described.

1) The application 92 in the mobile terminal 90 (smartphone) proposes a simple diagnosis of dementia and depression to the subject (S701).
The suggestions will come from exercise recommendation apps to prevent dementia in the elderly and health check apps, so subjects can receive a simple diagnosis without feeling uncomfortable.

2) The subject takes a photograph of his/her face according to the guidance of the application 92 (S702). It is preferable to take two photographs of the face, one with a blank expression and one with a smiling face, because this can improve the accuracy of estimating dementia and depression.
By taking a picture while viewing the image of the user on the mobile terminal 90, the user can take a picture while being conscious of his/her own expressionless face and smiling face, and therefore can take a more appropriate facial picture.

3) The application 92 transmits the facial photograph to the management unit 80 via the Internet N.
The management unit 80 processes the facial photograph of the subject by removing the background, adjusting the size of the face, etc., so that the facial photograph data becomes similar to the sample data (S703).

4) The management unit 80 inputs the photo data to the input unit 51 of the machine learning system 50, which is the core of the AI (S704).
5) Check the values of the output section 53 of the machine learning system 50 and determine the estimated values of the dementia level and depression level (S705).
6) The estimated value is transmitted to the mobile terminal 90 via the Internet N (S706).

7) The subject will be notified of the degree of dementia and depression estimated by the AI, and advice based on the AI estimate will be displayed on the screen. For example, the advice could be, "The degree of dementia is estimated to be 70%. We recommend that you seek medical attention as soon as possible."
The contents of the advice may be generated by the management unit 80, or the contents may be displayed on the mobile terminal 90 in advance according to the estimated values.
In addition, in the case of the second or subsequent simple diagnosis, the past estimated values may also be displayed, and advice may be given according to the increase or decrease in the value (S707). For example, the advice may be, "The degree of dementia is estimated to be 30%. This is better than the previous estimated value of 40%. Please continue to exercise to prevent dementia."

By making such suggestions, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and/or depression.

The dementia and/or depression prediction AI system and training data creation method of the present invention can of course be used as technology for building dementia and/or depression prediction AI outside of hospitals, but can also be applied to other cases in which the illness is manifested as a characteristic facial expression. Therefore, it is believed that the present invention has great industrial applicability.

1. Dementia and/or depression prediction AI
10 Pharmacy 11 Photography equipment 12 User's face photo 13 Expressionless face photo 14 Smiling face photo 20 Face photo data 21 Expressionless face photo data 22 Smiling face photo data 30 Medicine notebook 31 Prescription 32 Medicine data 40 Teacher data 41 Example data 42 Answer data 43 Calculation table 50 Machine learning system 51 Input section 52 Intermediate section 53 Output section 54 Answer section 60 User's face 61 Degree of drooping of corners of mouth 62 Degree of drooping of eyelids 63 Degree of drooping of cheeks 64 Degree of drooping of eyebrows 65 Mouth opening degree 70 Feature amount 71 Expressionless feature amount 72 Smile feature amount 80 Management section 81 Server 82 Server management data 83 User information 84 Information acquired from user 90 Mobile terminal 91 Display section 92 Application 93 Health level check question column 94 Information provision request screen for AI learning 95 Face photo guidance screen 96 Drug information shooting guidance screen P User D Error N Internet

Claims (13)

A method for creating training data used in an AI that estimates the degree of dementia and/or depression from a user's facial photograph, comprising:
1. A system for estimating the degree of dementia and/or depression,
The information that serves as training data for the AI can be obtained from the location where the user receives the medicine according to the prescription or from an application on the mobile device used by the user.
The teacher data includes at least a facial photograph of the user, a prescription, an instruction sheet, or information that can estimate the user's dementia and/or depression,
Using either a facial photograph of the user with a neutral expression or a facial photograph with a smiling expression or both, extracting facial features from the facial photograph and generating them as example data of training data;
The degree of dementia and the degree of depression are estimated from the type and amount of the medicine described in the prescription or instruction sheet and used as correct answer data for training data;
A method for creating teacher data, comprising creating the teacher data for machine learning . The method for creating teacher data according to claim 1, characterized in that the information that can be used to estimate the user's dementia and/or depression is obtained by asking the user questions. 2. The method for creating training data according to claim 1 , wherein the example data of the training data further includes age, sex, medical history, and family medical history. 2. The method for generating training data according to claim 1 , wherein a feature amount extracted from a facial photograph of the user is used as the example data of the training data. 5. The method for generating training data according to claim 4 , wherein the feature values include at least one of the following: degree of drooping of the corners of the mouth, degree of drooping of the eyelids, degree of drooping of the cheeks, degree of drooping of the eyebrows, and degree of mouth opening. Estimating the feature amount predicted at a specific dementia level and depression level;
The teacher data creation method described in claim 4, characterized in that the estimated features are used as the example data, and the predicted dementia level and depression level are used as the correct answer data, and machine learning is performed. An AI system using teacher data created by the teacher data creation method according to any one of claims 1 to 6 ,
A dementia and/or depression estimation AI system that estimates the degree of dementia and depression of a subject based on a facial photograph of the subject. The dementia and/or depression prediction AI system of claim 7, characterized in that a facial photo of the subject is taken at the pharmacy and advice is provided based on the results of the AI system's analysis of the photo. The dementia and/or depression prediction AI system as described in claim 7, characterized in that a photo of the subject's face is taken and, depending on the results of the AI system's response to the photo, the subject is guided to video content. The dementia and/or depression prediction AI system of claim 7, characterized in that a facial photo of the subject is taken and, depending on the results of the AI system's analysis, a referral to a specialized medical institution is made. The dementia and/or depression prediction AI system of claim 7, characterized in that a facial photo of the subject is taken and advice is provided to the family depending on the results of the AI system's response to the photo. The dementia and/or depression prediction AI system of claim 7, characterized in that a facial photo of the subject is taken at an OTC medical product store, and advice is provided based on the results of the AI system's analysis of the photo. The dementia and/or depression prediction AI system of claim 7, characterized in that a facial photograph of the subject taken with a mobile device used by the subject can be obtained via a network, and advice is provided depending on the results of the AI system's use of the obtained facial photograph.