JP7650509B2 - HEALTH MANAGEMENT SYSTEM, HEALTH MANAGEMENT DEVICE, HEALTH MANAGEMENT PROGRAM, AND HEALTH MANAGEMENT METHOD - Google Patents

Description

The present invention relates to a health management system, a health management device, a health management program, and a health management method.

Since the World Health Organization (WHO) advocated healthy lifespan in 2000, there has been growing interest not just in extending lifespan, but in how to extend the period during which people can live independently and healthily both physically and mentally.

In Japan, the difference between average life expectancy and healthy life expectancy (the period during which daily life is not restricted by health problems) was 8.84 years for men and 12.34 years for women in 2016 (see non-patent document 1).

Maintaining the health of citizens is the basis of individual happiness, and at the same time, it prevents a decline in productivity due to the worsening of illness, prevents an increase in the period during which medical expenses and nursing care costs are incurred, and curbs the increase in social security burdens.

On the other hand, except for special cases such as athletes, individuals often only realize the importance of health when they become ill. In other words, since people are not normally very health conscious, early detection can be difficult.

For example, in the case of health checkups (including medical examinations, health checkups, and medical check-ups), which are semi-mandatory company health checkups, the attendance rate is 77%, whereas the attendance rate for self-employed persons, family employees, housekeepers, contract employees, etc. is around 50%. There are various reasons for not getting a checkup, such as not being able to make time, the cost, or not knowing that a checkup existed, but ultimately it can be said that this is due to low health literacy and an underestimation of the importance of preventing or detecting illness early.

On the other hand, there is a problem that mental and neurological disorders are rarely included in the test items even if people undergo health checkups. This is because there are no highly accurate biomarkers, and expensive equipment is required to perform the tests. Furthermore, although there are questionnaires for depression that are commonly used around the world, such as the BDI and PHQ-9, there are limitations to their objectivity due to the possibility of the subject's conscious concerns about the evaluation, unconscious defenses, and the examiner's subjectivity being included.

In addition, people with mental or neurological illnesses may not want others to know that they may have the illness, or may feel reluctant to visit a psychiatrist, so it is thought that they need to be able to manage their own health.

In recent years, technologies have been disclosed that estimate emotions and mental states by analyzing spoken voice (see Patent Documents 1 and 2), making it possible to measure and quantify a person's state by analyzing voice.

Furthermore, there are also situations where voice is actively used for purposes other than interpersonal communication, such as the disclosure of a technology that performs personal authentication using voiceprints and grants the right to access devices (see Patent Document 3) and a voice recognition technology that enables the operation of machines, such as smart home compatible home appliances, by voice (see Patent Document 4).

In addition, with the spread of smartphones, everyone carries a calling device, making it possible to speak whenever necessary.

Furthermore, if voice is recorded and stored as electronic data, it does not deteriorate like blood or urine, and therefore has the advantage that it can be analyzed retroactively at any time if necessary.

JP 2007-296169 A International Publication No. 2006/132159 U.S. Publication No. 2016/0119338 JP 2014-206642 A

Ministry of Health, Labour and Welfare Science Research Grant FY2018 Research Report, pp. 26-39 "Research on the calculation and evaluation of nationwide trends in healthy life expectancy"

In order to increase the rate of medical examinations that lead to the prevention and early detection of diseases, there is a need for tests that are easy to perform by oneself, inexpensive, and do not require the user to make time for them in their daily lives. Therefore, the object of the present invention is to provide a health management system, a health management device, a health management program, and a health management method that allow anyone, anywhere, to measure and understand their own health condition in a short time, non-invasively, and without the knowledge of others.

The inventors discovered that by using the voice as a biomarker indicating the state of the mind and body, it is possible to continuously manage one's health from a healthy state to contracting an illness, recovering, and then returning to a healthy state, and thus arrived at the present invention.

The health management system according to the present application is a health management system comprising a first server that continuously stores voice data acquired from a user, a second server that stores identification data of the user, a third server that stores output data for the user, and an estimation device that infers a predetermined disease using an acoustic feature extracted from the voice data as an input, the estimation device comprising an extraction unit that extracts an acoustic feature based on the voice data, an estimation unit that infers the predetermined disease using the acoustic feature as an input, and a selection unit that selects output data of the third server based on an estimation result of the estimation unit and the identification data of the user, and the selection unit: The system is characterized in that, when the estimation unit estimates that the user is not suffering from any disease, it is configured to select first output data from the third server for the user before the disease, and when the estimation unit estimates that the user is suspected of suffering from any disease, it is configured to select second output data from the third server for the user who is suspected of suffering from the disease, and after the user is diagnosed with any disease by a doctor's diagnosis, the identification data of the second server is updated, third output data from the third server for the user after the disease has been developed, and the third output data is continuously selected even after remission.

According to the present invention, it is possible to provide a health management system, a health management device, a health management program, and a health management method that allow anyone, anywhere, to measure and understand their own health condition in an extremely simple manner, in a short time, non-invasively, and without others knowing.

FIG. 1 is a hardware configuration diagram showing an example of a computer that realizes the functions of a health management system 200 of the present invention. FIG. 2 is a configuration diagram of a health management system 200 according to an embodiment of the present application. FIG. 3A illustrates a feature equation F(a) for calculating one or more acoustic features. FIG. 3B is an image diagram showing that the intensity of a certain acoustic feature varies for each disease. FIG. 3C is a distribution image of disease predictive values obtained by combinations of three acoustic parameters. FIG. 4A is a graph showing an ROC curve indicating the performance of separating healthy subjects or subjects with a specific disease from others. FIG. 4B is a graph showing an ROC curve indicating the performance of separating healthy subjects or subjects with a specific disease from others. FIG. 5 shows example sentences read by patients and healthy subjects for use as learning data. FIG. 6 is a diagram showing the results of estimating whether a user is in a healthy state, a dementia group, or a mood disorder group. FIG. 7 shows the estimation results of the estimation program on a phrase-by-phrase basis in the classification of major depression and healthy subjects. FIG. 8A shows the results of collating for each person the judgment that each phrase is either major depression or healthy based on the estimation program. FIG. 8B shows the results of collating for each person the judgment that each phrase was judged to be either major depression or healthy based on the estimation program. FIG. 9 shows the results when the healthy judgment rate was calculated for each individual, and individuals with a healthy judgment rate of 60% or more were estimated to be healthy individuals, and individuals with a healthy judgment rate of less than 60% were estimated to be major depression patients. FIG. 10 shows the estimation results of a phrase-by-phrase estimation program in classifying dementia with Lewy bodies from healthy subjects. FIG. 11A is a diagram showing the estimation results of Lewy body dementia. FIG. 11B is a diagram showing the estimation results of Lewy body dementia. Figure 12 shows the results when the healthy judgment rate was calculated for each individual, and those with a healthy judgment rate of 60% or more were presumed to be healthy, and those with a healthy judgment rate of less than 60% were judged to have Lewy body dementia. FIG. 13 is a sequence diagram showing the operation of the health management system 200. FIG. 14 is a sequence diagram showing the operation of the health management system 200. FIG. 15 is a schematic diagram showing an overall image of the health management system 200. FIG. 16 is a diagram showing an example of registering a user using a voice platform device or another system (transmission/reception, output of unique determination results) or an embedded system. FIG. 17 is a diagram for explaining the flow of user registration. FIG. 18 is a diagram for explaining the flow of storing audio data. FIG. 19 is a diagram illustrating an example of a voice analysis of a user whose identification information has been registered. FIG. 20 is a diagram for explaining an example regarding the determination. FIG. 21 is a diagram for explaining an example related to action support. FIG. 22A is a diagram for explaining the flow of the system. FIG. 22B is a diagram illustrating the flow of the system. FIG. 23 is a diagram illustrating data transmission and reception, a voice platform, a voice analysis device, and an embedded system. FIG. 24 is a diagram illustrating the audio platform device. FIG. 25 is a diagram illustrating a voice analysis device. FIG. 26 is a diagram for explaining another system. FIG. 27A is a diagram illustrating another system. FIG. 27B is a diagram illustrating another system.

Hereinafter, a detailed description will be given of a form for carrying out the disease inference system and disease inference method according to the present application (hereinafter, referred to as an "embodiment") with reference to the drawings. Note that the disease inference system and disease inference method according to the present application are not limited to the embodiment. In addition, the same parts in each of the following embodiments are given the same reference numerals, and duplicated descriptions are omitted.
1. Program

A health management system 200 according to an embodiment is realized by a computer 100 having a configuration as shown in Fig. 1, for example. An example will be described below. Fig. 1 is a hardware configuration diagram showing an example of a computer that realizes the functions of the health management system 200. The computer 100 has a CPU 101, a RAM 102, a ROM 103, a HDD 104, a communication interface (I/F) 105, an input/output interface (I/F) 106, and a media interface (I/F) 107.

The CPU 101 operates based on a program stored in the ROM 103 or the HDD 104, and controls each unit. The ROM 103 stores a boot program executed by the CPU 101 when the computer 100 is started, programs that depend on the hardware of the computer 100, and the like.

The HDD 104 stores programs executed by the CPU 101, data used by the programs, etc. The communication interface 105 receives data from other devices via the network N and sends it to the CPU 101, and transmits data generated by the CPU 101 to other devices.

The CPU 101 controls output devices such as a display and a printer, and audio input devices such as a microphone, and input devices such as a keyboard and a mouse, via the input/output interface 106. The CPU 101 acquires data from the input devices via the input/output interface 106. The CPU 101 also outputs generated data to the output devices via the input/output interface 106.

The media interface 107 reads a program or data stored in the recording medium 108 and provides it to the CPU 101 via the RAM 102. The CPU 101 loads the program from the recording medium 108 onto the RAM 102 via the media interface 107, and executes the loaded program. The recording medium 108 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical Disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 100 functions as the health management system 200 according to the embodiment, the CPU 101 of the computer 100 executes a program loaded onto the RAM 102 to realize the function of the control unit. In addition, data in the storage unit is stored in the HDD 104. The CPU 101 of the computer 100 reads and executes these programs from the recording medium 108, but as another example, these programs may be obtained from another device.
2. Structure of the health management system

Next, the configuration of a health management system 200 according to an embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the health management system 200 is connected to a user terminal 201 and a web server 202 via a network N so as to be able to communicate with each other by wire or wirelessly. The health management system 200 may also be connected to a plurality of other user terminals 201 and a plurality of web servers 202. The web server 202 includes some of the functions of the health management system 200, and servers of health checkup facilities and medical facilities.

As shown in Fig. 2, the health management system 200 includes a communication unit 203, a first server 204, a second server 205, a third server 206, and a fourth server 210. Each of the first to fourth servers functions as a storage unit. Also, a part of the fourth server functions as a control unit. The communication unit 203 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 203 is connected to the network N by wire or wirelessly, and transmits and receives information between the communication unit 203 and the user terminal 201.

The storage unit is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

The control unit is realized by, for example, a central processing unit (CPU) or a micro processing unit (MPU) executing various programs stored in the storage device of the health management system 200 using a RAM as a working area. The control unit is realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

As described above, the functions of the storage unit and the control unit are realized by the cooperation of the first to fourth servers in Fig. 2. Here, the first to fourth servers will be described.

The first server 204 has a function of continuously storing voice data acquired from the user. The second server 205 has a function of storing user identification data, and the user log is continuously updated through communication with the user terminal 201 and the web server 202.

The third server 206 includes the first to third distribution data (207 to 209). The third server 206 has a function of storing distribution data for users. The function of the third server may be substituted by an external web server 202 via a communication unit 203.

The fourth server 210 includes a calculation unit 211, an estimation unit 212, a selection unit 213, and a storage unit 214. The fourth server performs estimation processing to estimate a predetermined disease using an acoustic feature calculated based on the voice data of the first server as an input, and also performs selection and distribution processing of distribution data stored in the third server, thereby functioning as an estimation device.

The first to fourth servers are assigned for the sake of convenience in order to explain the functions of the health management system 200, and these functions may be performed by part of the same server or by multiple servers.

The estimation program and distribution program stored in the fourth server 210 are executed using the RAM 102 in Fig. 1 as a working area, and thereby operate as a calculation unit 211, an estimation unit 212, and a distribution unit 213. The calculation unit 211, the estimation unit 212, and the distribution unit 213 function as control units within the fourth server. Meanwhile, the storage unit 214 stores the estimation program, the distribution program, and data on a second acoustic parameter, which will be described later.

Here, the calculation unit 211 calculates acoustic features based on the speech data. The estimation unit 212 estimates a predetermined disease using the acoustic features calculated by the calculation unit 211 as an input. The selection unit 213 selects distribution data to be stored in the third server based on the estimation result of the estimation unit 212 and with reference to the identification data of the second server.

Here, the health management system 200 executes an estimation process for estimating disease information and the like of the user based on an estimation program and a distribution program stored in the fourth server 210, and a distribution process for selecting and distributing voice data, video data, or text data (hereinafter, voice data, video data, or text data will be simply referred to as "distribution data") stored in the third server 206 based on the estimated information. The distribution data can be any content, such as characters, figures, symbols, voice, video, games, or any other content, as long as it is capable of generating interest in the user and widely informing the information contained in the distribution data.

The distribution data stored in the third server 206 includes data including distribution content that prompts the user to input voice, and distribution data related to the user's health management, including a message that prompts the user to visit a medical institution or undergo a health check. The distribution data related to the user's health management is distribution data related to information for improving cognitive and social skills that determine an individual's motivation and ability to access, understand, and use information related to how to improve and maintain better health, and includes, for example, information for improving health literacy. The distribution data may also be a numerical result of the estimation process of the fourth server.

In the third server 206 in Fig. 2, the first distribution data 207, the second distribution data 208, and the third distribution data 209 are conveniently classified and stored. The first distribution data 207 includes health information for users before they become ill. The second distribution data 208 includes health information including a message encouraging a user who is suspected of having any disease by the estimation unit 212 of the fourth server 210 to visit a medical institution or undergo a health check. The third distribution data 209 includes health information for users after they become ill.

The first to third distribution data (207 to 209) will be described in more detail. The user to whom the first distribution data 207 is distributed is assumed to be a person who is relatively good at health care, disease prevention, and health promotion in daily life. Therefore, the first distribution data 207 includes information on health promotion such as information encouraging regular medical checkups and recreational information tailored to the situation and place where the user terminal 201 is used. The frequency at which the first distribution data 207 is distributed is less than the frequency at which the second and third distribution data are distributed. For example, the first distribution data 207 may be distributed at the timing when the user inputs voice via the user terminal 201.

The user to whom the second distribution data 208 is distributed is assumed to be a person suspected of suffering from some mental or neurological disorder. Therefore, the second distribution data 208 includes information encouraging the user to visit a specific medical institution. However, there is a risk that the user will not admit that he or she is suspected of suffering from a mental or neurological disorder. In addition, even if the patient is aware of his or her condition, there is a risk that the patient will not receive medical care due to prejudice against mental illness and reluctance to visit a psychiatrist or psychosomatic clinic. Therefore, the distribution data for persistently persuading the patient to seek social services such as those of a medical institution may be distributed more frequently than the first distribution data.

In addition, since mental and neurological diseases have common symptoms, it may be difficult for an individual to identify the disease. For example, it is difficult to distinguish between Alzheimer's dementia and frontotemporal dementia, Alzheimer's dementia and Lewy body dementia, Lewy body dementia and Parkinson's disease, and bipolar disorder and major depression. In addition, in the case of depression in elderly people, attention, concentration, judgment, and memory are reduced due to depressive symptoms such as depressed mood and inhibition of thinking, and there is pseudodementia that appears to be dementia at first glance, and it is necessary to distinguish whether it is a type of depression or a type of dementia in order to receive appropriate treatment. The present invention is advantageous in that it can provide distribution data by distinguishing medical institutions recommended according to the identified disease based on the disease estimated via the estimation unit 212 as in the present invention.

The user to whom the third distribution data 209 is distributed is assumed to be a person who has a history of suffering from some mental/nervous disease, or a person who is undergoing treatment for some mental/nervous disease. In general, it is difficult to completely cure a mental/nervous disease. Therefore, although it cannot be said that the disease is completely cured, it is common to continue taking medicine or being monitored by a doctor to prevent recurrence even after the symptoms have disappeared after the condition has improved and entered a state of mild remission. Therefore, the third distribution data 209 includes a message or the like that encourages the user to check the symptoms of the disease by continuous voice input.

Even after the third distribution data 209 has been distributed to the user, if the estimation unit 212 subsequently estimates that there is a suspicion of recurrence of the disease or that the user is suspected of suffering from another mental/neurological disease, the second distribution data 208 is distributed.

In this manner, the selection unit 213 of the fourth server 210 selects and distributes the first to third distribution data in accordance with the estimation result of the estimation unit 212 and the health condition of the user.

In addition, after a doctor diagnoses that the user has any disease, the identification data of the second server 205 is updated by a server of the medical institution to which the doctor belongs via the web server 202, or by the user himself/herself via the user terminal 201. After updating of the identification data is completed, the selection unit 213 is configured to select the second or third distribution data (208, 209) for the user.

Returning to the description of the health management system 200, the health management system 200 is an information processing device such as a web server that collects voice data of a user using a user terminal 201 used by the user as a voice input means and performs data communication via the user terminal 201. For example, when the health management system 200 receives a distribution request for requesting data distribution from the user terminal 201, the health management system 200 selects target distribution data based on a disease estimation result based on the voice data of the user using the user terminal 201, and transmits the selected distribution data to the user terminal 201.

The user's identification data is stored in the second server 205, the log is continuously updated via the user terminal 201, and the medical examination institution or medical institution continuously updates the medical examination results and diagnosis results via the web server 202. The user's identification data includes health information having personal information including whether the user has ever suffered from a specific disease, and various information indicating the user's profile and background, such as the user's gender, age, address, hobbies, interests, behavioral history, and web page browsing history. The health information having personal information includes personal information including the user's medical examination results or the results of a doctor's diagnosis of a disease.

The user terminal 201 is a communication terminal such as a smartphone or tablet having a means for inputting voice and a means for outputting distribution data, and can communicate with any server device via a wireless communication network such as 3G (Generation) or LTE (Long Term Evolution). The user terminal 201 may be an information processing device such as a desktop PC (Personal Computer), a notebook PC, a netbook, or a server device, as long as it has a means for inputting voice and a means for outputting distribution data.

The user terminal 201 may have some of its functions built into an electronic device having a means for inputting voice or a means for outputting distribution data. For example, a mobile device such as an automobile, a hardware device for operating a computer game such as a home game machine, or an amusement facility such as a karaoke booth or a running machine may have the function for inputting voice or the function for outputting distribution data.

The health management system 200 can display web content that prompts the user to input voice via the user terminal 201. The web content that prompts the user to input voice is distribution content stored in the third server 206, and includes text, images, videos, animations, or audio playback. In such a case, the user terminal 201 transmits a web content distribution request to a web server that distributes web content automatically via an application or by operation of the user. The distribution request may be automatically transmitted to the web server when the user starts an application on the user terminal 201.

The health management system 200 delivers web content that prompts the user to input voice to the user terminal 201. The user terminal 201 then displays the web content received from the health management system 200.

For example, when a user is playing a game via the user terminal 201, a character in the game may be configured to deliver a message via the application encouraging the user to input voice input.

When a user inputs voice via the user terminal 201, the health management system 200 stores the voice data in the first server 204 of the distribution server 10. Then, when a distribution request for requesting distribution of data stored in the third server 206 is received, a disease estimation process is performed based on the voice data of the user using the user terminal 201, and further a process of selecting distribution data is performed, and the selected distribution data is transmitted to the terminal device 100.
3. Processing Executed by Health Management System 200

When distributing data stored in the third server 206, the health management system 200 selects the distribution data to be distributed from among the first to third distribution data (207 to 209) stored in the third server based on the estimation result of the estimation unit 212 of the fourth server 210 and the user identification data stored in the second server 205.

In other conventional technologies, the distribution data related to health management to be distributed is based only on the identification data of the user stored in the second server 205. To give a more specific example, in the conventional technologies, if the user to be distributed is a man, health information similar to the health information provided to many men is distributed.

However, there are cases where there is no log of attribute information of a newly registered user. In addition, there are cases where the attribute information of a user becomes unknown, such as when cookies or the like are deleted and an ID (Identifier) for identifying a user becomes unknown. In such cases, the conventional technology is unable to select appropriate distribution data to be distributed.

In addition, for users who do not receive regular health checkups or who do not visit a medical institution even when they develop a mental or neurological disorder, the distribution data that the health management system 200 should select is not desirable, and there is a risk that the information provided to improve health literacy may actually have a negative impact on the users.

As described above, in the conventional technology, when there is no user log stored in the second server 205 or when the user's attribute information is unknown, it is not possible to select appropriate distribution data. Even if a user develops some kind of mental or neurological disease and the abnormality can be detected from the user's behavior history obtained via the user terminal 201, if the predicted disease cannot be identified to some extent, there is a risk that incorrect health information will be selected and provided, which may have a negative impact on the user.

Therefore, the health management system 200 calculates the acoustic feature of the user using a calculation unit 211 of a fourth server 210 that calculates the acoustic feature from the voice data stored in the first server 204. Then, the distribution server 10 executes an estimation process in which an estimation unit 212 of the fourth server estimates a disease of the user based on the calculated feature.

Furthermore, when a user whose attribute information is unknown inputs voice data, the health management system 200 calculates acoustic features of the voice data using a calculation unit 211 of a fourth server 210. The health management system 200 also executes an estimation process in which an estimation unit 212 of the fourth server estimates a disease of the user based on the calculated features. The health management system 200 then executes a distribution process in which a selection unit 213 uses the estimated disease information to select distribution data to be distributed and distributes the data.
4. Calculation Unit and Estimation Unit

Here, the calculation unit 211 and the estimation unit 212 used by the health management system 200 when estimating a disease will be described.
4-1. Construction of estimation model

The following describes how the estimation model of the fourth server of the health management system is constructed. First, a stage is taken in which multiple acoustic parameters are extracted from the subject's voice data. The acoustic parameters are parameterized features of sound transmission. For example, acoustic parameters include the zero-crossing rate and the Hurst exponent. The zero-crossing rate is calculated as the number of times per unit time that the waveform of the sound pressure of the voice crosses the reference pressure, as a measure of the intensity of changes in the waveform of the voice. The Hurst exponent indicates the correlation of changes in the waveform of the voice.

The acoustic parameters are classified into first acoustic parameters and second acoustic parameters. Here, the first acoustic parameters are acoustic parameters extracted from the voice of a user whose specific disease is to be estimated. The first acoustic parameters are extracted based on the voice data stored in the first server 204.

The second acoustic parameters are acoustic parameters recorded in advance in a database. Data on the second acoustic parameters is stored in the storage unit 214 of the fourth server 210 or another server that substitutes for the function of the database. The second acoustic parameters are extracted from speech data of a patient with Alzheimer's disease, Lewy body dementia, Parkinson's disease, major depression, atypical depression, or bipolar disorder, and each acoustic parameter is linked in advance to each disease.

Specific examples of the acoustic parameters used in the present invention include the following items.
1) Volume envelope (attack time, decay time, sustain level, release time)
2) Waveform fluctuation information (Shimmer, Jitter)
3) Zero point crossing rate 4) Hurst index 5) VOT (Voice Onset Time)
6) Statistics of the intra-utterance distribution of certain coefficients of the Mel-frequency cepstrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.)
7) Statistics of the intra-utterance distribution of the rate of change of the frequency spectrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.)
8) Statistics on the intra-utterance distribution of the time-varying coefficients of the Mel-frequency cepstrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.)
9) Statistics on the intra-utterance distribution of the time change of a certain coefficient of the Mel-frequency cepstrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.)
10) Squared error for a quadratic regression approximation of the intra-utterance time change of the frequency spectrum 90% roll-off. 11) Arithmetic error for a quadratic regression approximation of the intra-utterance time change of the frequency spectrum center of gravity. Other examples include pitch rate, probability of being a voiced sound, power of a given range of frequencies, scale, speaking rate (number of moras in a given time), pauses/intervals, and volume.

Such acoustic parameters are extracted for each speech phrase by acoustic parameter extraction software (e.g., openSMILE, etc.). In the present invention, 7,440 acoustic parameters were extracted, acoustic parameters dependent on the recording environment were first removed, and one or more acoustic parameters suitable for expressing the health condition were selected from the remaining parameters.

The estimation program stored in the fourth server 210 has a learning function based on artificial intelligence and performs estimation processing using the learning function. Neural network-type deep learning may be used, or reinforcement learning that partially strengthens the learning field may be used, or other techniques such as genetic algorithms, cluster analysis, self-organizing maps, and ensemble learning may be used. Of course, other artificial intelligence techniques may be used. In ensemble learning, a classification algorithm may be created using a method that combines boosting and decision trees.

In the stage of creating the estimation program, the algorithm creator performs stepwise analysis to find the best combination of any acoustic parameters to be used as the variable f(n) from among the items of the second acoustic parameters, and selects one or more of them. Next, a coefficient is assigned to the selected acoustic parameter to create one or more acoustic features. Furthermore, these acoustic features are combined to create an acoustic feature F(a).

There are three types of stepwise methods: forward, backward, and increment/decrement methods, and any of them may be used. Regression analysis used in the stepwise method includes linear classification such as linear discriminant and logistic regression analysis. The variables f(n) and their coefficients, i.e., the coefficients xn of the formula F(a) shown in FIG. 3A, are called regression coefficients, and are weights given to the function f(n).

After the creator of the learning algorithm selects the regression coefficients, the quality may be improved by machine learning to increase the estimation accuracy based on disease information accumulated in a database.

FIG. 3A is a feature equation F(a) for calculating one or more acoustic features. A predicted value of the subject's disease can be calculated based on the equation F(a). Here, f(n) is any one or more second acoustic parameters arbitrarily selected from the above acoustic parameter items (1) to (11). xn is a regression coefficient specific to the disease. f(n) and xn may be stored in advance in the storage unit 214 of the fourth server 210 as part of the estimation program. The regression coefficient of the feature F(a) may be improved during the machine learning process of the estimation program.

2 calculates a feature quantity for distinguishing between healthy subjects and subjects with a disease, or between diseases, based on the combination of the second acoustic parameters. From the feature quantity, a reference range and a scoring for calculating how far the subject's value is from the reference range are calculated, thereby calculating a predicted value of the subject's disease.

FIG. 3B is an image showing that the intensity of a certain acoustic feature varies for each disease. It shows that the subject has the highest score for disease A. Therefore, the predicted value for disease A of the subject is calculated to be higher than that of other disease groups. In addition, for example, by setting the intensity of 50 as the threshold, it is possible to classify into a group of diseases A, D, and E and a group of diseases B and C.

3B, a disease prediction value is calculated based on the intensity of one acoustic feature, but in reality, it is difficult to classify diseases using only one acoustic feature. Therefore, diseases may be classified by calculating a feature F(a) obtained by combining several acoustic features.

Based on the feature F(a), which is a combination of these acoustic features, a predicted value of the disease is calculated for the voice of the labeled subject, and a distribution of the predicted values for each disease is obtained. This makes it possible to classify each disease.

FIG. 3C is a distribution image of disease predictive values (denoted as "mental values" in FIG. 3C) obtained by combining three acoustic parameters.

3C shows that the distribution of predicted values for the Lewy body dementia patient group can be separated from the distribution of predicted values for the other disease patient groups and the healthy subject group. The present invention sets a combination of acoustic parameters for each disease so that it can be distinguished from other diseases, calculates a feature F(a), and uses the feature F(a), which is a combination of acoustic features, as an input to determine which disease the voice of each subject corresponds to.

As another method, a feature F(a), which is a combination of acoustic features for each disease, is extracted from the voice of each patient, and it is determined which disease features the patient has more of. The predicted values of the diseases are then compared with each other to infer the disease that the patient is suffering from.

In this case, the predicted value of a disease can be regarded as the degree of the disease. By comparing and recalculating the predicted values of each disease, it is possible to express the probability of having each disease.

In this way, feature F(a), which is a combination of acoustic features associated with each disease, is extracted from the voices of patients with six diseases, including Alzheimer's disease, dementia with Lewy bodies, Parkinson's disease, major depression, atypical depression, and bipolar disorder, as well as the voices of healthy individuals, and a predicted value for each disease is calculated.

Furthermore, a prediction program may be created from the voices of patients with ten diseases, including four more diseases, namely, vascular dementia, frontotemporal dementia, cyclothymia, and dysthymia, as the target diseases.

Finally, by analyzing the voice spoken by the subject, it is estimated whether the subject has one of the above-mentioned six to ten diseases, or is healthy.

Regarding the estimation flow of the estimation program, as described above, for each individual disease, the feature (a) of each disease may be extracted and a predicted value of the disease may be calculated. Alternatively, first, a combination of acoustic features related to a group of diseases may be created, and the feature F(a) related to the group of diseases may be used as an input to the estimation unit, and input/estimation may be performed in multiple stages to finally estimate each disease or healthy state.
[4-2. Estimation accuracy]

4A and 4B are graphs of ROC curves showing the separation performance between healthy subjects or specific diseases and others. The horizontal axis shows 1-specificity, and the vertical axis shows sensitivity. In other words, the horizontal axis shows the false positive rate, and the vertical axis shows the true positive rate. Both of the ROC curves in FIG. 4A and FIG. 4B showed high true positive rates at points where the false positive rate was low.

In addition, the AUC (Area under an ROC curve) was higher than 0.5 in all cases, and a significant difference from the case of random classification was confirmed. The diseases for which the separation performance was verified were Lewy body dementia, Alzheimer's dementia, Parkinson's disease, major depression, bipolar disorder, and non-specific depression. The AUC in each ROC curve was 0.794 for Lewy body dementia, 0.799 for Alzheimer's dementia, 0.771 for Parkinson's disease, 0.869 for major depression, 0.86 for bipolar disorder, and 0.86 for non-specific depression. The diseases that can be estimated using the present invention are not limited to the above. From the AUC results, it can be confirmed that the accuracy of disease estimation is high.

The prediction program of the present invention can classify dementia with Lewy bodies as healthy or as having another disease.

The prediction program of the present invention can classify Alzheimer's disease as healthy or as having another disease.

The prediction program of the present invention can classify Parkinson's disease from healthy subjects or from other diseases.

The prediction program of the present invention can classify major depression as healthy or as having another disease.

The prediction program of the present invention can classify bipolar disorder from health or from other diseases.

The prediction program of the present invention can classify non-specific depression as healthy or as having another disease.

The prediction program of the present invention can classify a healthy state into the above-mentioned diseases.

The estimation program created as described above can be used without any restrictions by those suspected of having a mental or neurological disorder, or those presumed to be healthy. It can also be used as a tool for doctors to examine the subjects, or as a test item for health checkups and medical examinations, simply by acquiring the subjects' voices.

The voice data analysis device of the present invention can be equipped with several different voice analysis programs.

For example, if a healthy person wants to know their daily health condition, MIMOSSYS (registered trademark of PST Corporation) can tell them the level of vitality from their "vitality pressure" and "activity value." MIMOSSYS is a product equipped with a voice analysis program that uses voice analysis to calculate the type and magnitude of emotions expressed by the user and the level of vitality.

In addition, if the user himself/herself is aware of a decline in vitality, or if the MIMOSYS measurement indicates a tendency for the level of vitality to decline, it is preferable to use an analysis program that determines the possibility of a mental or neurological disorder.

Examples of such voice analysis programs include programs that estimate whether a person is healthy, has a possibility of having a dementia group disease, or has a possibility of having a mood disorder group disease.

Furthermore, if the user is considered to have a high possibility of having some kind of mental or neurological disorder, it is preferable to use a program that predicts either the mental or neurological disorder.

Here, specific examples of psychiatric and neurological disorders include dementia with Lewy bodies, Alzheimer's disease, and Parkinson's disease. Vascular dementia can also be included. Mood disorders include major depression, bipolar disorder, and atypical depression. Cyclothymia and dysthymia can also be included.

In addition, the disease to be estimated is preferably a mental or nervous system disease, but any disease that affects speech may be possible to estimate, such as heart disease, respiratory disease, etc.
5. Example of Estimation Program Creation

In the above item 4, the performance as a classifier of the estimation program executed in the estimation unit 212 of the fourth server 210 has been sufficiently proven. In this item 5, an example of creating an estimation program will be introduced.
[5-1. Example 1 of Estimation Program Creation]

In this section 5-1, we will explain a prediction program for predicting whether a user is healthy, in the dementia group, or in the mood disorder group when diseases are classified into a dementia group including dementia with Lewy bodies, Alzheimer's disease, and Parkinson's disease, and a mood disorder group including major depression, bipolar disorder, and atypical depression. First, we will describe the procedure for creating the prediction program.

The speech samples used as training data were 15 patients with Alzheimer's disease, 12 patients with bipolar disorder, 14 patients with atypical depression, 15 patients with Lewy body dementia, 15 patients with major depression, 15 patients with Parkinson's disease, and 15 healthy individuals.

The test data consisted of speech samples from five patients with Alzheimer's disease, four patients with bipolar disorder, five patients with atypical depression, five patients with dementia with Lewy bodies, five patients with major depression, five patients with Parkinson's disease, and five healthy subjects. The data was randomly divided into training data and test data.

The speech used for the learning data was obtained by each patient and healthy subject speaking the 17 sentences introduced in Figure 5 (numbers 1 to 13 spoken twice each, and numbers 14 to 17 spoken once each), resulting in 30 utterances for each person.

Characteristic acoustic parameters were extracted from each of these utterances, and the characteristics of the acoustic parameters, such as their frequency and intensity, were correlated with the subject's disease, thereby obtaining various groups of acoustic parameters associated with the above six diseases and seven types of healthy subjects.

Characteristic acoustic parameters were selected for each disease, and the selected acoustic parameters were then combined to infer for each person's spoken sentence whether they had one of the six diseases or were a healthy person. Then, an estimation program 10 was created based on the features to infer whether the person had a dementia group, a mood disorder group, or was healthy.

As test data, eight phrases (16 in total, two for each) were used from the 17 sentences in Figure 5: 2 "aiueokakikukeko", 3 "It's a fine day today", 6 "I'm very healthy", 8 "I have an appetite", 9 "I'm calm", 10 "I'm angry", 12 "Let's look up and walk", and 13 "Let's do our best". For each utterance, it was estimated whether it was a disease or healthy. Among the estimated diseases, those estimated to be Alzheimer's disease, dementia with Lewy bodies, and Parkinson's disease were determined to be diseases in the dementia group. In addition, those estimated to be major depression, bipolar disorder, and atypical depression were determined to be diseases in the mood disorder group.

Here, the 16 utterances from each person were tallied to see whether they were judged to be a dementia group disease, a mood disorder group disease, or healthy, and the one that received the most judgments was regarded as the final judgment result.
(Result 1) Patient with Alzheimer's disease (patient code AD03):

Thirteen were judged to be dementia group, one was judged to be mood disorder group, and two were judged to be healthy. Therefore, they were finally estimated to be dementia group. This was consistent with the dementia group to which the actual disease Alzheimer's disease belongs.

The results of the similar judgments for 34 people are shown in Figure 6. The accuracy rate of the prediction program was 85.3%. This fully proves that the program can predict whether a user is healthy, in the dementia group, or in the mood disorder group, when classified into a dementia group including Lewy body dementia, Alzheimer's dementia, and Parkinson's disease, and a mood disorder group including major depression, bipolar disorder, and atypical depression.
[5-2. Example 2 of Estimation Program Creation]

In this section 5-2, as examples of creating a prediction program, a prediction program that classifies major depression as healthy or another disease, and a prediction program that classifies Lewy body dementia as healthy or another disease will be described. Note that a similar procedure can be used to create a program that classifies Alzheimer's disease, Parkinson's disease, bipolar disorder, or atypical depression as healthy or a disease other than the selected disease, but this will be omitted because it is obvious from the results of section 4 above.
[5-2-1. Example of creating a prediction program for major depression]

First, the procedure for creating the estimation program (5-2-1A) for classifying major depression and healthy subjects will be described.

7440 acoustic features were extracted from each phrase (approximately 30 utterances from each person) by 20 healthy subjects and 20 patients with major depression using openSMILE, a software that extracts acoustic features. Features with significant differences depending on the facility where the voice was obtained were excluded, and 1547 features were selected. Then, using a method that combines boosting and decision trees, a disease prediction value for discriminating between healthy subjects and patients with major depression was generated from the features, and a classification algorithm was created. The estimation results for each phrase are shown in Figure 7A.

As shown in FIG. 7A, the prediction program for classifying major depression from healthy subjects had a sensitivity of 81.4%, a specificity of 85.5%, and an accuracy rate of 83.5% in determining whether major depression was present.

The results of determining whether each phrase was major depression or healthy for each person based on the prediction program are shown in FIGS. 8A and 8B.

Furthermore, the healthy judgment rate for each person was calculated, and those with a healthy judgment rate of 60% or more were estimated to be healthy, and those with a healthy judgment rate of less than 60% were estimated to be major depression patients. The results are shown in Figure 9. All 20 healthy people were estimated to be healthy, and all 20 major depression patients were estimated to be major depression patients. In other words, the specificity, sensitivity, and accuracy rate were all 100%.

Next, a procedure for creating the estimation program 5-2-1B for classifying major depression from other psychiatric and neurological disorders will be described.

The estimation program 5-2-1B extracts and estimates major depression from among psychiatric and neurological disorders including at least Alzheimer's disease, Lewy body dementia, Parkinson's disease, major depression, atypical depression, and bipolar disorder, which have similar symptoms.

The types of utterances used in the analysis were the 17 types of utterances (phrases) shown in FIG.

First, 7440 acoustic features were extracted for each spoken phrase using openSMILE, a software for extracting acoustic features. Data with significant differences depending on the facility where the voice was acquired were excluded, and a classification algorithm was created using the features of 1547 data by a method that combined boosting and decision trees, and a disease prediction value for distinguishing major depression from other diseases was generated.

In addition, 2236 speech phrases from the five patients other than those with major depression were used, while the number of speech phrases from the patients with major depression was 940. Therefore, in order to avoid an imbalance in the number of data, pseudo-data of the speech of the patients with major depression was created using the Synthetic Minority Over-sampling Technique (SMOTE), and the number of speech phrases from the patients with major depression was set to 3139.

The data was divided into 10 parts, and 10-fold cross-validation of the classification algorithm of the estimation program 5-2-1B was performed.

The results of the estimation based on the estimation program 5-2-1B are shown in Figure 7B. When using the estimation program 5-2-1B to distinguish between major depression and other psychiatric and neurological disorders, the recall rate for major depression was 91.1%, the recall rate for other psychiatric and neurological disorders was 93.8%, and the accuracy rate was 92.7%. The AUC was 0.977.
[5-2-2. Example of creating a prediction program for Lewy body dementia]

First, we will explain the procedure for creating a prediction program for classifying Lewy body dementia and healthy subjects.

The procedure for creating the estimation program 5-2-2A was the same as that for the estimation program in item 5-2-1 above, except that speech phrases from 20 healthy subjects and 20 patients with dementia with Lewy bodies were used as the voices. The estimation results for each phrase are shown in FIG. 10A.

The prediction program 5-2-2A had a sensitivity of 81.5%, a specificity of 83.1%, and an accuracy rate of 82.2% in determining whether or not there is Lewy body dementia.

Based on the estimation program 5-2-2A, the results of estimating whether each phrase was dementia with Lewy bodies or healthy for each person are shown in Figures 11A and 11B.

Furthermore, the healthy judgment rate for each person was calculated, and those with a healthy judgment rate of 60% or more were estimated to be healthy, and those with a healthy judgment rate of less than 60% were estimated to be patients with dementia with Lewy bodies. The results are shown in Figure 12. All 20 healthy people were estimated to be healthy, and of the 20 patients with dementia with Lewy bodies, 19 were estimated to have dementia with Lewy bodies, and 1 was estimated to be a healthy person. That is, the specificity was 100%, the sensitivity was 95%, and the accuracy rate was 97.5%.

Next, we will explain the procedure for creating a prediction program (5-2-2B) that classifies Lewy body dementia from other psychiatric and neurological disorders.

The estimation program 5-2-2B extracts and estimates dementia with Lewy bodies from among psychiatric and neurological disorders including at least Alzheimer's disease, dementia with Lewy bodies, Parkinson's disease, major depression, atypical depression, and bipolar disorder, which have similar symptoms. The types of utterances used in the analysis are 17 types of utterances (phrases), the same as in the estimation program 1.

First, 7440 acoustic features were extracted for each spoken phrase using openSMILE, a software for extracting acoustic features. Features with significant differences depending on the facility where the voice was acquired were excluded, and a classification algorithm was created using a method that combined boosting and decision trees using 1547 features, generating disease prediction values for distinguishing between Lewy body dementia and other diseases.

In addition, 2,987 speech phrases from the five patients other than Lewy body dementia were used, while 583 speech phrases from patients with Lewy body dementia were used. In order to avoid an imbalance in the amount of data, pseudo-data of the speech of patients with Lewy body dementia was created using the Synthetic Minority Over-sampling Technique (SMOTE), resulting in 2,696 speech phrases from patients with Lewy body dementia.

The data was divided into 10 parts, and 10-fold cross-validation of the classification algorithm of the estimation program 5-2-2B was performed.

The estimation results based on the estimation program 5-2-2B are shown in Figure 10B. When using the estimation program 5-2-2B to distinguish between dementia with Lewy bodies and other psychiatric and neurological disorders, the recall rate for dementia with Lewy bodies was 88.6%, the recall rate for other psychiatric and neurological disorders was 89.8%, and the accuracy rate was 89.2%. The AUC was 0.959.

By combining a prediction program that classifies a specific disease from the other five diseases with a prediction program that classifies the specific disease from healthy individuals in a similar manner to the above procedure, it is possible to predict with a high probability whether a patient has Alzheimer's disease, dementia with Lewy bodies, Parkinson's disease, major depression, atypical depression, or bipolar disorder, or whether the patient is a healthy individual.
6. Operation of the Health Management System 200

13 and 14 are sequence diagrams showing the operation of the health management system 200. Each will be described in order below.

FIG. 13 is a sequence diagram showing the operation of the health management system 200 when the user has not yet visited a medical institution or has gone into remission after visiting a medical institution once.

First, in step S1001, the user's identification information or a delivery request input from the user terminal 201 is transmitted to the second server 205 via the network N and the communication unit 203. A user who has achieved remission of a psychiatric or neurological disorder may include this fact in the identification information.

Next, in step S 1002 , the second server transmits a message to the third server 206 notifying that a distribution request has been made from the user terminal 201 .

Next, in step S1003, when the third server 206 receives the distribution request, it transmits the distribution data to the user terminal 201. This distribution data includes a message prompting the user to input voice. The distribution data is transmitted to the user terminal 201 via the communication unit 203 and the network N, and is output by the means for outputting the distribution data.

Next, in step S 1004 , voice data input by the user via the voice input means of the user terminal 201 is transmitted to the first server 204 .

Next, in step S1005, the first server stores a copy of the received voice data and transmits the voice data to the fourth server.

Next, in step S1006, the fourth server preprocesses the received speech data to calculate a first acoustic feature, and calculates a feature F(a) in the calculation unit 211 based on the second acoustic feature previously stored in the storage unit 214 and the estimation program. The estimation unit 212 performs a process of estimating a psychiatric/neurological disorder based on the feature F(a) calculated by the calculation unit 211.

Here, when the estimation unit 212 estimates that the user does not suffer from any psychiatric/neurological disorder, the first distribution data 207 of the third server 206 is distributed via the selection unit 213. When the estimation unit 212 estimates that the user is suspected of suffering from any psychiatric/neurological disorder, the second distribution data 208 of the third server 206 is distributed via the selection unit 213.

In addition, if the identification information of a user registered in the second server 205 includes a history of a psychiatric/neurological disorder but a record indicating that the disorder is already in remission, and the estimation unit 212 estimates that the user is not suffering from any psychiatric/neurological disorder, the third distribution data 209 of the third server 206 is distributed via the selection unit 213.

Next, in step S1007, the third server 206 transmits the predetermined distribution data to the user terminal 201. The user terminal 201 outputs the received distribution data from an output means, and ends the process.

13, the data distributed from the third server 206 is changed depending on the identification information of the user registered in the second server 205 and how the user's voice data is estimated by the estimation unit 212 of the fourth server 210. This is useful in that health information suited to the individual health condition of the user can be distributed as distribution data. In addition, the steps shown in FIG. 13 can be repeatedly executed in part or in whole to check changes in condition in chronological order.

Next, we move on to the explanation of Fig. 14. Fig. 14 is a sequence diagram after a user is diagnosed by a doctor, and includes mutual communication between the health management system 200 and a server of a medical institution in addition to the functions of the health management system 200 in Fig. 13. Note that in Fig. 14, the steps will be explained on the assumption that the user has been diagnosed with some kind of mental/neurological disorder.

First, in step S2001, the medical institution server 202M, which is a type of the web server 202, accesses the second server 205 via the network N and the communication unit 203, and updates the user's identification information.

Next, in step S2002, the second server 205 transmits a message to the third server 206 indicating that the user's identification information has been updated.

Next, in step S2003, when the medical institution server 202M requests the user to input voice, or when the user terminal accesses the health management system 200, the third server 206 sends a distribution message to the user terminal 201 prompting the user to input voice.

Next, in step S 2004 , the user's voice data input via the voice input means is transmitted to the first server 204 .

Next, in step S 2005 , the first server 204 copies and stores the voice data, and also transmits the voice data to the fourth server 210 .

Next, in step S2006, the fourth server 210 preprocesses the received voice data to calculate a first acoustic feature, and calculates a feature F(a) in the calculation unit 211 based on the second acoustic feature previously stored in the storage unit 214 and an estimation program. The estimation unit 212 performs a process of estimating a psychiatric/neurological disorder based on the feature F(a) calculated by the calculation unit 211.

Here, the processes of steps S2001 to S2006 may be repeated. For example, in FIG. 14, the user has already been diagnosed with some kind of mental or neurological disease, but even after the diagnosis, the user can continue to input the prognosis as voice data periodically and the estimation process can be continued in the fourth server. The voice data in the first server and the results of the estimation process in the fourth server are stored each time. The stored data can be used as a basis for determining the prognosis in the diagnosis at the next visit to the hospital.

After the diagnosis, it is also possible to check the voice data of the first server before the diagnosis and the result of the estimation process of the fourth server acquired in the step of Fig. 13. This can increase the accuracy of the diagnosis.

Next, in step S2007, the third server 206 selects distribution data via the selection unit based on the result of the estimation process of the fourth server 210, and transmits the distribution data to the prognostic user. The second or third distribution data is selected as the distribution data.

In addition, there is a case where the second or third distribution data includes question data for the user, including a medical questionnaire provided by a doctor. In that case, proceed to step S2007. The operations from step S2007 onwards may be repeatedly executed independently of the operations from step S2001 to step S2006.

Next, in step S2007, the medical questionnaire is transmitted to the user terminal 201. Next, when the user completes inputting the information into the medical questionnaire from the user terminal 201, in step S2008, the input data is transmitted to the second server 205 and stored therein.

Next, in step S2009, the input data is transmitted to the medical institution server 202M. The operation of step S2009 may be combined with the operation of step S2008 performed by the second server 205. The medical institution server 202M outputs the input data from an output means and ends the process.

In this manner, by using the health management system 200 in Fig. 13 or 14, the user can use the health management system 200 as a personal health management tool when the user is not yet ill. After the onset of a mental or neurological disease, the health management system 200 can be used as a diagnostic support tool for doctors. For users who have fully recovered or are in remission, the health management system 200 can be used as a personal health management tool and as a monitoring means by doctors as necessary.

So far, an example of the operation of the health management system 200 has been introduced, but the present invention is not limited to this. In addition, in Fig. 14, the medical institution server 202M is taken up as an example of the combined use of the web server 202 and the health management system 200, but instead of a medical institution using this system, a market institution may use the health management system 200 including the medical institution server 202M for the purpose of conducting a consumer trend survey.
7. Features of the present invention

The health management system 200 can display the output every time an utterance is acquired, and can grasp the health condition each time, but it can also acquire voice continuously, compile and analyze it, and view changes over time, as shown in Figures 13 and 14. For example, it is possible to view the history of a person's health periodically, such as once a month or once every three months, or whenever a person experiences some kind of health problem.

As a result, the health management system has the following advantages over conventional systems:

(1) By looking at changes over time, you can see trends in your health condition, and if there is a trend toward illness, you can take preventive measures.

(2) By continuously recording audio, for example, even once a week, it is possible to grasp the health status every week. This is much more frequent than a health checkup once or twice a year, so that the timing for prevention and early detection is not missed.

(3) In the case of mental illnesses such as depression, various factors such as genetic factors, personal qualities (stress tolerance, resilience, etc.), the physical environment of the residential area, and close human relationships with family and colleagues at work overlap to cause the illness. If the causes are not removed, there is a high risk of recurrence even after recovery. Therefore, by analyzing the illness over time, it is possible to determine the timing at which the health condition deteriorates, and by considering the life events at that time, it is possible to identify the main cause. This makes it easier to remove the causes, making it less likely that the illness will recur after recovery or remission.

(4) As dementia diseases such as Alzheimer's progress, the individual's awareness of the disease decreases and they tend to stop acknowledging that they have the disease. This makes it more difficult to receive treatment and the disease progresses more quickly, and increases the mental burden on family and others around the individual. Since anyone is more likely to develop dementia as they age, if the individual is aware of the risk of dementia progressing while the disease is still mild, they can actively receive training to prevent the progression of dementia and prepare for the future. It also allows the family to prepare for the future and make it easier to deal with the disease.
[8. Voice Acquisition Scene]

2, it has been described that the user terminal 201 includes a communication terminal such as a smartphone or a tablet having a means for inputting voice and a means for outputting distribution data. Here, other examples of the means for inputting voice will be introduced.

(1) When semi-forcefully instructing a user to speak and acquiring audio recordings: Confirming that a user has clocked in by audio recording instead of by using a time card when clocking in.
・When working remotely, a certain utterance is made from home etc. as a signal to start work. The same applies when work ends.
- Used as a pre-work examination (to understand health status) for transport drivers, pilots, etc.
・During follow-up monitoring of treatment, patients will be asked by their doctor to record their voice. For example, voice recordings will be made when patients take their medicine after each meal.

(2) When a user is in a situation where he or she regularly speaks and that voice is to be acquired, the voice of a call center operator's conversation is to be acquired.
・Capture audio of customer service conversations at convenience stores, fast food restaurants, and other retail businesses.
-Capture audio spoken in the course of business by announcers, voice actors, presenters, station staff, etc.
- When smart home appliances are operated by voice, the voice is acquired.
- Capturing audio when chatting within a team in online games, etc.
- When playing a smartphone app game, set it so that spells are cast using actual audio.
- Capture audio of radio conversations used by police officers, security companies, taxis, forestry workers, etc.
-Capture audio when medical and care workers speak to patients, etc.
- Acquire voice spoken at the counter of an insurance or financial store when introducing a product.

(3) When obtaining voice that the user voluntarily speaks in order to obtain a benefit by giving the user an incentive. In a smartphone game, the system is set up so that the user can obtain an item that can be used in the game by making a sound.
- When accessing a coupon distribution site, set it up so that you can receive a more preferential coupon than usual by speaking.

(4) Creating a situation in which the user can speak by accessing the user from another person. Day care staff, security companies, etc., can periodically call elderly people living alone to check on their health and record the voice responses they make.
- The conversational robot periodically speaks to a user who owns the conversational robot as a pet, and the audio response is captured.
- If the user is hospitalized, their body temperature, pulse, and blood pressure are measured, and their speech is recorded.

(5) Suggest speaking to the user in places the user regularly visits, and capture the voice.
- For health and beauty conscious customers at gyms, yoga salons, swimming schools, beauty salons, dance studios, etc., audio recording will be taken as part of health management, similar to blood pressure and other measurements.

(6) When a user voluntarily and regularly records speech for the purpose of managing his/her own health.
By setting up a variety of voice capture situations, it is possible to create an environment in which voice capture can be continued. For each person, even if the voice capture situation changes, voice can be continuously captured as long as any of the above conditions are met.
[9. Example of distribution data]

An example of distribution data distributed from the third server 206 will be introduced.
[9-1. Output when healthy]

In the health management system 200, when the patient is healthy, the voice analysis engine is used in a non-medical field, and therefore it is considered that the engine for determining the risk of illness and mental and physical state is mainly used, rather than an engine for determining illness. Specifically, for example, a voice determination engine that classifies the patient into three categories: "healthy", "presenting depressive symptoms", and "presenting cognitive decline".

It is also possible to combine an engine that determines whether a person is "healthy" or "possibly ill" with an engine that determines whether the possible illness is "presenting depressive symptoms" or "presenting cognitive decline."
9-2. Examples of quantification by the calculation unit 211 or the estimation unit 212 and examples of conversion into language

The captured voice is analyzed and classified into three categories: "healthy," "exhibiting depressive symptoms," and "exhibiting cognitive decline." The confidence level of which category the person belongs to is calculated as a number that sums up to 1.

In this case, if the value representing "health" is the "H value," the certainty value of a depressed state is the "M value," and the certainty value of dementia is the "C value," when analyzing the voice of a certain user, the analysis results by the analysis device (engine) may be, for example, H value 0, M value 0.75, and C value 0.25.
H value 0.5, M value 0.25, C value 0.25, etc.

The numerical values obtained by the analysis device may be output as they are, but in order to make them easier for the user to understand, they may be converted into some kind of words and then displayed.

For example, if both the M and C values are 0.3 or less, the system judges the user to be healthy and outputs the message, "Your body and mind are in a healthy state." If the M or C value is between 0.3 and 0.5, the system suggests that the user try a program to maintain, strengthen, or improve their physical and mental health.

For example, examples of phrases that can be converted into language include, "Exercise can help relieve stress. How about going to the gym or dancing?", "If you have something worrying, try relieving it with mindfulness", "Massages, beauty treatments, exfoliation, and other things that involve contact with the skin are soothing", "Let's go on a trip to change your mood!", and "If you let your childlike side out, you'll be able to make the most of your natural talents. How about a party or a disco?"

When the M value or C value exceeds 0.5, feedback is given to the user for the category exceeding 0.5. For example, when the M value exceeds 0.5, feedback is given that can trigger awareness, such as "Are you feeling weak mentally?", "Are you under a lot of stress?", "It seems like you're trying too hard", "Are you having trouble with interpersonal relationships?", "It's important not to push yourself too hard", etc.
[9-3. When the possibility of illness emerges]

Furthermore, if the M or C value exceeds 0.5, it can be subjected to further analysis to determine the presence or absence of disease, and further feedback can be provided based on the numerical value.

For example, if the certainty factor indicating that a particular disease is present is 0.7 or higher using an analysis device that determines the disease, and the disease is one in which depressive symptoms are the main symptom, such as major depression, bipolar disorder, or atypical depression, then feedback such as "We recommend that you visit a mental clinic or a psychosomatic medicine specialist" can be added.Also, if the certainty factor is 0.7 or higher and the disease is one in which cognitive or motor decline due to brain dysfunction is the main symptom, such as Alzheimer's disease, Lewy body dementia, or Parkinson's disease, then feedback such as "We recommend that you visit a neurologist" can be added.
[9-4. When used by doctors]

When used by doctors, the engine is used as a regular diagnostic device to determine illness. By also using an engine that determines the severity of the patient's physical and mental condition or the severity of the illness, treatment progress can be monitored. In addition, by enabling data to be acquired over time through voice input from the patient, the effort required for doctors and nurses to visit patients' homes in person can be reduced, which can lead to reduced medical costs.
[10. Overall System Description]

15 is a schematic diagram showing an overall image of the health management system 200. The main system flow will be explained using steps S3001 to S3012.

First, in step S3001, voice input from the user is performed in various usage scenarios. In addition to voice input from the user terminal 201, the various usage scenarios include the scenarios listed in the section "8. Situations in which voice is acquired." In the health management system 200 having first to fourth servers, the input voice data is stored mainly in the first server 204. At the same time, the identification information input by the user via the user terminal 201 is stored in the second server 205 of the health management system 200.

Next, based on the input voice data, the fourth server 210 performs voice analysis, measurement, and estimation processing, and in step S3002, the estimation result is transmitted to the user terminal 201. In addition, the estimation result may be transmitted to the non-medical service institution 202A via the web server 202 (S3003). In this case, in step S3004, the estimation result is transmitted from the non-medical service institution 202A to the third server 206. Note that in FIG. 15, part of the function of the third server 206 is described separately from the health management system 200. Then, in step S3005, a message recommending a visit to a medical institution is transmitted from the third server 206 to the user terminal 201.

In addition, in steps S3006 and S3007, management information relating to the user's identification information is transmitted and received between the user terminal 201 and the health management system 200.

The user then undergoes a medical interview by visiting the facility, etc.

Next, in step S3008, the health management system 200 can transmit the voice data stored in the first server 204 and the estimation result data stored in the fourth server 210 to a facility. The facility can use the data when taking a medical interview, performing an examination, and making a diagnosis.

After the examination and diagnosis, the facility provides the diagnosis results to the health management system 200 (step S3009). The health management system 200 transmits and receives the voice data stored in the first server 204 and the estimated result data stored in the fourth server 210 (steps S3010 and S3011) during treatment and during the prognosis period, and can use them for treatment. Even after the patient goes into remission, the progress can be monitored by periodically acquiring the user's voice data.

In step S3012, the voice data and the time-series data of the estimation result can be fed back and used in various usage scenarios.
[11. Other Commentaries]

Other aspects of the present invention will be explained below. Note that the present invention is not limited to the following explanation.

FIG. 16 shows an example of registering a user using a voice platform device or another system (transmission/reception, output of unique determination results) or an embedded system.

The process for using this device is as follows:
(1) input the voice and other data of the registered user and store it in the user data storage device; (2) select the stored voice and other associated data;
(3) Select the service you wish to use,
(4) The voice data is analyzed using a voice data analysis device used in the service, and the results are stored in a user data storage device.
(5) Based on the analysis results, search for the discrimination results from the discrimination master prepared in advance based on other data,
(6) Transmit the results of the discrimination and any analysis results or other data required by the service.
(7) The service provider or system receives the data and provides the user with an output or service that is useful for behavioral support.

For example, currently, sports gyms and diet programs provide services aimed at improving physical health. At these health promotion facilities, goals are set and action plans are drawn up through services from activity trainers and other professionals. On the other hand, some organizations, particularly athletes, aim to achieve effective goals by improving not only the physical aspects but also the mental aspects. By using an audio platform device or a system with the functions of an audio platform device built in, it is possible to encourage the use of information that is effective in improving the mental aspects.

FIG. 17 explains the flow of user registration. The user checks the explanation of data usage from the consent screen for starting use, and if the user agrees to the contents, presses the start use button. After pressing the start use button, the initial registration screen is displayed. Input the necessary information (age, sex, medical history, etc.) displayed on the initial registration screen, and press the input completion button. When the input completion button is pressed, if a code that identifies the device used for input can be obtained, it is entered as the terminal identification number mainly used. After pressing the input completion button, the user himself selects "free speech" or "free speech" displayed on the screen, and inputs voice according to one or both input guides. When it is confirmed that the voice required for voiceprint extraction has been input, a screen indicating completion of voice input is displayed, and the input data is transmitted to the voice platform device as confirmed information by pressing the confirmation button.

For example, when a user wishes to use a voice platform device or a system incorporating the functions of a voice platform device to formulate a training plan that also includes mental aspects, the user registers. At this time, voice is acquired as user information.

FIG. 18 explains the flow of voice data storage. A user registers for the first time or inputs voice data at any time directly to the voice platform device, or inputs voice acquired by another system via the voice platform device. After completing the input confirmation, the voice data is sent from the state temporarily stored in the voice platform device to the data storage device. The transmitted voice and associated data are received by the data storage device constituting the voice platform device and stored in the database of the data storage device together with the voice acquisition date and time, the terminal used, and data associated with the use. The stored voice and other data cannot be searched, viewed, output, or deleted by anyone other than the user himself or the service or service provider approved by the user. Data sets that are difficult to uniquely identify individuals as described in the consent at the time of registration can be viewed and messages can be sent and received by businesses that have obtained permission to use the voice platform device from businesses that provide services using the voice platform device. When a user requests the voice platform operator to stop or terminate use in a predetermined procedure and the process is completed, the data that can identify an individual can be erased (logically or physically) based on consent, or if the voice platform operator determines that the user has violated the terms of use. Data that cannot uniquely identify individuals may continue to be used by voice platform operators as statistical information for those operators.

The data stored on this platform includes sensitive personal information, and the data storage environment will include measures to protect personal information, such as encryption and data distribution.

For example, voice data is input as part of an activity record when applying for use at a sports gym, when making a plan, during the execution of the plan, and when the plan is completed. If the voice platform device is used with a mobile terminal device such as a smartphone, it is possible to save and utilize voices captured by the user's own device in addition to those captured by the microphone at the sports gym.

Examples of audio acquisition methods include the following:

(1) When a user receives health promotion services, voice input is used to create an activity plan aimed at improving not only the physical condition but also the mental condition.

(2) When a user wishes to store or utilize voice or other data, the user inputs the data using the functions of the voice platform device, or using other apps, systems, etc. that utilize the functions of the voice platform device, or via wearable devices, etc. to which they are connected.

(3) With the user's consent, employers etc. obtain the user's voice in advance during business activities (such as call center response records, communication radio communications in the transportation industry, or when a singer is recording or performing live, or when an actor or voice actor is performing, or when taking medical interviews, providing nursing care, speaking to customers or counseling in medical, nursing care or health promotion services, or when selling insurance products, or when serving customers at a retail store, etc.), convert it into a form that can be handled by the voice platform device, and input it using the functions of the voice platform device.

(4) When businesses provide or implement incentives for the purpose of improving marketing or customer loyalty or obtaining information, or when using a system in an application, etc. to purchase cards or virtual items to be used in a game by lottery (so-called gacha), or when using the functions of a device that is mainly operated by voice, such as a smart speaker, or when operating a device with a voice input device such as a smartphone, or when operating an application on a device such as a computer or smartphone,

(5) Using voice for amusements (such as karaoke) or provision of services that involve voice input, or inputting voice obtained in connection with explaining or providing a service (such as voice obtained as part of a health check or for the purpose of understanding health status when purchasing an insurance product, or for the purpose of planning nursing care or health promotion services and evaluating the performance of such services).

(6) Audio is collected when trying on or wearing clothes or accessories, when riding in a vehicle such as a car, when viewing or living in a home in connection with a real estate transaction, or when trying out or using other services or goods.

(7) Services provided via the Internet (online shopping, remote medical care, counseling, education, financial product referrals, employment and career change support services, matching apps that arrange meetings with members of the opposite sex, etc.).

(8) When conveying the voice content or status of the information sender, the voice is input using a device such as a computer or smartphone.

(9) Daily voice input is performed in order to know changes in one's own condition at a more appropriate time when a mental illness or dementia-related disease relapses after remission.

FIG. 19 illustrates an example of voice analysis of a user whose identification information has been registered. When a user requests the use of the voice platform device's functions or other service provider's services using voice data and associated data that have been previously saved or registered or imported when using the service, and the service provider receives the request, the voice and other data required for the service are transmitted to the analysis device that uses the data required for the user's use of the service as an analysis data set. After confirming permission to use the service, the analysis device preprocesses and analyzes the received data set, and outputs the analysis result. The output analysis result data set is transmitted to the voice platform device. The voice platform device receives the analysis result of the analysis data set transmitted by the voice platform device, checks the result, and if the analysis is completed at the expected service level, a completion notice is transmitted to the analysis device, and if it is not completed, a reanalysis request is transmitted to the analysis device, or if it is due to a defect in the analysis data set, a notice is transmitted to the voice platform device, or if it is an unexpected error of any device, a predetermined message based on the error log is transmitted to the operator providing the service of the platform device, the user, and the administrator of the analysis device.

For example, the voice input in each scene is stored as the person's own voice in the data storage device of the voice platform device. The stored voice is stored together with a voiceprint value that indicates the person's identity by a voiceprint verification device that can be executed when the voice is acquired or when the voice is saved. The stored data is extracted from the data storage device and a data set required for analysis is created by a service selected by the user, and the analysis data set is transmitted from the voice platform device to an analysis device.

The analysis device preprocesses and analyzes the received analysis data set, and transmits the analysis results. By viewing the analysis results, the mental state (mental value) and the state of cognitive function (cognitive value) can be confirmed in chronological order for the voice recorded in chronological order.

20 illustrates an example of discrimination. When a user requests the use of a voice platform device function or other service provider service using the analysis results of voice data and associated data that have been previously stored or registered or imported when using a service, and the service provider accepts the request, the voice analysis result and other data required for the service are extracted as a discrimination data set and transmitted to a discrimination device that uses data required for discrimination based on discrimination conditions set based on the user's service usage request. The discrimination device searches for a corresponding discrimination result from a discrimination result data storage device that corresponds to data such as a voice analysis result that has been registered in advance, and outputs the data discrimination result to an output destination and a voice platform data storage device that are predetermined for the service.

For example, based on the analysis results data, indicators and information are identified to better understand the state of mental health and cognitive function. By checking the chronological status of the user from the results of the identification, the user can consider future solutions or services based on objective information.

21 illustrates an example of behavioral support. When a user requests the use of the voice platform device's functions or other service provider's services using the discrimination results of the voice data and associated data stored in the voice platform device or registered or imported when using the service, and the service provider accepts the request, the voice analysis results and other data and discrimination results required for the service are extracted as an information utilization data set and transmitted to an information utilization method selection device used in the service based on the information utilization conditions set or selected by the user or set in advance by the service provider as data constituting a part of the service.

The information utilization method selection device searches for a corresponding information utilization method from an information utilization method data storage device that corresponds to the received data set from pre-registered information utilization methods, and outputs the information utilization method selection result to an output destination predetermined by the service and to the voice platform data storage device.

The result of information utilization based on the output result is received. The received information utilization result is transmitted to the voice data platform data storage device. The voice data platform data storage device stores the received information utilization result as a result of the selected information utilization method.

For example, by displaying search results for information such as recommended training menus, counseling that is also effective for disease prevention, or feedback on diet (and other information held by platform operators and service providers), based on the results of the discrimination, it is possible to obtain progress on plans, opportunities for new support, or opportunities to take action to prevent disease.

22A and 22B explain the flow of the system. When the embedded system is introduced or when changes to the registered information occur, the service provider such as a medical facility registers facility information, employee information, etc. In addition, it is possible to share data and perform mutual input and output by combining the conditions of the examination and other services used in the embedded system from a combination of settings that are predetermined, settings by an administrator, or a connection from a system already owned by the service provider.

The target users are those who have previously registered as users of the service or facility that uses the present invention (in the case of medical care, this means that their information is registered as a patient at a medical facility and they have been issued a patient card, or they have joined social insurance and been issued a social insurance number, etc.).

By storing medical card-related data and social insurance-related data in the voice data platform device, it will be possible to perform identity verification and data exchange, etc., by using devices such as smartphones, IC card technology, and contactless communication technology.

When the user first starts using the embedded system, the user can check the terms of use and decide whether or not to accept them. After accepting the terms of use, the user's information is registered by the user himself/herself or his/her relatives, or by the medical facility staff requested by the user or his/her relatives, or by the system. If the user information is stored in a hospital system or the like in advance, it is also possible to register the user information after receiving prior consent. If the user inputs user information or voice, etc. using a device such as a smartphone, it is also possible to obtain information on the input device. In addition, identity verification is possible by incorporating a function to verify the identity of the user using voiceprints. The consent to use and the results of the registration of the user information are stored in the data storage device of the embedded system, and the results are output to the user and the service provider as electronic data, paper, or other format.

After the user is registered, the service provider, who has received instructions from the user or a relative, etc., either in writing or using an embedded system, etc., selects a service (analysis, discrimination, conditions for information utilization, etc.) and inputs voice data, etc., by capturing voice data, etc., stored in advance in a voice platform device or using a voice input device such as a microphone of the device.

After inputting the voice data to be analyzed, the data is checked to see if it meets the conditions of the service. If it does, it is sent to the data storage device of the embedded system; if it does not, a corresponding message or the like is output and the input screen is displayed again.

After confirming that the voice data has been input correctly, the voice data is output to a predetermined analysis device in accordance with the terms of the service. After confirming that the voice data has been received correctly, the analysis device performs pre-processing necessary for analysis, analyzes the voice data, and outputs the analysis results to a data storage device.

The discrimination device extracts data necessary for discrimination, including the stored analysis results, from the data storage device, and selects a corresponding data discrimination result from a previously set discrimination result data storage device. The data discrimination result is output to an information utilization method selection device and data storage device set in the service. The information utilization method selection device selects an information utilization method based on the contents previously set in the service based on the input data discrimination result, and outputs the selection result to the output destination and data storage device defined in the information utilization method. The information utilization result is received by the information utilization device and output to the data storage destination.

The data stored in the data storage device of the embedded system can be retrieved at the request of the user using a pre-registered smartphone terminal, an IC card with a magnetic storage device, contactless communication, etc. In addition, the results of using a specific service from the embedded system with the user's consent can also be retrieved via the embedded system's functions.

The health care system of the present invention can be used in the following situations, for example.
A patient who had been told by people around him that he had been feeling low recently, had low motivation in everything he did, and was making more and more simple mistakes, was taken by his family to a nearby general internal medicine clinic. General internal medicine clinics are not specialists in mental illnesses or dementia, so it can be difficult to decide on a treatment plan, or it can take a long time to arrive at a treatment plan that will be effective. However, when the patient visited this general internal medicine clinic, which has a built-in system, he was instructed to take a voice test as well as a simple measurement of cognitive function during the preliminary interview. Since the patient's motivation was declining, he and his family members confirmed the terms of use and agreed to the voice test. Since the family members handled everything except the patient's voice input, the burden on the patient was low, and the doctor checked the results of the voice test before entering the examination room.

At the time of the examination, the patient's cognitive function was also impaired, but based on the results of the voice test, the patient was diagnosed with a high possibility of major depression, a depressive disorder, and a treatment plan was made, which the patient and her family agreed to. Except when visiting the facility, the patient's voice was recorded using a treatment support app linked to the voice platform device on her smartphone before and after taking medication. During hospital visits, the voice data and changes in condition outside the hospital were confirmed, and the time-series voice data was imported from the smartphone to the built-in system using a non-contact communication function, and then re-examined using the built-in system. At that time, the voice data and medical data obtained during past hospital visits were saved as the patient's data in the data storage device on the voice platform device. Since the condition outside the hospital suggested that the treatment effect may be lower than expected, the treatment plan was revised and the prescription was changed. When the symptoms improved and gradual improvement was seen at work, the patient was transferred to a distant location. In order to continue treatment, the voice platform device was used to search for hospitals that could be visited from the new address and that used the voice platform device, a referral letter was created at the current hospital, and the history of treatment to date was saved as data on the patient's own voice platform device. The data allowed the patient to continue treatment without interruption at the new hospital. The treatment was more effective and the patient's symptoms improved, so the timing for tapering was detected, which would be difficult to detect with a regular examination alone. When symptoms were stable, the patient was allowed to adjust the dosage themselves, and was able to tapering between visits while checking the audio data. The detailed tapering, which took into account daily fluctuations, was effective, and the patient went into remission. Although the relapse rate of mental illnesses is high after remission, the patient was able to use the audio platform device to grasp his condition chronologically, and was able to find activities that had a positive effect on his mental health through data, which helped him to remain stable. However, when his workload temporarily increased, he noticed a drop in his mental health values, so he went back to the hospital. Because the consultation was received before his symptoms worsened, he went into remission with shorter visits, mainly cognitive behavioral therapy.

FIG. 23 explains data transmission and reception, the voice platform, the voice analysis device, and the embedded system.

"300. Other systems (data transmission only)" acquires voice data that identifies the user and transmits it to the voice platform device. Voice data can be acquired from smartphones, smart speakers, karaoke, business radios, etc. The results can be confirmed on the voice platform device.

In "400. Other systems (transmission and reception, output of unique judgment results)," voice acquisition and judgment result output are performed by this device. The judgment results can be converted to the device's own judgment results based on the judgment results. This applies to systems that have individual requirements for training and care budget planning support systems and attendance and other business management systems.

With "100. Voice Platform Device," the device itself can record, select services, check results, check history, view information, and support secondary data use. If there are specific constraints or requirements for voice acquisition or feedback of discrimination results, data determined by this system based on the results of receiving and analyzing voice data acquired outside of this device can be output to a specific external system. Note that even when processing outside of this device, the consent of the user must be obtained using this device or the procedures and functions provided by this device.

"200. Voice analysis device" has the function of preprocessing and analyzing the voice data received from the voice platform device, and transmitting the analysis results to the voice platform device in a predetermined format. The analysis determines whether or not to perform the analysis process based on the service information received along with the voice data. After the analysis is performed, the results such as the possibility of a specific disease, mental values, cognitive values, etc. are transmitted, and the execution history of the process up to the completion of reception by the voice platform device is stored. After analysis, the voice data is automatically deleted.

In the case of "500. Embedded system", when a direct connection to the voice platform device is not desired due to restrictions such as portability of medical data, the intention of the service provider, and various regulations, the functions of this device can be embedded in an individual system to obtain voice and output the determination results. When information managed in a system of a medical facility or the like is to be taken out using an emergency connection using NFC or Bluetooth (registered trademark) technology, the information can be imported into the voice platform device by outputting it in a form that meets the format specifications that can be input into the voice platform device.

The data output at each step to the data storage destination of the voice platform device and the embedded system is stored as a record linking the extraction and selection results for each user. The stored results can be reused within the scope of the individual's consent.

The stored data, excluding information that can identify individuals to the extent that consent for use has been obtained, can be searched, viewed, or extracted as statistical information only if a third party has permission from the operator of the voice platform device. For users who meet certain conditions set by a third party without being identified as individuals, if the user so wishes, it is possible to disclose or provide some or all of the data along with the purpose of use of the personal information, or to respond to requests for disclosure from operators, etc., or, for example, to recruit volunteers for clinical trials, via the voice data platform.

FIG. 24 describes the voice platform device. This device is composed of a voice acquisition unit, an acquired voice confirmation unit, an acquired voice transmission unit, a data storage unit, an analysis data set extraction unit, an analysis data set transmission unit, an analysis result data reception unit, an analysis result discrimination device (including discrimination target data extraction, discrimination result search, and discrimination result output, etc.), an information utilization method selection device (including selection target data extraction, information utilization method search, and search result output, etc.), and a user and administrator use screen of each device, etc. In addition, the data stored in the data storage unit includes, for example, user data consisting of user data, service selection data, voice data, service utilization history, analysis data, discrimination data, information utilization method selection result, and information utilization implementation data, etc. Various master data consisting of service providers, information utilization methods, and service price lists, etc. Analysis engine utilization data consisting of analysis engine types, connection methods, connection settings, and connection history, etc. Secondary utilization data consisting of public minimum datasets, browsing history, data utilization application history, data utilization approval history, data lending period, and lending data deletion history, etc. Various data such as transaction data, master data, various settings, and utilization execution history can be cited.
This voice platform device is a new device that enables highly secure and unprecedented utilization of data, mainly voice, by the individual himself/herself, or by a person or business authorized by the individual.

The voice analysis device is explained in Fig. 25. It is composed of a voice analysis engine execution and management environment, an analytical voice input device, an analytical voice preprocessing device, a voice analysis device, an analysis result output device, an output destination reception result confirmation device, an analysis data set and analysis result deletion device, an analysis engine usage record acquisition device, a usage history acquisition and complement device, a device for calculating and outputting costs from the usage record by user and engine, and a connection method setting device by connection destination.

To provide an environment in which multiple analysis engines can be selected and executed upon request, personal information can be analyzed according to the request of the individual, and after the analysis, data analysis can be performed in a multifaceted manner without identifying individuals by having only a portion of the user data held by the requesting system or service.

26, 27A and 27B, other systems will be described.

A system that has some of the functions of a voice platform device (acquiring voice data, checking voice data, discriminating, searching for ways to utilize information, etc.). A form of use aimed at providing services that utilize systems already owned by individual service providers, or when it is necessary to meet unique requirements for security, usability, etc. After acquisition and confirmation, the voice is sent to the voice platform, and the results of analysis, discrimination, and searching for ways to utilize the information are received in a data format predetermined by the voice platform device.

When voice is acquired according to a unique specification, the acquired voice can be output according to a predetermined specification, enabling voice data input to the voice platform device. After the voice data is input, the functions of the voice platform device can be used.

This refers to a usage pattern when a network connection to a voice platform device is not possible, such as in a medical facility, and is an independent system device that handles everything from obtaining consent to use to outputting results. Data input from the voice platform device and data output to the voice platform device are not transmitted via the Internet, but through wireless technology such as a smartphone (Bluetooth (registered trademark),
This can be realized via a communication technology such as NFC, AirDrop, etc., or an IC card with memory.

Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section.

This application claims priority based on Japanese Patent Application No. 2019-232504, filed on December 24, 2019, and incorporates by reference all of the contents of that Japanese application.

200 Health management system 204 First server 205 Second server 206 Third server 210 Fourth server 211 Calculation unit 212 Estimation unit 213 Selection unit

Claims (7)

a first server for continuously storing voice data acquired from a user;
a second server for storing user identification data;
a third server for storing output data for users; and
Estimation device,
A health management system comprising:
The estimation device includes:
A calculation unit that calculates an acoustic feature based on the speech data;
an estimation unit that estimates a predetermined disease using the acoustic feature as an input; and a selection unit that selects output data of the third server based on an estimation result of the estimation unit and identification data of the user.
Equipped with
The selection unit is
when the estimation unit estimates that there is no suspicion of the subject suffering from the predetermined disease, the selection unit selects, from the third server, first output data including distribution data for healthy individuals;
When the estimation unit estimates that the patient is suspected of having the predetermined disease, the selection unit selects second output data including a message from the third server encouraging the patient to visit a medical institution,
After the user is diagnosed by a doctor as suffering from a predetermined disease, the identification data of the second server is updated, and the selection unit is configured to select third output data including distribution data for the user after the user is diagnosed with the disease from the third server,
The health management system wherein the estimation unit estimates at least one disease from a group of diseases including Alzheimer's disease, Lewy body dementia, Parkinson's disease, atypical depression, and bipolar disorder. The health management system of claim 1, wherein the estimation unit is also capable of estimating major depression. The estimation unit is capable of classifying the subject into a dementia group including Lewy body dementia, Alzheimer's disease, and Parkinson's disease, and a mood disorder group including major depression, bipolar disorder, and atypical depression.
The health management system according to claim 2 . The health management system according to claim 1 or 2, wherein the estimation unit is capable of estimating Lewy body dementia. The health management system according to claim 1 or 2, wherein the estimation unit is capable of estimating Alzheimer's dementia. The health management system according to claim 1 or 2, wherein the estimation unit is capable of estimating Parkinson's disease. The health management system according to claim 1 , wherein the estimation unit is capable of estimating bipolar disorder.

Images