JP7548846B2 - Analytical device, analytical method, and analytical program - Google Patents

Description

The present invention relates to an analysis device, an analysis method, and an analysis program.

Technology is known that performs mental analysis of a user based on voice data produced by the user.

JP 2015-90506 A

However, conventional technology has a problem in that it may not be possible to accurately analyze the stress state of a user living indoors. For example, it is believed that a user's stress state is affected by the surrounding environment. On the other hand, conventional technology may not be able to take into account the environment surrounding the user.

In order to solve the above-mentioned problems and achieve the objective, the analysis device is characterized by having an acquisition unit that acquires user information, which is information about the user's behavior, and environmental information, which is information about the user's environment, a calculation unit that calculates a score representing the level of stress of the user based on the user information and the environmental information, and a determination unit that determines whether the level of stress calculated by the calculation unit exceeds a threshold value.

The present invention makes it possible to accurately analyze the stress state of a user living indoors.

FIG. 1 is a diagram illustrating an example of the configuration of an analysis system according to the first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of the analysis device according to the first embodiment. FIG. 3 is a diagram illustrating an example of the configuration of the learning device according to the first embodiment. FIG. 4 is a flowchart showing a process flow of the analysis apparatus according to the first embodiment. FIG. 5 is a flowchart showing a flow of processing of the learning device according to the first embodiment. FIG. 6 is a diagram illustrating an example of a computer that executes an analysis program.

Below, embodiments of the analysis device, analysis method, and analysis program according to the present application are described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments described below.

[Configuration of the first embodiment]
First, the configuration of an analysis system including an analysis device will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an analysis system device according to a first embodiment. As shown in Fig. 1, an analysis device 10 exchanges information with devices provided in a building 2 via a network N. For example, the network N is the Internet. The analysis device 10 may be provided outside the building 2 or may be provided inside the building 2.

As shown in FIG. 1, the analysis system 1 has an analysis device 10. The analysis system 1 also has a microphone 20 and a thermometer/hygrometer 30 provided in a building 2. Furthermore, a user U1 and a user U2 exist in the building 2. For example, the users U1 and U2 are in a relationship of family members or partners living together. For example, the building 2 is a house in which the users U1 and U2 live.

In this embodiment, the microphone 20 and the thermometer/hygrometer 30 are IoT (Internet of Things) devices equipped with a communication function, and are capable of performing data communication with the analysis device 10 via the network N. The microphone 20 and the thermometer/hygrometer 30 may also perform data communication via other communication devices.

The analysis device 10 receives the information collected by the microphone 20 and the thermometer/hygrometer 30. The analysis device 10 then analyzes the stress conditions of the users U1 and U2 based on the input information and outputs the analysis results.

The microphone 20 collects audio and transmits the collected audio to the analysis device 10 as a signal in a specified format. The thermo-hygrometer 30 also transmits the measured temperature and humidity to the analysis device 10 at regular intervals.

The configuration of the analysis device 10 will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of the configuration of the analysis device according to the first embodiment. As shown in FIG. 2, the analysis device 10 has an interface unit 11, a memory unit 12, and a control unit 13.

The interface unit 11 is an interface for inputting and outputting data. For example, the interface unit 11 is a NIC (Network Interface Card). The interface unit 11 can transmit and receive data to and from other devices via the network N.

The memory unit 12 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or an optical disk. The memory unit 12 may be a semiconductor memory in which data can be rewritten, such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM). The memory unit 12 stores the operating system (OS) and various programs executed by the analysis device 10.

The memory unit 12 stores model information 121. The model information 121 is information for constructing a model. For example, if the model includes a neural network, the model information 121 is parameters such as a weight matrix and bias values for constructing the neural network.

The control unit 13 controls the entire analysis device 10. The control unit 13 is, for example, an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The control unit 13 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory. The control unit 13 also functions as various processing units by the operation of various programs. For example, the control unit 13 has an acquisition unit 131, a calculation unit 132, and a determination unit 133.

The acquisition unit 131 acquires user information, which is information about the user's behavior, and environmental information, which is information about the user's environment. For example, the acquisition unit 131 acquires user information from the microphone 20 and environmental information from the thermometer/hygrometer 30.

The acquisition unit 131 acquires, as user information, voice obtained by a microphone 20 installed in the house where the user lives. The acquisition unit 131 also acquires, as environmental information, the indoor temperature and humidity obtained by a sensor installed in the user's home.

The calculation unit 132 calculates a score representing the user's level of stress based on the user information and environmental information. The calculation unit 132 inputs the user information and environmental information into the model constructed from the model information 121, and obtains a score indicating the level of stress. In the example of Figure 1, the model outputs a score of "75."

The calculation unit 132 performs calculations by referring to environmental information as well as user information, so it can calculate the degree of stress more accurately. For example, indoor conditions such as high temperature, low temperature, high humidity, and low humidity can cause a sudden change in mentality compared to normal, which can increase a person's level of stress.

The calculation unit 132 can adjust the score depending on the degree of comfort indicated by the environmental information. For example, the calculation unit 132 may multiply the score by a predetermined weight greater than 1 when the temperature and humidity are outside a predetermined range. Also, for example, the calculation unit 132 may add a value proportional to the discomfort index calculated from the temperature and humidity to the score.

The calculation unit 132 can perform calculations such that the score increases as the difference between the characteristics of the voice to be analyzed obtained by the microphone 20 and the characteristics of normal voice prepared in advance increases. For example, the calculation unit 132 may calculate the distance between a vector representing the characteristics of normal voice and a vector representing the characteristics of the voice to be analyzed as the score.

In this case, the model may be trained using an unsupervised learning method. That is, the model learns the voice features that occur when each user is not stressed (normal). Then, the calculation unit 132 uses the model to calculate the degree of deviation of the input voice features from the learned features as a score. Specifically, the calculation unit 132 may calculate the score using an anomaly detection method that uses a generative model and a clustering analysis method such as k-means.

In addition to the voice features, environmental information may also be input to the model. In this case, the model learns normal voice features and environmental information, and calculates a score from the input voice features and environmental information.

The voice includes not only the user's voice but also all sounds that occur in daily life, such as breathing sounds, footsteps, and even sounds made when moving an object. The calculation unit 132 may also input features extracted from the voice (for example, the fast Fourier transform (FFT) of the voice signal) to the model.

The normal features used to calculate the score may be extracted from audio collected regardless of date or time, or may be extracted from audio collected on a specific date and time period.

Here, it is considered that the tendency of the voice characteristics to occur varies depending on the date and time period. If the normal voice characteristics are extracted from voice collected on a specific date and time period, it becomes possible to calculate a score according to the date and time period when the voice to be analyzed was collected. For example, if the voice to be analyzed was collected between 18:00 and 19:00 on a Monday, the calculation unit 132 calculates the difference between the characteristics of the voice to be analyzed and the characteristics of normal voice collected between 18:00 and 19:00 on a Monday different from the day when the voice to be analyzed was collected, as the score.

It is also considered that the tendency of the generated voice characteristics will differ depending on the user's behavior. Therefore, by extracting the normal voice characteristics for each user behavior, it becomes possible to calculate a more accurate score. For example, if the voice to be analyzed is a voice collected while the user is eating, the calculation unit 132 calculates the difference between the voice characteristics of the user eating a meal that was collected in advance and the voice characteristics to be analyzed as the score.

Furthermore, the audio may be separated into voice and non-voice sounds (footsteps, sounds of moving objects, etc.) using a specified method. In this case, the calculation unit 132 calculates a score for each separated audio.

A user who is stressed may take certain actions to relieve stress. Therefore, the calculation unit 132 calculates, as a score, the degree of similarity between the characteristics of the voice picked up by the microphone 20 and the characteristics of the voice resulting from a certain action taken in response to increased stress.

Increased stress may occur along with irritation, haste, fatigue, etc. Therefore, for example, if a user is stressed, it is conceivable that they may speak louder due to irritation, handle things roughly and cause more collision noises, shorten the intervals between footsteps due to haste, or lengthen the intervals between footsteps due to fatigue.

It is also conceivable that sound may be generated when the user performs a specific action to relieve stress. For example, a stressed user may sigh, bang a part of their body such as a finger on a desk, scream, or make small movements of a part of their body (known as fidgeting).

The determination unit 133 determines whether the degree of stress calculated by the calculation unit 132 exceeds a threshold value. For example, when the score calculated by the calculation unit 132 exceeds a threshold value, the determination unit 133 determines that the user corresponding to the score is in a stressed state, and outputs the determination result to an administrator or another system, etc.

Here, regardless of whether a user is stressed or not, it is conceivable that the characteristics of the generated voice will differ if the user's physical characteristics, personality, lifestyle, etc. differ. Therefore, a model may be prepared individually for each of multiple users. In this case, the model is assumed to have learned the characteristics of each user's normal voice. This makes it possible to calculate the user's level of stress with high accuracy.

In this case, the acquisition unit 131 acquires the voice obtained by the microphone 20 in association with identification information that identifies the user who produced the voice. The calculation unit 132 then performs calculations using a model that outputs a score from the input user information and environmental information, and that is identified by the identification information from among the models associated with each user.

Furthermore, a separate microphone may be provided for each user. This allows the acquisition unit 131 to use information identifying the microphone from which the voice is sent as information identifying the user.

In addition, an individual microphone for each user may be provided in a mobile device (e.g., a smartphone), a wearable device, or the like owned by each user. In this case, the acquisition unit 131 acquires voice via each terminal, and further identifies the user by using identification information (e.g., a MAC (Media Access Control) address) of each terminal that is associated with the user in advance.

The configuration of a learning device for training a model will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the configuration of a learning device according to the first embodiment. Note that the analysis device 10 may perform the same processing as the learning device. As shown in FIG. 2, the learning device 40 has an interface unit 41, a memory unit 42, and a control unit 43.

The interface unit 41 is an interface for inputting and outputting data. For example, the interface unit 41 is a NIC. The interface unit 41 can transmit and receive data to and from other devices via the network N.

The storage unit 42 is a storage device such as an HDD, SSD, or optical disk. The storage unit 42 may be a semiconductor memory in which data can be rewritten, such as a RAM, flash memory, or NVSRAM. The storage unit 42 stores the OS and various programs executed by the learning device 40.

The memory unit 42 stores model information 421. The model information 421 is information for constructing a model. For example, if the model includes a neural network, the model information 421 is parameters such as a weight matrix and bias values for constructing the neural network.

The model information 121 of the analysis device 10 is information on a trained model, whereas the model information 421 of the learning device 40 may be untrained or in the process of being trained. For example, the learning device 40 can repeatedly update the model information 421 and then pass it on to the analysis device 10.

The control unit 43 controls the entire learning device 40. The control unit 43 is, for example, an electronic circuit such as a CPU, MPU, or GPU, or an integrated circuit such as an ASIC or FPGA. The control unit 43 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory. The control unit 43 also functions as various processing units by running various programs. For example, the control unit 43 has an acquisition unit 431, a calculation unit 432, and an update unit 433.

The acquisition unit 431 and the calculation unit 432 perform the same processes as the acquisition unit 131 and the calculation unit 132 of the analysis device 10, respectively. The update unit 433 updates the model information 421 so that the difference between the score calculated by the calculation unit 432 and the correct score becomes smaller.

For example, the correct score can be obtained by directly asking the user about the level of stress they were under when they uttered the sound for which the score was calculated.

[Processing of the First Embodiment]
The flow of processing by the analysis device 10 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing by the analysis device according to the first embodiment. First, the analysis device 10 acquires a voice inside a building (step S101). For example, the building is a house where a user lives.

Next, the analysis device 10 inputs the features extracted from the voice into the model (step S102). The analysis device 10 then uses the model to calculate a score indicating the degree of stress (step S103). The analysis device 10 then performs a stress assessment (step S104).

The processing flow of the learning device 40 will be described using FIG. 5. FIG. 5 is a flowchart showing the processing flow of the learning device according to the first embodiment. First, the learning device 40 acquires the voice inside the building and the correct score (step S201).

For example, the correct score is the degree of stress actually felt by the user in the building at the time the voice was generated. Next, the learning device 40 inputs the features extracted from the voice into the model (step S202).

Next, the learning device 40 uses the model to calculate a score indicating the degree of stress (step S203). Then, the learning device 40 updates the model based on the difference between the calculated score and the correct score (step S204).

[Effects of the First Embodiment]
As described above, the acquisition unit 131 acquires user information, which is information about the user's behavior, and environmental information, which is information about the user's environment. The calculation unit 132 calculates a score representing the user's stress level based on the user information and the environmental information. The determination unit 133 determines whether the stress level calculated by the calculation unit 132 exceeds a threshold value. In this manner, the analysis device 10 calculates the score and determines the stress level using both the user information and the environmental information. Therefore, according to this embodiment, the stress state of a user living indoors can be analyzed with high accuracy.

The acquisition unit 131 acquires, as user information, voice obtained by a microphone 20 installed in the house in which the user lives. For example, it is possible to analyze the user's stress based on images acquired by a camera. However, some users may dislike being monitored by a camera from the standpoint of privacy, etc. In contrast, in this embodiment, there is no need to install a camera, so information can be collected without the user being displeased.

The calculation unit 132 performs calculations so that the score increases as the difference between the voice characteristics obtained by the microphone 20 and the pre-prepared normal voice characteristics increases. This makes it possible to improve the accuracy of score calculation using only normal voice, even when voices under high stress cannot be obtained in advance (unsupervised learning).

The calculation unit 132 calculates the degree of similarity between the voice characteristics obtained by the microphone 20 and the voice characteristics resulting from a specific behavior performed in response to increased stress as a score. This makes it possible to detect behaviors performed in response to increased stress and calculate the score more accurately.

The acquisition unit 131 acquires the voice obtained by the microphone 20 in association with identification information that identifies the user who produced the voice. The calculation unit 132 is a model that outputs a score from the input user information and environmental information, and performs calculations using a model identified by the identification information from among the models associated with each user. This makes it possible to calculate a score taking into account differences in how each user feels stress.

The acquisition unit 131 acquires indoor temperature and humidity obtained by a sensor installed in the user's home as environmental information. This makes it possible to calculate a score taking into account the impact that temperature and humidity have on the user.

[Other embodiments]
In the above embodiment, the analysis device 10 has been described as acquiring information from the microphone 20 and the thermo-hygrometer 30, but the analysis device 10 may acquire information from devices equipped with other sensors. In addition, the model may learn sounds generated in the life cycle of each user, and determine an increase in stress based on a disruption in the cycle.

In addition, when the analysis device 10 determines that the score of any one of the multiple users present in the building 2 is equal to or greater than a threshold, the analysis device 10 may notify the other users of the determination result. For example, when the analysis device 10 determines that the score of user U1 is equal to or greater than a threshold, the analysis device 10 notifies the terminal owned by user U2 that the stress of user U1 is increasing. This allows user U2 to take measures to reduce the stress of user U1.

[System configuration, etc.]
In addition, each component of each device shown in the figure is functionally conceptual, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in part by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. Note that the program may be executed not only by the CPU but also by other processors such as a GPU.

Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[program]
In one embodiment, the analysis device 10 can be implemented by installing an analysis program that executes the above-mentioned analysis process as package software or online software on a desired computer. For example, the above-mentioned analysis program can be executed by an information processing device, causing the information processing device to function as the analysis device 10. The information processing device referred to here includes desktop or laptop personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).

The analysis device 10 can also be implemented as an analysis server device that provides services related to the above-mentioned analysis processing to a client, the client being a terminal device used by the user. For example, the analysis server device is implemented as a server device that provides an analysis service that takes user information and environmental information as input and outputs a stress state assessment result. In this case, the analysis server device may be implemented as a web server, or may be implemented as a cloud that provides services related to the above-mentioned analysis processing by outsourcing.

Figure 6 is a diagram showing an example of a computer that executes an analysis program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. Each of these components is connected by a bus 1080.

The memory 1010 includes a read only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define each process of the analysis device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to the functional configuration of the analysis device 10 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

The setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary, and executes the processing of the above-described embodiment.

The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

N network U1, U2 user 1 analysis system 2 building 10 analysis device 11, 41 interface unit 12, 42 storage unit 13, 43 control unit 20 microphone 30 thermo-hygrometer 40 learning device 131, 431 acquisition unit 132, 432 calculation unit 133 determination unit 433 update unit

Claims (6)

an acquisition unit that acquires user information, which is a voice collected on a specific day of the week and in a specific time period by a microphone installed in a house where the user lives , and environmental information, which is information about the user's environment;
a calculation unit that calculates, based on the user information and the environmental information , a score representing a degree of stress of the user, the score increasing as a difference between a feature of the voice collected by the microphone and a feature of a normal voice prepared in advance and collected by the microphone on the specific day of the week and in the specific time period increases ;
a determination unit that determines whether the score calculated by the calculation unit exceeds a threshold;
An analytical device comprising: The analysis device according to claim 1 , characterized in that the calculation unit calculates as a score the degree of similarity between the characteristics of the voice obtained by the microphone and the characteristics of the voice produced by a specific action performed in response to increased stress. The acquisition unit acquires the voice obtained by the microphone in association with identification information that identifies a user who uttered the voice,
The analysis device according to claim 1 or 2, characterized in that the calculation unit performs calculations using a model that outputs the score from the input user information and environmental information, and that is identified by the identification information among models associated with each of the users. The analysis device according to claim 1 , wherein the acquisition unit acquires, as the environmental information, indoor temperature and humidity obtained by a sensor provided in the user's home. An analytical method performed by an analytical device, comprising:
an acquisition step of acquiring user information, which is a voice collected on a specific day of the week and in a specific time period by a microphone installed in a house where the user lives , and environmental information, which is information about the user's environment;
a calculation step of calculating, based on the user information and the environmental information, a score representing a degree of stress of the user, the score increasing as a difference between a feature of the voice collected by the microphone and a feature of a normal voice prepared in advance and collected by the microphone on the specific day of the week and in the specific time period increases ;
a determination step of determining whether the score calculated by the calculation step exceeds a threshold;
An analytical method comprising the steps of: An analysis program for causing a computer to function as the analysis device according to any one of claims 1 to 4 .

Images