JP7615772B2 - Relaxation level determination device, relaxation level determination method, and relaxation induction device - Google Patents

Description

The present invention relates to a relaxation level determination device, a relaxation level determination method, and a relaxation induction device.

A technique for estimating human emotions is known as one technique for controlling machines that takes human emotions into account. One such technique is to estimate the emotion of a subject based on the facial expression and biometric information of the subject, and combine the obtained estimation results based on weighting by the subject to obtain a final estimation result of the subject's emotion (see, for example, Patent Document 1).

JP 2012-59107 A

However, the above-mentioned conventional techniques leave room for further study in terms of estimating the subject's degree of relaxation (relaxation level). For example, the above-mentioned Patent Document 1 does not disclose a method for determining the subject's degree of relaxation. Furthermore, with the technique of the above-mentioned Patent Document 1, the final estimation result may lack objectivity because it includes weighting by the subject for the estimation results based on each index. For this reason, if the subject is not aware of the relationship between the subject's emotions and the detected indexes, inappropriate weighting may be applied, which may reduce the accuracy of the final estimation result.

One aspect of the present invention aims to realize a technology that accurately determines the level of relaxation from the subject's biometric information and facial expression information.

In order to solve the above problem, a relaxation level assessment device according to one aspect of the present invention includes a heart rate measurement value acquisition unit that acquires a heart rate measurement value of the person being assessed, a facial expression information acquisition unit that acquires facial expression information of the person being assessed, a relaxation level estimation unit that estimates the relaxation level of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit, and a relaxation level correction unit that corrects the relaxation level estimated by the relaxation level estimation unit based on the facial expression information acquired by the facial expression information acquisition unit, and judges the relaxation level corrected by the relaxation level correction unit to be the relaxation level of the person being assessed.

In order to solve the above problem, a relaxation level assessment method according to one aspect of the present invention includes a heart rate measurement value acquisition step of acquiring a heart rate measurement value of the person being assessed, a facial expression information acquisition step of acquiring facial expression information of the person being assessed, a relaxation level estimation step of estimating the relaxation level of the person being assessed by referring to the heart rate measurement value acquired in the heart rate measurement value acquisition step, and a relaxation level correction step of correcting the relaxation level estimated in the relaxation level estimation step based on the facial expression information acquired in the facial expression information acquisition step, and the relaxation level corrected in the relaxation level correction step is assessed as the relaxation level of the person being assessed.

With these configurations, it is possible to determine the degree of relaxation by appropriately using the measured heart rate value and facial expression information of the person being assessed, thereby realizing a technology that can accurately determine the degree of relaxation from the biometric information and facial expression information of the person being assessed.

The relaxation level correction unit may correct the degree of relaxation of the person being assessed to be higher in accordance with facial expression information of the person being assessed that is more strongly correlated with a high level of relaxation, and may correct the degree of relaxation of the person being assessed to be lower in accordance with facial expression information that is less strongly correlated with a high level of relaxation.

This configuration is even more effective in terms of increasing the accuracy of determining the degree of relaxation by appropriately reflecting facial expression information in the determination of the degree of relaxation.

The relaxation level correction unit may further classify the type of the highly relaxed state of the person being assessed according to facial expression information of the person being assessed that is more strongly correlated with a high level of relaxation.

This configuration is even more effective in terms of increasing the accuracy of determining the level of relaxation by appropriately classifying the type of relaxation level from facial expression information.

The relaxation level assessment device may further include a frequency component extraction unit that extracts ultra-low frequency components and high frequency components of the heart rate fluctuations of the person being assessed by referring to the heart rate measurement values acquired by the heart rate measurement value acquisition unit, and the relaxation level estimation unit may estimate the relaxation level of the person being assessed based on the ultra-low frequency components and high frequency components extracted by the frequency component extraction unit.

This configuration is more effective in reducing the information processing load in the relaxation level estimation process and in increasing the accuracy of the relaxation level estimation.

The frequency component extraction unit may determine the magnitude of the heart rate fluctuation of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit, and the relaxation level estimation unit may estimate the relaxation level of the person being assessed based on the result of the heart rate fluctuation determination by the frequency component extraction unit, based on the ultra-low frequency components and the high frequency components when the heart rate fluctuation is large, and may estimate the relaxation level of the person being assessed based on the ultra-low frequency components when the heart rate fluctuation is small.

This configuration is even more effective in reducing the information processing load in the relaxation level estimation process and in increasing the accuracy of the relaxation level estimation.

The frequency component extraction unit may extract ultra-low frequency components of the subject's heart rate fluctuations based on a moving average of the subject's heart rate fluctuations.

This configuration can clarify the characteristics of the heart rate fluctuations of the person being assessed, and is even more effective in terms of improving the accuracy of estimating the degree of relaxation.

The relaxation level estimation unit may estimate the degree of relaxation of the subject by referring to the reference data acquired by the reference data acquisition unit.

This configuration is even more effective in terms of accurately grasping the state of the subject's heart rate measurement value when estimating the degree of relaxation and increasing the accuracy of estimating the degree of relaxation.

The relaxation level assessment device may further include an external factor information acquisition unit that acquires external factor information including at least one of environmental information indicating the environment in which the person to be assessed is present and date and time information, and the relaxation level estimation unit may estimate the relaxation level of the person to be assessed by referring to the external factor information acquired by the external factor information acquisition unit.

This configuration is even more effective in terms of improving the accuracy of estimating the degree of relaxation, since it allows the influence of external factors to be appropriately reflected in the estimation of the subject's degree of relaxation.

The subject may be a passenger in a moving body, the heart rate measurement value acquisition unit may acquire the heart rate measurement value of the subject from a heart rate sensor mounted on the moving body, and the facial expression information acquisition unit may acquire facial expression information of the subject from a camera mounted on the moving body.

This configuration is even more effective in terms of making it easier to determine the degree of relaxation of the person being assessed in the moving body, and in terms of constantly utilizing the results of the assessment of the degree of relaxation of the person being assessed in the moving body in the operation of the moving body.

Furthermore, in order to solve the above problem, a relaxation induction device according to one aspect of the present invention includes a heart rate measurement value acquisition unit that acquires a heart rate measurement value of the person being assessed, a facial expression information acquisition unit that acquires facial expression information of the person being assessed, a relaxation level estimation unit that estimates the relaxation level of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit, a relaxation level correction unit that corrects the relaxation level estimated by the relaxation level estimation unit based on the facial expression information acquired by the facial expression information acquisition unit, and a stimulus provision unit that provides the person being assessed with a stimulus according to the relaxation level corrected by the relaxation level correction unit.

With this configuration, it is possible to determine the degree of relaxation by appropriately using the measured heart rate value and facial expression information of the person being assessed, thereby realizing a technology that can accurately determine the degree of relaxation from the biometric information and facial expression information of the person being assessed. Furthermore, with the above configuration, it is possible to guide the person being assessed into a more desirable state of relaxation by providing the person being assessed with a stimulus that is determined by referring to the highly accurate degree of relaxation.

According to one aspect of the present invention, it is possible to realize a technology that can accurately determine the degree of relaxation from the subject's biometric information and facial expression information.

1 is a block diagram showing a schematic functional configuration of a relaxation degree determination device according to a first embodiment of the present invention; 4 is a flowchart showing a flow of an example of a process for determining a relaxation level by the relaxation level determination device according to the first embodiment of the present invention. 10 is a flowchart showing the flow of a first example of a process for correcting a relaxation level based on facial expression information according to the first embodiment of the present invention. 10A and 10B are diagrams illustrating an example of a screen showing a result of determining a relaxation level according to a first example of correction of a relaxation level in the first embodiment of the present invention. 13 is a flowchart showing the flow of a second example of a process for correcting a relaxation level based on facial expression information according to the first embodiment of the present invention. 13A and 13B are diagrams illustrating an example of a screen showing a result of determining a relaxation level according to a second example of correction of a relaxation level in the first embodiment of the present invention. FIG. 11 is a block diagram illustrating a functional configuration of a relaxation degree determination device according to a second embodiment of the present invention. 13 is a flowchart showing the flow of a first example of a process for estimating a relaxation level based on a heart rate measurement value according to a second embodiment of the present invention. 13 is a flowchart showing the flow of a second example of a process for estimating a relaxation level based on a heart rate measurement value according to the second embodiment of the present invention. FIG. 11 is a block diagram illustrating a functional configuration of a relaxation degree determination device according to a third embodiment of the present invention. FIG. 13 is a diagram illustrating an example of an arrangement of devices around a subject in a third embodiment of the present invention. 13 is a flowchart showing a flow of an example of a process for determining a relaxation level by a relaxation level determination device according to a third embodiment of the present invention. FIG. 11 is a block diagram illustrating a functional configuration of a relaxation induction device according to a fourth embodiment of the present invention. 13 is a flowchart showing an example of a flow from a relaxation degree determination process to a relaxation induction process by a relaxation induction device according to a fourth embodiment of the present invention. 13 is a flowchart showing the flow of an example of a relaxation induction process according to a fourth embodiment of the present invention.

[Embodiment 1]
Hereinafter, one embodiment of the present invention will be described in detail.

[Functional configuration]
Fig. 1 is a block diagram showing a schematic functional configuration of a relaxation level assessment device according to a first embodiment of the present invention. As shown in Fig. 1, the relaxation level assessment device of this embodiment is composed of a control unit 1. The control unit 1 includes a heart rate measurement value acquisition unit 11 that acquires a heart rate measurement value of a person to be assessed, a facial expression information acquisition unit 12 that acquires information on a facial expression of the person to be assessed, a relaxation level estimation unit 13 that estimates a relaxation level of the person to be assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit 11, and a relaxation level correction unit 14 that corrects the relaxation level estimated by the relaxation level estimation unit 13 based on the facial expression information acquired by the facial expression information acquisition unit 12.

The control unit 1 is connected to a heart rate measuring device 15, a camera 16, a memory unit 17, and a notification device 18. The heart rate measuring device 15 can be appropriately selected within a range in which the heart rate of the person being assessed can be measured. Examples of the heart rate measuring device 15 include heart rate sensors arranged in accordance with each seat of a moving body (vehicle) described below, and a wristwatch-type vital information measuring device that can be worn by the person being assessed.

Camera 16 is a device that captures the facial expression of the person being assessed. Camera 16 is usually placed in front of the person being assessed so that it can capture the entire face of the person being assessed. The image captured by camera 16 may be a still image of the person being assessed's face or a video.

The memory unit 17 stores information for executing the determination (estimation and correction) of the degree of relaxation. Examples of the information include a program for performing the determination of the degree of relaxation, and numerical values, formulas, and maps used in the determination. The memory unit 17 can also store the measured values and acquired values in the determination of the degree of relaxation in association with the person being determined.

The notification device 18 is a device for notifying the person being assessed. Examples of the notification device 18 include an audio output device that outputs audio to notify the person being assessed, and a display device that displays text and images to notify the person being assessed.

[Example of relaxation level assessment]
2 is a flowchart showing an example of a process for determining a degree of relaxation performed by the relaxation degree determination device according to the first embodiment of the present invention. The heart rate measurement device 15 measures the heart rate of the person being determined. The camera 16 captures a front image of the facial expression of the person being determined.

In step S101, the heart rate measurement value acquisition unit 11 obtains output information of the subject's heart rate being measured by the heart rate measurement device 15. In this way, the heart rate measurement value acquisition unit 11 acquires the subject's heart rate measurement value. The "heart rate measurement value" is information on the measurement results of the subject's heart rate and its components. More specifically, the heart rate measurement value may be the heart rate data itself measured by the heart rate measurement device 15, or may be data extracted from the heart rate data. Examples of the latter include the RR-I value and HF value described below.

In step S102, the facial expression information acquisition unit 12 obtains output information of a front image of the face of the person being judged captured by the camera 16. In this way, the facial expression information acquisition unit 12 acquires information on the facial expression of the person being judged. "Facial expression information" is information that indicates the facial expression of the person being judged, and is, for example, information on an image of a main part or substantially the entirety of the person being judged.

In step S103, the relaxation level estimation unit 13 estimates the relaxation level of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit 11. The estimation of the relaxation level in step S103 may be performed by any method within the scope of methods that refer to the heart rate measurement value of the person being assessed. The method of estimating the relaxation level in step S103 may be a method in which the quantitatively expressed relaxation level is uniquely determined by the heart rate measurement value. For example, the relaxation level estimation unit 13 may use the heart rate of the person being assessed, which is substantially unchanged, as a reference value, and estimate a lower relaxation level as the heart rate increases, and estimate a higher relaxation level as the heart rate decreases. The estimation of the relaxation level using the heart rate measurement value will be described in more detail in a later embodiment.

In step S104, the relaxation degree correction unit 14 corrects the relaxation degree estimated in step S103 based on the facial expression information acquired in step S102. The control unit 1 then determines the relaxation degree corrected by the relaxation degree correction unit 14 as the relaxation degree of the person being assessed.

<First Example of Correcting Relaxation Level Based on Facial Expression Information>
Here, the correction of the relaxation level based on the facial expression information will be described. Fig. 3 is a flow chart showing the flow of a first example of the process of correcting the relaxation level based on the facial expression information in the first embodiment of the present invention.

In step S201, the relaxation level correction unit 14 determines the emotion of the person being assessed based on their facial expression.

Here, we will explain how to determine emotions from facial expressions. There are no limitations on the method for determining emotions from facial expressions, and any known method can be used. For example, emotions can be determined from facial expressions using "faceproc." "faceproc" is a facial expression API (application programming interface), and is a facial expression recognition API that uses artificial intelligence to infer basic emotions from facial images.

The types and number of emotions to be determined from facial expressions can be determined as appropriate within the range in which the degree of relaxation of the person being assessed can be determined. The emotions may be only emotions that indicate that the person being assessed is in a relaxed state, or may include both emotions that indicate that the person being assessed is in a relaxed state and emotions that indicate that the person being assessed is not relaxed. Examples of types of emotions used in determining facial expressions include the seven emotions of anger, disgust, fear, happiness, neutral or emotionless or indifferent (hereinafter also referred to as "neutral, etc."), sadness, and surprise, or in some cases eight emotions that further include contempt.

In step S202, the relaxation degree correction unit 14 obtains a correction value for the relaxation degree according to the result of the emotion judgment based on the facial expression of the person being judged. The result of the emotion judgment based on the facial expression and the correction of the relaxation degree are described below, and an example of obtaining the correction value is also described below.

Normally, when the person being assessed is relaxed, the parasympathetic nervous system becomes dominant and the person's heart rate slows down. In a relaxed state, the accuracy of judgment based on heart rate measurements tends to be higher than the accuracy of judgment based on facial expressions. On the other hand, when the person being assessed is irritated, the heart rate increases, but in everyday life, feelings of irritation strong enough to actually cause the heart rate to increase rarely occur. Therefore, feelings of irritation tend to be judged more accurately based on facial expressions than on heart rate measurements.

In this way, when judging the degree of relaxation of the person being assessed, for emotions for which the accuracy of judging emotions based on facial expressions tends to be higher, the judgment results based on heart rate measurements are actively corrected by the judgment results based on emotions based on facial expressions. This makes it possible to further increase the accuracy of judging the degree of relaxation.

When emotion assessment is performed for multiple emotions, the result can be expressed as a ratio of the emotion expressed by the facial expression. In this case, the total number of assessment values for each emotion will be 1. The highest assessment value among the multiple emotion assessment values will be the result of the emotion assessment based on the facial expression of the person being assessed. The quantification of emotions will be described later.

On the other hand, the types of emotions are classified into two types: facial expression information that is more strongly correlated with the level of relaxation, and facial expression information that is less strongly correlated with the level of relaxation. For example, the former is information on emotions that represent a state in which the person being assessed is relaxed (hereinafter also referred to as "positive emotions"), and the latter is information on emotions that represent a state in which the person being assessed is not relaxed (hereinafter also referred to as "negative emotions"). Emotions can be classified in this manner, for example, by classifying emotions that are more strongly related to a relaxed state as positive emotions, and other emotions as negative emotions. In the case of the eight emotions mentioned above, it is possible to classify the two emotions, happiness and neutral, as positive emotions, and the other emotions as negative emotions.

Then, set a correction value for positive emotion and a correction value for negative emotion. For example, set the correction value to +0.5 for positive emotion and -0.5 for negative emotion.

The result of the facial expression-based emotion determination is used to determine whether the emotion is positive or negative, and a correction value according to the determination result is selected, thereby obtaining a correction value for the degree of relaxation according to the result of the facial expression-based emotion determination.

The following describes an example of the process for correcting the degree of relaxation based on facial expression information. In step S203, the degree of relaxation is corrected by adding the correction value acquired in step S202 to the degree of relaxation based on the heart rate measurement value estimated in step S103. The corrected degree of relaxation is determined to be the degree of relaxation of the person being assessed.

For example, if the relaxation level based on the heart rate measurement value is 0.4 and the correction value for positive emotions is +0.5, the person's relaxation level is determined to be 0.9. Also, for example, if the upper limit of the relaxation level is 1 and the lower limit is -1, and the relaxation level after adding the correction value exceeds the upper limit, the corrected relaxation level may be set to the upper limit, i.e., 1, and if the relaxation level after adding the correction value falls below the lower limit, the corrected relaxation level may be set to the lower limit, i.e., -1.

The degree of relaxation can be notified to the person being assessed by the notification device 18. If the notification device 18 is a display device such as a liquid crystal display, the result of the assessment of the degree of relaxation can be notified to the person being assessed by an image, as shown below.

<First example of notification of judgment result>
FIG. 4 is a diagram showing an example of a screen showing a result of determining the degree of relaxation in accordance with a first example of correction of the degree of relaxation in the first embodiment of the present invention.

As shown in FIG. 4, screen 180 is the screen of notification device 18, which is a liquid crystal display device. Screen 180 is vertically divided into two sections according to the level of relaxation.

The character image 181 represents a state in which the person being assessed is relaxed, and is displayed when the degree of relaxation is greater than a predetermined value, i.e., when the person being assessed is determined to be in a relaxed state. The character image 181 is displayed, for example, according to the determined degree of relaxation. The character image 181 may be displayed, for example, at a position in the vertical direction of the screen 180 according to the degree of relaxation, or may be displayed at a predetermined position at the top of the screen 180, but may be displayed larger the higher the degree of relaxation.

Character image 182 represents a state in which the person being assessed is not relaxed, and is displayed when the degree of relaxation is less than a predetermined value. Like character image 181, character image 182 may be displayed at a position in the vertical direction of screen 180 that corresponds to the degree of relaxation, or may be displayed at a predetermined position on screen 180 but be displayed larger the lower the degree of relaxation.

In this way, in a first example of the correction of the degree of relaxation in this embodiment, the relaxation degree correction unit 14 corrects the degree of relaxation of the person being assessed to be higher in accordance with facial expression information of the person being assessed that is more strongly correlated with the level of relaxation, and corrects the degree of relaxation of the person being assessed to be lower in accordance with facial expression information that is less strongly correlated with the level of relaxation. Therefore, since facial expression information that is more strongly correlated with the level of relaxation is reflected in the result of the relaxation degree judgment, it is possible to judge the degree of relaxation with higher accuracy compared to judging the degree of relaxation based only on heart rate measurement values or only on facial expression information.

<Second Example of Correction of Relaxation Level Based on Facial Expression Information>
In this embodiment, the correction of the relaxation level based on the facial expression information may involve classifying the judgment result according to the type of emotion based on the facial expression of the person being judged. Fig. 5 is a flowchart showing the flow of a second example of the process of correcting the relaxation level based on the facial expression information in embodiment 1 of the present invention. The flowchart in Fig. 5 is the same as the flowchart shown in Fig. 4 except that step S301 is included between steps S201 and S202.

In step S301, the relaxation level correction unit 14 classifies the judgment result of the emotion determined by the facial expression in step S201 according to the type of the emotion. For example, in step S301, the relaxation level correction unit 14 classifies the judgment of a high degree of relaxation, described below, into active relaxation and calm relaxation.

A relaxed state typically includes a state in which one is enjoying something. In this case, the heart rate measurement increases, but there is essentially no feeling of irritation. Even for emotions such as this, which are more strongly correlated with a relaxed state, even though the heart rate increases, the accuracy of judgment based on facial expression tends to be higher than that based on heart rate measurement.

For example, talking with a like-minded friend or watching comedy videos or other content that makes you laugh can make you feel calm and refreshed. The previously mentioned state of relaxation in which the parasympathetic nervous system is dominant and the heart rate slows can be described as a calm relaxation that includes drowsiness, whereas the above-mentioned state of relaxation in which you are having fun can be described as active relaxation. Reflecting such states of relaxation that can be classified according to emotions in determining the degree of relaxation is also included in the embodiments of the present invention.

For example, of the two emotions, happiness and neutral, which are classified as positive emotions, happiness tends to be expressed as a clear smile on the face. Therefore, when the judgment value for a positive emotion is the judgment value for happiness, the state of high relaxation in this case can be classified as a state of active relaxation among high relaxation states. Note that, when the judgment value for a positive emotion is the judgment value for neutral, the state of high relaxation in this case can be classified as a state of calm relaxation among high relaxation states.

<Second example of notification of judgment result>
Fig. 6 is a diagram showing an example of a screen showing the result of determining the relaxation level according to a second example of the correction of the relaxation level in embodiment 1 of the present invention. In Fig. 6, the diagram on the left side of the page is an example of a screen when notifying the relaxation level of the above-mentioned sedative relaxation, and the diagram on the right side is an example of a screen when notifying the relaxation level of the above-mentioned active relaxation.

In this example, character image 181 represents a state of calm relaxation among the states in which the person being assessed is relaxed. In this example, character image 183 represents a state of active relaxation among the states in which the person being assessed is relaxed. Like character images 181 and 182, character image 183 may be displayed at a position in the vertical direction of screen 180 that corresponds to the degree of relaxation, or it may be displayed at a specified position at the top of screen 180 but with a relative size that corresponds to the numerical value of the degree of relaxation.

In this way, in the second example of the relaxation level correction in this embodiment, the relaxation level correction unit 14 further classifies the type of high relaxation level of the person being assessed according to the facial expression information of the person being assessed that is more strongly correlated with the level of relaxation level. Therefore, it becomes possible to more clearly grasp the high relaxation level of the person being assessed from the relaxation level assessment, and the accuracy of the relaxation level assessment is improved.

[Summary of the First Embodiment]
The control unit (1, relaxation level assessment device) according to this embodiment includes a heart rate measurement value acquisition unit (11) that acquires the heart rate measurement value of the person being assessed, a facial expression information acquisition unit (12) that acquires information on the facial expression of the person being assessed, a relaxation level estimation unit (13) that estimates the relaxation level of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit, and a relaxation level correction unit (14) that corrects the relaxation level estimated by the relaxation level estimation unit based on the facial expression information acquired by the facial expression information acquisition unit, and judges the relaxation level corrected by the relaxation level correction unit to be the relaxation level of the person being assessed.

The relaxation level assessment method according to this embodiment includes a heart rate measurement value acquisition step (step S101) for acquiring the heart rate measurement value of the person being assessed, a facial expression information acquisition step (step S102) for acquiring information on the facial expression of the person being assessed, a relaxation level estimation step (step S103) for estimating the relaxation level of the person being assessed by referring to the heart rate measurement value acquired in the heart rate measurement value acquisition step, and a relaxation level correction step (step S104) for correcting the relaxation level estimated in the relaxation level estimation step based on the facial expression information acquired in the facial expression information acquisition step, and the relaxation level corrected in the relaxation level correction step is judged to be the relaxation level of the person being assessed.

Therefore, according to this embodiment, it is possible to determine the degree of relaxation by appropriately using both the heart rate measurement value and facial expression information of the person being assessed. As a result, it is possible to realize a technology that can accurately determine the degree of relaxation from the biometric information and facial expression information of the person being assessed.

In this embodiment, the relaxation level correction unit corrects the facial expression information of the person being assessed to increase the relaxation level of the person being assessed in accordance with facial expression information that is more strongly correlated with the level of relaxation, and corrects the facial expression information of the person being assessed to decrease the relaxation level of the person being assessed in accordance with facial expression information that is less correlated with the level of relaxation. This configuration is even more effective in terms of improving the accuracy of the relaxation level assessment by appropriately reflecting facial expression information in the assessment of the relaxation level.

In addition, in this embodiment, the relaxation level correction unit further classifies the type of high relaxation level of the person being assessed according to the facial expression information of the person being assessed that is more strongly correlated with the level of relaxation level. This configuration is even more effective in terms of increasing the accuracy of the relaxation level assessment by appropriately classifying the type of relaxation level from the facial expression information.

[Embodiment 2]
Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

Fig. 7 is a block diagram showing a schematic functional configuration of a relaxation level assessment device according to a second embodiment of the present invention. As shown in Fig. 7, the control unit 2 has the same functional configuration as the control unit 1 in the first embodiment, except that it further includes a frequency component extraction unit 21 that extracts ultra-low frequency components and high frequency components of the heart rate fluctuation of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit 11.

[Example of relaxation level assessment]
The control unit 2 determines the relaxation level in the same manner as the control unit 1 of embodiment 1, except that the control unit 2 performs a process of estimating the relaxation level based on heart rate measurement values (step S103 in FIG. 2) as described below. The estimation of the relaxation level based on heart rate measurement values in this embodiment will be described below.

<First Example of Estimating Relaxation Level Based on Heart Rate Measurement>
FIG. 8 is a flowchart showing the flow of a first example of a process for estimating a relaxation level based on a heart rate measurement value according to the second embodiment of the present invention.

The frequency component extraction unit 21 judges the magnitude of the heart rate fluctuation of the subject by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit 11. For example, in step S401, the frequency component extraction unit 21 extracts the ultra-low frequency component (RR-I value) and high frequency component (HF value) of the heart rate fluctuation of the subject by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit 11.

The RR-I value is the length (in seconds) of the heartbeat interval of the person being assessed. The RR-I value of heartbeat fluctuation is the heartbeat fluctuation expressed in the band of 0.0033 to 0.04 Hz. The HF value is a value that represents the high-frequency components (0.15 to 0.4 Hz) in the heartbeat fluctuation of the person being assessed, and is the sum of the power values in the range of 0.15 to 0.4 (Hz) (the area of the corresponding range in the spectrum data). The RR-I value and HF value can be extracted from the heartbeat data using known techniques.

In step S402, the frequency component extraction unit 21 determines whether the acquired heart rate fluctuation is greater than or less than a threshold. The "threshold" here is a threshold used to stratify data into data with large heart rate fluctuation and data with small heart rate fluctuation from a moving average of the measurement results of multiple RR-I values, which will be described later.

The relaxation level estimation section 13 estimates the relaxation level of the person being assessed based on the RR-I value and the HF value extracted by the frequency component extraction section 21. For example, if the heartbeat fluctuation is equal to or greater than the threshold value (YES in step S402), the control section 2 reads out the following formula 1 from the storage section 17, and the relaxation level estimation section 13 calculates the relaxation level using the following formula 1 in step S403. In the following formulas 1 and 2, "D _relax " represents the relaxation level, "V _RR-I " represents the RR-I value, and "V _HF " represents the HF value. In addition, α _1, α ₂ , β ₁ , β _{2, and} γ ₁ are all positive or negative constants.
D _relax =α ₁ ×V _RR−I +β ₁ ×V _HF +γ ₁ (Formula 1)

If the heart rate fluctuation is less than the threshold value (NO in step S402), the control unit 2 reads out the following formula 2 from the memory unit 17, and the relaxation level estimation unit 13 calculates the relaxation level using the following formula 2 in step S404.
D _relax =α ₂ ×V _RR−I +β ₂ (Formula 2)

The above formulas 1 and 2 are obtained as follows. First, the moving average of the measurement results of multiple RR-I values is stratified into data with large heartbeat fluctuations and data with small heartbeat fluctuations. The moving average of the RR-I value can be obtained by calculating the moving average of the RR-I at an appropriate interval (for example, 20 seconds). Then, 0.04 seconds is set as a threshold value for the amplitude of the smoothed RR-I value per 160 seconds, and data whose amplitude per 160 seconds is appropriately extracted and exceeds the threshold value is classified as data with large heartbeat fluctuations, and data below the threshold value is classified as data with small heartbeat fluctuations. The above formulas 1 and 2 are derived by performing multiple regression analysis on each stratified data group. Alternatively, the above formulas 1 and 2 are derived by performing multiple regression analysis on each stratified data group using the difference from the reference value.

As mentioned above, when one is relaxed, the parasympathetic nervous system becomes dominant and the heart rate tends to slow down. In the first example of this embodiment, as described above, the degree of relaxation of the person being assessed is estimated by utilizing the correlation between the magnitude of the amplitude of the heart rate fluctuation of the person being assessed and the degree of relaxation. Therefore, the above first example omits frequency analysis in the calculation of HF, which is advantageous in further reducing the information processing load in the process of estimating the degree of relaxation and further increasing the accuracy of the estimation of the degree of relaxation.

<Second Example of Estimating Relaxation Level Based on Heart Rate Measurement>
In this embodiment, the estimation of a relaxation level based on heart rate measurement values may be performed as follows: Fig. 9 is a flowchart showing the flow of a second example of a process for estimating a relaxation level based on heart rate measurement values in embodiment 2 of the present invention. The second example differs from the first example described above in the process after the RR-I value and the HF value are acquired in step S401.

In step S501, the frequency component extraction unit 21 smoothes the RR-I value. For example, the frequency component extraction unit 21 calculates a moving average of the RR-I value at an appropriate interval (e.g., 20 seconds). By calculating the moving average at an appropriate time interval, the RR-I value of the person being assessed is appropriately and clearly represented.

In step S502, the relaxation level estimation unit 13 judges whether the RR-I value smoothed by the frequency component extraction unit 21 is greater than or less than a threshold value. For example, the relaxation level estimation unit 13 sets the threshold value to, for example, 0.04 seconds for the amplitude of the RR-I value per 160 seconds, appropriately extracts the amplitude of the RR-I value per 160 seconds, and judges whether the extracted amplitude exceeds the threshold value.

If the amplitude per unit time of the RR-I value is greater than the threshold value in step S502 (YES in step S502), then in step S503 the relaxation level estimation unit 13 judges the relaxation level of the person being assessed based on the HF value. More specifically, if the amplitude is greater than the threshold value, the relaxation level estimation unit 13 reads out an equation showing the correlation between the HF value and the relaxation level from the storage unit 17, and calculates the relaxation level of the person being assessed using this equation. Note that this "equation showing the correlation" can be found, for example, by the least squares method from multiple measurement results.

If the amplitude per unit time of the RR-I value is equal to or less than the threshold value in step S502 (NO in step S502), in step S504, the relaxation level estimation unit 13 judges the relaxation level of the person being assessed based on the RR-I value. More specifically, when the amplitude is equal to or less than the threshold value, the relaxation level estimation unit 13 reads out from the storage unit 17 an equation indicating the correlation between the RR-I value and the relaxation level, and calculates the relaxation level of the person being assessed using the equation.

Thus, in the second example, the frequency component extraction unit 21 extracts the RR-I value of the heart rate fluctuation of the person being assessed based on the moving average of the heart rate fluctuation of the person being assessed. Then, using this as an index, the relaxation level estimation unit 13 estimates the relaxation level of the person being assessed based on the RR-I value and HF value extracted by the frequency component extraction unit 21.

In the second example above, the type of heart rate measurement value that has a higher correlation with the subject's degree of relaxation is determined from the RR-I value and HF value of the subject depending on the magnitude of the amplitude of the subject's heart rate fluctuation. The subject's degree of relaxation is then estimated using the correlation between the determined type of heart rate measurement value and the degree of relaxation. Thus, like the first example described above, the second example above is advantageous in further reducing the information processing load in the relaxation level estimation process and further increasing the accuracy of the relaxation level estimation.

[Summary of the second embodiment]
The control unit (2) of this embodiment further includes a frequency component extraction unit (21) that extracts ultra-low frequency components and high frequency components of the heart rate fluctuations of the subject by referring to the heart rate measurement values acquired by the heart rate measurement acquisition unit.The relaxation level estimation unit estimates the relaxation level of the subject based on the ultra-low frequency components and high frequency components extracted by the frequency component extraction unit.Such a configuration is more effective from the viewpoints of reducing the information processing load in the relaxation level estimation process and increasing the accuracy of the relaxation level estimation.

In addition, in this embodiment, the frequency component extraction unit may determine the magnitude of the heart rate fluctuation of the person being assessed by referring to the heart rate measurement value acquired by the heart rate measurement value acquisition unit. In this case, the relaxation level estimation unit may estimate the relaxation level of the person being assessed based on the result of the heart rate fluctuation determination by the frequency component extraction unit, based on the ultra-low frequency components and high frequency components when the heart rate fluctuation is large, and may estimate the relaxation level of the person being assessed based on the ultra-low frequency components when the heart rate fluctuation is small. Such a configuration is even more effective in terms of reducing the information processing load in the relaxation level estimation process and increasing the accuracy of the relaxation level estimation.

In addition, in this embodiment, the frequency component extraction unit may extract ultra-low frequency components of the heart rate fluctuation of the person being assessed based on a moving average of the heart rate fluctuation of the person being assessed. Such a configuration can clarify the characteristics of the heart rate fluctuation of the person being assessed, and is even more effective in terms of improving the accuracy of estimating the degree of relaxation.

[Embodiment 3]
Another embodiment of the present invention will be described below. For ease of explanation, the same reference numerals are used for members having the same functions as those described in the above embodiment, and the description thereof will not be repeated. In this embodiment, the technology for determining the relaxation level of the present invention is applied to a passenger in a vehicle, which is a moving object.

[Functional configuration]
Fig. 10 is a block diagram showing a schematic functional configuration of a relaxation degree assessment device according to embodiment 3 of the present invention. As shown in Fig. 10, the control unit 3 has the same functional configuration as the control unit 1 of embodiment 1 described above, except that it further includes a reference data acquisition unit 31 that acquires reference data that is a reference for the heart rate measurement value of the person to be assessed, and an external factor information acquisition unit 32 that acquires external factor information. The external factor information is information that includes at least one of environmental information that indicates the environment in which the person to be assessed exists and date and time information.

The control unit 4 is further connected to an on-board sensor 33 and an input device 34. The on-board sensor 33 is a sensor mounted on the vehicle to control the operation of the vehicle and to detect the environment inside and outside the vehicle. Examples of the on-board sensor 33 include a temperature and humidity sensor, a seat pressure sensor, and a seat reclining angle sensor.

The input device 34 is a device for inputting information to the control unit 3 from the subject or from outside the control unit 3, such as the Internet. Examples of the input device 34 include a touch panel and a voice input device, and a receiving device for Internet communication.

Figure 11 is a diagram showing a schematic example of the arrangement of devices around a subject 100 in embodiment 3 of the present invention. As shown in Figure 11, the subject 100 is a passenger in a moving body and is seated in a seat 35 of the vehicle.

The heart rate measuring device 15 is a heart rate sensor mounted on the vehicle, and is, for example, wrapped around the left wrist of the subject 100. The heart rate measuring device 15 measures the heart rate of the subject 100, and transmits the acquired heart rate information, that is, the heart rate measurement value, to the control unit 4 via wired or wireless communication.

The camera 16 is a camera mounted on the vehicle (on-board camera) and is positioned, for example, in a position to capture the face of the person being assessed 100 from a position slightly above and in front of the person being assessed. The camera 16 continuously captures images of the facial expressions of the person being assessed 100 and transmits facial expression information, which is information about the acquired facial images, to the control unit 4 via wired or wireless communication.

[Example of relaxation level assessment]
FIG. 12 is a flowchart showing an example of a process for determining a relaxation degree performed by the device for determining a relaxation degree according to the third embodiment of the present invention.

In step S601, the heart rate measurement value acquisition unit 11 starts acquiring the heart rate measurement value from the heart rate measurement device 15. In this manner, in this embodiment, the heart rate measurement value acquisition unit 11 acquires the heart rate measurement value of the subject from a heart rate sensor mounted on the moving body.

In step S602, the facial expression information acquisition unit 12 starts acquiring image data of facial expressions from the camera 16. In this manner, in this embodiment, the facial expression information acquisition unit 12 acquires facial expression information of the person being determined from the camera 16 mounted on the moving body.

In step S603, the external factor information acquisition unit 32 acquires external factor information from the in-vehicle sensor 33 and the input device 34. For example, the external factor information acquisition unit 32 acquires information on the temperature outside the vehicle, the temperature and humidity inside the vehicle, and the seat reclining angle from the in-vehicle sensor 33, and acquires information on the date and time that the person to be determined is riding in the moving object from the input device 34.

In step S604, the control unit 3 determines whether the heart rate of the person being assessed is stable by referring to the heart rate measurement value acquired by the heart rate measurement acquisition unit 11. For example, the control unit 3 determines that the heart rate is stable when the fluctuation in the heart rate at a specific time is equal to or less than a threshold value, and determines that the heart rate is not stable when the fluctuation in the heart rate at a specific time exceeds the threshold value.

If the control unit 3 determines in step S604 that the heart rate is stable (YES in step S604), the reference data acquisition unit 31 acquires reference data for the heart rate measurement value in step S605. For example, the reference data acquisition unit 31 acquires the heart rate measurement value for a specific period of time (e.g., 2 minutes) acquired by the heart rate measurement value acquisition unit 11 as basic data.

On the other hand, if the control unit 3 determines in step S604 that the heart rate has not stabilized, in step S606, the control unit 3 waits for a predetermined time (e.g., one minute) and returns to step S604.

In step S607, the relaxation level estimation unit 13 executes a process of estimating a relaxation level using the reference data acquired by the reference data acquisition unit 31 as a heart rate measurement value. The relaxation level estimation process is performed in the same manner as the process in step S103 described above, except that the basic data is used as the heart rate measurement value. In this manner, in this embodiment, the relaxation level estimation unit 13 estimates the relaxation level of the person being assessed by referring to the reference data acquired by the reference data acquisition unit 31.

In addition, in estimating the degree of relaxation in step S607, the relaxation degree estimation unit 13 estimates the degree of relaxation of the person being assessed by referring to the external factor information acquired by the external factor information acquisition unit 32. For example, the relaxation degree estimation unit 13 calculates the degree of relaxation by adding a constant to the formula read out from the storage unit 17 described above or by changing a coefficient, depending on the external factor information acquired by the external factor information acquisition unit 32.

Next, the relaxation degree correction unit 14 corrects the relaxation degree estimated in step S607 based on the facial expression information. This process of correcting the relaxation degree based on the facial expression information is performed in the same manner as the process in step S104 described above.

In this embodiment, reference data for the person being assessed is obtained when estimating the degree of relaxation, making it possible to more accurately assess the degree of relaxation of the person being assessed who is in a moving body such as a vehicle. When the person being assessed is a driver, it is possible to more accurately assess the degree of relaxation of the person being assessed who is driving a car, which is advantageous from the perspective of realizing a stable driving state of the car.

In addition, in this embodiment, external factor information is acquired when estimating the degree of relaxation of the person being assessed. This is advantageous in that the estimation of the degree of relaxation can appropriately reflect the influence of external factors acting on the person being assessed on the degree of relaxation of the person being assessed.

[Summary of the Third Embodiment]
The control unit (3) of this embodiment further includes a reference data acquisition unit (31) that acquires reference data that is a standard for the heart rate measurement value of the person being assessed. The relaxation level estimation unit estimates the relaxation level of the person being assessed by further referring to the reference data acquired by the reference data acquisition unit. This configuration is even more effective in terms of accurately grasping the state of the heart rate measurement value of the person being assessed when estimating the relaxation level and improving the accuracy of the estimation of the relaxation level.

In this embodiment, the control unit further includes an external factor information acquisition unit (32) that acquires external factor information including at least one of environmental information indicating the environment in which the person to be assessed is present and date and time information. The relaxation level estimation unit further references the external factor information acquired by the external factor information acquisition unit to estimate the relaxation level of the person to be assessed. This configuration is even more effective in terms of improving the accuracy of the estimation of the relaxation level, since it allows the influence of external factors to be appropriately reflected in the estimation of the relaxation level of the person to be assessed.

In this embodiment, the person to be assessed is a passenger in the moving body. The heart rate measurement value acquisition unit acquires the heart rate measurement value of the person to be assessed from a heart rate sensor (heart rate measurement device 15) mounted on the moving body, and the facial expression information acquisition unit acquires facial expression information of the person to be assessed from a camera (16) mounted on the moving body. This configuration is even more effective from the perspective of more easily judging the relaxation level of the person to be assessed in the moving body, and from the perspective of constantly utilizing the judgment result of the relaxation level of the person to be assessed in the moving body in the operation of the moving body.

[Embodiment 4]
Another embodiment of the present invention will be described below. For ease of explanation, the same reference numerals are used for components having the same functions as those described in the above embodiment, and the description thereof will not be repeated. In this embodiment, the technology of the present invention for inducing a subject to a relaxed state is applied to a passenger in a moving body, such as an automobile.

[Functional configuration]
Fig. 13 is a block diagram showing a schematic functional configuration of a relaxation induction device according to embodiment 4 of the present invention. As shown in Fig. 13, the relaxation induction device of this embodiment includes a control unit 4. The control unit 4 is configured similarly to the control unit 3 of embodiment 3 described above, except that it further includes a stimulus determination unit 41. The stimulus determination unit 41 determines a stimulus according to the relaxation degree corrected by the relaxation degree correction unit 14.

The control unit 4 is further connected to a stimulus providing device 42. The stimulus providing device 42 is a device that provides the stimulus determined by the stimulus determination unit 41 to the subject. The stimulus providing device 42 may be a standalone device, or may be any of a variety of in-vehicle devices. Examples of stimuli provided by the stimulus providing device 42 include light, sound, vibration, heat, humidity, and smell, and the stimulus providing device 42 may be any device that is capable of appropriately controlling and presenting such stimuli.

For example, a stimulus providing device 42 that presents light may be a device that can change the brightness or color of light presented in a vehicle, and examples of such devices include interior lights, displays, and control panels. An example of a stimulus providing device 42 that presents sound includes car audio. An example of a stimulus providing device 42 that presents vibration includes a seat with a massage function. An example of a stimulus providing device 42 that presents heat includes an air conditioner and a seat with a heating function. An example of a stimulus providing device 42 that presents humidity includes an air conditioner. An example of a stimulus providing device 42 that presents an odor includes a deodorizing device or a fragrance generating device that can be attached to a cigarette lighter socket.

[Example of relaxation induction processing]
Fig. 14 is a flow chart showing an example of the flow from the relaxation degree determination process to the relaxation induction process by the relaxation induction device according to embodiment 4 of the present invention. The relaxation induction process is executed in the same manner as the relaxation degree determination process in embodiment 3 shown in Fig. 12, except that it includes a process of providing a stimulus based on the result of the relaxation degree determination. In the relaxation induction process in this embodiment, as shown in Fig. 14, in step S701, the control unit 4 executes a process of providing the subject with a stimulus based on the result of the relaxation degree determination determined by the correction process of the relaxation degree based on facial expression in step S104.

Figure 15 is a flowchart showing an example of the flow of relaxation induction processing in embodiment 4 of the present invention.

In step S801, the stimulus determination unit 41 selects a stimulus to be provided to the subject based on the result of the assessment of the subject's degree of relaxation. For example, the stimulus determination unit 41 refers to a response model stored in the memory unit 17 to select a stimulus to be provided to the subject. The response model is a model that specifies the relationship between the subject's degree of relaxation and the content and intensity of the stimulus to be provided to the subject. Examples of response models include model formulas. The model formulas specify at least the relationship between the subject's degree of relaxation and the content and intensity of the stimulus to be provided. By referring to such a reference model, the stimulus determination unit 41 determines the content and intensity of the stimulus to be provided to the subject.

In step S802, the stimulus providing device 42 provides the stimulus selected by the stimulus determining unit 41 to the subject in accordance with the information of the stimulus selected by the stimulus determining unit 41.

After providing the stimulus, the control unit 4 performs a determination of the degree of relaxation of the subject to whom the stimulus was provided. This determination of the degree of relaxation is performed in the same manner as steps S103 and S104 of FIG. 2 described above, as shown in FIG. 15.

In step S803, the control unit 4 generates notification information of the result of the relaxation degree judgment and transmits it to the notification device 18. The notification device 18 notifies the person being judged of the notification information as sound or an image.

In step S804, the control unit 4 inquires of the person being assessed as to whether or not to end the relaxation induction process. This inquiry is made by sending inquiry information to the notification device 18, and is made by the notification device 18 using audio or images. When the person being assessed inputs information to the effect that the person is terminating the process via the input device 34 (in the case of YES), the control unit 4 terminates the relaxation induction process. When the person being assessed inputs information to the effect that the person is not terminating the relaxation induction process (in the case of NO), the control unit 4 returns to step S801 and executes the above-described process for relaxation induction again.

In this embodiment, it is possible to provide the subject with both a refreshing active relaxation state and a drowsy calm relaxation state. Therefore, during autonomous driving, it is possible to guide the subject, who is a passenger in a moving body, to a calm relaxation state on the way, and to an active relaxation state near the destination. This makes it possible to guide the subject into a state where he or she can be active as soon as he or she arrives at the destination.

In addition, in this embodiment, it is possible to induce the subject to be assessed into an active or calm relaxed state periodically over a long period of time, thereby reducing the stress of the subject traveling on a mobile object and improving the subject's mental health.

Furthermore, if only sedative relaxation is provided, the parasympathetic nervous system will always be dominant. If the parasympathetic nervous system is dominant and overactive, the body may feel heavy and lethargic, or abnormally sleepy. By providing both sedative relaxation and active relaxation, the autonomic nervous system can be kept in a good state.

[Summary of the fourth embodiment]
The control unit (4, relaxation induction device) of this embodiment includes, in addition to the heart rate measurement value acquisition unit, facial expression information acquisition unit, relaxation level estimation unit, and relaxation level correction unit described above, a stimulus provision unit (21) that provides the subject with a stimulus according to the relaxation level corrected by the relaxation level correction unit. Thus, the relaxation level of the subject can be accurately determined from the biometric information and facial expression information of the subject, and further, by providing the subject with a stimulus determined with reference to the relaxation level, the subject can be induced to a more desirable relaxed state.

[Modifications]
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in the different embodiments are also included in the technical scope of the present invention.

In the first to fourth embodiments of the present invention, the facial expression information may be information on an image of a part of the face of the person being judged. For example, the facial expression information may be information on an image of key parts of the person being judged, such as the eyes and mouth.

In addition, in the first to fourth embodiments of the present invention, the correction value in correcting the relaxation level based on facial expression information may be different for each type of emotion. Furthermore, the absolute value of the correction value may be larger for emotions that are more easily expressed by facial expressions than by measured heart rate values, so that the result of the judgment based on facial expressions is more strongly reflected in the judgment of the person's relaxation level.

In addition, in the first to fourth embodiments of the present invention, other information than emotion-based information may be used as facial expression information in the relaxation level determination process. For example, "eyes closed" may be added as facial expression information, and the "eyes closed" information may be referenced in the relaxation level determination process. As an example of the reference of such information, when the measurement result of the "eyes closed" information (e.g., a percentage or number of consecutive occurrences) exceeds a specific threshold, the control unit may end the relaxation determination process and provide a notification or a stimulus to wake up the person being determined.

In addition, in the "second example of correction of relaxation level based on facial expression information" in the above-mentioned embodiment 1, the relaxation level in the above-mentioned active relaxation state may be lower than the relaxation level in the sedative relaxation state, when the relaxation level is estimated based on the heart rate measurement value. Therefore, when classified as an active relaxation state, the correction value may be set to a larger value than when classified as a sedative relaxation state. In this way, setting the correction value appropriately depending on the type of relaxation level is also included in preferred embodiments of the present invention.

In addition, in the "first example of estimating a degree of relaxation based on heart rate measurement values" in the above-mentioned embodiment 2, the RR-I value may be smoothed in the same manner as in the "second example of estimating a degree of relaxation based on heart rate measurement values" in the same embodiment. In this case, in the first example as well, the characteristics of the subject's heart rate fluctuations can be made clearer, as in the second example.

In addition, in the above-mentioned fourth embodiment, when the result of the relaxation level assessment is notified to the person being assessed by the notification device 18, the change in the relaxation level after the provision of the stimulus relative to the relaxation level before the provision of the stimulus may also be notified to the person being assessed.

In the above-mentioned fourth embodiment, the stimulus determination unit may determine the stimulus based on input of stimulus information (content, intensity, etc.) from the subject.

In addition, in the above-mentioned fourth embodiment, the control unit may select a response model based on one or more factors indicated by the external factor information. By selecting a response model based on the factors, a response model that is appropriately weighted by the factors is selected. Therefore, it is possible to guide the subject to a relaxed state that appropriately reflects the external factor information.

In addition, in the above-mentioned fourth embodiment, the control unit may further include a reaction model update unit that updates the reaction model according to the result of the assessment of the degree of relaxation of the person being assessed and the stimulus provided to the person being assessed at that time. Such a configuration is more advantageous from the viewpoint of increasing the accuracy in guiding the person being assessed to a desired relaxed state in the relaxation induction process.

[Software implementation example]
The control blocks (control units 1 to 4) of the relaxation level assessment device or relaxation induction device according to the embodiments of the present invention may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the relaxation level assessment device or relaxation induction device is equipped with a computer that executes the instructions of a program, which is software that realizes each function. This computer is equipped with, for example, one or more processors, and a computer-readable recording medium that stores the program. The object of the present invention is achieved by having the processor in the computer read and execute the program from the recording medium.

The processor may be, for example, a CPU (Central Processing Unit). The recording medium may be a "non-transitory tangible medium" such as a ROM (Read Only Memory), as well as a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The device may further include a RAM (Random Access Memory) for expanding the program.

The program may be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). One aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[Experimental example for acquiring information on the subject's emotions]
In relation to the first to fourth embodiments of the present invention, an experiment for acquiring information on the emotion of the person to be assessed, which is used in the first to fourth embodiments of the present invention, will be described below. The emotion information of the person to be assessed can be acquired by the following experiment. Note that the "subject" in the following description may or may not be the person to be assessed.

In the first step, the subject is allowed to rest for three minutes and the first psychological evaluation is performed. Then, in the second step, the subject's emotions are controlled to a specific emotion for three minutes and the second psychological evaluation is performed. The emotion is controlled by providing the subject with a specific stimulus, or by controlling the subject to reach a specific emotion on his or her own. In emotion control, for example, the subject is controlled to a neutral emotional state or an irritated state. Then, in the third step, a specific stimulus is provided to the subject for ten minutes and the third psychological evaluation is performed. The stimulus provided to the subject is, for example, thermal vibration, hot and cold, or a comedy video.

The subject's heart rate is measured throughout each step: the initial three minutes of rest, followed by three minutes of emotional control, and then the subsequent ten minutes of stimulation.

When measuring the degree of relaxation using heart rate measurements, the degree of relaxation is calculated based on the heart rate measurements for the last two minutes of each step, for example using the above-mentioned formulas 1 and 2.

To measure emotions based on facial expressions, the subject's facial expressions are recorded on video at a frame rate of 30 fps for the last two minutes of each step, and the emotion estimation API obtains a measurement value of the subject's emotions based on facial expressions based on the image data of each frame of the captured facial images. The average value of the measurement value of each emotion for every seven frames over the two minutes of recording is calculated. The highest average value is then used as the measurement result of the emotion based on facial expressions for that step.

In the psychological evaluation after the completion of each step, the subject's emotions are quantitatively evaluated. The psychological evaluation can be performed using the Two-Dimensional Mood Scale (TDMS). TDMS is a test that quantifies the subject's psychological state through self-monitoring of the psychological state.

At each step, the degree of relaxation is measured using heart rate measurements, or emotions are measured using facial expressions, or both are performed. This makes it possible to obtain individual results for measuring the degree of relaxation using heart rate measurements, measuring emotions using facial expressions, and judging the degree of relaxation based on these.

[Reference Experimental Example]
Among the items of the psychological evaluation in the third step, the score of the ratio of positive emotions minus the score of the ratio of negative emotions was corrected between -1 and 1 for each individual, and was determined as the first result (actual value) of the relaxation level in the experiment. The relaxation level calculated from the heart rate measurement value in the third step using the above-mentioned formula 1 or formula 2 was determined as the second result (estimated result of the relaxation level based on the heart rate measurement value). Furthermore, the above-mentioned correction value was calculated from the result of the emotion estimation API based on the facial expression image in the third step, and the above-mentioned measurement result of the relaxation level based on the heart rate measurement value was corrected to obtain the third result (determined result of the relaxation level). The tolerance for the first result was set to ±0.25, and if the second result and the third result were within the above-mentioned tolerance range, they were determined as correct, and if they were outside the above-mentioned tolerance range, they were determined as incorrect.

In 24 experiments, the accuracy rate of the second result compared to the first result was 62.5% (15 correct answers). Of the nine incorrect answers, six were results that estimated that the subject was in an irritated state. On the other hand, the accuracy rate of the third result compared to the first result was 83.3% (20 correct answers). In this way, the accuracy of judging the relaxation level was improved by correcting the estimation result of the relaxation level based on the heart rate measurement value based on facial expression information.

1 to 4 Control unit 11 Heart rate measurement value acquisition unit 12 Facial expression information acquisition unit 13 Relaxation level estimation unit 14 Relaxation level correction unit 15 Heart rate measurement device 16 Camera 17 Memory unit 18 Notification device 21 Frequency component extraction unit 31 Reference data acquisition unit 32 External factor information acquisition unit 33 Vehicle-mounted sensor 34 Input device 35 Reclining seat 41 Stimulus determination unit 42 Stimulus provision device 100 Subject 180 Screen 181 to 183 Character images

Claims (9)

a heart rate measurement value acquisition unit for acquiring a heart rate measurement value of a subject;
A facial expression information acquisition unit that acquires information on the facial expression of the person to be determined;
a frequency component extraction unit that extracts ultra-low frequency components and high frequency components of the heartbeat fluctuation of the subject by referring to the heartbeat measurement value acquired by the heartbeat measurement value acquisition unit, and judges the magnitude of the heartbeat fluctuation of the subject;
Based on the result of the determination of heartbeat fluctuation by the frequency component extraction unit,
When the heart rate fluctuation is large, a relaxation level of the person to be determined is estimated based on the extremely low frequency component and the high frequency component;
a relaxation level estimation unit that estimates a relaxation level of the subject based on the ultra-low frequency component when the heart rate fluctuation is small;
a relaxation level correction unit that corrects the relaxation level estimated by the relaxation level estimation unit based on the facial expression information acquired by the facial expression information acquisition unit,
The relaxation degree determination device determines the relaxation degree of the person to be determined as the relaxation degree corrected by the relaxation degree correction unit. The relaxation degree correction unit is configured to:
correcting the degree of relaxation of the person to be determined to be higher in accordance with facial expression information that is more strongly correlated with the degree of relaxation;
correcting the degree of relaxation of the person to be determined to be lower in accordance with facial expression information that is less correlated with the level of relaxation;
The relaxation level determination device according to claim 1 . The relaxation level assessment device according to claim 1 or 2, wherein the relaxation level correction unit further classifies the type of the highly relaxed state of the person being assessed according to facial expression information that is more strongly correlated with the level of relaxation among the facial expression information of the person being assessed. The relaxation level assessment device according to claim 1, wherein the frequency component extraction unit extracts ultra-low frequency components of the subject's heart rate fluctuations based on a moving average of the subject's heart rate fluctuations. A reference data acquisition unit that acquires reference data that is a reference for the heart rate measurement value of the subject,
the relaxation level estimation unit estimates a relaxation level of the person to be determined by referring to the reference data acquired by the reference data acquisition unit;
The relaxation degree determination device according to any one of claims 1 to 4. An external factor information acquisition unit that acquires external factor information including at least one of environmental information indicating an environment in which the person to be determined exists and date and time information,
the relaxation level estimation unit estimates a relaxation level of the person to be determined by referring to the external factor information acquired by the external factor information acquisition unit;
The relaxation degree determination device according to any one of claims 1 to 5. The subject is a passenger of a moving body,
the heartbeat measurement value acquisition unit acquires a heartbeat measurement value of the subject from a heartbeat sensor mounted on a moving object;
7. The relaxation level determination device according to claim 1, wherein the facial expression information acquisition unit acquires the facial expression information of the person to be determined from a camera mounted on a moving object. A relaxation level determination method executed by a relaxation level determination device having a processor,
The processor performs the following processes (1) to (6):
(1) A process of acquiring a heart rate measurement value of a subject;
(2) acquiring information on the facial expression of the person to be determined;
(3) referring to the heart rate measurement value, extracting ultra-low frequency components and high frequency components of the heart rate fluctuation of the subject, and determining the magnitude of the heart rate fluctuation of the subject;
(4) Based on the result of the determination of the heartbeat fluctuation,
When the heart rate fluctuation is large, a relaxation level of the person to be determined is estimated based on the extremely low frequency component and the high frequency component;
When the heart rate fluctuation is small, a process of estimating a relaxation level of the person to be determined based on the ultra-low frequency component;
(5) a process of correcting the estimated relaxation level based on the facial expression information;
(6) executing a process of determining the corrected relaxation degree as the relaxation degree of the person to be determined;
A method for assessing relaxation level. a heart rate measurement value acquisition unit for acquiring a heart rate measurement value of a subject;
A facial expression information acquisition unit that acquires information on the facial expression of the person to be determined;
a frequency component extraction unit that extracts ultra-low frequency components and high frequency components of the heartbeat fluctuation of the subject by referring to the heartbeat measurement value acquired by the heartbeat measurement value acquisition unit, and judges the magnitude of the heartbeat fluctuation of the subject;
Based on the result of the determination of heartbeat fluctuation by the frequency component extraction unit,
When the heart rate fluctuation is large, a relaxation level of the person to be determined is estimated based on the extremely low frequency component and the high frequency component;
a relaxation level estimation unit that estimates a relaxation level of the subject based on the ultra-low frequency component when the heart rate fluctuation is small;
a relaxation level correction unit that corrects the relaxation level estimated by the relaxation level estimation unit based on the facial expression information acquired by the facial expression information acquisition unit;
a stimulus providing unit that provides the subject with a stimulus according to the degree of relaxation corrected by the relaxation degree correcting unit;
A relaxation induction device comprising:

Images