JP6509778B2 - Learning device, estimation device, methods thereof and program - Google Patents

Description

  The present invention relates to a technique for estimating information according to at least one of a heart beat and a pulse.

  There is a technique of estimating a target variable related to a heartbeat using information on a past heartbeat as an explanatory variable (see, for example, Non-Patent Document 1).

Yokoyama Kyoko, Mokichi Masanori, Watanabe Yasusaku, Yoshioka Kiyoshi, Takada Kazuyuki, "Analysis of heart rate variability by impulse response estimated from autoregressive model of RR interval time series", December 16, 1994, Electronic Corporation IEICE technical report, IEICE technical report, MBE, ME and Bio Cybernetics 94 (416), 1-6.

  However, in the prior art, only the information related to the heartbeat is used, and there is a problem that the estimation accuracy is low when there is a rapid heartbeat fluctuation based on various factors. Such a problem is common not only to estimating the information on the heartbeat but also to estimating the information on the pulse.

  An object of the present invention is to estimate information on at least one of a heart beat and a pulse with high accuracy even if there is a sudden heart beat fluctuation or pulse fluctuation due to various factors.

  The feature of at least one of the heartbeat and the pulse obtained in the first time interval and the feature of the heartbeat and the pulse using the feature for learning based on the feature of at least one of the heartbeat and the pulse and the physiological feature other than the feature of the heartbeat and the pulse A prediction model for obtaining estimation information according to at least one of the heartbeat and the pulse in the second time interval is obtained from input feature quantities based on other physiological characteristics.

  Because it uses not only heart rate and / or pulse characteristics but also physiological characteristics other than heart and pulse characteristics, heart beats with high accuracy even in the case of rapid heart rate fluctuations and heart rate fluctuations based on various factors. It is possible to estimate information on at least one of:

FIG. 1 is a block diagram for explaining a system configuration of the embodiment. FIG. 2 is a flowchart for explaining the process of the embodiment. FIG. 3A is a diagram for explaining the process of the embodiment. FIG. 3B is a diagram illustrating the relationship between the viewing angle and the heartbeat. FIG. 4 is a figure for demonstrating the feature-value of a micro saccade. FIG. 5 is a diagram illustrating changes in pupil diameter of both eyes. FIG. 6A is a block diagram for illustrating the configuration of the feature quantity extraction unit of the second embodiment. FIG. 6B is a flowchart for illustrating the process of the feature quantity extraction unit of the second embodiment.

Hereinafter, embodiments of the present invention will be described.
[principle]
First, the principle of this embodiment will be described. The “learning process” of each embodiment uses “features for learning” based on at least one of heart rate and pulse characteristics and physiological features other than heart rate and pulse characteristics, and is obtained in the “first time interval”. In order to obtain estimation information according to at least one of the heartbeat and the pulse in the "second time interval" from the "input feature" based on the feature of at least one of the heartbeat and the pulse and physiological features other than the features of the heartbeat and the pulse Get a "prediction model". In the "estimation process" of each embodiment, the "input feature amount" based on a feature of at least one of the heartbeat and the pulse obtained in the "first time interval" and a physiological feature other than the feature of the heartbeat and the pulse is To obtain estimated information according to at least one of the heartbeat and the pulse in the “second time interval”. Thus, various factors (a sudden psychological condition caused by the external or internal of the individual) are used because not only the characteristics of the heart rate and the pulse but also physiological characteristics other than the heart and pulse characteristics are used. Even in the case of rapid heart rate fluctuations or pulse rate fluctuations based on fluctuations (for example, surprise, sadness, fear, etc.), it is possible to estimate the information on the heart rate and / or the pulse with high accuracy.

  The "first time interval" may be a time interval after (future) the "second time interval" or may be a time interval before (the past) before the "second time interval" And may be a time interval included in the “second time interval”. The “first time interval” may be a time interval immediately after or immediately before the “second time interval”, or may be a time interval apart from the “second time interval”.

  The “heart beat and / or pulse feature” may be a feature of heart beat only, may be a feature of pulse only, or may be a feature of heart beat and pulse. Examples of the “characteristics of heartbeat” are heartbeat interval, occurrence time and occurrence time of heartbeat, heart rate per predetermined time, amplitude of heartbeat waveform, RR interval, PQ time, QRS width, QT time and the like. Similarly, examples of the "characteristics of the pulse" are a pulse interval, a pulse generation time and a pulse generation time, a pulse rate per predetermined time, an amplitude of a pulse waveform, and the like. The "physiological characteristics other than heart and pulse characteristics" may be anything. For example, this "physiological feature" may include "feature of dynamic change of eye", may include "feature of respiratory movement", or may include "feature of movement appearing on the surface of living body". It may be good, may include "muscle activity features", may include "body temperature features", may include "perspiration features", may include "electroencephalogram features", etc. It may also include biological features.

  "The characteristics of at least one of heart rate and pulse and physiological characteristics other than the characteristics of heart rate and pulse" are determined according to the occurrence time of "dynamic change of the eye" and at least one of the occurrence time of heart beat and pulse. It may include features based on relative values of the corresponding values. For example, “at least one of heart rate and pulse characteristics and physiological characteristics other than heart rate and pulse characteristics” are the “first interval” and “eye movement” from the occurrence of heart beat to the occurrence of “dynamic change of eye”. Characteristics in response to the "second interval" from the occurrence of the "typical change" to the occurrence of the next heartbeat, and "the third interval" between the occurrence of the pulse and the "dynamic change of the eye" At least one of the features may be included according to the “fourth interval” from the occurrence of “dynamic change of the eye” to the occurrence of the next pulse. The example of the feature according to the "first interval" and the "second interval" is a relative value of the "first interval" and the "second interval", and for the "third interval" and the "fourth interval" An example of the corresponding feature is a relative value between the "third interval" and the "fourth interval". The “value according to α” may be α itself or a function value of α. The “relative value between α and β” is, for example, the difference between α and β, the value obtained by subtracting β from α, the value obtained by subtracting α from β, the value obtained by dividing α by β, and dividing β by α Value, difference between function value of α and function value of β, value obtained by subtracting function value of β from function value of α, value obtained by subtracting function value of α from function value of β, α divided by (α + β) Or the value β / (α + β) obtained by dividing β by (α + β), or any function value thereof.

  "Physical features" are estimated from the features of "dynamic change of eye" in "third time interval" and the feature of "dynamic change of eye" in "fourth time interval" The “feature according to the degree of difference” may be included in the “three-hour interval” feature. For example, the feature quantity obtained based on the "dynamic change of the eye" in the "third time interval" and the "dynamic change of the eye" in the "fourth time interval" The feature quantity may be represented according to the degree of difference between the feature quantity in the “third time interval” estimated from the time-series information of the feature quantity obtained on the basis of. The “feature according to the degree of difference” may be a larger value as the “difference” is larger, or may be a smaller value as the “difference” is larger, and other “difference” function values It may be. "Difference" may be rephrased as "distance". The case where "features of the eye's dynamic change in the third time interval" and "features in the third time interval estimated from the features of the eye's dynamic change in the fourth time interval" are expressed by scalars The "differences" of these "features" can be represented by differences or distances of values representing those "features". The case where “features of the eye's dynamic change in the third time interval” and “features in the third time interval estimated from features of the eye's dynamic change in the fourth time interval” are represented by vectors The "differences" of these "features" can be represented by the distance between vectors representing those "features". Here, the “feature according to the degree of difference” is actually compared to the feature in the “third time interval” predicted using the feature of the “dynamic change of the eye” in the “fourth time interval”. It indicates how different the characteristics of the "dynamic change of the eye" in the "third time interval" of. That is, the “feature according to the degree of difference” indicates how abrupt the “third time interval” is from the tendency of the features predicted from the “fourth time interval”. Such “feature according to the degree of difference” represents a state of mind such as surprise, and is highly relevant to rapid heart rate fluctuations and pulse rate fluctuations. Therefore, by using such features, it is possible to estimate information on at least one of the heart rate and the pulse with higher accuracy. Note that the "third time interval" may be a time interval after (future) the "fourth time interval" or may be a time interval before (the past) before the "fourth time interval" It may be a time interval included in the “fourth time interval”. The “third time interval” may be a time interval immediately after or immediately before the “fourth time interval”, or may be a time interval apart from the “fourth time interval”. It is desirable that the “fourth time interval” be a time interval longer than the “third time interval”. This is because the trend of the feature amount can be accurately predicted if the “fourth time interval” is longer, and the novelty of the “third time interval” with respect to the trend can be evaluated with high accuracy. However, if the trend of feature quantities can be predicted from the “fourth time period”, the lengths of the “fourth time period” and the “third time period” may be equal, or the “fourth time period” May be a time interval shorter than the “third time interval”.

  The “physiological characteristics” are values derived from dynamic changes in one eye (eg, the right eye) of the same individual (human or non-human animal) and dynamic values of the other eye (eg, the left eye) It may include features based on relative values with values derived from such changes. The relative value of the dynamic change of both eyes shows the attribute and individuality of the individual, and by using the feature based on such relative value, it is possible to estimate the information on the heart rate and / or the pulse with high accuracy.

  The “dynamic change of the eye” may be the movement of the eye itself (the change of the position of the eye over time) or the movement of the pupil (the change of pupil diameter over time). An example of "movement of the eye itself" is the saccade of the eye. The "saccade" may be a micro saccade or a large saccade. The characteristic of the saccade of the eye can be exemplified by the movement direction of the eye, the absolute value of the amplitude of the eye movement, the attenuation coefficient of the eye movement, the natural angular frequency of the eye movement, the generation timing of the eyelid saccade, and the like. Examples of "pupil movement" are miosis and mydriasis. As the characteristics of miosis, the amplitude of mydriasis, the duration of mydriasis, the speed of average mydriasis, the number of occurrence of mydriasis, etc. can be exemplified. As the features of miosis, the amplitude of miosis, the duration of miosis, the velocity of mean miosis, and the number of occurrence of miosis can be exemplified.

  The “learning feature amount” or the “input feature amount” may be a scalar consisting of one element or a vector consisting of a plurality of elements. The “learning feature amount” and the “input feature amount” are, for example, time-series information including scalars or vectors in each discrete time or each time interval. However, the “learning feature amount” and the “input feature amount” may not be time-series information. Further, as described above, the “learning feature amount” and the “input feature amount” are feature amounts based on at least one of the heartbeat and pulse characteristics and physiological features other than the heartbeat and pulse characteristics. The “feature amount based on the feature α and the feature β” may be any feature amount in which a feature including the feature α and the feature β appears. For example, “a feature amount based on the feature α and the feature β” may be (1) a scalar representing the feature α or a feature representing the vector representing the feature α and a scalar representing the feature β or (2) the feature α It may be a scalar or vector representing a feature, a scalar or vector representing a feature β, and other features consisting of a scalar or vector, or (3) a scalar or vector representing a feature α and a scalar or vector function representing a feature β It may be a value, or (4) a scalar or vector representing feature α, a scalar or vector representing feature β, and other scalar or vector function values. For example, each element included in "feature for learning" and "input feature" may correspond to each of "feature of at least one of heartbeat and pulse" and "physiological feature other than heartbeat and pulse feature". It may be good (for example, an element that represents each feature) or may correspond to a plurality of "features" (for example, a function value of a value that represents a plurality of features). However, the “learning feature amount” and the “input feature amount” are feature amounts of the same type.

  The “prediction model” may be a linear regression model or a non-linear regression model. For example, a multiple regression model such as a vector autoregression model can be used as a linear regression model, and an artificial neural network can be used as a non-linear regression. Besides, a probability model may be used as a "prediction model". Also, the "prediction model" may be obtained in the "learning process", and then the "estimation process" may be executed without the "prediction model" being updated, or the "prediction model" may be updated by the "learning process". The “estimation process” may be performed. The “feature for learning” and the “input feature” may correspond to the same individual, or the individual corresponding to the “feature for learning” may be different from the individual corresponding to the “input feature”.

  The “estimated information according to at least one of the heartbeat and the pulse in the second time interval” may be any information as long as it corresponds to at least one of the heartbeat and the pulse in the second time interval. For example, the heartbeat interval or the pulse interval in the second time interval may be "estimated information", or the change in the heartbeat interval or the pulse interval in the second time interval may be "estimated information". The “change in heart rate interval” may be a difference or a ratio of two consecutive heart rate intervals, or may be a time-series derivative value of the heart rate interval. Similarly, the "change in pulse interval" may be a difference or a ratio of two consecutive pulse intervals, or may be a time-series derivative value of the pulse interval. When a probability model is used as the “prediction model”, the probability distribution of the heart rate interval or pulse interval in the second time interval may be used as “estimate information”, or the heart rate interval or pulse interval (probability) selected based on the probability distribution The one with the highest value, the one with the probability above the threshold value, etc. may be used as "estimated information", or the change in the selected heartbeat interval or pulse interval may be used as "estimated information".

First Embodiment
Next, a first embodiment will be described using the drawings.
<Configuration and processing>
As illustrated in FIG. 1, the system of the present embodiment includes a learning device 11 that performs learning processing and an estimation device 12 that performs estimation processing. The learning device 11 includes a biological information acquisition unit 111, a beat information acquisition unit 112, a feature extraction unit 113, and a prediction model generation unit 114. The estimation device 12 includes an estimation unit 121. Each of the learning device 11 and the estimation device 12 includes, for example, a processor (hardware processor) such as a central processing unit (CPU) and a memory such as a random-access memory (RAM) or a read-only memory (ROM). A general-purpose or dedicated computer is an apparatus configured by executing a predetermined program. The computer may have one processor or memory, or may have a plurality of processors or memory. This program may be installed in a computer or may be stored in advance in a ROM or the like. Further, instead of an electronic circuit (circuitry) that realizes a functional configuration by reading a program like a CPU, a part or all of the processing units are configured using an electronic circuit that realizes a processing function without using a program. May be Also, the electronic circuit that constitutes one device may include a plurality of CPUs.

<< Biometric Information Acquisition Unit 111 (FIG. 2: Step S111) >>
The biological information acquisition unit 111 acquires time-series information on “dynamic change of the eye” of each discrete time of the individual 100, and outputs the acquired time-series information on dynamic change of the eye to the feature amount extraction unit 113 Do. The "dynamic change of the eye" to be acquired may be the movement of the eye 100 itself of the individual 100, the movement of the pupil, or both of them. The biological information acquisition unit 111 may acquire time-series information on dynamic change of both eyes, or may acquire time-series information on dynamic change of one of the eyes. The individual 100 may be a human or an animal other than a human.

  The time-series information on the “movement of the eye itself” of the individual 100 is obtained based on an image obtained by photographing the eye of the individual 100 with an imaging device (for example, an infrared camera). The biological information acquisition unit 111 performs image processing on the captured video, for example, to obtain a time series of eyeball positions for each frame (for example, a sampling interval of 1000 Hz) which is a predetermined time section, a time series related to eyeball movement. Acquire as information. The biometric information acquisition unit 111 may be realized by an imaging device and a computer that executes an image processing algorithm, or a computer that executes an algorithm that processes an image input from the imaging device using the imaging device as an external device. It may be realized by Alternatively, the biological information acquisition unit 111 may measure the movement of the eye using a potential measurement method using an electrode, and may acquire time-series information on “the movement of the eye itself” based on the measurement result. In this case, the biological information acquisition unit 111 may be realized by a measurement device (including an electrode) and a computer that executes an algorithm for calculating the position of the eyeball based on the potential measured by the measurement device. The external device may be realized by a computer or the like that executes an algorithm that calculates the position of the eyeball based on the potential input from the measurement device. The upper diagram of FIG. 3B exemplifies time-series information representing the movement of one eyeball. The horizontal axis of this figure represents time [s], and the vertical axis represents viewing angle [°].

  The time-series information on the “pupil movement” of the individual 100 is obtained based on an image obtained by photographing the eye of the individual 100 with an imaging device (for example, an infrared camera). For example, one individual 100 is made to gaze at one point, and the pupil at that time is imaged by an infrared camera. The biological information acquisition unit 111 performs image processing on the captured video to acquire a time series of pupil sizes for each frame (for example, a sampling interval of 1000 Hz). The biological information acquisition unit 111 can, for example, fit a circle to the pupil of an image obtained by photographing the pupil, and use the radius of the fitted circle as the pupil diameter. Since the pupil diameter finely fluctuates, it is preferable that the biological information acquisition unit 111 use the value of the pupil diameter smoothed for each predetermined time interval. FIG. 5 exemplifies time-series information indicating the movement (change in pupil diameter) of the pupils of the right eye and the left eye. The horizontal axis in FIG. 5 represents time [seconds], and the vertical axis represents pupil diameter. The pupil diameter is expressed by z-score when the average of all data of pupil diameter acquired at each time is 0 and the standard deviation is 1. However, “the time series information on the movement of the pupil” acquired by the biological information acquisition unit 111 may not be the time series of the pupil diameter represented by z-score, but is the time series of the pupil diameter value itself It may be a time series of the area or diameter of the pupil, and may be any time series of values corresponding to the size of the pupil.

<< Beat Information Acquisition Unit 112 (Step S112) >>
The beat information acquisition unit 112 acquires time-series information on “at least one of the heartbeat and the pulse” at each discrete time of the individual 100, and outputs the acquired time-series information to the feature extraction unit 113. The example of the beat information acquisition unit 112 is a wearable terminal wearable on the individual 100, a heart rate monitor, a pulse rate monitor, or the like. "Time-series information on at least one of a heart beat and a pulse" may be time-series information of a value (for example, a voltage value) representing the amplitude of a heart beat or a pulse, or is time-series information of a heart beat pulse or a pulse pulse It may be time-series information of a function value of a value representing the amplitude of a heart beat and / or a pulse, or may be time-series information of a function value of a heart beat and / or a pulse pulse. It may be any function value of them. The bottom of FIG. 3B exemplifies time-series information of voltage values representing the amplitude of a heartbeat. The horizontal axis of this figure represents time [s], and the vertical axis represents voltage value [V] that represents the amplitude of the heartbeat.

<< Feature Amount Extraction Unit 113 (Step S113) >>
Feature amount extraction unit 113 inputs the time-series information regarding time-series information and "heartbeat and at least one of the pulse" relating to the acquired "dynamic changes in the eye", for each predetermined time interval F _t, heart rate and Obtain feature values y _t = (y _{t, 1} ,..., Y _{t, d} ) (learning features and input features) based on at least one feature of the pulse and physiological features other than the features of the heart and pulse Output. Here, t is a time index representing a discrete time corresponding to the time interval F _t , and d is the number of elements of the feature amount, that is, a positive integer representing a dimension. As t increases in value, it corresponds to a new discrete time. When the feature amount y _t is a scalar, d = 1, and when the feature amount y _t is a vector, d is an integer of 2 or more. There is no limitation on how to determine the time interval F _t . The length of the time interval F _t may be fixed or variable. For example, the heartbeat interval or pulse interval may be each time period F _t, may be set a time interval F _t as in each time period F _t is "at least one of heart rate and pulse" includes one or more . Alternatively, the time interval F _t may be set such that each time interval F _t includes one or more “dynamic changes of the eye (micro saccade, large saccade, miosis, mydriasis, etc.)”. So that each time interval F _t includes one or more of “dynamic change of eye (microsaccade, large saccade, miosis, mydriasis, etc.)” and “at least one of heartbeat and pulse”. A time interval F _t may be set.

The elements y _{t, i} (where i = 1,..., D) of the feature quantities y _t = (y _{t, 1} ,..., Y _t , d) are heart beat characteristics, pulse characteristics, and other physiological characteristics. The feature may be a feature based on any of the features, or may be a feature based on a combination of any of the plurality of features. Specific examples are shown below.

Feature quantities based on "at least one of heart rate features and heart rate features":
The feature quantity based on the feature of the heartbeat in the time interval F _t is, for example, the heartbeat interval in the time interval F _t , the occurrence time or occurrence time of the heartbeat, the heart rate, the amplitude of the heartbeat waveform, the RR interval, the PQ time, the QRS width , QT time, heart rate per unit time, etc. The feature quantity based on the feature of the pulse in the time interval F _t is, for example, the pulse interval in the time interval F _t , the generation time or generation time of the pulse, the pulse rate, the amplitude of the pulse waveform, the pulse rate per unit time, etc. is there. Feature based on features of the characterizing and pulse beat of a time interval F _t is, for example, functions for either one and the feature based on the feature of the pulse of the feature based on the characteristics of the heart beat in a time interval F _t It is a value. The feature quantity based on “at least one of heart beat feature and pulse feature” in such a time interval F _t is used as an element of the feature quantity y _t = (y _{t, 1} ,..., Y _{t, d} ) It is also good.

Feature quantities based on "movement of the eye itself":
As the feature amount based on the “movement of the eye itself”, for example, feature amounts of “micro saccade” and “large saccade” can be illustrated. At this time, in order to increase the accuracy of the model, it is desirable that there is no false detection or omission of detection of micro saccades or large saccades as much as possible.

Microsaccade features:
"Micro-saccade" refers to fine jumping eye movements that appear in eye movement. When a human is gazing at a certain point, the eyeball is not completely stopping movement, and three types of eye movement called drifting (also called drift, trend), trema, It is a micro-saccade (it may be called Flick). Drift is a small smooth movement, Trema is a very small high frequency oscillation, a micro saccade is a small jumping movement. A micro-saccade is an eye movement that appears (involuntarily) about once in 1 to 2 seconds while focusing on a certain point, and it is a small jumping movement It is The microsaccade can be obtained from either the horizontal or vertical component of the motion. In the present embodiment, only the horizontal component is used for simplicity, based on the property of the microsaccade to be deflected in the horizontal direction. However, the directional component of the microsaccade that can be used in the present invention is not limited to the horizontal direction. The “horizontal direction” does not mean to limit the direction parallel to the ground, but the horizontal direction with respect to the face of the individual 100 (the arrangement direction of the eyeballs, which may be referred to as the lateral direction or the width direction) This is a concept including the direction defined as the horizontal direction in the biometric information acquisition unit 111.

  The feature quantity extraction unit 113 calculates, for example, a first-order difference series for the time series of the position of the eyeball, and the micro-saccade start time is the time when the absolute value of the first-order series exceeds the predetermined first threshold. It may be detected as (occurrence time). However, if the length of time in which the absolute value of the primary difference sequence exceeds a predetermined threshold does not last longer than a predetermined value (usually about 3 ms), it is excluded from the detection. Moreover, when the below-mentioned reference amplitude A is more than a predetermined | prescribed threshold value (normal viewing angle 2 degree extent), it is excluded from detection as a large saccade. The feature quantity extraction unit 113 may use a moving average value within an appropriate range when calculating the first-order difference series, when it is determined that much noise is included in the acquired position information of the eyeball. It is preferable to use a value of about six times the standard deviation of the difference series as the threshold used for detection.

The microsaccade features include a value Z based on the generation timing of the microsaccade, a value D corresponding to the movement direction, an absolute value | A | of the reference amplitude A, a maximum velocity V _max , a duration D _m , and an overshoot. Absolute value | A _o | of the amplitude A _o , overshoot velocity V _o , rise time K, damping rate λ, damping coefficient ζ, natural angular frequency ω _n , per unit time of microsaccade (for example, 1 second) It is possible to illustrate the occurrence frequency R _{m and the} like. As described above, it is preferable to use the value Z based on the generation timing of the microsaccade, the value D corresponding to the movement direction, the absolute value | A | of the reference amplitude A, and the damping coefficient 特 に.

The value Z based on the generation timing of the micro-saccades that occur in a time interval F _t is, for example, may be a start time M _t time interval F _t, micro saccade corresponding to the reference become time interval RF _t time difference between the occurrence time RM _t and start time M _t | RM _t -M _t |, or may be a starting time M _t or the time difference | RM _t -M _t | function value g (M _t) or g It may be (| RM _t −M _t |). The reference time interval RF _t corresponds to the time interval F _t , for example RF _t = F _t−1 . Although there is no limitation on the function value g (M _t ) or g (| RM _t −M _t |), for example, the larger the representative value M _t or the time difference | RM _t −M _t | Function values of functions having no singularity can be used. Examples of the function value g (M _t ) include 1 / M _t and exp (-M _t ). "Exp" represents an exponential function based on the number of Napiers. Examples of g (| RM _t -M _t |) include 1 / | RM _t -M _t | and exp (-| RM _t -M _t |). However, when M _t and | RM _t −M _t | are 0, 1 / M _t and 1 / | RM _t −M _t | are ∞. Therefore, g (M _t ) or g (| RM _t −M _t |) may be set to 0 when M _t or | RM _t −M _t | becomes zero. The value Z based on the generation timing of the microsaccade generated in such a time interval F _t may be used as one of the elements of the feature amount y _t = (y _{t, 1} ,..., Y _{t, d} ).

The value D according to the movement direction of the micro _saccade generated in the time interval F _t is a value determined for each movement direction. For example, the value D corresponding to the movement direction may take either of two values corresponding to the left and right movement directions, or may take one of four values corresponding to the left, right, upper and lower movement directions. Any of n values corresponding to the motion direction may be taken. As an example, a value D corresponding to the movement direction in the right direction (the direction from the left eye to the right eye) is taken as a first value (for example, −1), and the movement direction in the left direction (the direction from the left eye to the right eye) A value D corresponding to T is set to a second value (for example, 1). The value D corresponding to the movement direction of the microsaccade generated in such a time interval F _t may be used as one of the elements of the feature amount y _t = (y _{t, 1} ,..., Y _{t, d} ).

と表される。 Referring now to FIG. 4, the microsaccade reference amplitude A, maximum velocity V _max , duration D _m , overshoot amplitude A _o , overshoot velocity V _o , rise time K, and decay rate λ will be described. Do.
(1) Reference amplitude A: The amount of movement when the movement of the eyeball due to the micro saccade converges.
(2) Maximum velocity V _max : This is the maximum velocity to reach the reference amplitude A + the amplitude A _o of the overshoot.
(3) Duration time D _m : The length of the time interval in which the microsaccade occurs. The start time of the microsaccade is the time when the absolute value of the primary difference sequence exceeds a predetermined threshold, and the end time of the microsaccade is the time when it returns to the reference amplitude A for the first time after reaching the overshoot amplitude. .
(4) the amplitude A _o overshoot (overshoot phenomenon): reference amplitude A has been exceeded (excesses) by a micro saccade is the amount of parts. The overshoot is a phenomenon in which the waveform protrudes beyond the reference amplitude A at the rising portion of the waveform or a waveform that protrudes. In other words, the overshoot amplitude is the amount of the protruding portion.
(5) Overshoot velocity V _o : This is the maximum velocity when trying to converge from the reference amplitude A + overshoot amplitude A _o to the reference amplitude A.
(6) Rise time K: time taken to reach (rise) reference amplitude A + overshoot amplitude A _o . The time taken to reach the reference amplitude A + the amplitude A _o of the overshoot is equal to the time taken to reach the speed V _o of the overshoot from the maximum speed V _max .
(7) Attenuation rate λ: A ratio of overshoot amplitude A _o to reference amplitude A. It may be the ratio of the overshoot speed V _{o to} the maximum speed V _max ,

It is expressed as

と表される。固有角振動数ω_ｎはマイクロサッカードの応答の速さを表す指標に相当し、減衰係数ζはマイクロサッカードの応答の収束性を表す指標に相当する。 Damping coefficient マ イ ク ロ of microsaccade, natural angular frequency ω _n is

It is expressed as The natural angular frequency ω _n corresponds to an index representing the speed of response of the microsaccade, and the damping coefficient ζ corresponds to an index representing the convergence of the response of the microsaccade.

The feature amount extraction unit 113 optimizes the attenuation coefficient マ イ ク ロ of the microsaccade, the natural angular frequency ω _n , and the reference amplitude A by the least square method etc. by fitting the position of the eye while the microsaccade is occurring. You may calculate by doing.

  Since the attenuation coefficient 113 of the microsaccade tends to change in value depending on the movement in the left and right direction, the feature quantity extraction unit 113 is a representative value of the attenuation coefficient of the microsaccade in the left direction, The representative value of the saccade attenuation coefficient may be calculated separately.

Absolute value | A | of the reference amplitude A of the above _microsaccade generated in the time interval F _t , maximum velocity V _max , duration D _m , absolute value of overshoot amplitude A _o | A _o | At least one of the shoot speed V _o , the rise time K, the attenuation factor λ, the attenuation coefficient 角, and the natural angular frequency ω _n , any one of the feature quantities y _t = (y _{t, 1} ,..., Y _{t, d} ) It may be an element of

Large saccade features:
The term "large saccade" refers to jumping eye movement with amplitude greater than that of microsaccades. Generally, large saccades with amplitudes greater than 2 degrees are considered microsaccades with less than 2 degrees. The feature amount extraction unit 113 may detect the time when the above-mentioned reference amplitude A becomes equal to or more than the predetermined threshold as the start time of occurrence of the large saccade. Examples of large saccade feature values are a value Z based on the occurrence timing of the large saccade, a value D according to the motion direction, a reference amplitude A, a maximum velocity V _max , a duration D _m , an overshoot amplitude A _o , The overshoot speed V _o , the rise time K, the attenuation factor λ, the attenuation coefficient 角, the natural angular frequency ω _n , the number of occurrences R _m per unit time, the number of occurrences, and the like. These specific examples replace "micro saccade" of the example of the feature-value of the micro saccade mentioned above with "large saccade". At least one of the feature quantities of the large saccade generated in the time interval F _t may be an element of the feature quantities y _t = (y _{t, 1} ,..., Y _{t, d} ). For the reasons described above, it is preferable to use the value Z based on the occurrence timing of the large saccade, the value D corresponding to the movement direction, the absolute value | A | of the reference amplitude A, and the damping coefficient 特 に.

Feature quantities based on "pupil movement":
As illustrated in FIG. 5, the pupil size is not constant but is changing. The size of the pupil is enlarged (mydriasis) by the pupil dilator under the control of the sympathetic nervous system, and contracted (miosis) by the pupil sphincter under the control of the parasympathetic nervous system. The change in pupil size is mainly divided into three: light reflex, convergence reflex and emotional change. The light reflection is a reaction in which the size of the pupil changes to control the amount of light incident on the retina, resulting in miosis for strong light and mydriasis in the dark. The convergence reflex is a reaction in which the diameter of the pupil changes as the eyes involuntary or abduction (convergence motion) when focusing, and when looking at a close eye, miosis or mydriasis It occurs. Emotional change is the response to external stress regardless of any of the above, and mydriasis occurs when sympathetic dominance becomes dominant with anger, surprise, or active activity When the parasympathetic nerve dominates, miosis occurs. As feature quantities based on "pupil movement", feature quantities of miosis and mydriasis can be used.

Mimic feature:
The start time of miosis (hereinafter, miosis start point) is detected by extracting the maximum point from the time series of pupil sizes. The time when miosis ends (hereinafter, miosis end point) is set as the earlier one of the point where mydriasis starts for the first time after the start of miosis or the point where blink starts for the first time after the start of miosis. The amplitude A _c of miosis is the difference in pupil diameter from the miosis start point to the miosis end point. The miosis duration D _c is the time difference from the miosis start point to the miosis end point. The average miosis velocity V _c is (amplitude A _c ) / (duration D _c ). Amplitude A _c of miosis occurring in a time interval F _t, the duration D _c of miosis, velocity V _c of the average miosis, based on the "movements of the pupil" of the like number of occurrences of miosis in a time interval F _t It can be used as a feature quantity. The feature amount extraction unit 113 calculates at least one of the amplitude A _{c of} the miosis, the duration D _{c of} the miosis, the velocity V _{c of the} average miosis, and the number of occurrences of miosis in the time interval F _{t. It} may be any element of _t = (y _{t, 1} ,..., y _{t, d} ). When a plurality of pupils are detected in the time interval F _t , representative values of the pupillary amplitude A _c , the pupillary duration D _c , and the average pupillary velocity V _{c determined} for each pupil are shown Any of the above may be an element of one of the feature quantities y _t = (y _{t, 1} ,..., Y _{t, d} ). In order to prevent false detection due to noise, when the duration of miosis is less than a predetermined threshold (for example, 10 ms) or when the amplitude of miosis is less than a predetermined threshold, the miosis is excluded from detection. May be

Feature amount of mydriasis:
The start time of mydriasis (hereinafter referred to as mydriatic start point) is detected by extracting the minimum point from the time series of pupil diameter. The time when the mydriasis ends (hereinafter, the mydriatic end point) is the earlier one of the point where mydriasis starts for the first time after the mydriasis start or the point where the blink starts for the first time after the mydriasis start. Amplitude A _d mydriatic is the difference between the pupil diameter from mydriasis start point to mydriasis end point. The mydriatic duration D _d is the time difference from the mydriatic start point to the mydriatic end point. The average velocity V _{d of} mydriasis is (amplitude A _d ) / (duration D _d ). The amplitude A _d of mydriasis generated in the time interval F _t , the duration D _d of mydriasis, the velocity V _{d of the} average mydriasis, the number of occurrences of mydriasis, etc. are based on the “pupil movement” in the time interval F _t It can be used as a feature quantity. The feature amount extraction unit 113 determines at least one of the amplitude A _{d of} the mydriasis generated in the time interval F _t , the duration D _d of mydriasis, the velocity V _{d of the} average mydriasis, and the number of occurrences of mydriasis _It may be any element of _t = (y _{t, 1} ,..., y _{t, d} ). When a plurality of mydriasis are detected in the time interval F _t , representative values of an amplitude A _d of mydriasis, a duration D _d of mydriasis, and a velocity V _d of an average mydriasi obtained for each mydriasi Any of the above may be an element of one of the feature quantities y _t = (y _{t, 1} ,..., Y _{t, d} ). In order to prevent false detection due to noise, when the duration of mydriasis is less than a predetermined threshold (for example, 10 ms) or when the amplitude of mydriasis is less than a predetermined threshold, the mydriasis is excluded from detection. May be

Feature quantities based on relative quantities of binocular features:
One eye of an individual 100 (e.g., right eye) values and the other eye derived from dynamic changes (e.g., left) the feature amount of the relative amounts of the value derived from the dynamic changes in y _t = It may be any element of (y _{t, 1} ,..., y _{t, d} ). As the values derived from the dynamic change of the eye, the above-mentioned feature amounts of micro saccade, large saccade features, miosis features, mydriasis features and the like can be used. However, the value derived from the dynamic change of one eye and the value derived from the other dynamic change are the same feature quantities. Specific examples of the "relative amount" are as described above. The “relative amount between α and β” is, for example, the difference between α and β, a value obtained by subtracting β from α, a value obtained by subtracting α from β, a value obtained by dividing α by β, and dividing β by α Value, difference between function value of α and function value of β, value obtained by subtracting function value of β from function value of α, value obtained by subtracting function value of α from function value of β, or any one of them It is a value etc.

Feature quantities based on relative amounts of "at least one feature of heart rate and pulse" and "feature of dynamic change of eye":
The relative amount of the value corresponding to the occurrence time of “dynamic change of the eye” in the time interval F _t and the value according to the occurrence time of “at least one of heart rate and pulse” is the feature value y _t = (y _It may be any element of _{t, 1} , ..., yt _{, d} ). For example, from the occurrence of the “first interval” from the occurrence of “heart beat” in the time interval F _t to the occurrence of “dynamic change of the eye” and the occurrence of the “dynamic change of eye” A feature according to the "second interval" until the occurrence, and "third interval" from the occurrence of the "pulse" in the time interval F _t to the occurrence of the "dynamic change of the eye" and "movement of the eye" Of at least one of the features according to the “fourth interval” from the occurrence of the “change” to the occurrence of the next “pulse”, the feature quantity y _t = (y _{t, 1} ,..., Y _{t, d} ) It may be any element. For example, in the example of FIG. 3B, a microsaccade occurs at time points MS ₁ , MS ₂ and MS ₃ , and a heart beat occurs at time points B ₁ and B ₂ . For example, a time interval F _t heartbeat interval from time B ₁ to time B _2, the time interval from the time point B ₁ to the time MS ₃ and T ₁ (MS _3), time from MS ₃ to time B ₂ Let the section be T ₂ (MS ₃ ). Function values of T ₁ (MS ₃ ), T ₂ (MS ₃ ), T ₁ (MS ₃ ) or T ₂ (MS ₃ ), and at least function values of T ₁ (MS ₃ ) and T ₂ (MS ₃ ) Any one of them is a feature based on the relative amount of the feature of the heartbeat at time points B ₁ and B ₂ and the feature of the microsaccade at time point MS ₃ , and feature value y _t = (y _{t, 1} ,. _It may be any element of _{t, d} ). As function values of T ₁ (MS ₃ ) and T ₂ (MS ₃ ), for example, T ₁ (MS ₃ ) / {T ₁ (MS ₃ ) + T ₂ (MS ₃ )}, T ₂ (MS ₃ ) / {T ₁ (MS ₃ ) + T ₂ (MS ₃ )}, {T ₁ (MS ₃ ) + T ₂ (MS ₃ )} / T ₁ (MS ₃ ), {T ₁ (MS ₃ ) + T ₂ (MS ₃ ) } / T ₂ (MS ₃ ) can be illustrated. Similar values may be obtained for the other _{microsaccades} included in the time interval F _t, and may be used as one of the features y _t = (y _{t, 1} ,..., Y _{t, d} ). In addition, an interval wider than one heartbeat interval (for example, an interval including a plurality of heartbeat intervals) may be set as the time interval F _t , or an interval narrower than one heartbeat interval (for example, one micro saccade) A section including _T may be taken as the time section F _t .

選択されたモデルパラメータφ（例えば、係数c₁,…,c_d，φ_1,1 ⁽¹⁾,…,φ_d,d ⁽¹⁾,…,φ_1,1 ^(p),…,φ_d,d ^(p)）は、推定装置１２の推定部１２１に送られる。 << Prediction Model Generation Unit 114 (Step S114) >>
The feature quantities y _t = (y _{t, 1} ,..., Y _{t, d} ) of each time interval F _t are sent to the prediction model generation unit 114. The prediction model generation unit 114 sets and outputs model parameters φ of the prediction model, using the plurality of feature quantities y _t sent so far as “explanatory variables” and “target variables” (FIG. 3A). The prediction model is estimated information according to at least one of the heartbeat and the pulse in the second time interval τ ₂ from the feature quantity y _τ (where τ∈τ ₁ ) (explanatory variable) obtained in the first time interval τ ₁ It is a prediction model for obtaining η _θ (where θ∈τ ₂ ) (objective variable) (FIG. 3A). As mentioned above, constraints on the relationship of the first time interval tau ₁ and the second time interval tau ₂ is not, there is no limitation to the predictive model. Here, as an example, first time period tau ₁ than the second time interval tau ₂ is a past time interval, illustrating the case of using a vector autoregressive model as a prediction model. In this case, the prediction model generator 114, which until the time interval F _t of _', ..., obtained by _{F t'-p (y t'} , 1, ..., y t ', d), ..., ( y _{t'-p, 1} , ..., y _{t'-p, d} ) are input, and coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d of the} following vector autoregressive model _{, d} ⁽¹⁾ ,..., φ _1,1 ^(p) ,..., φ _{d, d} ^(p) (model parameter φ) is calculated and output. Here, p is a positive integer, and may be predetermined or adaptively determined. Also, t ≧ t '. For example, t ′ = t, t−1, t−2,..., Tw, and w is an integer of w <t. For example, tw is the initial value of the time index (for example, tw = 1). The coefficients c ₁ ,..., C _d , φ _1,1 ⁽¹⁾ ,..., Φ _{d, d} ⁽¹⁾ ,..., Φ _1,1 ^(p) ,..., Φ _{d, d} ^(p) are, for example, To minimize the magnitudes of the prediction errors (prediction residuals) ε _{t ′, 1} ,..., Ε _{t ′, d} for all t ′ = t, t−1, t−2,. Is selected. For example, the coefficient c _{1 is set} so that the sum of ε _{t ′, 1} ,..., Ε _{t ′, d} for t ′ = t, t−1, t−2,. ,..., C _d , φ _1,1 ⁽¹⁾ ,..., Φ _{d, d} ⁽¹⁾ ,..., Φ _1,1 ^(p) ,..., Φ _{d, d} ^(p) are set. Also, the feature amount may be normalized in advance by a method such as Box-Cox conversion.

Selected model parameters φ (for example, coefficients c ₁ ,..., C _d , φ _1,1 ⁽¹⁾ ,..., Φ _{d, d} ⁽¹⁾ ,..., Φ _1,1 ^(p) ,..., Φ _{d , d} ^(p) ) are sent to the estimation unit 121 of the estimation device 12.

<< estimation part 121 (step S121) >>
The feature amount y _t obtained by the feature amount extraction unit 113 and the model parameter φ obtained by the prediction model generation unit 114 are sequentially input to the estimation unit 121. The estimation unit 121 updates the prediction model using the model parameter φ that is sequentially input, and determines from the feature value y _τ (where τ∈τ ₁ ) (explanatory variable) obtained in the first time interval τ ₁ . Estimated information _{θ θ} (where θ∈τ ₂ ) (target variable) according to at least one of the heartbeat and pulse in the 2-hour interval τ ₂ is obtained and output. For example, using a vector autoregressive model as a prediction _{model, τ 1 = F t "-1} , ..., F t" -p obtained in the feature amount _{y t "-1, ..., y} t" 2 from _-p tau = when estimating the _"feature quantity y ^ _{t in"} F _t, estimating section 121, _"obtained. Incidentally," y ^ _{t "feature} quantity y ^ _t as follows superscript" on letter " “^” Should be written right above “y” as follows, but it is written as “y _{t” because} of the restriction of written description.

The estimation unit 121 may output all elements of the feature amount y ^ _{t "} as the estimated information _{θ θ} at τ ₂ = _Ft" , and at least the heartbeat and the pulse among the elements of the feature amount y ^ _{t "} Only an element according to one may be output as the estimation information _{θ θ} , or a function value of an element according to all elements of the feature amount ^ _{t ′} ′ or at least one of the heartbeat and the pulse is output as the estimation information _{θ θ} May be For example, T ₂ (MS τ2) and T ₁ which is feature value y ^ _{t "corresponding} to the micro saccade when MS _.tau.2 belonging to ₂ tau second time period _{(MS τ2) / {T 1} (MS τ2) + T if it contains ₂ (MS _τ2)}, the estimation unit 121 may output the T ₂ (MS τ2) and _{T 1 (MS τ2) / {} T 1 (MS τ2) + T 2 (MS τ2)}. for example If the feature amount y ^ _{t "includes T 1 (MS τ2) + T} 2 (MS τ2) can identify the elements, the estimation unit 121 is _{T 1 (MS τ2) + T} 2 (MS τ2) the heartbeat interval It may be output as estimated information 出力_θ . The change of the estimated heart rate section may be output as the estimated information _{θ θ} . In addition, when using a probability model as a prediction model, the probability distribution of the feature amount y ^ _{t "} may be output as the estimation information _{θ θ} , or the feature amount y ^ _t" selected based on the obtained probability distribution Or an element thereof (for example, an element corresponding to at least one of a heart beat and a pulse) may be obtained and output.

Second Embodiment
"Physical features other than heart rate and pulse features" are the features of "dynamic change of the eye" in the "third time interval" and "dynamic change of the eye in the fourth time interval" And a feature according to the degree of difference between the feature in the third time interval estimated from the feature of. Hereinafter, differences from the matters described in the first embodiment will be mainly described, and the same reference numerals will be used for the matters already described to simplify the explanation.

<Configuration and processing>
As illustrated in FIG. 1, the system of the present embodiment includes a learning device 21 that performs learning processing and an estimation device 12 that performs estimation processing. The learning device 21 includes a biological information acquisition unit 111, a beat information acquisition unit 112, a feature quantity extraction unit 213, and a prediction model generation unit 114. As illustrated in FIG. 6A, the feature quantity extraction unit 213 includes a primary feature quantity extraction unit 213a, a prediction model generation unit 213b, and a prediction error feature quantity calculation unit 213c. The difference from the first embodiment of the present embodiment is only that step S213 by the feature amount extraction unit 213 is executed instead of step S113 by the feature amount extraction unit 113. Hereinafter, only the process of step S213 by the feature amount extraction unit 213 will be described.

<< Feature Amount Extraction Unit 213 (Step S213) >>
Feature quantity extracting unit 213 inputs the time-series information regarding time-series information and "heartbeat and at least one of the pulse" relating to the acquired "dynamic changes in the eye", for each predetermined time interval F _t, heart rate and Obtain feature values y _t = (y _{t, 1} ,..., Y _{t, d} ) (learning features and input features) based on at least one feature of the pulse and physiological features other than the features of the heart and pulse Output. The difference from step S113 of the first embodiment is that the feature amount obtained based on the “dynamic change of the eye” in the “third time interval” as the element of the feature amount y _t , and “the fourth The feature quantity according to the degree of difference between the feature quantity in the “third time interval” estimated from the time-series information of the feature quantity obtained based on the “dynamic change of the eye” in the “time interval” It is a point that contains. There is no restriction on the relationship between the third time interval and the fourth time interval, and there is no restriction on the method of estimating feature quantities in the third time interval. Here, as an example, a case where the fourth time interval is a time interval before the third time interval and the feature amount in the third time interval is estimated using a vector autoregression model is illustrated.

<< Primary feature amount extraction unit 213a (FIG. 6B: step S213a) >>
Primary feature extraction unit 213a inputs the time-series information on "at least one of heart rate and pulse" time-series information and a "dynamic change of the eye", for each predetermined time interval F _t, heart rate and pulse Feature quantities (x _{t, 1} ,..., X _{t, d ′} ) (learning feature quantities and input feature quantities) based on at least one feature and physiological features other than heartbeat and pulse features are obtained and output. However, d 'is a positive integer that satisfies 0 <d'<d. The primary feature quantity extraction unit 213a obtains an element of the feature quantity (x _{t, 1} ,..., X _{t, d ′} ), for example, by the same processing as the feature quantity extraction unit 113 of the first embodiment.

選択された係数c₁’,…,c_d’’，η_1,1 ⁽¹⁾,…,η_d’,d’ ⁽¹⁾,…,η_1,1 ^(p’),…,η_d’,d’ ^(p’)は、予測誤差特徴量計算部２１３ｃに送られる。 << prediction model generation unit 213b (step S213b) >>
Prediction model generating unit 213b has previously each time period of F _i, ..., _'obtained in _{(x i, 1, ...,} x i, d' F i-p), ..., (x i- The coefficients c ₁ ', ..., c _d' ',, _1,1 ⁽¹⁾ , ...,, of the following vector autoregressive model, with _{p', 1} , ..., x _{i-p ', d'} ) as input _{d ', d'} ⁽¹⁾ , ..., η _1,1 ^{(p ')} , ...,, _{d', d '} ^(p') are calculated and output. Here, t ≧ i. For example, i = t, t−1, t−2,..., T−w ′, and w ′ is an integer of w ′ <t. For example, tw 'is the initial value of the time index (eg, tw' = 1). Also, p ′ is a positive integer, and may be predetermined or adaptively determined. The coefficients c ₁ ', ..., c _d' ', _{1,1 1,1} ⁽¹⁾ , ..., _{d d', d '} ⁽¹⁾ , ..., _{1,1 1,1} ^(p') , ..., _{d d ', d '} ^(p') , for example, the magnitudes of the prediction errors _{i i, 1} ,..., _{i i, d '} for all i = t, t-1, t-2, ..., t-w' Is chosen to minimize For example, the coefficient c is such that the sum of _{i i, 1} ,...,, _{I, d ′} for i = t, t−1, t−2,. ₁ ', ..., c _d' ', η _1,1 ⁽¹⁾ , ..., η _{d', d '} ⁽¹⁾ , ..., η _1,1 ^(p') , ..., η _{d ', d'} ^{( p ')} is set.

Selected coefficients c ₁ ', ..., c _d' ', _{1,1 1,1} ⁽¹⁾ , ..., _{d d', d '} ⁽¹⁾ , ..., _{1,1 1,1} ^(p') , ..., _{d d ', d'} ^{(p ')} is sent to the prediction error feature quantity calculator 213 _c .

ただし、

であり、

であり、［・］^Ｔは［・］の転置である。Σ_ｔはx^_tの分散共分散行列であり、Σ_ｔのi行j列の要素Σ_ｉｊは以下の通りである。

ただし、Ｅ［・］は［・］の期待値を表す。 << Prediction Error Feature Amount Calculator 213 c (Step S 213 c) >>
Prediction error feature quantity calculation unit 213c, the coefficients _{c 1 ', ..., c d} '', η 1,1 (1), ..., η d', d '(1), ..., η 1,1 (p' _{^{), ..., η d ',}} d' (p '), and the feature amount _{(x j, 1, ...,} x j, d') ( however, j = t, t-1 , ..., t-p ' ) Is input, and the feature quantity St corresponding to _t is obtained and output as follows.

However,

And

And [·] ^T is transpose of [·]. Σ _t is a variance-covariance matrix of x _t , and elements Σ _ij of i rows and j columns of _{t t} are as follows.

However, E [•] represents the expected value of [•].

The feature quantity extraction unit 213 outputs the feature quantity S _t as an element of the feature quantity y _t = (y _{t, 1} ,..., Y _{t, d} ). Furthermore, the feature quantity extraction unit 213 may select any or all of the feature quantities (x _{t, 1} ,..., X _{t, d ′} ) output from the primary feature quantity extraction unit 213 a as feature quantities y _t = (y _{t, It} may be output as an element of ₁ ,..., Y _{t, d} ). The subsequent processing is the same as in the first embodiment.

Here, the coefficient c ₁ prediction model generating unit 213b is the real-time _{', ..., c d''} , η 1,1 (1), ..., η d', d '(1), ..., η 1,1 ( ^{p ')} , ..., _{d d', d '} ^(p') were updated. However, in advance the coefficients c ₁ ', ..., c _d' ', _{1,1 1,1} ⁽¹⁾ , ...,, _{d', d '} ⁽¹⁾ , ..., _{1,1 1,1} ^(p') , ..., η _{It is possible} to calculate _{d 'and d'} ^{(p ')} and not update them.

Third Embodiment
In the first and second embodiments, the feature of "dynamic change of the eye" is used as the "physiological feature other than the feature of heart rate and pulse". However, in addition to the feature of "dynamic change of eye", other "physiological features other than the feature of heart rate and pulse" may be used. Alternatively, other "physiological characteristics other than heart rate and pulse characteristics" may be used instead of the "dynamic change of the eye" feature. Other “physiological characteristics other than heart and pulse characteristics” include “respiratory movement characteristics” “movement characteristics appearing on the surface of the living body” “muscle activity characteristics” “body temperature characteristics” “perspiration characteristics” “electroencephalogram Feature of the above can be illustrated. In this case, the biometric information acquisition unit 111 of the learning device 11 or 21 may be provided with a known means for measuring such a feature.

Fourth Embodiment
In the first and second embodiments, the coefficient c ₁ prediction model generation unit 114 in real _{time, ..., c d, φ 1,1} (1), ..., φ d, d (1), ..., φ 1,1 ( ^p) , ..., φ _{d, d} ^(p) were updated. However, the coefficients c ₁ , ..., c _d , φ _1,1 ⁽¹⁾ , ..., φ _{d, d} ⁽¹⁾ , ..., φ _1,1 ^(p) , ..., φ _{d, d} ^(p) in advance You may calculate. In this case, the estimation device 12 further includes a storage unit 422, and the coefficients c ₁ ,..., C _d , φ _1,1 ⁽¹⁾ ,..., Φ _{d, d} calculated in advance in the storage unit 422. ⁽¹⁾ ,..., Φ _1,1 ^(p) ,..., Φ _{d, d} ^(p) may be stored. Estimation unit 121, the coefficient c ₁ is stored in the storage unit _{422, ..., c d, φ} 1,1 (1), ..., φ d, d (1), ..., φ 1,1 (p), ... , φ _{d, d} ^(p) to construct a prediction model, to which the feature quantity y _t obtained by the feature quantity extraction unit 113 is applied to obtain and output estimated information _{θ θ} . Others are the same as the first and second embodiments.

[Other modifications etc]
The present invention is not limited to the above-described embodiment. For example, the learning device and the estimation device may be configured in the same housing. In addition, the feature amount used in the estimation process may be obtained by a biological information acquisition unit, a beat information acquisition unit, and a feature amount extraction unit configured outside the learning device. In addition, individuals who acquire heart and pulse characteristics and other physiological characteristics for learning processing differ from individuals who acquire heart and pulse characteristics and other physiological characteristics for estimation processing. May be The physiological characteristics other than the heart rate and the pulse may include features based on relative values of values derived from dynamic change of one eye of the same individual and values derived from dynamic change of the other eye. .

  The various processes described above may be performed not only in chronological order according to the description, but also in parallel or individually depending on the processing capability of the apparatus that executes the process or the necessity. It goes without saying that other modifications can be made as appropriate without departing from the spirit of the present invention.

  When the above configuration is implemented by a computer, the processing content of the function that each device should have is described by a program. The above processing functions are realized on a computer by executing this program on a computer. The program describing the processing content can be recorded in a computer readable recording medium. An example of a computer readable recording medium is a non-transitory recording medium. Examples of such recording media are magnetic recording devices, optical disks, magneto-optical recording media, semiconductor memories and the like.

  This program is distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. At the time of execution of processing, this computer reads a program stored in its own storage device and executes processing in accordance with the read program. As another execution form of this program, the computer may read the program directly from the portable recording medium and execute processing in accordance with the program, and further, each time the program is transferred from the server computer to this computer Alternatively, processing may be performed sequentially according to the received program. The configuration described above is also executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to this computer. Good.

  In the above embodiment, the processing function of the present apparatus is realized by executing a predetermined program on a computer, but at least a part of these processing functions may be realized by hardware.

11, 21 learning device 12 estimation device

Claims (10)

The feature of at least one of the heartbeat and the pulse obtained in the first time interval and the feature of the heartbeat and the pulse using the feature for learning based on the feature of at least one of the heartbeat and the pulse and the physiological feature other than the feature of the heartbeat and the pulse A learning apparatus for obtaining a prediction model for obtaining estimation information according to at least one of a heartbeat and a pulse in a second time interval from an input feature based on a physiological feature other than the above ,
The characteristics of at least one of the heart rate and pulse and physiological characteristics other than the heart and pulse characteristics are values according to the occurrence time of the dynamic change of the eye and values according to the occurrence time of the heart and / or pulse. A learning device that includes features based on relative values of and. The feature of at least one of the heartbeat and the pulse obtained in the first time interval and the feature of the heartbeat and the pulse using the feature for learning based on the feature of at least one of the heartbeat and the pulse and the physiological feature other than the feature of the heartbeat and the pulse A learning apparatus for obtaining a prediction model for obtaining estimation information according to at least one of a heartbeat and a pulse in a second time interval from an input feature based on a physiological feature other than the above ,
The features of at least one of the heartbeat and the pulse and physiological features other than the features of the heartbeat and the pulse are from the first interval from the occurrence of the heartbeat to the occurrence of the dynamic change of the eye and the occurrence of the dynamic change of the eye The second pulse from the next pulse to the next heartbeat from the occurrence of the third pulse from the occurrence of the dynamic change of the eye and the third pulse from the occurrence of the dynamic change of the eye A learning device including at least one of a fourth interval according to occurrence and a feature according to the fourth interval. The feature of at least one of the heartbeat and the pulse obtained in the first time interval and the feature of the heartbeat and the pulse using the feature for learning based on the feature of at least one of the heartbeat and the pulse and the physiological feature other than the feature of the heartbeat and the pulse A learning apparatus for obtaining a prediction model for obtaining estimation information according to at least one of a heartbeat and a pulse in a second time interval from an input feature based on a physiological feature other than the above ,
The physiological feature is a feature of the dynamic change of the eye in the third time interval and a feature of the third time interval estimated from the feature of the dynamic change of the eye in the fourth time interval A learning device comprising features according to the degree of difference. An input feature based on at least one of heart rate and pulse characteristics obtained in the first time interval and a physiological feature other than heart rate and pulse characteristics is applied to the prediction model, and at least the heart rate and heart rate in the second time interval. An estimation apparatus for obtaining estimation information according to one of the following:
The characteristics of at least one of the heart rate and pulse and physiological characteristics other than the heart and pulse characteristics are values according to the occurrence time of the dynamic change of the eye and values according to the occurrence time of the heart and / or pulse. An estimation device that includes features based on relative values of and. An input feature based on at least one of heart rate and pulse characteristics obtained in the first time interval and a physiological feature other than heart rate and pulse characteristics is applied to the prediction model, and at least the heart rate and heart rate in the second time interval. An estimation apparatus for obtaining estimation information according to one of the following:
The features of at least one of the heartbeat and the pulse and physiological features other than the features of the heartbeat and the pulse are from the first interval from the occurrence of the heartbeat to the occurrence of the dynamic change of the eye and the occurrence of the dynamic change of the eye The second pulse from the next pulse to the next heartbeat from the occurrence of the third pulse from the occurrence of the dynamic change of the eye and the third pulse from the occurrence of the dynamic change of the eye An estimation apparatus, comprising at least one of a feature according to a fourth interval to occurrence. An input feature based on at least one of heart rate and pulse characteristics obtained in the first time interval and a physiological feature other than heart rate and pulse characteristics is applied to the prediction model, and at least the heart rate and heart rate in the second time interval. An estimation apparatus for obtaining estimation information according to one of the following:
The physiological feature is a feature of the dynamic change of the eye in the third time interval and a feature of the third time interval estimated from the feature of the dynamic change of the eye in the fourth time interval An estimation device comprising features according to the degree of difference. An input feature based on at least one of heart rate and pulse characteristics obtained in the first time interval and a physiological feature other than heart rate and pulse characteristics is applied to the prediction model, and at least the heart rate and heart rate in the second time interval. a estimating method for obtaining the other hand the estimation information corresponding to,
The characteristics of at least one of the heart rate and pulse and physiological characteristics other than the heart and pulse characteristics are values according to the occurrence time of the dynamic change of the eye and values according to the occurrence time of the heart and / or pulse. An estimation method that includes features based on relative values of and.   An input feature based on at least one of heart rate and pulse characteristics obtained in the first time interval and a physiological feature other than heart rate and pulse characteristics is applied to the prediction model, and at least the heart rate and heart rate in the second time interval. An estimation method for obtaining estimation information according to one of
  The features of at least one of the heartbeat and the pulse and physiological features other than the features of the heartbeat and the pulse are from the first interval from the occurrence of the heartbeat to the occurrence of the dynamic change of the eye and the occurrence of the dynamic change of the eye The second pulse from the next pulse to the next heartbeat from the occurrence of the third pulse from the occurrence of the dynamic change of the eye and the third pulse from the occurrence of the dynamic change of the eye An estimation method including at least one of a fourth interval to occurrence and a feature according to the fourth interval to the occurrence.   An input feature based on at least one of heart rate and pulse characteristics obtained in the first time interval and a physiological feature other than heart rate and pulse characteristics is applied to the prediction model, and at least the heart rate and heart rate in the second time interval. An estimation method for obtaining estimation information according to one of
  The physiological feature is a feature of the dynamic change of the eye in the third time interval and a feature of the third time interval estimated from the feature of the dynamic change of the eye in the fourth time interval An estimation method that includes features according to the degree of difference. A program for causing a computer to function as the estimation device according to any one of claims 4 to 6 .

Images