JP7644466B2 - Detection device and detection method - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act. Disclosure fact 1: Published on the website of the proceedings of The ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2019) on September 4, 2019 (https://dl.acm.org/conference/ubicomp). Disclosure fact 2: Presented at the workshop of The ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2019) on September 9, 2019. Public fact 3: Published on the website of the Information Processing Society of Japan Interaction 2020 proceedings (https://www.interaction-ipsj.org/proceedings/2020/) on March 2, 2020. Public fact 4: Presented at Information Processing Society of Japan Interaction 2020 (held online (https://www.youtube.com/user/interactionipsj/live)) on March 11, 2020.

This disclosure relates to a detection device and a detection method.

Eye activity can be an indicator of mental and physical activity. Mental and physical activity includes the activity of a person's mind and body. Therefore, the results of sensing eye activity can be useful information in many fields. In recent years, there has been a desire for eye activity sensing technology to be available to the general public at all times in many situations in their daily lives.

Eye activity includes eyelid movement and eyeball movement, and the number and interval of blinks, the degree of eye opening, and the movement and direction of the eyeballs are sensed. As an example of technology for sensing eye activity, International Publication WO2016/080042 (hereinafter, Patent Document 1) discloses a glasses-type detection device with multiple infrared sensors attached to the frame, which detects eyeball movement by measuring the eyeballs of the user wearing the glasses with infrared rays.

International Publication No. WO2016/080042

Infrared sensors such as those disclosed in Patent Document 1 directly detect eye movements to detect the number of blinks, eye vibrations, eye direction when the eyes are closed, and the like, but they are known to have low recognition and detection rates.

On the other hand, while it is possible to improve detection accuracy by using a high-precision detection device, eye activity is widely useful information, and so there is a demand to be able to detect and obtain it on a daily basis, and so a device that is simple both mechanically and in terms of cost is desirable. For this reason, there is a demand for a detection device and detection method that can detect eye activity with high accuracy using a simple device.

According to one embodiment, the detection device detects eye activity and includes a sensor capable of detecting the movement of the skin at multiple locations near the eye, and a calculation unit that calculates eye activity using a signal from the sensor.

According to one embodiment, the detection method senses the movement of the skin at multiple locations near the eye and determines the eye activity based on the sensing results.

Further details are described in the embodiments below.

FIG. 1 is a schematic diagram showing a specific example of the configuration of a detection device according to an embodiment. FIG. 2 is a schematic block diagram showing an example of the functional configuration of a calculation device that functions as a calculation unit of the detection device. FIG. 3 is a flowchart illustrating an example of a detection method according to an embodiment. FIG. 4 is a diagram showing a method for learning learning data when the eye activity to be detected is a blink. FIG. 5 is a graph showing the sensing results by the sensor during blinking. FIG. 6 is a diagram for explaining an outline of an evaluation experiment conducted by the inventor to verify the blink detection accuracy. FIG. 7 is a graph showing the results of an evaluation experiment conducted by the inventor to verify the blink detection accuracy. FIG. 8 is a diagram showing a method of learning learning data when the eye activity to be detected is the direction of the eyeball. FIG. 9 is a diagram for explaining an outline of an evaluation experiment conducted by the inventor to verify the detection accuracy of the eyeball direction. FIG. 10 is a graph showing the results of an evaluation experiment conducted by the inventor to verify the detection accuracy of the eyeball direction. FIG. 11 is a graph showing the results of an evaluation experiment conducted by the inventor to verify the detection accuracy of the eyeball direction. FIG. 12 is a diagram showing a method of learning learning data when the eye activity to be detected is the eyeball movement direction. FIG. 13 is a diagram for explaining an outline of an evaluation experiment conducted by the inventor to verify the detection accuracy of the eyeball movement direction. FIG. 14 is a graph showing the results of an evaluation experiment conducted by the inventor to verify the detection accuracy of the eyeball movement direction. FIG. 15 is a graph showing the results of an evaluation experiment conducted by the inventor to verify the detection accuracy of the eyeball movement direction.

＜1. Overview of the detection device and detection method＞

(1) A detection device according to one embodiment is a detection device that detects eye activity, and includes a sensor capable of detecting the movement of the skin at multiple locations near the eye, and a calculation unit that calculates the eye activity using a signal from the sensor. Eye activity includes eyelid movement, eyeball movement, and the like. Specifically, it includes blinking, eyeball direction, and eyeball movement direction. The multiple locations near the eye are, for example, around the eye, such as above and below the eye, or multiple locations above and below the eye.

The skin at multiple locations near the eye is affected by eye activity. The inventors have found that it is therefore possible to detect eye activity by detecting the movement of this skin. The movement of the skin can be obtained, for example, by measuring the distance to the skin. In this case, the sensor required to detect eye activity is a sensor for measuring the distance to the skin at multiple locations near the eye, such as an infrared distance sensor. This makes it cheaper than devices that use electrical potential or perform image analysis to detect eye activity. Furthermore, the fact that it can be detected with high accuracy has been verified through evaluation experiments conducted by the inventors, which will be described later.

(2) Preferably, the calculation of eye activity by the calculation unit includes using one or more detection results of skin movement from the skin movements at multiple locations detected by the sensor for use in calculating the eye activity. This allows eye activity to be detected with high accuracy.

(3) Preferably, the eye activity includes eye movement, and the calculation unit determines eye movement from the skin movements at multiple locations detected by the sensor using the detection results of multiple skin movements.

(4) Preferably, determining eye movement includes inputting feature values obtained from the detection results of multiple skin movements into a computational model and obtaining an output from the computational model, the computational model being a machine learning model obtained using feature values representing multiple skin movements as input values and eye movement corresponding to the multiple skin movements as output values as learning data. Eye movement is, for example, the orientation of the eye and the direction of eye movement. By using the machine learning model, eye movement can be detected easily and with high accuracy.

(5) Preferably, the sensor includes a distance sensor that detects the distance to the skin. The distance sensor is, for example, an infrared distance sensor. This allows the detection device to be constructed inexpensively.

(6) Preferably, the multiple locations near the eyes include above the eyes and below the eyes. This allows eye activity to be detected with high accuracy. This allows eye activity to be detected with high accuracy.

(7) Preferably, the multiple locations near the eyes include multiple locations above and below the eyes. This allows eye activity to be detected with high accuracy.

(8) Preferably, the detection device is in the form of glasses that can be worn on the subject's face so that the detection range of the sensor is in the vicinity of the eyes. By making the device in the form of glasses, the subject can easily wear the device and the burden of wearing it for long periods of time can be reduced. This makes it possible to detect eye activity over a long period of time.

(9) A detection method according to one embodiment senses the movement of the skin at multiple locations near the eyes and determines eye activity based on the sensing results. By using this detection method, the detection device can be made inexpensive, and the fact that eye activity can be detected with high accuracy has been verified through evaluation experiments conducted by the inventors described below.

2. Examples of detection devices and methods

The detection device 100 according to this embodiment is a detection device that detects eye activity. Eye activity includes eyelid movement and eyeball movement. Specifically, it includes blinking, eyeball direction, and eyeball movement direction.

Referring to FIG. 1, the detection device 100 according to this embodiment has a plurality of sensors 31-1 to 32-12. These sensors 31-1 to 32-12 constitute a glasses-type sensor. Specifically, as shown in FIG. 1, the sensors 31-1 to 31-12 are attached to the rim of the right-eye lens LA of the glasses 30, and the sensors 32-1 to 32-12 are attached to the rim of the left-eye lens LB of the glasses 30. The glasses-type sensors are easy for the user to wear and can reduce the burden of wearing them for long periods of time. This makes it possible to detect eye activity over a long period of time.

Sensors 31-1 to 32-12 are sensors capable of detecting the movement of skin at multiple locations near the eye. The multiple locations near the eye include at least the areas above and below the eye. Preferably, the multiple locations near the eye include multiple locations above and below the eye. As an example, as shown in FIG. 1, sensors 31-1 to 31-12 and sensors 32-1 to 32-12 are arranged on the rims of the right-eye lens LA and the left-eye lens LB of the glasses 30 so as to be approximately evenly distributed around the eye. Specifically, sensors 31-1 to 31-5 are arranged evenly above the right eye from the inner corner to the iris, sensors 31-7 to 31-11 are arranged evenly below the right eye from the inner corner to the inner corner, sensor 31-6 is arranged next to the iris, and sensor 31-12 is arranged next to the inner corner.

Detecting skin movement with sensors 31-1 to 32-12 includes, as an example, measuring the distance to the skin. As a specific example, sensors 31-1 to 32-12 are infrared distance sensors. By using infrared distance sensors and detecting eye activity using the measured distance as described below, a device can be constructed more inexpensively than when using electrical potentials or performing image analysis.

Sensors 31-1 to 31-12 are positioned near the eyes to measure the distance to the skin at multiple points near the eyes. As a specific example, as shown in the A-A cross-sectional view of Figure 1, sensor 31-3 positioned above the right eye measures the distance to the upper eyelid of the right eye, and sensor 31-9 positioned below the right eye measures the distance to the lower eyelid of the right eye.

The detection device 100 further includes a calculation device 10. The calculation device 10 is connected wirelessly or by wire to the sensors 31-1 to 32-12 and receives sensor signals indicating sensing results from the sensors 31-1 to 32-12. The sensor signals indicate the distance to the skin at each position. The calculation device 10 functions as a calculation unit that calculates eye activity using the signals from the sensors 31-1 to 32-12.

The arithmetic device 10 may be a general computer as shown in FIG. 1, or may be a server connected to the sensors 31-1 to 32-12 via a communication network such as the Internet. The arithmetic device 10 may also be a terminal device such as a smartphone, or may be a small arithmetic device that can be mounted on the glasses 20.

Referring to FIG. 2, the computing device 10 is configured as a computer having a processor 11 and a memory 12. The processor 11 is, for example, a CPU. The memory 12 includes a flash memory, an EEPROM, a ROM, a RAM, etc. Alternatively, the memory 12 may be a primary storage device or a secondary storage device. The memory 12 stores a program 122 to be executed by the processor 11.

The computing device 10 further includes a communication device 13 for receiving sensor signals from the sensors 31-1 to 32-12. The communication device 13 receives the sensor signals from the sensors 31-1 to 32-12 and passes them to the processor 11. In other words, the communication device 13 functions as an input unit for the sensor signals from the sensors 31-1 to 32-12.

Preferably, the memory 12 stores user data 121. The user data 121 includes identification information of a learning model 115 used in the determination process 114 described below for each user (hereinafter, subject) who wears the glasses 30 to detect eye activity. In this case, the processor 11 can use the learning model 115 corresponding to the subject when executing the determination process 114.

The processor 11 executes a program 122 stored in the memory 12 to execute a detection process for detecting the eye activity of the subject. The detection process includes a feature extraction process 111. The feature extraction process 111 is a process for extracting features from the sensor signals received by the communication device 13 from the sensors 31-1 to 32-12. The processor 11 extracts features according to the type of eye activity to be detected. At this time, the processor 11 uses multiple sensor signals from the sensors 31-1 to 32-12, and as an example, uses sensor signals from all the sensors. Details will be described later.

The detection process further includes a determination process 114. The determination process 114 is a process for determining the presence or absence and content of eye activity of the detection target. In detail, if the eye activity of the detection target is blinking, the determination of eye activity includes determining the blinking, and the determination of the blinking includes determining whether a blink occurred and determining the number of blinks based on whether a blink occurred. If the eye activity of the detection target is eye movement, the determination of eye activity includes determining the eye movement, and the determination of the eye movement includes determining the orientation of the eye and the direction in which the eye moves.

Preferably, the determination process 114 uses a learning model 115. Specifically, the determination process 114 includes inputting the features extracted in the feature extraction process 111 to the learning model 115 and obtaining an output from the learning model 115.

The learning model 115 is a machine learning model trained by machine learning using training data of input data T1 and output data T2, as shown in FIG. 4 etc. The input data T1 is a feature obtained from the sensor signals of the sensors 31-1 to 32-12, and is a feature corresponding to the eye activity of the detection target, as described below. Details will be described later for each eye activity of the detection target. Preferably, the learning model 115 is trained by machine learning using training data corresponding to the subject. This is expected to improve detection accuracy.

Preferably, the detection process includes an output control process 117. The output control process 117 is a process for controlling the output device 14 included in the computing device 10 so that the output device 14 outputs the determination result of the determination process 114. The output device 14 is a display, a speaker, a transmission device to another device, etc. This allows the detection result of the detection device 100 to be grasped from the outside.

The detection device 100 detects the eye activity of the subject using the detection method shown in the flowchart of Figure 3. In more detail, referring to Figure 3, the sensors 31-1 to 32-12 measure the distance to the skin at multiple points near the eyes of the subject wearing the glasses 30 (step S1). The sensor signals from the sensors 31-1 to 32-12 indicating the measured distances are input to the calculation device 10 (step S2).

The detection device 100 is capable of detecting multiple types of eye activity, such as blinking, eye direction, and eye movement direction, and has verified that each of these can be detected with high accuracy in evaluation experiments performed by the inventors described below. Therefore, preferably, the calculation device 10 accepts a designation of the eye activity to be detected, and differs in the calculations in subsequent processing depending on the designation.

In the computing device 10 that has received the sensor signal input, the processor 11 executes the program 122 to perform a process of extracting features from the sensor signal (step S3). Preferably, in step S3, the processor 11 extracts features using multiple sensor signals from the sensors 31-1 to 32-12. As described above, when a designation of the type of eye activity to be detected is received, in step S3 the processor 11 extracts features according to the type of eye activity to be detected.

The processor 11 uses the features extracted in step S3 to determine the eye activity of the subject (step S4). Specifically, in step S4, the features extracted in step S3 are input to the learning model 115 (step S41), and an output value is obtained from the learning model 115 (step S42). At this time, if a designation of the type of eye activity to be detected is accepted, the processing of step S4 includes selecting a learning model 115 that has been machine-learned according to the type of eye activity to be detected. Also, if a learning model 115 is prepared for each subject, the processing of step S4 includes selecting a learning model 115 according to the subject.

(blink detection)

Here, we will explain the case where blinking is detected as eye activity. With reference to FIG. 4, when blinking is detected as eye activity, as an example of the teacher data used for machine learning in the learning model 115, the feature amount obtained from the sensor signals of the sensors 31-1 to 32-12 that sense the area around the eyes at the time of blinking is used as input data T1, and the determination result of "blinking occurred" is used as output data T2.

Curve L1 in Figure 5 represents the sensing results from one of sensors 31-1 to 32-12 during a blink. In Figure 5, the vertical axis represents the normalized measurement results of the distance to the skin, and the horizontal axis represents time. In other words, Figure 5 represents the change over time in the distance to the skin at a position near the eye during a blink. Curve L2 represents the differential value of curve L1.

By analyzing curves L1 and L2, the inventors discovered that in the section where the differential value shown by curve L2 is negative, the eyelids close, and in the section where the differential value is positive, the eyelids open. From this, they discovered that the point where the differential value switches between positive and negative marks the turning point of the eyelid opening and closing movement. Specifically, in Figure 5, point P1 corresponds to the start of blinking, point P2 corresponds to the end of blinking, and point P3 corresponds to the turning point, i.e., the point where the eyelids open.

In the detection device 100 according to the present embodiment, the learning model 115 is machine-learned by utilizing this principle discovered by the inventor. That is, the curve L1 itself of the sensing result or the value of the curve L2, which is its differential value, is used as the input data T1 as the feature, and the point where the differential value changes from positive to negative is the blink start point P1, the point where it changes from negative to positive is the blink turning point P3, and the point where the positive change becomes smaller than a predetermined value is the blink end point P2, which are machine-learned as one blink.

Note that during machine learning, a threshold may be set for the value of curve L1 in the range where the value of curve L1 is region R, and points above the threshold that are detected as turning points of blinking may be removed as noise during machine learning. This is because the range where the value of curve L1 is region R is when the eyelids are open, and therefore there are no turning points of blinking. This can improve the accuracy of blink detection.

Preferably, among the sensors 31-1 to 32-12, sensors for which the difference in sensed distance is equal to or greater than a threshold value are used for machine learning. This makes it possible to use the sensing results of positions near the eyes where the distance to the skin is likely to be affected by blinking, improving the accuracy of blink detection. The sensor used may be one or more positions experimentally determined for blinking, or may be experimentally determined for each subject. In the latter case, the user data 121 includes identification information of the learning model 115 corresponding to the subject.

When detecting blinks, the processor 11 extracts feature amounts, which are differential values, from the sensor signals of a specific sensor among the sensors 31-1 to 32-12, and inputs them to the learning model 115 that has been trained as shown in FIG. 4. The processor 11 detects blinks by obtaining an output value of "blink occurred" from the learning model 115. The processor 11 also detects the number of blinks by inputting a series of feature amounts obtained from a series of sensor signals and counting the output values of "blink occurred" obtained from the learning model 115.

The inventors conducted an evaluation experiment to verify the blink detection accuracy of the detection device 100. Specifically, as shown in FIG. 6, subject H sat facing the display 200 and looked straight ahead, gazing at each of the 15 points displayed on the display 200 without changing the direction of his face, blinked 20 times at a rate of once per second, and the duration of blinking was sensed.

At this time, the sensor with the highest blink detection accuracy among the sensors 31-1 to 32-12 was selected as the detection sensor based on the sensing results when the subject H blinked. The sensing results of the subject H using the detection sensor were used to train the learning model 115.

Figure 7 shows the accuracy rate for each of the 15 points as a result of the evaluation experiment. The accuracy rate is the ratio of the number of blinks detected to the number of blinks (20), with 100% indicating that all 20 blinks were detected. As shown in Figure 7, with the detection device 100, the accuracy rate was 100% or close to 100% at almost all points. Therefore, it was verified that blinks can be detected with high accuracy by using this detection device 100.

(Detection of eye movement (eye direction))

Next, we will explain the case of detecting eye movement, particularly the direction of the eye, as eye activity. The direction of the eye corresponds to the direction of gaze. The inventors discovered that when the eye is facing a specific direction, the skin near the eye changes in a way that is specific to that direction, and therefore the direction of the eye is expressed by the sensing result of the distance to the skin.

When detecting eye direction as eye activity, referring to FIG. 8, the training data used for machine learning in the learning model 115 uses the feature values obtained from the sensor signals of the sensors 31-1 to 32-12 that sense the area around the eyes with the eyes facing in a specific direction as input data T1, and uses that specific direction as output data T2.

As an example, referring to FIG. 8, the eye direction is divided into nine parts by directing the eye to each of the multiple marks 1-1 to 3-3 arranged on the display 200, and the feature amount in each direction is set as input data T1. The feature amount used in this case represents the condition of the skin in multiple locations near the eye, and as an example, is the sensor value obtained at the same time from each of the sensors 31-1 to 32-12, as shown in the line graph in FIG. 8. Preferably, the sensor signals are smoothed to reduce noise before use. The graph in FIG. 8 shows the sensor values from each sensor for each direction.

When detecting the direction of the eyeball, the processor 11 uses the sensor values obtained simultaneously from the sensors 31-1 to 32-12 as features and inputs them to the machine-learned learning model 115 as shown in FIG. 8. The processor 11 detects the direction of the eyeball by obtaining the directional output value from the learning model 115.

The inventors conducted an evaluation experiment to verify the accuracy of eyeball orientation detection by the detection device 100. Specifically, as shown in FIG. 9, subject H sat facing the display 200 and looked straight ahead, gazing at each of 15 points displayed on the display 200 for 30 seconds without changing the orientation of his face, and this period was sensed. The angular change between each point was approximately 11 degrees. The sensing results of subject H using the detection sensor were used to train the learning model 115.

Figure 10 shows the recognition accuracy for each of the 15 points as a result of the evaluation experiment. The recognition accuracy is expressed as the F-score when using each of the classifiers: K Nearest Neighbor (Knn), support vector machine (SVM), decision tree, and random forest. The right side of Figure 10 shows the average of the 15 points. Figure 11 shows the average F-score of the 15 points for each classifier, and is an enlarged version of the results at the right side of Figure 10.

The results in Figure 10 show that the F-measure for the highly accurate classifier for each direction was approximately 0.9. The results in Figure 11 also show that there is variation in the evaluation by the classifier. Therefore, this evaluation experiment verified that the eye direction can be detected with high accuracy by using this detection device 100.

(Detection of eye movement (direction of eye movement))

Next, we will explain the case of detecting eye movement, particularly the direction of eye movement, as an example of eye activity. From the perspective that the direction of eye movement is represented by a change in the orientation of the eye, the inventors have discovered that the direction of eye movement can be obtained from the amount of change in the sensor value over a short time interval, based on the principle they discovered that the orientation of the eye is represented by the sensing results of the distance to the skin.

Therefore, when detecting the direction of eye movement as eye activity, referring to FIG. 12, the training data used for the machine learning of the learning model 115 is, for example, a feature value when the eye is moved in a specific direction, such as up, down, left, or right, and the like, is used as input data T1. The feature value used in this case represents a change in the condition of the skin at multiple locations near the eye, and is, for example, the difference between the first sensor value and the last sensor value for a certain period for each of the sensors 31-1 to 32-12, as shown in the line graph of FIG. 12. Preferably, the sensor signal is smoothed to reduce noise before use. The certain period is, for example, about 0.3 seconds. The graph in FIG. 12 shows the difference between the first sensor value and the last sensor value for a certain period from each sensor for up, down, left, and right.

When detecting the eye movement direction, the processor 11 extracts the above-mentioned features from the sensor values obtained from the sensors 31-1 to 32-12 over a certain period of time, and inputs them into the machine-learned learning model 115 as shown in FIG. 12. The processor 11 detects the eye movement direction by obtaining an output value of the movement direction from the learning model 115.

The inventors conducted an evaluation experiment to verify the accuracy of eye direction detection by the detection device 100. Specifically, as shown in FIG. 13, subject H, sitting facing the display 200 and facing forward, moved his/her gaze between two of the four points shown on the display 200 without changing the direction of his/her face, and the period of movement was sensed. Each movement took 2-3 seconds, and was repeated a total of 20 seconds in each direction. The sensing results of subject H using the detection sensor were used to train the learning model 115.

Figure 14 shows the recognition accuracy for the downward, leftward, rightward, and upward directions as a result of the evaluation experiment. The recognition accuracy is expressed as the F-score when using each of the following classifiers: K Nearest Neighbor (Knn), support vector machine (SVM), decision tree, and random forest. Figure 15 shows the average F-score for each classifier in the four directions.

The results in Figure 14 show that the F-measure for the highly accurate classifier in each direction was approximately 0.9. The results in Figure 15 also show that there is not much variation in the evaluation by the classifier when averaging the four directions. Therefore, this evaluation experiment verified that the eye movement direction can be detected with high accuracy by using this detection device 100.

<3. Notes>
The present invention is not limited to the above-described embodiment, and various modifications are possible.

10: Calculation device 11: Processor 12: Memory 13: Communication device 14: Output device 30: Glasses 31-1: Sensor 31-2: Sensor 31-3: Sensor 31-4: Sensor 31-5: Sensor 31-6: Sensor 31-7: Sensor 31-8: Sensor 31-9: Sensor 31-10: Sensor 31-11: Sensor 31-12: Sensor 32-1: Sensor 32-2: Sensor 32-3: Sensor 32-4: Sensor 32-5: Sensor 32-6: Sensor 32-7: Sensor 32-8: Sensor 32-9: Sensor 32-10: Sensor 32-11: Sensor 32-12: Sensor 100: Detection device 111: Feature extraction process 114: Judgment process 115: Learning model 117 : Output control process 121 : User data 122 : Program 200 : Display H : Subject LA : Right eye lens LB : Left eye lens P1 : Start point P2 : End point P3 : Turning point R : Region T1 : Input data T2 : Output data

Claims (5)

1. A detection device for detecting eye activity, comprising:
a plurality of infrared distance sensors for detecting distances to a plurality of skin locations near the eye;
a calculation unit that calculates the eye activity using signals from the plurality of infrared distance sensors;
The eye activity calculated by the calculation unit includes a gaze movement direction,
The calculation unit is
receiving a plurality of sensor signals indicative of respective sensing results of the distances to the plurality of skin locations proximate the eye;
extracting features representing changes in skin condition at the plurality of locations from the plurality of sensor signals;
the feature amount includes a difference between an initial sensor signal and a final sensor signal during a certain period from the plurality of infrared distance sensors;
The feature amount representing the change in the state of the skin at the plurality of locations is input to a computation model, and a determination result of the movement direction of the gaze is obtained as an output of the computation model,
The computational model is a machine learning model that has, as learning data, feature values representing changes in skin condition at the multiple locations as input values and gaze movement directions corresponding to the feature values as output values;
the determination result includes a determination result of an upward direction, a downward direction, a leftward direction, and a rightward direction as a determination result of a movement direction of the line of sight,
The plurality of locations near the eye include a plurality of locations above the eye and below the eye in the left and right directions, and locations on the left and right sides of the eye,
The feature amount represents a state change of a plurality of locations above and below the eye in the left and right directions, and each location on the left and right sides of the eye.
Detection device. The detection device according to claim 1 , wherein the eye activity calculated by the calculation unit further includes blinking and gaze direction . The detection device according to claim 2 , wherein the feature amount input to the computational model is a feature amount corresponding to a type of the eye activity to be determined. The detection device according to claim 1 , wherein the detection device is in the form of glasses that can be worn on the subject's face so that the detection range of the infrared distance sensor is in the vicinity of the eyes. sensing distances to a plurality of skin points near the eye using a plurality of infrared distance sensors;
a calculation unit receiving a plurality of sensor signals indicating the results of the sensing by the plurality of infrared distance sensors;
The calculation unit extracts feature amounts representing changes in skin conditions at the plurality of locations from the plurality of sensor signals,
the calculation unit inputs the feature amounts representing the changes in the skin condition at the plurality of locations into a calculation model, and obtains a determination result of the gaze movement direction as an output of the calculation model;
Prepare for this.
the feature amount includes a difference between an initial sensor signal and a final sensor signal during a certain period from the plurality of infrared distance sensors;
The computational model is a machine learning model that has, as learning data, feature values representing changes in skin condition at the multiple locations as input values and gaze movement directions corresponding to the feature values as output values;
the determination result includes a determination result of an upward direction, a downward direction, a leftward direction, and a rightward direction as a determination result of a movement direction of the line of sight,
The plurality of locations near the eye include a plurality of locations above the eye and below the eye in the left and right directions, and locations on the left and right sides of the eye,
The feature amount represents a state change of a plurality of locations above and below the eye in the left and right directions, and each location on the left and right sides of the eye.
Detection methods.

Images