JP6904269B2 - Auditory attention estimation device, auditory attention estimation method, program - Google Patents

Description

The present invention relates to a technique for estimating the auditory attention of a subject to whom a plurality of sounds are presented.

When trying to objectively measure which position the subject is paying attention to among the information existing at various positions in the space, for example, "look and listen in the direction in which you want to pay attention". When accompanied by a conscious movement such as "moving", it is relatively easy to estimate the direction of attention from that movement. However, it is difficult to presume that "consciousness is concentrated in a particular direction" if the movements that can be grasped are not expressed from such an appearance.

Non-Patent Document 1 discloses that the direction in which visual attention is directed can be estimated from the information on the size of the pupil of the eye when it is not expressed as a directional movement. In Non-Patent Document 1, under the condition that the subject is presented with images having different brightness in the right half and the left half and is instructed to look at one point in the central part, conscious attention is paid to either the right side or the left side. It has been reported that when instructed to turn, the size of the pupil changes according to the brightness of the image in the direction in which the attention is directed. For example, when the right half is a black image and the left half is a white image, the size of the pupil increases when the right side is focused, and the size of the pupil decreases when the left side is focused.

Conventionally, it has been known that the size of the pupil increases in a dark place (called mydriasis) and decreases in a bright place (called miosis). In Non-Patent Document 1, the size of the pupil was adapted to the darkness in advance by consciously moving the eye in the dark direction (let's pay attention), although the eye was not actually moved in the dark direction or the bright direction. It is thought that something like a preparatory action that is about to change in size is appearing.

Sebastiaan Mathot, Lotje van der Linden, Jonathan Grainger, Francoise Vitu, “The Pupillary Light Response Reveals the Focus of Covert Visual Attention”, PLOS ONE, Vol. 8, Issue 10, e78168, 2013.

Using Non-Patent Document 1, it is possible to estimate the position (or direction) of attention in the visual information presented as two-dimensional information. In other words, it is possible to estimate the direction in which attention is visually directed. However, since the method of estimating the direction of auditory attention is not known, there is a problem that the direction of auditory attention cannot be estimated.

In addition, it is difficult to objectively evaluate not only the estimation of the direction in which attention is paid aurally, but also the fact that attention is not paid or cannot be heard in the first place. For example, as a conventional hearing test method, a method of performing a task in which the subject keeps pressing a button while listening to a sound is known. In this case, even if the subject can hear the sound, the subject can perform the task. It is possible to behave as if it is not heard by intentionally not pressing a button, and it is difficult to evaluate whether or not it is really heard. In addition, there is a similar problem in the evaluation of hearing of a subject whose task is difficult to understand, such as an infant.

Hereinafter, "auditory attention" will be used as meaning the direction of auditory attention and whether or not it is auditory attention (including the case where it is not heard). It can be said that the above two problems are problems of estimating auditory attention.

Therefore, an object of the present invention is to provide a technique for estimating the auditory attention of a subject.

One aspect of the present invention includes first biometric information, which is time-series information on the size of the pupil in a state where the subject does not pay attention to the sound perceived by the left and right ears, and the subject is either left or right. Regarding the biometric information acquisition unit that acquires the second biometric information, which is the time-series information of the pupil size in the state of paying attention to the sound perceived by the ear, and the two or more regions included in the subject's visual field. When the second biological information is acquired by using the region information acquisition unit that acquires the region information which is an index value indicating the brightness of the region, the first biological information, the second biological information, and the region information. It includes an auditory attention estimation unit that estimates the auditory attention direction of the subject.

One aspect of the present invention is an acoustic information presenting unit that presents different sounds to the subject from two or more different directions, and a size of the pupil of the subject in a state where no sound is presented from the acoustic information presenting unit. Acquisition of biometric information that acquires the first biometric information, which is the time-series information, and the second biometric information, which is the time-series information of the size of the pupil of the subject in the state where the sound is presented by the acoustic information presenting unit. A region information acquisition unit that acquires region information that is an index value indicating the brightness of the region, the first biological information, the second biological information, and the above for two or more regions included in the visual field of the subject. It includes an auditory attention estimation unit that estimates the sound most easily attracting auditory attention among the sounds presented by the acoustic information presentation unit using the area information.

One aspect of the present invention is an acoustic information presenting unit that presents sound to one of the left and right ears of the subject and does not present sound to the other ear, and either the left or right of the subject from the acoustic information presenting unit. The first biological information, which is the time-series information of the size of the pupil of the subject in the state where the sound is not presented to the ears of the subject, and the sound from the acoustic information presentation unit to either the left or right ear of the subject. Regarding the biometric information acquisition unit that acquires the second biometric information, which is the time-series information of the pupil size of the subject in the presented state, and the left and right regions included in the subject's visual field, The first ear is a region information acquisition unit that acquires region information that is an index value indicating brightness, and the ear to which sound is presented from the acoustic information presentation unit is defined as the first ear, and the first biometric information and the second biometric information. The auditory attention estimation unit includes an auditory attention estimation unit that estimates whether or not the sound presented to the first ear is heard by the subject using the region information, and the auditory attention estimation unit is the first living body. When there is a change in the relative amount of change in the information and the second biometric information based on the first biometric information calculated from the second biometric information, the sound presented to the first ear is heard by the subject. Presumed to be.

According to the present invention, it is possible to estimate the auditory attention of the subject.

The figure which shows the procedure of an experiment. The figure which shows the state of the task of an experiment. The figure which shows the experimental result. The figure which shows an example of the still image which the brightness differs on the left and right. The block diagram which shows an example of the structure of the auditory attention estimation device 100/101/102. The flowchart which shows an example of the operation of the auditory attention estimation device 100/101/102. The block diagram which shows an example of the structure of the auditory attention estimation device 103. The flowchart which shows an example of the operation of the auditory attention estimation device 103. The block diagram which shows an example of the structure of the auditory attention estimation device 200. The flowchart which shows an example of the operation of the auditory attention estimation device 200. The block diagram which shows an example of the structure of the auditory attention estimation device 201. The flowchart which shows an example of the operation of the auditory attention estimation device 201. The block diagram which shows an example of the structure of the auditory attention estimation device 300. The flowchart which shows an example of the operation of the auditory attention estimation device 300. The block diagram which shows an example of the structure of the auditory attention estimation device 301. The flowchart which shows an example of the operation of the auditory attention estimation device 301.

Hereinafter, embodiments of the present invention will be described in detail. The components having the same function are given the same number, and duplicate explanations will be omitted.

<Technical background>
An embodiment of the present invention, that is, an auditory attention estimator for estimating human auditory attention, is based on the discovery of the law of nature that there is a correlation between auditory attention and the size of the human pupil. Is.

Therefore, here, the discovery of the correlation between the auditory attention and the size of the human pupil, which is a premise of the embodiment of the present invention, will be described.

The procedure of the experiment is as follows (see FIG. 1).
(1) Place a display in front of the subject and present the image. As the image to be presented, as shown in FIG. 1, an image having different brightness in the right side region and the left side region is used. In addition, the subject should wear headphones.
(2) Instruct the subject to look near the center of the image. After that, different sounds (“dog bark” and “telephone sound” in Fig. 1) are played simultaneously from both ears, so it corresponds to the sound heard from either the right ear or the left ear. Instruct them to press the button to press as soon as possible.
(3) Instruct the subject to use the right or left ear, and after a predetermined time (5 seconds in FIG. 1), generate different sounds from the right and left of the headphones. During this period, the pupil diameter of the subject is measured.

FIG. 2 is a diagram showing a state of a task performed by a target person. In the task of FIG. 2A, for example, the following instructions are given.
(Instruction) First, you will hear the "right" or "left" instruction. Next, you will hear different sounds from both ears, so press the button for the sound presented to the instructed ear as soon as possible. Press the "M" button for "dog barking" and the "N" button for "phone ringtone".

In this task, after the instruction "right", the "dog bark" is presented to the right ear and the "phone sound" is presented to the left ear, so that the subject presses the "M" button.

Further, in the task of FIG. 2B, for example, the following instructions are issued.
(Instruction) First, you will hear the "right" or "left" instruction. Next, you will hear different sounds from both ears, so press the button for the sound presented to the instructed ear as soon as possible. Press the "N" button for "dog barking" and the "M" button for "phone ringtone".

In this task, after the instruction "right", the "dog bark" is presented to the right ear and the "phone sound" is presented to the left ear, so that the subject presses the "N" button.

That is, in the task of FIG. 2 (A) and the task of FIG. 2 (B), the buttons to be pressed are exchanged. The subject is asked to perform such a set of tasks a predetermined number of times, and the size of the pupil is measured.

FIG. 3 is a diagram showing the results of the above experiment. The horizontal axis of the upper figure of FIGS. 3 (A) and 3 (B) is the time, the "instruction" is the time when the instruction of the right ear or the left ear is issued, and the "reproduced sound" reproduces the sound from both ears. It is the time. The vertical axis is an index value indicating the size of the pupil, and the size of the pupil is expressed by z-score. Further, FIG. 3A is a result when an image in which the left half is black and the right half is white (that is, an image in which the right side is brighter than the left side) is presented, and FIG. 3B is the left half. Shows the result when an image with white and black on the right half (that is, the image on the left side is brighter than the image on the right side) is presented.

From this result, it can be seen that there is the following tendency.
(1) When the right ear is audibly focused and the image on the right is darker than the image on the left, the size of the pupil of the eye is large.
(2) When the image on the right side is brighter than the image on the left side when the right ear is focused aurally, the size of the pupil of the eye becomes smaller.
(3) When the left ear is audibly focused and the left image is darker than the right image, the size of the pupil of the eye is large.
(4) When the left ear is audibly focused and the left image is brighter than the right image, the size of the pupil of the eye becomes smaller.

In other words, the pupil size when the image on the auditory attention side is dark tends to be larger than the pupil size when the image on the auditory attention side is bright. I understand. It can also be seen that when about 2 seconds have passed from the time when the person was instructed to pay attention, the change in the size of the pupil according to the brightness of the image appears to the extent that it can be discerned.

From this result, it can be seen that the size of the pupil changes depending on the brightness in the direction in which the auditory attention is directed (although the eyes are not actually directed in that direction). .. In embodiments of the present invention, this discovery of the laws of nature is utilized.

In this experiment, the right half and the left half of the image were painted in white and black, but the images to be presented are not limited to this. That is, any image may have different brightness on the right side and the left side. For example, an image as shown in FIG. 4 may be used. In this image, the area on the left side is brighter than the area on the right side. Further, instead of dividing the image into two, the image may be divided into three or more regions having different brightness.

In addition, although a simple black-and-white still image is used in this experiment, the image to be presented is not limited to the still image, and different brightness (for example, brightness) can be set to close values in a certain region for each position. If it is (that is, each area can be a dark area, a bright area, etc.), it may be a moving image or an actual environment.

Hereinafter, still images, moving images, and the real environment are referred to as visual information.

<First Embodiment>
The auditory attention estimation device of the first embodiment estimates the direction in which the subject's auditory attention is directed, such as which of the left and right ears the subject is paying attention to.

Hereinafter, the auditory attention estimation device 100 will be described with reference to FIGS. 5 to 6. FIG. 5 is a block diagram showing the configuration of the auditory attention estimation device 100. FIG. 6 is a flowchart showing the operation of the auditory attention estimation device 100. As shown in FIG. 5, the auditory attention estimation device 100 includes a visual information presentation unit 110, an acoustic information presentation unit 120, a biological information acquisition unit 130, an area information acquisition unit 140, and an auditory attention estimation unit 150. Includes recording unit 190. The recording unit 190 is a component unit that appropriately records information necessary for processing.

Hereinafter, the operation of the auditory attention estimation device 100 will be described with reference to FIG.

[Visual information presentation unit 110]
In step S110, the visual information presenting unit 110 presents to the subject a still image composed of two or more regions having different index values (for example, brightness) indicating brightness. Therefore, the visual field of the subject includes two or more regions having different index values indicating brightness. A display device such as a display may be used for presenting the still image.

[Acoustic information presentation unit 120]
In step S120, the acoustic information presenting unit 120 presents different sounds to the subject from two or more different directions. In addition, information (specific information) for identifying the sound presented from each direction is output to the auditory attention estimation unit 150. For example, the acoustic information presentation unit 120 presents different sounds from the two directions of the right ear side and the left ear side, and information (sound) for identifying the presented sounds from the right ear side and the left ear side. Specific information) is output to the auditory attention estimation unit 150.

[Biological information acquisition unit 130]
In step S130, the biometric information acquisition unit 130 includes first biometric information, which is time-series information on the size of the pupil in a state where the subject is not paying attention to the sound perceived by the left and right ears, and the subject is left and right. The second biological information, which is the time-series information of the size of the pupil in the state of paying attention to the sound perceived by any of the ears, is acquired. Here, the state in which the subject does not pay attention to the sounds perceived by the left and right ears means that the subject's attention is neutral to any sound perceived by the left and right ears. For example, the sound is not presented to the target person by the acoustic information presentation unit 120. Further, the state in which the subject pays attention to the sound perceived by either the left or right ear is, for example, a state in which the sound is presented to the subject by the acoustic information presenting unit 120. The biological information acquisition unit 130 measures the size of the pupil of the subject's eye, and specifically, a predetermined time in a state where sound is not presented to the subject by the acoustic information presentation unit 120. First biometric information, which is time-series information on the size of the pupil of the subject for each time of the section, and each time of a predetermined time section in a state where sound is presented to the subject by the acoustic information presentation unit 120. The second biological information, which is the time-series information of the pupil size of the subject, is measured and output to the auditory attention estimation unit 150.

[Area information acquisition unit 140]
In step S140, the area information acquisition unit 140 acquires area information which is an index value indicating the brightness of the area for the area of the still image presented in S110, that is, each area included in the visual field of the target person, and the area is concerned. The information is output to the auditory attention estimation unit 150.

[Aural attention estimation unit 150]
In step S150, the auditory attention estimation unit 150 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S140 to acquire the second biological information, which is the auditory hearing of the subject. Estimate the direction in which attention is directed (auditory attention direction). Specifically, the following procedure is executed.

First, using the first biological information and the second biological information, the relative change amount of the second biological information based on the first biological information is calculated. The relative change amount is an index (hereinafter referred to as pupil diameter change information) indicating the magnitude of the change in the pupil diameter. For example, from the first biometric information, statistical information indicating the size of the pupil of the subject in a state where the subject does not pay attention to the sound perceived by the left and right ears (state in which no sound is presented) (here, the size of the pupil of the subject). , Using the mean and standard deviation), and using that information (that is, the mean and standard deviation), the z-score of the pupil size included in the second biometric information is the pupil diameter change information. Calculate as. When the second biological information includes the size of the pupil at multiple times, the z-score at each time is calculated, and the representative value (for example, the average value or the median value) of the z-score is changed in the pupil diameter. It should be information.

Next, using the region information, the auditory attention direction is estimated from the relative change amount (pupil diameter change information) and output. Specifically, if the pupil diameter change information is a positive value, the area of the still image presented by the visual information presenting unit 110 starts from a relatively dark area, that is, an area where the area information is small (for example, the brightness is low). The determined direction is estimated as the auditory attention direction. On the other hand, if the pupil diameter change information is a negative value, it is determined from the relatively bright region, that is, the region where the region information is large (for example, the brightness is high) among the regions of the still image presented by the visual information presentation unit 110. The direction is presumed to be the auditory attention direction. This estimation can be made by directing auditory attention toward relatively bright areas compared to the size of the pupil when the subject is not paying attention to the sounds perceived by the left and right ears. When the pupil size is small, the z-score is negative, and when the auditory attention is directed toward a relatively dark area, the pupil size is large. , Z-score is a positive value.

It should be noted that the pupil diameter change information does not estimate the auditory attention direction based on whether the pupil diameter change information is a positive value or a negative value, but allows a certain amount of error and the pupil diameter change information has a predetermined threshold value θ ₁ or more. In the case of, the direction determined from the relatively dark area is defined as the auditory attention direction, and when the pupil diameter change information is _{equal to or less than the predetermined threshold value θ 2} (however, θ ₂ <θ ₁ ), the direction determined from the relatively bright area is defined as It may be presumed to be the auditory attention direction.

Similarly, when the still image presented by the visual information presenting unit 110 includes three or more regions, the auditory attention direction is estimated based on the pupil diameter change information and the magnitude relationship of the threshold values using a plurality of threshold values. be able to. That is, it may be estimated that the larger the pupil diameter change information (indicating that the pupil becomes larger), the more the direction determined from the darker region becomes the auditory attention direction. For example, in the case of a still image containing three regions with different brightness, if the pupil diameter change information is θ _{1 or more using two thresholds θ 1} and θ 2 (however, θ ₁ > θ ₂ ), it is the most. The direction determined from the dark area is estimated to be the auditory attention direction _{, if it is θ 2} or more and _{less than θ 1} , the direction determined from the dark area is estimated to be the second auditory attention direction, and if it is less than _{θ 2, it is estimated to be the third.} The direction determined from the dark (that is, the brightest) area is estimated as the auditory attention direction.

[Modification 1]
Although the visual information presentation unit 110 presents a still image to the target person, the target person may view the actual environment (real environment) in which he / she is present.

Hereinafter, the auditory attention estimation device 101 will be described with reference to FIGS. 5 to 6. FIG. 5 is a block diagram showing the configuration of the auditory attention estimation device 101. FIG. 6 is a flowchart showing the operation of the auditory attention estimation device 101. As shown in FIG. 5, the auditory attention estimation device 101 includes an acoustic information presentation unit 120, a biological information acquisition unit 130, an area information acquisition unit 141, an auditory attention estimation unit 151, and a recording unit 190. That is, the auditory attention estimation device 101 includes the area information acquisition unit 141 and the auditory attention estimation unit 151 instead of the visual information presentation unit 110, the area information acquisition unit 140, and the auditory attention estimation unit 150. It is different from the estimation device 100.

Hereinafter, the operation of the auditory attention estimation device 101 will be described with reference to FIG. Here, the area information acquisition unit 141 and the auditory attention estimation unit 151 will be described.

However, the actual environment that the subject sees shall include a certain continuous range, that is, two or more regions, in which the index values indicating different brightness for each point are close to each other. As the index value indicating the brightness, for example, the illuminance measured by the illuminometer and the brightness measured by the luminance meter can be used.

[Area information acquisition unit 141]
In step S141, the area information acquisition unit 141 acquires area information which is an index value indicating the brightness of the area for the area of the real environment that the target person is viewing, that is, each area included in the field of view of the target person. The area information is output to the auditory attention estimation unit 150. The area information acquisition unit 141 measures the brightness of a point (representative point) representing the area included in the area, and sets the brightness of the representative point as the brightness of the area, that is, the area information.

[Aural attention estimation unit 151]
In step S151, the auditory attention estimation unit 151 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S141 to acquire the second biological information, which is the auditory hearing of the subject. Estimate the direction in which attention is directed (auditory attention direction). Specifically, the following procedure is executed.

First, using the first biological information and the second biological information, the relative change amount of the second biological information based on the first biological information is calculated. The calculation method may be the same as that of the first embodiment.

Next, using the region information, the auditory attention direction is estimated from the relative change amount (pupil diameter change information) and output. Specifically, it may be estimated that the larger the pupil diameter change information (indicating that the pupil becomes larger), the more the direction determined from the darker region becomes the auditory attention direction. For example, when the brightness is measured at points representing three regions, the first point P1, the second point P2, and the third point P3 are set in the order of darkness, and the two thresholds θ ₁ and θ ₂ (however, θ). _{Using 1} > θ ₂ ), if the pupil diameter change information is θ ₁ or more, the direction determined from the region including the darkest point P 1 is estimated as the auditory attention direction, and if it is _{θ 2} or more and _{less than θ 1, 2} The direction determined from the area including the darkest point P2 is estimated as the auditory attention direction, and _{if it is less than θ 2} , the direction determined from the area containing the third darkest (that is, the brightest) point P3 is the auditory attention direction. presume.

[Modification 2]
Although the visual information presenting unit 110 presents a still image to the subject, a moving image may be presented instead of the still image.

Hereinafter, the auditory attention estimation device 102 will be described with reference to FIGS. 5 to 6. FIG. 5 is a block diagram showing the configuration of the auditory attention estimation device 102. FIG. 6 is a flowchart showing the operation of the auditory attention estimation device 102. As shown in FIG. 5, the auditory attention estimation device 102 includes a visual information presentation unit 112, an acoustic information presentation unit 120, a biological information acquisition unit 130, an area information acquisition unit 142, and an auditory attention estimation unit 152. Includes recording unit 190. That is, the auditory attention estimation device 102 replaces the visual information presentation unit 110, the area information acquisition unit 140, and the auditory attention estimation unit 150 with the visual information presentation unit 112, the area information acquisition unit 142, and the auditory attention estimation unit 152. It differs from the auditory attention estimation device 100 in that it includes.

Hereinafter, the operation of the auditory attention estimation device 102 will be described with reference to FIG. Here, the visual information presentation unit 112, the area information acquisition unit 142, and the auditory attention estimation unit 152 will be described.

[Visual information presentation unit 112]
In step S112, the visual information presenting unit 112 presents to the subject a moving image including two or more regions having different index values (for example, brightness) indicating brightness. The moving image to be presented may be any moving image as long as the brightness is perceived differently for each area.

[Area information acquisition unit 142]
In step S142, the area information acquisition unit 142 acquires area information which is an index value indicating the brightness of the area for each area included in the visual field of the target person, that is, the area of the moving image presented in S122, and the area is concerned. The information is output to the auditory attention estimation unit 150. The area information acquisition unit 142 acquires an index value indicating the brightness of each area of the image of each frame constituting the moving image. In this case, the area information becomes the time series information of the index value. Instead of directly outputting the area information to the auditory attention estimation unit 150, the auditory attention estimation unit 150 may appropriately read the area information after recording it in the recording unit 190.

[Aural Attention Estimator 152]
In step S152, the auditory attention estimation unit 152 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S142 to acquire the second biological information, which is the auditory hearing of the subject. Estimate the direction in which attention is directed (auditory attention direction). Specifically, the following procedure is executed.

First, using the first biological information and the second biological information, the relative change amount of the second biological information based on the first biological information is calculated. The calculation method may be the same as that of the first embodiment.

Next, using the region information, the auditory attention direction is estimated from the relative change amount (pupil diameter change information) and output. Here, two methods will be described.
(Method 1)
For each frame, the auditory attention direction is estimated from the pupil diameter change information corresponding to the time of the frame. Since the auditory attention direction is estimated for each frame, the auditory attention direction can be estimated by the same method as the method used in the first embodiment. The time-series information of the auditory attention direction for each frame may be used as it is as the auditory attention direction, or the statistical value of the auditory attention direction for each frame included in a predetermined time interval (for example, the direction determined by a majority vote) may be used. It may be an auditory attention direction.
(Method 2)
Here, it is assumed that the moving image presented by the visual information presenting unit 112 is a moving image whose brightness changes in a different time pattern for each area. For example, it is a moving image in which the time pattern of the change in brightness in the left half region of the moving image and the time pattern of the change in brightness in the right half region are different.

In this case, based on the findings explained in <Technical Background>, when the attention is given to the left side aurally, the time pattern of the pupil diameter change information and the change in brightness of the left half region of the moving image When the correlation (or similarity) with the time pattern is higher than the correlation (or similarity) between the time pattern of the pupil diameter change information and the time pattern of the change in brightness in the right half region of the moving image. Conceivable. Similarly, when audibly focusing on the right side, the correlation (or similarity) between the time pattern of pupil diameter change information and the time pattern of brightness change in the right half of the moving image is better. , It is considered that the correlation (or similarity) between the time pattern of the pupil diameter change information and the time pattern of the change in brightness in the left half region of the moving image is higher.

Therefore, based on the correlation between the time pattern of the pupil diameter change information and the time pattern of the change in brightness of each region of the moving image (the time pattern of the region information), the time pattern of the pupil diameter change information is the most correlated. The direction determined from the area of the high moving image is estimated to be the auditory attention direction.

For example, the direction determined from the region where the brightness fluctuates in a time pattern having a high correlation with the pattern obtained by frequency analysis of the pupil diameter change information is estimated as the auditory attention direction.

Alternatively, as another method, the cross-correlation function of the pupil diameter change information (series of pupil diameters) and the area information of a certain region of the moving image (series of index values indicating brightness) is calculated. In general, the higher the cross-correlation function of the two series (referred to as series A and series B) has a peak near the time difference of 0, the higher the correlation between series A and series B. Therefore, here, the direction in which the cross-correlation function is determined from the region of the region information having a high peak near the time difference of 0 is estimated as the auditory attention direction.

Not limited to the method illustrated above, any method may be used as long as it can evaluate the correlation and similarity between the two series.

[Modification 3]
As described above, the auditory attention direction can be estimated by comparing the relative change amount and the threshold value calculated from the first biological information and the second biological information, but the relative change amount (or the first one). To estimate the auditory attention direction even by using a model learned by machine learning or a neural network so that the biological information and the second biological information) and the area information are input and the auditory attention direction is output. Can be done.

Hereinafter, a method of constructing an auditory attention direction estimation model will be described with respect to the first embodiment, the modified example 1, and the modified example 2. The auditory attention direction estimation model is the first biological information and the second biological information acquired from one or more subjects, the region information which is an index value indicating the brightness of the region included in the visual field of the subject, and the second biological information. Information based on the first biometric information and the second biometric information and area information learned using a set of learning data obtained based on the correct answer of the subject's auditory attention direction when the information was acquired are input. , A model that outputs the auditory attention direction. Here, the first biological information and the second biological information, the area information which is an index value indicating the brightness of the region included in the visual field of the subject, and the auditory attention direction of the subject when the second biological information is acquired. The learning data obtained based on the correct answer is a set of correct answers in the first biological information, the second biological information, the area information, and the auditory attention direction, or a relative calculated from the first biological information and the second biological information. It is a set of correct answers for the amount of change, area information, and auditory attention direction. Further, the information based on the first biological information and the second biological information is a set of the first biological information and the second biological information, or a relative change amount calculated from the first biological information and the second biological information. That is.

The learning data may be prepared by conducting an experiment similar to the experiment described in <Technical Background> for a plurality of subjects. For the first modification, the actual environment may be viewed instead of the still image, and for the second modification, the moving image may be presented instead of the still image.

Hereinafter, the auditory attention estimation device 103 will be described with reference to FIGS. 7 to 8. FIG. 7 is a block diagram showing the configuration of the auditory attention estimation device 103. FIG. 8 is a flowchart showing the operation of the auditory attention estimation device 103. As shown in FIG. 7, the auditory attention estimation device 103 includes the visual information presentation unit 110/112, the acoustic information presentation unit 120, the biological information acquisition unit 130, the area information acquisition unit 140/141/142, and the auditory information. The attention estimation unit 153 and the recording unit 190 are included. The recording unit 190 may record an auditory attention direction estimation model that has been learned in advance. In the auditory attention estimation device 103, when the experiment used to generate the learning data uses a still image, the visual information presentation unit 110 and the area information acquisition unit 140 are used in the actual environment. In the case of, the area information acquisition unit 141 is included, and in the case of using a moving image, the visual information presentation unit 112 and the area information acquisition unit 142 are included.

Hereinafter, the operation of the auditory attention estimation device 103 will be described with reference to FIG. Here, the auditory attention estimation unit 153 will be described.

[Aural attention estimation unit 153]
In step S153, the auditory attention estimation unit 153 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S140 / S141 / S142 to acquire the second biological information. Estimate the direction in which a person's auditory attention is directed (auditory attention direction). Specifically, it is as follows.
(When the input of the auditory attention direction estimation model is a set of the first biological information, the second biological information, and the area information)
The auditory attention estimation unit 153 estimates the auditory attention direction from the first biological information, the second biological information, and the area information by using the auditory attention direction estimation model.
(When the input of the auditory attention direction estimation model is a set of relative change amount and area information)
First, the auditory attention estimation unit 153 calculates the relative change amount of the second biological information based on the first biological information by using the first biological information and the second biological information. The calculation method may be the same as that of the first embodiment. Next, the auditory attention estimation unit 153 estimates the auditory attention direction from the relative amount of change and the area information by using the auditory attention direction estimation model.

The first embodiment, the modified example 1, the modified example 2, and the modified example 3 can be summarized as follows. In the auditory attention estimation unit 150/151/152, the relative change amount of the second biological information based on the first biological information calculated from the first biological information and the second biological information is reduced in the pupil of the subject. In the case of corresponding to the above, the relative amount of change in the direction corresponding to the relatively bright region information among the two or more regions included in the subject's visual field is that the subject's pupil is mydriasis. In the case of corresponding to the above, the direction corresponding to the relatively dark area information among the two or more areas included in the target person's visual field is estimated as the auditory attention direction.

In addition, the auditory attention estimation unit 150/151/152/153 estimates the auditory attention direction using a predetermined relationship between the first biological information, the second biological information, and the area information. Here, the above-mentioned predetermined relevance means that the auditory attention estimation unit 150/151/152 refers to the second biological information based on the first biological information calculated from the first biological information and the second biological information. The more the relative change amount indicates that the pupil diameter of the subject becomes smaller when the second biological information is acquired, the brighter the region information among the two or more regions included in the visual field of the subject becomes. The visual field of the subject is such that the corresponding direction indicates the auditory attention direction, or the relative amount of change indicates that the pupil diameter of the subject becomes larger when the second biological information is acquired. Of the two or more regions included in, the direction corresponding to the darker region information is one of the auditory attention directions, and the auditory attention estimation unit 153 is the auditory attention direction. It is given by an estimation model.

According to the invention of the present embodiment, it is possible to estimate the auditory attention of the subject. Specifically, it is possible to estimate the direction in which the auditory attention of the subject presented with two or more different sounds is directed.

<Second embodiment>

The phenomenon explained in <Technical Background> is not limited to the case where the subject consciously pays attention to either the left or right ear, but also the case where the subject unconsciously draws attention to either the left or right ear. It is a phenomenon. In the first embodiment, the auditory attention estimator that estimates the direction in which the subject consciously tries to pay attention has been described, but here, the auditory one that estimates the direction in which the subject is unconsciously attracted attention. The attention estimation device will be described. Unconsciously attracting attention can be presumed to be that the sound presented by the attracted side is easier to attract auditory attention than the other sounds. That is, the auditory attention estimation device of the second embodiment estimates the sound that is most likely to attract auditory attention among the two or more sounds.

Hereinafter, the auditory attention estimation device 200 will be described with reference to FIGS. 9 to 10. FIG. 9 is a block diagram showing the configuration of the auditory attention estimation device 200. FIG. 10 is a flowchart showing the operation of the auditory attention estimation device 200. As shown in FIG. 9, the auditory attention estimation device 200 includes a visual information presentation unit 110, an acoustic information presentation unit 120, a biological information acquisition unit 130, an area information acquisition unit 140/141, and an auditory attention estimation unit 250. And the recording unit 190 is included. The auditory attention estimation device 200 includes either the visual information presentation unit 110 and the area information acquisition unit 140, or the area information acquisition unit 141.

Hereinafter, the operation of the auditory attention estimation device 200 will be described with reference to FIG. Here, the auditory attention estimation unit 250 will be described.

[Aural attention estimation unit 250]
In step S250, the auditory attention estimation unit 250 uses the first biological information and the second biological information acquired in S130 and the area information acquired in S140 / S141 to among the sounds presented by the acoustic information presentation unit 120. Estimate the sound that most easily attracts the subject's auditory attention. Specifically, the following procedure is executed.

First, using the first biological information and the second biological information, the relative change amount of the second biological information based on the first biological information is calculated. The calculation method may be the same as that of the first embodiment.

Next, using the region information, the sound that is most likely to attract auditory attention is estimated from the relative amount of change (pupil diameter change information), and the information for identifying the sound (specific information) is output. Specifically, if the pupil diameter change information is a positive value, the sound presented from the relatively dark region of the visual information region, that is, the region where the region information is small, attracts the most auditory attention. Estimated to be an easy sound. On the other hand, if the pupil diameter change information is a negative value, the sound presented from the relatively bright region of the visual information region, that is, the region where the region information is large, is regarded as the sound that attracts the most auditory attention. presume. If there is no significant difference between the first biometric information and the second biometric information, the subject is not paying attention to any sound, in other words, the sound is heard in the same way. It can also be presumed to be.

[Modification 1]
Similar to the third modification of the first embodiment, it is possible to estimate the sound that is most likely to attract auditory attention by using a model learned by using machine learning, a neural network, or the like. The model used here is the auditory attention direction estimation model described in the modified example 3 of the first embodiment.

Hereinafter, the auditory attention estimation device 201 will be described with reference to FIGS. 11 to 12. FIG. 11 is a block diagram showing the configuration of the auditory attention estimation device 201. FIG. 12 is a flowchart showing the operation of the auditory attention estimation device 201. As shown in FIG. 11, the auditory attention estimation device 201 includes a visual information presentation unit 110, an acoustic information presentation unit 120, a biological information acquisition unit 130, an area information acquisition unit 140/141, and an auditory attention estimation unit 251. And the recording unit 190 is included. The recording unit 190 may record an auditory attention direction estimation model that has been learned in advance. In the auditory attention estimation device 201, when the experiment used to generate the learning data uses a still image, the visual information presentation unit 110 and the area information acquisition unit 140 are used in the actual environment. If is, the area information acquisition unit 141 is included.

Hereinafter, the operation of the auditory attention estimation device 201 will be described with reference to FIG. Here, the auditory attention estimation unit 251 will be described.

[Aural attention estimation unit 251]
In step S251, the auditory attention estimation unit 251 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S140 / S141 to among the sounds presented by the acoustic information presentation unit 120. Estimate the sound that most easily attracts the subject's auditory attention. Specifically, it is as follows.
(When the input of the auditory attention direction estimation model is a set of the first biological information, the second biological information, and the area information)
The auditory attention estimation unit 251 estimates the auditory attention direction from the first biological information, the second biological information, and the area information by using the auditory attention direction estimation model, and the sound presented from the direction is the most auditory attention. Is estimated as a sound that is easy to attract.
(When the input of the auditory attention direction estimation model is a set of relative change amount and area information)
First, the auditory attention estimation unit 251 calculates the relative change amount of the second biological information based on the first biological information by using the first biological information and the second biological information. The calculation method may be the same as that of the first embodiment. Next, the auditory attention estimation unit 251 estimates the auditory attention direction from the relative amount of change and the area information using the auditory attention direction estimation model, and attracts the most auditory attention to the sound presented from the direction. Estimate as an easy sound.

The second embodiment and the first modification are summarized as follows. The auditory attention estimation unit 250 indicates that the pupil of the subject is miotic in the relative amount of change in the second biological information based on the first biological information calculated from the first biological information and the second biological information. In the case of, the relative change amount of the sound presented from the direction corresponding to the relatively bright region information among the two or more regions included in the subject's visual field is mydriasis in the subject's pupil. Of the two or more areas included in the subject's pupil, the sound presented from the direction corresponding to the relatively dark area information is estimated as the sound that attracts the most auditory attention. To do.

In addition, the auditory attention estimation unit 250/251 estimates the sound that is most likely to attract auditory attention by using a predetermined relationship between the first biological information, the second biological information, and the area information. Here, the above-mentioned predetermined relationship is the relative change amount of the second biological information based on the first biological information calculated from the first biological information and the second biological information for the auditory attention estimation unit 250. However, the smaller the pupil diameter of the subject when the second biological information is acquired, the brighter the region of the two or more regions included in the subject's visual field, from the direction corresponding to the information. The more the presented sound indicates that it is the sound that attracts the most auditory attention, or the relative amount of change indicates that the pupil diameter of the subject becomes larger when the second biological information is acquired. Of the two or more areas included in the subject's visual field, the sound presented from the direction corresponding to the relatively dark area information is one of the sounds that are most likely to attract auditory attention, and is auditory. The attention estimation unit 251 is given by an auditory attention direction estimation model.

According to the invention of the present embodiment, it is possible to estimate the auditory attention of the subject. Specifically, it is possible to estimate the sound that is most likely to attract the auditory attention to the subject among the two or more different sounds presented to the subject.

<Third Embodiment>
It is also possible to present the sound to only one of the left and right ears of the subject and estimate whether or not the subject can hear the sound. Here, an auditory attention estimation device that estimates whether or not the sound presented to one of the left and right ears of the subject is heard will be described.

Hereinafter, the auditory attention estimation device 300 will be described with reference to FIGS. 13 to 14. FIG. 13 is a block diagram showing the configuration of the auditory attention estimation device 300. FIG. 14 is a flowchart showing the operation of the auditory attention estimation device 300. As shown in FIG. 13, the auditory attention estimation device 300 includes a visual information presentation unit 110, an acoustic information presentation unit 320, a biological information acquisition unit 130, an area information acquisition unit 140/141, and an auditory attention estimation unit 350. And the recording unit 190 is included. The auditory attention estimation device 300 includes either the visual information presentation unit 110 and the area information acquisition unit 140, or the area information acquisition unit 141.

Hereinafter, the operation of the auditory attention estimation device 300 will be described with reference to FIG. Here, the visual information presentation unit 110, the acoustic information presentation unit 320, the area information acquisition unit 140/141, and the auditory attention estimation unit 350 will be described.

[Visual information presentation unit 110]
In step S110, the visual information presenting unit 110 presents to the subject a still image composed of two regions (left region and right region) having different index values (for example, brightness) indicating brightness.

[Acoustic information presentation unit 320]
In step S320, the acoustic information presenting unit 320 presents sound to either the left or right ear of the subject and does not present sound to the other ear (that is, presents a silent state). Any sound can be used as the sound to be presented, but for example, a sound estimated by the auditory attention estimation device 200/201 to easily attract auditory attention may be used.

[Biological information acquisition unit 130]
In step S130, the biological information acquisition unit 130 and the first biological information which is time-series information of the size of the pupil in a state where sound is not presented to any of the left and right ears of the subject from the acoustic information presentation unit 320. , The second biological information which is the time-series information of the size of the pupil in the state where the sound is presented to either the left or right ear of the subject is acquired from the acoustic information presentation unit 320.

[Area information acquisition unit 140/141]
In step S140 / 141, the area information acquisition unit 140/141 determines the brightness of the area of the still image presented in S110 or the area of the real environment seen by the target person, that is, each area included in the field of view of the target person. The area information which is an index value indicating the above is acquired, and the area information is output to the auditory attention estimation unit 150.

However, the actual environment that the subject sees shall include a certain continuous range, that is, two areas where the index values indicating different brightness for each point are close to each other, and these two areas are on the left side. It shall be recognized as an area and an area on the right side.

[Aural attention estimation unit 350]
In step S350, the auditory attention estimation unit 350 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S140 / S141 to present the sound from the acoustic information presentation unit 320. It is estimated whether or not the sound presented to the subject (hereinafter referred to as the first ear) is heard by the subject. Specifically, when there is no change in the relative change amount of the second biological information based on the first biological information calculated from the first biological information and the second biological information, it is presented to the first ear. It is presumed that the subject is not hearing the sound, and if there is a change, it is presumed that the subject is hearing the sound presented to the first ear.

This change may be consistent with the trends (1) to (4) described in <Technical Background>, or opposite to the trends (1) to (4) described in <Technical Background>. It may show the tendency of. Originally, if the subject can hear the sound, the subject should be directed to the auditory attention in the direction in which the sound is presented, so that the pupil diameter should change according to the direction of the auditory attention. Therefore, when the change in the pupil diameter matches the tendencies (1) to (4) explained in <Technical Background>, it can be determined that the first ear is directed to auditory attention. Further, when the change in the pupil diameter shows a tendency opposite to the tendency (1) to (4) explained in <Technical Background>, that is, when the direction in which the sound is originally presented is bright. If the pupil is miotic and the direction in which the sound is presented is dark, the pupil should be dilated, but the direction in which the sound is presented is bright but the pupil is dilated, or the direction in which the sound is presented is the same. If the sound is dark but the pupils tend to shrink, it can be inferred that the subject may be consciously trying to pay attention to the opposite ear against the first ear. In other words, the change in pupil diameter tends to be explained in <Technical Background> even though no sound is presented to the ear that does not present sound (hereinafter referred to as the second ear) (1). If it can be determined that the subject is paying attention to the second ear according to (4), the subject consciously tries to pay attention to the opposite ear against the first ear. It is presumed that there is. In other words, even if the subject declares that he / she is "inaudible," it can be presumed that he / she is actually audible (that is, pretending to be inaudible).

[Modification 1]
Similar to the modified example 3 of the first embodiment, it is also possible to estimate whether or not the target person can hear the sound by using the model learned by using machine learning, a neural network, or the like. The model used here is the auditory attention direction estimation model described in the modified example 3 of the first embodiment.

Hereinafter, the auditory attention estimation device 301 will be described with reference to FIGS. 15 to 16. FIG. 15 is a block diagram showing the configuration of the auditory attention estimation device 301. FIG. 16 is a flowchart showing the operation of the auditory attention estimation device 301. As shown in FIG. 15, the auditory attention estimation device 301 includes a visual information presentation unit 110, an acoustic information presentation unit 320, a biological information acquisition unit 130, an area information acquisition unit 140/141, and an auditory attention estimation unit 351. And the recording unit 190 is included. The recording unit 190 may record an auditory attention direction estimation model that has been learned in advance. In the auditory attention estimation device 301, when the experiment used to generate the learning data uses a still image, the visual information presentation unit 110 and the area information acquisition unit 140 are used in the actual environment. If is, the area information acquisition unit 141 is included.

Hereinafter, the operation of the auditory attention estimation device 301 will be described with reference to FIG. Here, the auditory attention estimation unit 351 will be described.

[Aural attention estimation unit 351]
In step S351, the auditory attention estimation unit 351 uses the first biological information and the second biological information acquired in S130, and the area information acquired in S140 / S141 to present the sound from the acoustic information presentation unit 320. It is estimated whether or not the sound presented to the subject (hereinafter referred to as the first ear) is heard by the subject. Specifically, it is as follows.
(When the input of the auditory attention direction estimation model is a set of the first biological information, the second biological information, and the area information)
The auditory attention estimation unit 351 estimates the auditory attention direction from the first biological information, the second biological information, and the area information by using the auditory attention direction estimation model, and the direction matches the direction of the first ear. If so, it is presumed that the subject is hearing the sound presented to the first ear, and if they do not match, it is presumed that the subject is not hearing the sound presented to the first ear.
(When the input of the auditory attention direction estimation model is a set of relative change amount and area information)
First, the auditory attention estimation unit 351 calculates the relative change amount of the second biological information based on the first biological information by using the first biological information and the second biological information. The calculation method may be the same as that of the first embodiment. Next, the auditory attention estimation unit 351 estimates the auditory attention direction from the relative amount of change and the area information using the auditory attention direction estimation model, and when the direction matches the direction of the first ear. Estimates that the subject hears the sound presented to the first ear, and if they do not match, presumes that the subject does not hear the sound presented to the first ear.

The third embodiment and the first modification are summarized as follows. When the relative change amount of the second biological information based on the first biological information calculated from the first biological information and the second biological information is changed, the auditory attention estimation unit 350 makes a change in the first ear. It is presumed that the presented sound is heard by the subject.

In addition, the auditory attention estimation unit 350/351 uses a predetermined relationship between the first biological information, the second biological information, and the area information, and the sound presented to the first ear is heard by the subject. Estimate whether or not. Here, the above-mentioned predetermined relationship is the relative change amount of the second biological information based on the first biological information calculated from the first biological information and the second biological information for the auditory attention estimation unit 350. However, the larger the value, the more the subject can hear the sound presented to the first ear, or the smaller the relative change amount, the more the subject can hear the sound presented to the first ear. It is one of those indicating that the information is not provided, and the auditory attention estimation unit 351 is given by the auditory attention direction estimation model.

According to the invention of the present embodiment, it is possible to estimate the auditory attention of the subject. Specifically, it is possible to estimate whether or not the subject can hear the sound.

<Supplement>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Communication unit to which can be connected, CPU (Central Processing Unit, cache memory, registers, etc.), RAM or ROM which is memory, external storage device which is hard disk, and input unit, output unit, communication unit of these , CPU, RAM, ROM, has a connecting bus so that data can be exchanged between external storage devices. Further, if necessary, a device (drive) or the like capable of reading and writing a recording medium such as a CD-ROM may be provided in the hardware entity. A physical entity equipped with such hardware resources includes a general-purpose computer and the like.

The external storage device of the hardware entity stores the program required to realize the above-mentioned functions and the data required for processing this program (not limited to the external storage device, for example, reading a program). It may be stored in a ROM, which is a dedicated storage device). Further, the data obtained by the processing of these programs is appropriately stored in a RAM, an external storage device, or the like.

In the hardware entity, each program stored in the external storage device (or ROM, etc.) and the data necessary for processing each program are read into the memory as needed, and are appropriately interpreted, executed, and processed by the CPU. .. As a result, the CPU realizes a predetermined function (each configuration requirement represented by the above, ... Department, ... means, etc.).

The present invention is not limited to the above-described embodiment, and can be appropriately modified without departing from the spirit of the present invention. Further, the processes described in the above-described embodiment are not only executed in chronological order according to the order described, but may also be executed in parallel or individually depending on the processing capacity of the device that executes the processes or if necessary. ..

As described above, when the processing function in the hardware entity (device of the present invention) described in the above embodiment is realized by a computer, the processing content of the function that the hardware entity should have is described by a program. Then, by executing this program on the computer, the processing function in the above hardware entity is realized on the computer.

The program describing the processing content can be recorded on a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disk, a photomagnetic recording medium, a semiconductor memory, or the like. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like as a magnetic recording device is used as an optical disk, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disk. Memory), CD-R (Recordable) / RW (ReWritable), etc., MO (Magneto-Optical disc), etc. as a magneto-optical recording medium, EP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

Further, the distribution of this program is performed, for example, by selling, transferring, renting, or the like a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in the storage device of the server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

A computer that executes such a program first temporarily stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, when the process is executed, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of this program, a computer may read the program directly from a portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be executed sequentially. In addition, the above processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition without transferring the program from the server computer to this computer. May be. The program in this embodiment includes information to be used for processing by a computer and equivalent to the program (data that is not a direct command to the computer but has a property of defining the processing of the computer, etc.).

Further, in this form, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of these processing contents may be realized in terms of hardware.

Claims (13)

The first biometric information, which is the time-series information of the pupil size when the subject does not pay attention to the sound perceived by the left and right ears, and the sound perceived by the subject by either the left or right ear. A biometric information acquisition unit that acquires the second biometric information, which is time-series information on the size of the pupil in the state of paying attention.
For two or more areas included in the field of view of the target person, an area information acquisition unit that acquires area information that is an index value indicating the brightness of the area, and an area information acquisition unit.
Auditory attention including an auditory attention estimation unit that estimates the auditory attention direction of the subject when the second biological information is acquired by using the first biological information, the second biological information, and the area information. Estimator. The auditory attention estimation device according to claim 1.
The auditory attention estimation unit
When the relative amount of change in the second biometric information based on the first biometric information and the first biometric information calculated from the second biometric information corresponds to the diminished pupil of the subject. Of the two or more regions, when the relative change amount corresponds to the dilated pupil of the subject in the direction corresponding to the relatively bright region information, the two or more regions Among them, an auditory attention estimation device characterized in that a direction corresponding to relatively dark area information is estimated as the auditory attention direction. The auditory attention estimation device according to claim 1.
A predetermined relationship between the first biometric information, the second biometric information and the area information,
The relative change in the second biometric information based on the first biometric information calculated from the first biometric information and the second biometric information reduces the pupil diameter of the subject when the second biometric information is acquired. The more the area indicates, the more the direction corresponding to the relatively bright area information among the two or more areas included in the subject's visual field is the auditory attention direction, or
The more the relative amount of change indicates that the pupil diameter of the subject becomes larger when the second biological information is acquired, the darker the region information among the two or more regions included in the visual field of the subject. It is assumed that the direction corresponding to is one of the directions of auditory attention.
The auditory attention estimation unit
An auditory attention estimation device for estimating the auditory attention direction using the predetermined association. The auditory attention estimation device according to claim 1.
A predetermined relationship between the first biometric information, the second biometric information and the area information,
First biological information and second biological information acquired from one or more subjects, area information which is an index value indicating the brightness of the region included in the visual field of the subject, and the subject's hearing when the second biological information is acquired. Hearing that outputs the auditory attention direction by inputting the information based on the first biological information and the second biological information and the area information learned using the set of learning data obtained based on the correct answer of the attention direction. Assuming that it is given by the attention direction estimation model
The auditory attention estimation unit
An auditory attention estimation device for estimating the auditory attention direction using the predetermined association. An acoustic information presentation unit that presents different sounds to the target person from two or more different directions,
The first biological information, which is time-series information on the size of the pupil of the subject in the state where the sound is not presented by the acoustic information presenting unit, and the subject in the state where the sound is presented by the acoustic information presenting unit. A biometric information acquisition unit that acquires second biometric information, which is time-series information on the size of the pupil,
For two or more areas included in the field of view of the target person, an area information acquisition unit that acquires area information that is an index value indicating the brightness of the area, and an area information acquisition unit.
Includes an auditory attention estimation unit that estimates the sound most easily attracting auditory attention among the sounds presented by the acoustic information presentation unit using the first biological information, the second biological information, and the region information. Auditory attention estimator. The auditory attention estimation device according to claim 5.
The auditory attention estimation unit
When the relative amount of change in the second biometric information based on the first biometric information and the first biometric information calculated from the second biometric information corresponds to the diminished pupil of the subject. When the relative amount of change corresponds to the dilated pupils of the subject, the sound presented from the direction corresponding to the relatively bright region information among the two or more regions is described in 2. An auditory attention estimation device characterized in that a sound presented from a direction corresponding to relatively dark region information among the above regions is estimated as a sound that is most likely to attract auditory attention. The auditory attention estimation device according to claim 5.
A predetermined relationship between the first biometric information, the second biometric information and the area information,
The relative change in the second biometric information based on the first biometric information calculated from the first biometric information and the second biometric information reduces the pupil diameter of the subject when the second biometric information is acquired. The more the area indicates, the more the sound presented from the direction corresponding to the relatively bright area information among the two or more areas included in the subject's visual field is the sound that is most likely to attract auditory attention. ,
The more the relative amount of change indicates that the pupil diameter of the subject becomes larger when the second biological information is acquired, the darker the region information among the two or more regions included in the visual field of the subject. It is assumed that the sound presented from the direction corresponding to is one of the sounds that most easily attracts auditory attention.
The auditory attention estimation unit
An auditory attention estimation device, characterized in that the sound that is most likely to attract auditory attention is estimated using the predetermined association. The auditory attention estimation device according to claim 5.
A predetermined relationship between the first biometric information, the second biometric information and the area information,
First biological information and second biological information acquired from one or more subjects, area information which is an index value indicating the brightness of the region included in the visual field of the subject, and the subject's hearing when the second biological information is acquired. Hearing that outputs the auditory attention direction by inputting the information based on the first biological information and the second biological information and the area information learned using the set of learning data obtained based on the correct answer of the attention direction. Assuming that it is given by the attention direction estimation model
The auditory attention estimation unit
An auditory attention estimation device, characterized in that the sound that is most likely to attract auditory attention is estimated using the predetermined association. An acoustic information presentation unit that presents sound to either the left or right ear of the subject and does not present sound to the other ear.
The first biological information, which is time-series information on the size of the pupil of the subject in a state where no sound is presented to either the left or right ear of the subject from the acoustic information presentation unit, and the target from the acoustic information presentation unit. A biometric information acquisition unit that acquires second biometric information, which is time-series information on the size of the pupil of the subject in a state where sound is presented to either the left or right ear of the subject.
For the left area and the right area included in the target person's field of view, an area information acquisition unit that acquires area information that is an index value indicating the brightness of the area, and an area information acquisition unit.
The ear to which sound is presented from the acoustic information presentation unit is set as the first ear.
Hearing including an auditory attention estimation unit that estimates whether or not the sound presented to the first ear is heard by the subject using the first biological information, the second biological information, and the region information. Attention estimation device
The auditory attention estimation unit
When there is a change in the relative amount of change in the second biological information based on the first biological information calculated from the first biological information and the second biological information, the sound presented to the first ear is heard. An auditory attention estimation device that estimates that the subject is hearing. The first biological information, which is the time-series information of the pupil size when the subject is not paying attention to the sound perceived by the left and right ears, and the subject is either left or right. The biometric information acquisition step of acquiring the second biometric information, which is the time-series information of the pupil size in the state of paying attention to the sound perceived by the ear,
A region information acquisition step in which the auditory attention estimation device acquires region information, which is an index value indicating the brightness of the region, for two or more regions included in the visual field of the subject.
The auditory attention estimation device estimates the auditory attention direction of the subject when the second biological information is acquired by using the first biological information, the second biological information, and the area information. Auditory attention estimation methods, including estimation steps. An acoustic information presentation step in which the auditory attention estimator presents different sounds to the subject from two or more different directions.
The auditory attention estimation device presents first biological information, which is time-series information of the size of the pupil of the subject in a state where no sound is presented in the acoustic information presentation step, and sound in the acoustic information presentation step. The biometric information acquisition step of acquiring the second biometric information, which is the time-series information of the pupil size of the subject in the state of being
A region information acquisition step in which the auditory attention estimation device acquires region information, which is an index value indicating the brightness of the region, for two or more regions included in the visual field of the subject.
The auditory attention estimation device uses the first biological information, the second biological information, and the region information to estimate the sound most easily attracting auditory attention among the sounds presented in the acoustic information presentation step. An auditory attention estimation method that includes an auditory attention estimation step. An acoustic information presentation step in which the auditory attention estimator presents sound to either the left or right ear of the subject and does not present sound to the other ear.
The auditory attention estimation device and the first biological information which is time-series information of the size of the pupil of the subject in a state where no sound is presented to either the left or right ear of the subject in the acoustic information presentation step. , The biological information acquisition step of acquiring the second biological information which is the time-series information of the size of the pupil of the subject in the state where the sound is presented to either the left or right ear of the subject in the acoustic information presentation step. When,
A region information acquisition step in which the auditory attention estimation device acquires region information which is an index value indicating the brightness of the region for the left region and the right region included in the visual field of the subject.
The ear to which the sound is presented in the acoustic information presentation step is set as the first ear.
The auditory attention estimator estimates whether or not the sound presented to the first ear is heard by the subject by using the first biological information, the second biological information, and the region information. An auditory attention estimation method that includes an attention estimation step.
The auditory attention estimation step
When there is a change in the relative amount of change in the second biological information based on the first biological information calculated from the first biological information and the second biological information, the sound presented to the first ear is heard. An auditory attention estimation method that estimates that the subject is hearing. A program for operating a computer as an auditory attention estimation device according to any one of claims 1 to 9.

Images