JP6572841B2 - Information processing apparatus and program - Google Patents

Description

  The present invention relates to a technique for detecting a user's line of sight.

  2. Description of the Related Art Conventionally, a technique for detecting a user's line of sight from a captured image obtained by capturing a user's head is known. In order to reduce an error in line-of-sight measurement, a configuration for correcting the detected line of sight has been proposed. For example, Patent Document 1 proposes a technique for acquiring parameters for calibration by watching a plurality of standard points.

JP 2001-134371 A

  As described in Patent Document 1, conventionally, calibration is performed in accordance with a change in the position of the pupil. However, with this method, calibration cannot be performed properly, and the error may increase.

  The present disclosure provides a technique for appropriately performing gaze measurement calibration.

  A 1st aspect of this indication is an information processor (11) provided with a posture measurement part (40), a gaze measurement part (30), and a calibration part (30). The posture measuring unit is configured to measure the head posture of the subject. The line-of-sight measurement unit is configured to measure the line of sight of the subject from at least a captured image captured by the imaging unit (10) that captures the eye of the subject. The calibration unit is configured to calibrate the line of sight measured by the line-of-sight measurement unit using a predetermined calculation formula using a calibration value corresponding to the head posture measured by the posture measurement unit.

According to such a configuration, it is possible to reduce an error caused by a different head posture and perform appropriate calibration.
A second aspect of the present disclosure is a program for causing a computer to realize an attitude measurement function (40), a line-of-sight measurement function (30), and a calibration function (30). The posture measurement function is configured to measure the head posture of the subject. The line-of-sight measurement function is configured to measure the line of sight of the subject from at least the captured image captured by the imaging unit (10) that captures the eye of the subject. The calibration function is configured to calibrate the line of sight measured by the line-of-sight measurement function with a predetermined calculation formula using a calibration value corresponding to the head posture measured by the posture measurement function.

According to such a configuration, the same effects as those of the information processing apparatus according to the first aspect described above can be obtained.
In addition, the code | symbol in the parenthesis described in this column and a claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: The technical scope of this invention is shown. It is not limited.

It is a block diagram which shows the structure of a gaze measurement system. It is a block diagram which shows the structure of information processing apparatus. It is a flowchart of a calibration value calculation process. It is a figure explaining the acquisition method of a calibration value. It is a figure explaining the acquisition method of a calibration value. It is a figure which shows an example of a calibration value table. It is a figure explaining the acquisition method of a calibration value. It is a flowchart of a calibration process. It is a flowchart of the calibration value calculation process of a modification. It is a figure explaining the acquisition method of a calibration value.

Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings.
[1. Embodiment]
[1-1. Constitution]
A line-of-sight measurement system 1 shown in FIG. 1 is a device that is mounted on a vehicle such as an automobile and detects the line of sight of a passenger. In the following description, a passenger to be detected as a line of sight is referred to as a subject. The line-of-sight measurement system 1 includes an imaging unit 10, an information processing apparatus 11, a display 13, and a plurality of lamps 14a, 14b, 14c,. When the description is made for all of the plurality of lamps, the lamp 14 may be simply described.

  The imaging unit 10 is an imaging device provided in a vehicle, and for example, a known CCD image sensor or CMOS image sensor can be used. The imaging unit 10 captures the periphery of the vehicle at a predetermined time interval (for example, 1/15 s), and outputs the captured image to the information processing apparatus 11. The imaging target of the imaging unit 10 is the passenger's face in the passenger compartment and its peripheral part, and the imaging unit 10 is arranged so that at least the eyes of the passenger can be captured.

  As shown in FIG. 2, the information processing apparatus 11 is mainly configured by a known microcomputer having a CPU 15 and a semiconductor memory (hereinafter, memory 16) such as a RAM, a ROM, and a flash memory. Various functions of the information processing apparatus 11 are realized by the CPU 15 executing a program stored in a non-transitional physical recording medium. In this example, the memory 16 corresponds to a non-transitional tangible recording medium that stores a program. Further, by executing this program, a method corresponding to the program is executed. Note that the number of microcomputers constituting the information processing apparatus 11 may be one or plural. The memory 16 stores a calibration value table to be described later.

  As shown in FIG. 1, the information processing apparatus 11 includes a face detection unit 20, a line-of-sight measurement / calibration unit 30, and a head posture measurement unit 40 as functions configured by the CPU 15 executing a program. A personal authentication / glasses determination unit 50, a calibration value calculation unit 60, a calibration value storage unit 70, a calibration control unit 80, a calibration index presentation unit 90, and a head posture presentation unit 100. The line-of-sight measurement / calibration unit 30 corresponds to a line-of-sight measurement unit and a calibration unit. The personal authentication / glasses determination unit 50 corresponds to a personal authentication unit and a glasses determination unit.

  The technique for realizing these elements constituting the information processing apparatus 11 is not limited to software, and some or all of the elements may be realized using one or a plurality of hardware. For example, when the above function is realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit including a large number of logic circuits, an analog circuit, or a combination thereof.

The display 13 is a device having a display screen such as a liquid crystal display or an organic EL display capable of displaying an image.
The lamp 14 is a device that emits light including an LED light emitting element and the like, and a plurality of lamps 14 are arranged at intervals in the vehicle interior or the exterior of the vehicle interior. The position at which the lamp 14 is disposed is not particularly limited, but may be set to a place where there are many opportunities for the driver of the vehicle to view such as a rearview mirror or a side mirror.

[1-2. processing]
The information processing apparatus 11 in the line-of-sight measurement system 1 according to the present embodiment performs line-of-sight measurement by a known method and then calibrates to reduce the error in order to reduce errors in line-of-sight measurement caused by individual differences for each subject. I do. The calibration value used in the calculation formula for calibrating the line of sight is acquired in advance for each subject and for each posture of the subject's head (hereinafter, head posture).

<Outline of eye gaze measurement>
In the present embodiment, first, a line of sight is measured from a captured image obtained by capturing a face of a subject using a known method. The line of sight can be represented by, for example, an angle at which the line of sight faces the position of the eyeball or the like, coordinates of a position where the subject gazes on a predetermined plane, a straight line based on the position of the eyeball, or the like. In the present embodiment, the position of the eyeball or the like and the angle at which the line of sight is based on the position are used. Of course, the configuration may be such that the line of sight is expressed using other parameters.

  In the present embodiment, a reference point and a moving point are extracted from the captured image of the imaging unit 10 as eye feature points. The reference point is a point that is difficult to move even when the line of sight is changed, and can be, for example, the eye. The moving point is a point that moves when the line of sight is changed, and can be, for example, an iris. Then, the line of sight is calculated from the positional relationship between the reference point and the moving point. As the reference point, it is possible to use the Purkinje image, which is the reflected light from the cornea of the eyes, the corners of the eyes, and the irradiated light.

  Another method is to irradiate the eye with infrared rays and photograph the corneal reflection with an infrared camera, and calculate the line of sight based on the positional relationship with the pupil. That is, various known methods can be used as the method for detecting the line of sight.

Note that the line of sight measured here is a value that does not take into account the reduction in error caused by the difference in head posture.
Thus, in order to calibrate the line of sight calculated by a known method to a more accurate line of sight, calibration is performed by a predetermined calculation formula using a calibration value corresponding to the head posture. Specific contents of the calibration will be described later.

<Calibration value calculation process>
The calibration value calculation process executed by the CPU 15 of the information processing apparatus 11 will be described with reference to the flowchart of FIG. This calibration value calculation process is started, for example, when an instruction to calculate a calibration value is given by a passenger, for example, when a switch operation for the instruction is performed. However, it may be started on condition other than an instruction by the passenger. In the following description, the processing subject will be described as the functional configuration of the information processing apparatus 11 described above.

  In S <b> 10, the head posture presentation unit 100 presents the head posture to the subject using the lamp 14. The head posture here is a direction in which the position of the head of the subject and the face are directed. Only the direction in which the face is directed may be presented. Hereinafter, the presented head posture is also referred to as a presentation posture.

  As shown in FIG. 4, the head posture presenting unit 100 turns on only the lamp 14 a among the plurality of lamps 14. Further, the head posture presenting unit 100 causes the display 13 to display an image showing an instruction to turn the head of the subject 120 so that the lamp 14 faces the front.

  In S20, the calibration index presenting unit 90 presents the calibration index to the subject using the display 13. Specifically, as shown in FIG. 5, the calibration index presenting unit 90 should pay attention to any one of a plurality of calibration indices 122 whose positions to be displayed on the display 13 are predetermined. It is displayed in an aspect that can be recognized as an index. For example, it is conceivable that the notable calibration index 122 blinks or is changed to a color different from the other calibration indices 122. Further, only the calibration index 122 to be noted may be displayed.

  Note that when the process proceeds from S80 to S20, which will be described later, the calibration index 122 other than the calibration index 122 already displayed in the process of S20 when the presentation posture is the same is displayed as a notable index.

In S <b> 30, the face detection unit 20 acquires a captured image of the subject's head captured by the imaging unit 10. This captured image is an image of the subject's eyes.
In S40, the face detection unit 20 detects a face from the captured image acquired in S30. Specifically, the presence / absence of a face is determined by extracting a feature amount such as a shading change amount from a captured image and comparing it with the feature amount of a face image and a non-face image learned in advance. If it is determined that a face is photographed in the photographed image, facial organs such as eyes, nose and mouth are extracted in the same manner.

  Note that a method for extracting a face and a facial organ is not particularly limited, and various known methods for extracting a face and a facial organ from a captured image can be employed. For example, (i) A method of individually learning and detecting the entire face, eyes, nose, etc., Paul Viola and Michael J. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, CVPR (2001), ( ii) A method of learning the correlation between the arrangement of facial organ points and light and shade, Cootes, TF, Edwards, GJ, Taylor, CJ “Active Appearance Model”, ECCV (1998), (iii) Facial, such as the nostril Examples include a method using prior knowledge such as two black blocks arranged near the center, and Japanese Patent No. 4765008.

  In S50, the head posture measurement unit 40 measures the head posture. The head posture measuring unit 40 measures the head posture from the arrangement of the eyes, nose, mouth, etc. acquired in S40. The head posture here is information including three-dimensional coordinates of the head, a roll angle (hereinafter, roll angle), a pitch angle (hereinafter, pitch angle), a yaw angle (hereinafter, yaw angle), and the like. The roll angle is the amount of displacement in the direction of tilting the neck, the pitch angle is the amount of displacement in the direction of turning and the opposite direction, and the yaw angle is the amount of displacement in the direction of swinging the neck. In the present embodiment, the head posture is specified by the pitch angle and the yaw angle.

  The method for measuring the head posture from the position of the facial organ is not particularly limited, and various known methods for measuring the head posture can be employed. For example, it is conceivable to acquire the head posture by matching with a three-dimensional model.

  In S60, the calibration control unit 80 determines whether or not the fluctuation amount of the head posture from the time after the last S10 to the current time is less than a predetermined threshold value. The processing from S20 to S70, which will be described later, is repeatedly executed a plurality of times, thereby acquiring images of the eyes of the subject who have seen a plurality of different calibration indices 122 when the head posture is the same. If the head posture fluctuates, an image in which different calibration indices 122 are viewed in the same head posture is not obtained. Therefore, if the head posture has changed more than a predetermined threshold before acquiring an image in which the subject gazes at the calibration index 122 for all the calibration indices 122 in one head posture, the image is retaken, Redo the calibration. Note that if the head posture fluctuates, the accuracy of interpolation described later decreases.

If the fluctuation amount is not less than the predetermined threshold value in S60, the process returns to S10. On the other hand, if the variation is less than the predetermined threshold, the process proceeds to S70.
In S <b> 70, the line-of-sight measurement / calibration unit 30 performs line-of-sight measurement by the known method described above. The line-of-sight parameters acquired here are the three-dimensional coordinates of the eye position, the pitch angle θ, and the yaw angle φ.

  In S80, the calibration control unit 80 determines whether or not the processing up to S70 has been repeated for the head posture presented in S10 so as to cover all the preset calibration indices 122. If the above processing is performed for all the calibration indices 122, the processing proceeds to S90. On the other hand, if the above processing has not been performed for all the calibration indexes 122, the processing moves to S20, presents the calibration indexes 122 different from the calibration indexes 122 set so far, and performs the processing up to S70. Run.

In S90, the calibration value calculation unit 60 calculates a calibration value based on the line of sight acquired in S70 and the accurate line of sight calculated from the position of the calibration index 122.
The accurate line of sight is the line of sight when the calibration index 122 is viewed from the eye position calculated based on the head posture acquired in S50, and the pitch angle and the yaw angle are set to θ _A and φ _A , respectively. And If the line-of-sight parameters acquired in S70 are θ _B and φ _B , θ _A and φ _A can be expressed by the following equations.
Formula 1: θ _A = p ₁ · θ _B + p ₂
Formula 2: φ _A = p ₁ · φ _B + p ₂
The aforementioned p ₁ and p ₂ are calibration values. This calibration value is registered and held in the calibration value table shown in FIG. In the table, θ ₁ , θ ₂ ... Are the pitch angles of the head posture, and φ ₁ , φ ₂ ... Are the yaw angles of the head posture. That is, calibration values corresponding to the head posture are registered in this table.

  In S100, the calibration value storage unit 70 stores the calibration value calculated in S90 in the above-described calibration value table, thereby storing and storing the information configured by the memory 16 in a storage area where information can be stored. The calibration value storage unit 70 corresponds to a storage unit.

In S110, the calibration control unit 80 determines whether or not the calibration value has been repeatedly calculated and stored so as to cover all preset head postures.
If the above process is performed for all head postures, the process proceeds to S120. On the other hand, if the above processing has not been performed for all the head postures, the processing proceeds to S10, and as shown in FIG. 7, a lamp different from the lamp 14a, for example, the lamp 14b is turned on. The processing from S20 onward is repeated with a presentation posture different from the above. In the example of FIG. 7, the yaw angle of the head is changed compared to the example of FIG.

  In S120, the personal authentication / glasses discriminating unit 50 registers the facial feature amount in the database stored in the memory 16 based on the images of the subject photographed so far. In this database, feature quantities of facial organs are registered in association with the generated calibration value table. In addition, the specific method for authenticating a subject individual is not specifically limited. For example, it is possible to use a method of storing and comparing captured images of the subject themselves, a method of learning by combining the arrangement of facial organs, and the like.

<Calibration process>
The calibration process executed by the CPU 15 of the information processing apparatus 11 will be described with reference to the flowchart of FIG. This calibration process is started, for example, when the vehicle starts to travel.

In S <b> 210, the face detection unit 20 acquires a photographed image of the passenger's head photographed by the imaging unit 10.
In S220, the face detection unit 20 detects a face from the captured image acquired in S210. If it is determined that a face is photographed in the photographed image, facial organs such as eyes, nose and mouth are extracted. The specific process is the same as S40 of the calibration value calculation process.

  In S230, the personal authentication / glasses discriminating unit 50 performs personal authentication. Here, in the vicinity of the facial organs such as eyes, nose, mouth, etc., compared with the individual feature values registered in the database stored in the memory 16, the registered subjects are subject to gaze measurement. Identify a subject.

  In S240, the calibration control unit 80 determines whether or not the personal authentication is successful in S230. If the personal authentication is successful, the process proceeds to S250. On the other hand, if the personal authentication is not successful, the process proceeds to S310.

  In S250, the personal authentication / glasses determination unit 50 determines whether the subject is wearing glasses. Here, the state of the glasses is determined by comparing the image feature amount around the eyes with the pre-registered spectacle feature amount. The method for determining the presence or absence of glasses is not particularly limited, and various known methods can be used.

  In S260, the calibration control unit 80 determines whether or not the glasses have been successfully determined in S250, that is, whether or not glasses are being worn. If the eyeglass determination is successful, that is, if the eyeglasses are worn, the process proceeds to S270. On the other hand, if the eyeglass determination is not successful, that is, if the eyeglasses are not worn, the process proceeds to S300.

In S270, the head posture measurement unit 40 measures the head posture. The specific process is the same as S50 of the calibration value calculation process.
In S280, the calibration control unit 80 reads a calibration value corresponding to the head posture specified in S270 from the calibration value table of the subject authenticated in S240 stored in the memory 16.

  Further, as described above, the head posture is indicated by the pitch angle and the yaw angle, but there may be no calibration value in the calibration value table that matches the pitch angle and the yaw angle. Therefore, using the values registered in the calibration value table, the calibration values used in the case of the pitch angle and yaw angle for which the head posture is not registered are interpolated. The interpolation method is not particularly limited. For example, linear interpolation or nearest neighbor interpolation may be performed. Moreover, you may combine them.

  In S290, the line-of-sight measurement / calibration unit 30 performs line-of-sight measurement and calibration. Specifically, first, similarly to S70, line-of-sight measurement is performed by a known method, and the pitch angle θ and the yaw angle φ are calculated. Then, calibration is performed using the above-described formulas 1 and 2 using the calibration value table parameters or the interpolated parameters corresponding to the angles θ and φ. Thereby, it is possible to acquire a line of sight with reduced error. After this S290, this process ends.

  In S300, the line-of-sight measurement / calibration unit 30 performs line-of-sight measurement and calibration. However, the line-of-sight measurement is the same as S290, but unlike S290, calibration is performed using only one representative calibration value without changing the calibration value to be used for each head posture. As the representative calibration value, for example, a value obtained when the subject inclines his / her line of sight from the front of the face as shown in FIG. 5 can be used, but is not limited thereto.

  Note that the calibration value used is not limited to one. For example, the configuration may be such that the calibration value is changed depending on whether the direction in which the face faces is left or right or up and down. The calibration value used here corresponds to a predetermined calibration value.

In S310, the line-of-sight measurement / calibration unit 30 performs line-of-sight measurement. In S310, the line of sight is not calibrated.
[1-3. effect]
According to the embodiment detailed above, the following effects can be obtained.

  (1a) In this embodiment, the line-of-sight measurement / calibration unit 30 calibrates the line of sight measured by a predetermined calculation formula using a calibration value corresponding to the head posture. Therefore, it is possible to reduce errors that occur due to different head postures.

  In particular, when the subject wears spectacles, the shape of the eye in the captured image changes due to refraction of the lens of the spectacles, but the amount of change varies depending on the head posture. Therefore, there is a problem that an error in gaze measurement tends to increase depending on the head posture, but the gaze measurement system 1 can calibrate using a calibration value prepared for each head posture. The error can be reduced.

  (1b) In the present embodiment, when the subject is wearing glasses, calibration is performed with a calibration value corresponding to the head posture, while when the subject is not wearing glasses, one or Calibrate using a relatively small number of calibration values. When glasses are not worn, the error corresponding to the head posture is relatively small. Therefore, with the eye gaze measurement system 1, an increase in the processing load of the eye gaze measurement system 1 is suppressed while suppressing an increase in the error. be able to.

  (1c) In this embodiment, the calibration value for each subject registered in the database is read out and used based on the authentication result of the personal authentication / glasses discriminating unit 50. Therefore, it is possible to use an appropriate calibration value for each subject, and it is convenient that the subject who has already acquired the calibration value does not need to frequently perform work for acquiring the calibration value. .

[2. Other Embodiments]
As mentioned above, although embodiment of this indication was described, this indication is not limited to the above-mentioned embodiment, and can carry out various modifications.

  (2a) In the above embodiment, when the calibration value is acquired, the configuration not considering whether or not glasses are worn is exemplified. However, when glasses are not worn, calibration values for each head posture are not necessarily required. Therefore, the calibration value acquired according to the presence or absence of glasses may be configured to change.

A calibration value calculation process in which the calibration value acquired according to the presence or absence of glasses changes will be described using the flowchart of FIG.
In this process, S410 to S460 are the same processes as S210 to S260 of the calibration process, and thus description thereof is omitted. If it is determined in S440 that the personal authentication has not been successful, this process ends.

  If the determination is successful in S460, that is, if it is determined that the subject is wearing glasses, the process proceeds to S470. On the other hand, if the determination is not successful, the process proceeds to S480.

In S470, the CPU 15 executes S10 to S110 of the calibration value calculation process of FIG. 3 to acquire a calibration value for each head posture, and then the process ends.
In S480, the CPU 15 executes S20 to S100 of the calibration value calculation process of FIG. 3, and then the process ends. That is, when glasses are not worn, a calibration value is acquired for only one posture.

  With such a calibration value calculation process, a calibration value is acquired for each head posture when the subject is wearing glasses, and one head is used when the subject is not wearing glasses. A calibration value for the posture can be acquired. In S480, the calibration value may be acquired for two or more head postures.

  (2b) In the above embodiment, the configuration in which the line of sight is calibrated using one or a small number of calibration values when glasses are not worn is illustrated. However, even when glasses are not worn, the line of sight may be calibrated using the calibration values acquired for each head posture. In that case, more accurate line-of-sight calibration can be performed. Moreover, when not wearing glasses, it may be configured not to perform calibration.

  (2c) It is not necessary to obtain a calibration value using the same calibration index for all of the plurality of head postures, and at least a part of the head postures, the corresponding calibration indexes are different from those of other head postures. May be. In other words, the position of the calibration index to be measured may be set for each head posture.

  For example, in the case of FIG. 10, the rightmost calibration index 122 viewed from the subject is very difficult to visually recognize, so it is considered that the possibility of visual recognition in such a head posture is low. Therefore, by not measuring the calibration index 122 described above in the above-mentioned head posture, it is possible to reduce the burden on the subject, reduce the processing load, and reduce the storage area necessary for registering the calibration value. .

  (2d) In the above embodiment, the configuration in which the presentation of the head posture is executed using the lamp 14 is exemplified, but it may be executed using a specific display displayed on the display 13. For example, the display 13 may display an image for presenting the head posture. This image display may be the same display as the index for line-of-sight calibration.

  In the above-described embodiment, the configuration in which the calibration index 122 is displayed on the display 13 is illustrated. However, the display of the calibration index may be realized using an element other than the display 13. For example, any one of the lamps 14 may be turned on and the point may be used as a calibration index. Moreover, it is good also as a structure which instruct | indicates a head posture and a calibration parameter | index with an audio | voice. When an instruction is given by the location of the lamp 14 or by voice, the position of an existing object inside or outside the vehicle, such as a mirror, an instrument panel, or a seat, can be used as a reference.

  (2e) In the above embodiment, the configuration using the pitch angle and the yaw angle as the parameters for specifying the head posture is exemplified, but a configuration using parameters other than these may be used. For example, the head posture can be specified using at least one of the three-dimensional coordinates of the head and facial organs, the roll angle, the pitch angle, and the yaw angle.

  If the number of parameters specifying the head posture is three or more, not all calibration values can be registered with only a two-dimensional calibration value table as shown in FIG. Therefore, for example, when the head posture is indicated by three roll angles, pitch angles, and yaw angles, it is conceivable to prepare a calibration value table as shown in FIG. 6 for each roll angle.

  (2f) In the above-described embodiment, the configuration using the three-dimensional eye coordinates, the pitch angle θ of the line of sight, and the yaw angle φ as the parameters representing the line of sight is illustrated, but a configuration using parameters other than those may be used.

  (2g) In the above embodiment, a configuration is illustrated in which a line of sight is detected from a captured image by a known method, and calibration is performed using a calibration formula and a calibration value for each head posture. However, the method of measuring the calibrated line of sight is not limited to the configuration of the above embodiment.

For example, the equations used for calibration are not limited to Equations 1 and 2 described above, and various equations that can use calibration values according to the head posture can be applied.
Further, when the line of sight is indicated by the coordinates of the intersection of the line of sight vector and, for example, the plane which is the display screen of the display 13, when the coordinates of the intersection are the two-dimensional plane coordinates, It is also conceivable to correct with such a linear expression.

  Further, the calculation formula used when performing eye gaze measurement from the image and the calculation formula used for calibration may be combined into one calculation formula. Also in this case, by correcting the calculation formula using the calibration value corresponding to the head posture, it is possible to output a highly accurate line of sight that has been calibrated according to the head posture.

  (2h) When the personal authentication / glasses discriminating unit 50 cannot authenticate the subject in S230 of FIG. 8, the subject is prompted to obtain a calibration value by the calibration value calculation process. Also good. Specifically, it is conceivable to display the fact on the display 13 or to output the sound through a speaker.

  (2i) In the above embodiment, the personal authentication in S230 in FIG. 8 illustrates the configuration in which the personal authentication is performed without determining the presence or absence of the glasses. However, the subject is different depending on whether the glasses are worn or not. You may be comprised so that it may treat as an examiner. In that case, when registering the face feature amount in S120 of FIG. 3, it is possible to register the presence or absence of glasses.

  (2j) In the above embodiment, the configuration in which the calibration values are calculated for all the predetermined presentation postures and all the calibration indices 122 is exemplified. Here, in order to reduce the burden on the subject, as described above, apart from the mode for calculating all the calibration values as described above, the configuration has a simple mode in which the calibration values only need to be acquired in some head postures. Also good. In the simple mode, a calibration value that is not actually acquired can be estimated from the already acquired calibration value, or the already acquired calibration value can be used as it is.

  In the configuration having the simple mode described above, the calibration value is actually acquired for the head posture for which the calibration value has not been acquired by the instruction of the subject or the passenger or the proposal from the line-of-sight measurement system 1. Can be configured. The instruction by the subject or the like corresponds to, for example, that the subject or the like presses a switch for starting the calibration value acquisition. This switch can be pressed when the subject or the like recognizes that the accuracy of eye-gaze detection is low.

  In addition, the proposal from the line-of-sight measurement system 1 is, for example, when the subject takes a head posture for which a calibration value has not been acquired in the simple mode, when the accumulated time is long, or when the measured line of sight is used. This corresponds to the case where the line-of-sight measurement system 1 prompts the acquisition of the calibration value when the frequency of input operation mistakes is high. The case where the frequency of mistakes in the input operation is high corresponds to, for example, a case where the frequency of the operation for redoing the input is high in an application that performs the input operation with a line of sight, or a case where the time required for the input is long.

  (2k) The calibration value calculation process may not be required, and the calibration value may be acquired based on the operation during the driving operation of the subject. In this case, a position where the subject is assumed to be gazing at the position can be used as a calibration index. For example, the position of the switch when the subject performs the switch operation can be used as the configuration index.

(2l) Although the line-of-sight measurement system 1 of the present embodiment has been illustrated as being mounted on a vehicle, it may be used in a scene that is not mounted on the vehicle.
(2m) A plurality of functions of one constituent element in the above embodiment may be realized by a plurality of constituent elements, or a single function of one constituent element may be realized by a plurality of constituent elements. . Further, a plurality of functions possessed by a plurality of constituent elements may be realized by one constituent element, or one function realized by a plurality of constituent elements may be realized by one constituent element. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified from the wording described in the claims are embodiments of the present disclosure.

  (2n) In addition to the information processing apparatus 11 described above, a line-of-sight measurement system 1 including the information processing apparatus 11 as a constituent element, a program for causing a computer to function as the information processing apparatus 11, a semiconductor memory storing the program, and the like The present invention can also be realized in various forms such as a non-transition actual recording medium and a line-of-sight calibration method.

DESCRIPTION OF SYMBOLS 10 ... Imaging part, 11 ... Information processing apparatus, 30 ... Gaze measurement / calibration part, 40 ... Head posture measurement part

Claims (4)

A posture measurement unit (40) configured to measure the head posture of the subject;
A line-of-sight measurement unit (30) configured to measure the line of sight of the subject from a captured image captured by an imaging unit (10) that captures at least the eye of the subject;
A calibration unit (30) configured to calibrate the line of sight measured by the line-of-sight measurement unit according to a predetermined calculation formula using a calibration value corresponding to the head posture measured by the posture measurement unit; ,
A glasses discriminating unit (50) for discriminating whether or not the subject is wearing glasses;
The calibration unit uses the calibration value according to the head posture measured by the posture measurement unit when the eyeglass discrimination unit determines that the subject is wearing glasses. An information processing apparatus (11) that performs the calibration using a predetermined calibration value when it is determined by the glasses determination unit that the subject does not wear glasses . The information processing apparatus according to claim 1 ,
A storage unit (70) configured to store a plurality of calibration values;
A personal authentication unit (50) for identifying the subject who is the subject of eye gaze measurement from among the plurality of subjects registered in advance,
The calibration value is a value that is acquired in advance for each of the one or more head postures and stored in the storage unit for each of the plurality of subjects.
The calibration unit performs the calibration by reading the calibration value corresponding to the head posture measured by the posture measurement unit of the subject specified by the personal authentication unit from the storage unit. Processing equipment. The information processing apparatus according to claim 1 or 2 ,
The information processing apparatus, wherein the head posture includes at least one parameter of three-dimensional coordinates, a roll angle, a pitch angle, and a yaw angle. On the computer,
A posture measurement function (40) configured to measure the head posture of the subject;
A line-of-sight measurement function (30) configured to measure the line of sight of the subject from a captured image captured by an imaging unit (10) that captures at least the eye of the subject;
A calibration function (30) configured to calibrate the line of sight measured by the line-of-sight measurement function according to a predetermined calculation formula using a calibration value corresponding to the head posture measured by the posture measurement function; ,
A program for realizing a glasses discriminating function (50) for discriminating whether or not the subject is wearing glasses,
The calibration function uses the calibration value corresponding to the head posture measured by the posture measurement function when the subject is determined to be wearing glasses by the glasses discrimination function. The program is configured to perform the calibration using a predetermined calibration value when it is determined by the glasses determination function that the subject does not wear glasses .