JP7597217B2 - Learning device, estimation device, their methods, and programs - Google Patents

Description

The present invention relates to an estimation technology for estimating gaze angle and effective field of view from a facial image, and a learning technology for parameters used when making the estimation.

Gaze angle is the rotation angle that indicates the direction of the pupils of the left and right eyes. If the gaze angle can be estimated from an image, it is possible to understand the state of eye movement, such as what a person is gazing at or looking around, and to analyze the person's state and inner understanding. Gaze angle estimation is generally performed using a neural network. In conventional technology, only the gaze angle is learned by regression from the eye area image using a neural network model. The correct answer label for the gaze angle is vector data, which is continuous data of the rotation angle values of the horizontal and vertical components of the eyeball. In learning the parameters used for gaze angle estimation using a neural network, for example, the image features are extracted using a convolution layer and a pooling layer that are widely used in image recognition, and then the gaze angle can be estimated by regressing it to vector data using a fully connected layer.

Non-patent document 1 describes a method for estimating gaze angle using a neural network.

On the other hand, there is a method that uses the effective visual field as a measure to determine whether a person recognizes an object. The effective visual field refers to the range of horizontal angles of the eyeball from -15 degrees to 15 degrees and vertical angles from -12 degrees to 8 degrees (see Figure 1), and humans can clearly recognize objects within this range. Conversely, the ability to recognize objects outside the effective visual field drops drastically. For example, if the target is a robot as a use case, the effective visual field scale can be used to determine whether a robot equipped with a camera is recognized by a human captured on the camera. Specifically, even if a human is facing the robot, if the gaze angle is outside the effective visual field, it can be determined that the robot is not recognized, and if it is within the effective visual field, it can be determined that the robot is recognized. Figure 1 shows the visual field and visual field characteristics of a human. Figure 2 shows the effective visual field (the range in which the presence of an object can be recognized) and outside the effective visual field (the range in which the presence of an object cannot be recognized).

Xucong Zhang, Yusuke Sugano, Mario Fritz, Andreas Bulling, "Appearance-Based Gaze Estimation in the Wild", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015

In conventional model parameter learning for gaze angle estimation, a method is used in which only the gaze angle is regressed using a neural network, and whether the gaze angle is within or outside the effective visual field is determined by a rule judgment from the estimated gaze angle. Furthermore, conventional technology only minimizes the error in the gaze angle, and learning is performed without considering whether it is within or outside the effective visual field. The tendency of gaze angle estimation error is that the larger the absolute value of the correct angle, the larger the error, and there is a tendency for angle errors to be large at horizontal angles near -15 degrees and 15 degrees (areas surrounded by dashed lines in Figure 3), which are the boundaries of the effective visual field, and at vertical angles near -12 degrees and 8 degrees. For this reason, in a method that uses rules to judge the effective visual field from an estimated gaze angle, there is a problem in that the accuracy of the effective visual field judgment decreases near the boundaries of the effective visual field.

The present invention aims to provide an estimation device, a learning device, and methods and programs thereof that can perform highly accurate gaze angle estimation and effective visual field judgment that captures the characteristics of both the gaze angle and effective visual field judgment.

In order to solve the above problem, according to one aspect of the present invention, a learning device includes a shared network unit that uses model parameters ^ _θg or ^ _θp to convert a training face image ^Sb into an intermediate feature v by a neural network function; a gaze angle network unit that uses the model parameters ^ _θg to convert the intermediate feature v into an estimated gaze angle vector _Z'g by a neural network function; a gaze angle model parameter optimization unit that updates the model parameter ^ _θg by using the estimated gaze angle vector _Z'g and a gaze angle correct label for the training face image ^Sb ; an effective field of view network unit that uses the model parameters ^ _θp to convert the intermediate feature v _into an estimated effective field of view probability vector Z'p by a neural network function; and an effective field of view model parameter optimization unit that updates the model parameter ^ _θp by using the estimated effective field of view probability vector Z'p and the effective field of view correct label for the training face image ^Sb _, and obtains a learned model parameter _θg corresponding to the model parameter ^ _θg and a learned model parameter _θp corresponding to the model parameter ^ _θp .

The present invention has the effect of enabling highly accurate gaze angle estimation and effective field of view judgment that captures the characteristics of both the gaze angle and effective field of view judgment.

A diagram showing human visual field and visual field characteristics. FIG. FIG. 4 is a diagram for explaining an estimated angle error. FIG. 1 is a diagram showing an example of the configuration of an estimation system according to a first embodiment. FIG. 2 is a functional block diagram of the learning device according to the first embodiment. FIG. 4 is a diagram showing an example of a processing flow of the learning device according to the first embodiment. FIG. 1 is a functional block diagram of an estimation device according to a first embodiment. FIG. 4 is a diagram showing an example of a processing flow of the estimation device according to the first embodiment. FIG. 13 is a diagram showing an example of the configuration of a computer to which the present technique is applied.

Below, an embodiment of the present invention will be described. In the drawings used in the following description, components having the same functions and steps performing the same processing are given the same reference numerals, and duplicated explanations will be omitted. In the following description, symbols such as "^" used in the text should be written directly above the character immediately preceding it, but due to limitations in text notation, they are written immediately after the character in question. In formulas, these symbols are written in their original positions. Furthermore, unless otherwise specified, processing performed on an element-by-element basis of a vector or matrix is assumed to apply to all elements of that vector or matrix.

<Key Points of the First Embodiment>
In this embodiment, model parameters used to estimate the gaze angle and model parameters used to determine the effective visual field are simultaneously learned by a learning device. The learning data includes a face image for learning, a correct answer label ^Yg for the gaze angle, and a correct answer label ^Yp for the effective visual field. The neural network is configured with a shared network section, and a gaze angle network section and an effective visual field network section branched out at the rear. The shared network section takes a face image for learning as an input, and is expected to learn to acquire features of both the gaze angle and the effective visual field. The gaze angle network section at the rear regresses the gaze angle, and the effective visual field network section calculates the probability within the effective visual field, and minimizes the error of the estimated value using each correct answer label, and updates the model parameters. During estimation, a gaze angle estimation section and an effective visual field determination section are used, respectively. The gaze angle estimation section estimates the gaze angle using the network architecture of the shared network section and the subsequent gaze angle network section, and the model parameters learned by the learning device. Similarly, the effective visual field determination unit determines the effective visual field using the network architecture of the shared network unit and the subsequent effective visual field network unit, and the model parameters learned by the learning device.

First Embodiment
FIG. 4 shows an example of the configuration of an estimation system according to the first embodiment.

The estimation system includes a learning device 100 and an estimation device 200.

The learning device 100 and the estimation device 200 are special devices configured by loading a special program into a publicly known or dedicated computer having, for example, a central processing unit (CPU), a main storage device (RAM), etc. The learning device 100 and the estimation device 200 execute each process under the control of the central processing unit, for example. Data input to the learning device 100 and the estimation device 200 and data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out to the central processing unit as necessary and used for other processes. At least a part of each processing unit of the learning device 100 and the estimation device 200 may be configured by hardware such as an integrated circuit. Each storage unit provided in the learning device 100 and the estimation device 200 can be configured by, for example, a main storage device such as a RAM (Random Access Memory), or middleware such as a relational database or a key-value store. However, each storage unit does not necessarily need to be provided inside the learning device 100 and the estimation device 200, but may be configured as an auxiliary storage device made up of a hard disk, an optical disk, or a semiconductor memory element such as a flash memory, and may be configured to be provided outside the learning device 100 and the estimation device 200.

First, we will explain the learning device 100.

<Learning device 100>
The learning device 100 receives learning data D=(S ₁ , T ^g ₁ , T ^p ₁ ),...,(S _|M| , T ^g _|M| , T ^p _|M| ), learns model parameters θ _g , θ _p using the learning data D, and outputs the learned model parameters θ _g , θ _p . Here, M is the data size of the learning data D. S _m (m=1,...,M) is a face image for learning. The face image for learning may be, for example, either an image in which only the face is cut out, or an image in which only the eye region is cut out. For example, the resolution of the face image for learning is 224x224 pixels, and it has three RGB channels. ^{T g} _m (m=1,...,M) is a correct label of the gaze angle (hereinafter also referred to as "gaze angle correct label"), and is, for example, vector data that stores the horizontal and vertical rotation angles of the gaze angle, and has a data format such as [-35,56]. T ^p _m (m=1,...,M) is the correct label of the effective visual field (hereinafter also referred to as the "effective visual field correct label"), and is, for example, [1] if it is within the effective visual field and [0] if it is outside the effective visual field.

Figure 5 shows a functional block diagram of the learning device 100, and Figure 6 shows the processing flow.

The learning device 100 includes a shared network unit 120, a gaze angle network unit 130, a gaze angle model parameter optimization unit 140, an effective visual field network unit 150, and an effective visual field model parameter optimization unit 160.

Below, an overview of each part is provided.

The shared network unit 120 takes a training face image as input, learns the characteristics of both gaze angle and effective field of view, and acquires intermediate features using any neural network that outputs intermediate features.

The gaze angle network unit 130 obtains an estimate of the gaze angle using any neural network that takes intermediate features as input and outputs an estimate of the gaze angle.

Gaze angle model parameter optimization section 140 receives the gaze angle estimate and the gaze angle correct label as input, calculates the error in the gaze angle estimate, and updates the model parameter θ _g based on the error.

The effective visual field network unit 150 obtains the estimated effective visual field probability using any neural network that takes intermediate features as input and outputs the probability that they are within the effective visual field (estimated effective visual field probability).

The visual field model parameter optimization unit 160 receives the estimated visual field probability and the correct label of the visual field as input, calculates the error of the estimated visual field probability, and updates the model parameter θ _p based on the error.

The above process shows the learning procedure for one batch (a part of the training data that has been selected), and this can be repeated any number of times to learn all the data.

Details of each part are explained below.

<Shared Network Unit 120>
Input: face images for learning S ₁ , ..., S _|M| , updated model parameters ^θ _g or ^θ _p corresponding to the neural network constituting the shared network unit 120 Output: one batch of intermediate features v
The shared network unit 120 is configured with any neural network, for example, with four convolutional layers.

Prior to the conversion process, the shared network unit 120 receives the updated model parameters ^θ _g (output value of the gaze angle model parameter optimization unit 140) or ^θ _p (output value of the effective visual field model parameter optimization unit 160) corresponding to any neural network that constitutes the shared network unit 120.

The shared network unit 120 converts the training face images S b into intermediate features v by a function of an arbitrary neural network using the updated model parameters ^θ _g or ^θ _p corresponding to an arbitrary neural network constituting the shared network unit 120 (S120). The training face images S ^b are obtained by dividing the training face images S ₁ , ..., ^S _|M| into batch units. One batch of training face images S ^b is composed of, for example, 16 images. For example, when the training face images S ^b of a certain batch are composed of Q images, the intermediate feature corresponding to the qth image is defined as v _q , and v = [v ₁ , v ₂ , ..., v _Q ], where q = 1, 2, ..., Q.

<Gaze angle network unit 130>
Input: One batch of intermediate features v, and updated model parameters ^θ _g corresponding to the neural network that constitutes the gaze angle network unit 130. Output: Estimated gaze angle vector Z' _g
The gaze angle network unit 130 is configured with any neural network, for example, with two fully connected layers.

Prior to the conversion process, the gaze angle network unit 130 receives parameters corresponding to any neural network constituting the gaze angle network unit 130 from among the updated model parameters ^θ _g (output values of the gaze angle model parameter optimization unit 140).

The gaze angle network unit 130 converts the intermediate feature v into an estimated gaze angle vector Z' _g by using the received parameters and an arbitrary neural network function (S130). The estimated gaze angle vector is vector data that stores the estimated values of the horizontal rotation angle and the vertical rotation angle of the gaze angle. In other words, the estimated gaze angle vector is an estimated value of the gaze angle vector. The gaze angle vector is vector data that stores the horizontal rotation angle and the vertical rotation angle of the gaze angle. The rotation angle ranges from -180 degrees to 180 degrees, and for example, the gaze angle vector is a vector such as [-35, 56]. For example, the estimated gaze angle vector corresponding to the q-th intermediate feature v _q of a certain batch is Z' _g,q , and Z' _g = [Z' _g,1 , Z' _g,2 , ..., Z' _g,Q ]. The estimated gaze angle vector Z' _g,q is a vector such as [-35, 56].

<Gaze angle model parameter optimization unit 140>
Input: Estimated gaze angle vector Z' _g , gaze angle ground truth labels T ^g ₁ ,…,T ^g _|M|
Output: Updated model parameters ^θ _g or trained model parameters θ _g
The model parameters ^θ _g and θ _g are obtained by concatenating parameters corresponding to the neural network constituting the shared network unit 120 and parameters corresponding to the neural network constituting the gaze angle network unit 130 .

The gaze angle model parameter optimization unit 140 updates the model parameter ^ _θg using the estimated gaze angle vector _Z'g and _the gaze angle correct label ^Tg1 , ..., ^Tg _|M| (S140) and performs optimization. For example, the gaze angle model parameter optimization unit 140 calculates the error between the estimated gaze angle vector _Z'g and the gaze angle correct label ^Tg1 , ..., ^Tg _|M| , updates the model parameter ^ _θg so as to minimize the error, and performs optimization. For example, the error can be an MSE error or an MAE error, _and the parameter can be updated using a gradient descent method or the like.

<Effective Field of View Network Section 150>
Input: one batch of intermediate features v, and updated model parameters ^θ _p corresponding to the neural network that constitutes the visual field network unit 150. Output: estimated visual field probability vector Z' _p
The visual field network unit 150 is configured with any neural network, for example, with two fully connected layers.

Prior to the conversion process, the effective visual field network unit 150 receives parameters corresponding to any neural network that constitutes the effective visual field network unit 150 from among the updated model parameters ^θ _p (output values of the effective visual field model parameter optimization unit 160).

The effective visual field network unit 150 converts one batch of intermediate features v into an estimated effective visual field probability vector Z' _p by using the received parameters and an arbitrary neural network function (S150). The estimated effective visual field probability vector is a vector consisting of an estimated value of the effective visual field probability (estimated effective visual field probability), and the effective visual field probability is the probability of whether the gaze direction of the training face image is within the effective visual field. In other words, the effective visual field probability is the probability of whether the camera that captures the training face image is within the effective visual field of the subject of the training face image. For example, the estimated effective visual field probability corresponding to the q-th intermediate feature v _q of a certain batch is Z' _p,q , and Z' _p = [Z' _p,1 , Z' _p,2 , ..., Z' _g,Q ].

<Effective visual field model parameter optimization unit 160>
Input: Estimated effective field probability vector Z' _p , effective field correct labels T ^p ₁ ,…,T ^p _|M|
Output: Updated model parameters ^θ _p or trained model parameters θ _p
The model parameters ^θ _p and θ _p are a combination of parameters corresponding to the neural network constituting the shared network section 120 and parameters corresponding to the neural network constituting the effective visual field network section 150 .

The visual field model parameter optimization unit 160 updates the model parameters ^ _θp using the estimated visual field probability vector _Z'p and the visual field correct labels ^Tp1 _, ..., ^Tp _|M| (S160) and performs optimization. For example, the visual field model parameter optimization unit 160 calculates the error between the estimated visual field probability vector _Z'p and the visual field correct labels ^Tp1 , ..., ^Tp _|M|, updates the model _{parameters ^θp so} as to minimize the error, and performs optimization. For example, cross-entropy error can be used as the error, and gradient descent method can be used as the parameter update method.

The above-mentioned processes S120 to S160 are repeated until a predetermined condition is met (S170). The predetermined condition is a condition for determining whether the parameter update has converged or not, and may be, for example, (i) the number of updates has exceeded a predetermined number, or (ii) the difference between the parameters before and after the update is smaller than a predetermined value.

Furthermore, the above-mentioned processes S120 to S170 are performed on all batch data (learning data). For example, it is determined whether there is unprocessed batch data (S180), and if there is unprocessed batch data, the above-mentioned processes S120 to S170 are performed (NO in S180), and if there is no unprocessed batch data (YES in S180), the process ends.

After performing the above-mentioned processing on all the batch data, the finally obtained updated model parameters ^θ _g and ^θ _p are output as trained model parameters θ _g and θ _p .

Next, we will explain the estimation device 200.

<Estimation device 200>
Prior to the estimation process, the estimation device 200 receives trained model parameters _θg and _θp . The estimation device 200 receives a face image S of an estimation target as input, estimates a gaze angle using the trained model parameter _θg , estimates an effective visual field probability using the trained model parameter _θp , and outputs an estimated gaze angle vector _Zg and a determination result _Zp of the effective visual field.

Figure 7 shows a functional block diagram of the estimation device 200, and Figure 8 shows its processing flow.

The estimation device 200 includes a gaze angle estimation unit 210 and an effective field of view determination unit 220.

Details of each part are explained below.

<Gaze angle estimation unit 210>
Input: Face image S, model parameters θ _g
Output: Estimated gaze angle vector Z _g for face image S
Prior to the estimation process, the gaze angle estimation unit 210 receives the model parameter θ _g .

The gaze angle estimation unit 210 estimates the gaze angle from the face image S using the network architecture of the shared network unit 120 and the subsequent gaze angle network unit 130 and the model parameter _θg (S210) to obtain an estimated value (estimated gaze angle vector _Zg ). For example, if the shared network unit 120 is configured with an arbitrary neural network consisting of four convolution layers and the gaze angle network unit 130 is configured with an arbitrary neural network consisting of two fully connected layers, the gaze angle estimation unit 210 is configured with a neural network consisting of four convolution layers and two fully connected layers corresponding to the shared network unit 120 and the gaze angle network unit 130, and uses the model parameter _θg in this neural network.

<Effective visual field determination unit 220>
Input: Face image S, model parameters θ _p
Output: Judgment result Z _p of the effective visual field for face image S
Prior to the estimation process, the visual field determining unit 220 receives the model parameter θ _p .

The effective visual field determination unit 220 estimates the effective visual field probability from the face image S using the network architecture of the shared network unit 120 and the subsequent effective visual field network unit 150 and the model parameter θ _p , and determines whether or not the face image S is within the effective visual field based on the estimated value of the effective visual field probability (S220). For example, the effective visual field determination unit 220 determines whether or not the face image S is within the effective visual field based on the magnitude relationship between the estimated value of the effective visual field probability and a predetermined threshold value. For example, the effective visual field determination unit 220 outputs a determination result indicating that the face image S is within the effective visual field when the estimated value of the effective visual field probability is equal to or greater than a predetermined threshold value (e.g., 0.5), and outputs a determination result indicating that the face image S is outside the effective visual field when the estimated value of the effective visual field probability is less than the predetermined threshold value. However, the effective visual field determination unit 220 may output the estimated value of the effective visual field probability itself as the determination result. For example, if the shared network unit 120 is composed of an arbitrary neural network consisting of four convolution layers, and the effective visual field network unit 150 is composed of an arbitrary neural network consisting of two fully connected layers, the effective visual field determination unit 220 is composed of a neural network consisting of four convolution layers and two fully connected layers corresponding to the shared network unit 120 and the effective visual field network unit 150, and the model parameter _θp is used in this neural network.

＜Effects＞
The above configuration enables highly accurate gaze angle estimation and effective visual field determination that captures the characteristics of both the gaze angle and the effective visual field determination.

<Modification>
In this embodiment, the estimation device 200 estimates the gaze angle and the effective visual field probability, but may be configured to estimate only one of them. Even in this case, during learning, the model parameters used to estimate the gaze angle and the effective visual field are learned in a single neural network system, so that it is possible to perform a highly accurate gaze angle estimation or effective visual field judgment that captures the characteristics of both the gaze angle and the effective visual field judgment.

＜Effects＞
With the above configuration, it is possible to perform highly accurate gaze angle estimation and effective visual field determination that captures the characteristics of both the gaze angle and the effective visual field determination.

<Other Modifications>
The present invention is not limited to the above-mentioned embodiment and modified examples. For example, the above-mentioned various processes may be executed not only in chronological order as described, but also in parallel or individually depending on the processing capacity of the device executing the processes or as necessary. In addition, appropriate modifications are possible within the scope of the present invention.

<Program and recording medium>
The various processes described above can be implemented by loading a program that executes each step of the above method into the memory unit 2020 of the computer shown in Figure 9, and operating the control unit 2010, input unit 2030, output unit 2040, etc.

The program describing the processing can be recorded on a computer-readable recording medium. The computer-readable recording medium can be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other type of medium.

This program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, this program may be distributed by storing it in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

A computer that executes such a program, for example, first stores in its own storage device the program recorded on a portable recording medium or the program transferred from a server computer. Then, when executing a process, the computer reads the program stored in its own recording medium and executes the process according to the read program. As another execution form of this program, the computer may read the program directly from the portable recording medium and execute the process according to the program, or may execute the process according to the received program each time a program is transferred from the server computer to this computer. In addition, the server computer may not transfer the program to this computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes the processing function only by issuing an execution instruction and obtaining the results. Note that the program in this embodiment includes information used for processing by an electronic computer that is equivalent to a program (such as data that is not a direct command to the computer but has a nature that specifies the processing of the computer).

In addition, in this embodiment, the device is configured by executing a specific program on a computer, but at least a portion of the processing content may be realized in hardware.

Claims (8)

A shared network unit that converts the learning face image S ^b into an intermediate feature v by a neural network function using the model parameter ^θ _g or ^θ _p ;
a gaze angle network unit that converts the intermediate feature v into an estimated gaze angle vector Z' _g by a neural network function using the model parameter ^θ _g ;
a gaze angle model parameter optimization unit that updates the model parameter ^θ _g using the estimated gaze angle vector Z' _g and a gaze angle correct label for the learning face image S ^b ;
an effective visual field network unit that converts the intermediate feature v into an estimated effective visual field probability vector Z' _p by a neural network function using the model parameter ^θ _p ;
and an effective visual field model parameter optimization unit that updates the model parameter ^θ _p using the estimated effective visual field probability vector Z' _p and an effective visual field correct label for the learning face image S ^b,
Acquire a trained model parameter θ _g corresponding to the model parameter ^θ _g and a trained model parameter θ _p corresponding to the model parameter ^θ _p .
Learning device. An estimation device using a model parameter θ _g learned by the learning device of claim 1,
a gaze angle estimation unit that estimates a gaze angle from a face image S of an estimation target by using a network architecture of the shared network unit and the subsequent gaze angle network unit and the model parameter θ _g ;
Estimation device. An estimation device using a model parameter θ _p learned by the learning device of claim 1,
and an effective visual field determination unit that estimates an effective visual field probability from the face image S using a network architecture of the shared network unit and the subsequent effective visual field network unit and the model parameter θ _p , and determines whether or not the face image is within the effective visual field based on the estimated value of the effective visual field probability.
Estimation device. An estimation device that uses the model parameters θ _g and θ _p learned by the learning device of claim 1,
a gaze angle estimation unit that estimates a gaze angle from a face image S of an estimation target by using a network architecture of the shared network unit and the subsequent gaze angle network unit and the model parameter θ _g ;
and an effective visual field determination unit that estimates an effective visual field probability from the face image S using a network architecture of the shared network unit and the subsequent effective visual field network unit and the model parameter θ _p , and determines whether or not the face image is within the effective visual field based on the estimated value of the effective visual field probability.
Estimation device. A shared network step of converting the training face image S ^b into intermediate features v by a neural network function using the model parameter ^θ _g or ^θ _p ;
a gaze angle network step of converting the intermediate feature v into an estimated gaze angle vector Z' _g by a neural network function using the model parameter ^θ _g ;
a gaze angle model parameter optimization step of updating the model parameter ^θ _g using the estimated gaze angle vector Z' _g and a gaze angle correct label for the learning face image S ^b ;
a visual field network step of converting the intermediate feature v into an estimated visual field probability vector Z' _p by a neural network function using the model parameter ^θ _p ;
and an effective visual field model parameter optimization step of updating the model parameter ^θ _p using the estimated effective visual field probability vector Z' _p and an effective visual field correct label for the learning face image S ^b ;
Acquire a trained model parameter θ _g corresponding to the model parameter ^θ _g and a trained model parameter θ _p corresponding to the model parameter ^θ _p .
How to learn. An estimation method using a model parameter θ _g learned by the learning method of claim 5,
a gaze angle estimation step of estimating a gaze angle from a face image S of an estimation target by using a network architecture of the shared network step and the subsequent gaze angle network step and the model parameter θ _g ;
Estimation method. An estimation method using a model parameter θ _p learned by the learning method of claim 5,
and an effective visual field determination step of estimating an effective visual field probability from the face image S using a network architecture of the shared network step and the subsequent effective visual field network step and the model parameter θ _p , and determining whether or not the face image S is within the effective visual field based on the estimated value of the effective visual field probability.
Estimation method. A program for causing a computer to function as the learning device of claim 1 or as any one of the estimation devices of claims 2 to 4.

Images