JP6430987B2 - Reference point position determination device - Google Patents

Description

  The present invention relates to a reference point position determination device, and more particularly to a reference point position determination device that determines the position of a reference point in an input object image.

  Conventionally, there has been proposed a face image authentication apparatus that authenticates a target person by collating a two-dimensional face image acquired by photographing the face of the target person with a registered face image. Such a face image authentication device compares the registered face image of the subject person acquired in advance with the input face image acquired at the time of use to calculate the similarity, and performs a match determination using the similarity Whether to authenticate is determined based on the result. For this reason, if there is a difference in facial orientation or facial expression between the registered face image and the input face image, the degree of similarity is lowered and the authentication error rate is increased.

  As a method for verifying the identity by collating the registered face image and the input face image, as disclosed in Non-Patent Document 1, from the luminance value sequence of each pixel in the face image whose size and position are normalized. A unique space (Eigenface) method for generating a feature vector subspace by principal component analysis, projecting the feature vector of the input face image and the registered face image onto the partial space, calculating similarity, and performing matching determination Is widely known.

  According to the prior art of Non-Patent Document 1, partial distortion occurs in a face image due to changes in face orientation and facial expression, and pixels corresponding to each element of a feature vector between a registered face image and an input face image When the positional relationship between the two changes, the degree of similarity becomes low, and there is a problem that whether or not authentication is possible cannot be correctly determined.

  For these problems, feature points are set on the edges and contours of organs such as the eyes and nose and mouth, feature points are extracted from both the registered face image and the input face image, and feature vectors are obtained from pixel information around the feature points. Non-patent document 2 discloses a method of calculating similarity by comparing feature vectors and performing collation determination.

  According to the prior art of Non-Patent Document 2, since characteristic image features appear at the edges and contours of organs such as the eyes and nose and mouth, it is possible to extract feature points following changes in facial orientation and facial expressions. With this, it is possible to maintain the correspondence of the position where the feature vector is calculated between the registered face image and the input face image, and to alleviate the problems of the prior art of Non-Patent Document 1.

M. Turk and A. Pentland, “Face Recognition using Eigenfaces”, IEEE Conference on Computer Vision and Pattern Recognition, Maui, HI, June 1991. Wiskott, L et al., “Face Recognition by Elastic Bunch Graph Matching”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, p.775-779, 1997

  However, since the position of the feature point is limited to the periphery of an organ where a characteristic image feature such as an eye-nose mouth appears, the conventional technology of Non-Patent Document 2 described above can obtain a feature vector from a skin portion such as a cheek or forehead. There was a problem that feature vectors could not be obtained from the entire face.

  An object of the present invention is to provide a reference point position determination device that can stably determine a reference position for extracting features from the entire object regardless of the direction of the object.

  In order to achieve the above object, a reference point position determination apparatus according to the present invention is a reference point position determination apparatus that determines the position of each of a plurality of reference points in an input object image obtained by imaging an object. A plurality of standard 3D feature points representing each three-dimensional position of a plurality of feature points in the object model, and a plurality of standard 3D reference points representing each three-dimensional position of the plurality of reference points in the three-dimensional object model. The input means so that a shift in at least one of the two-dimensional space and the three-dimensional space between the storage means stored in advance and the plurality of extracted feature points extracted from the input object image and the plurality of reference 3D feature points is minimized. Alignment means for obtaining a parameter for correcting at least one of the direction and shape of the three-dimensional shape of the object represented by the object image as an alignment parameter; and the alignment parameter The input object image using the plurality of projecting the reference 3D reference points, is configured to include a, a reference point position determining means for determining the position of each of the plurality of reference points of the input object image.

  The alignment unit according to the present invention includes a reference 2D feature point obtained by projecting the reference 3D feature point onto the input object image using the parameter, and the extracted feature point corresponding to the reference 2D feature point on the image. The alignment parameter can be obtained so that the positional deviation is minimized.

  The alignment means according to the present invention provides a distance in a three-dimensional space between a straight line obtained by back projecting the extracted feature point into a three-dimensional space using the parameter and the reference 3D feature point corresponding to the extracted feature point. Using the sum, the alignment parameter can be determined so that the deviation is minimized.

  The alignment means according to the present invention includes a plurality of 3D feature points corresponding to the plurality of reference 3D feature points and a part image of each of the plurality of 3D feature points arranged on a predetermined three-dimensional shape model. Based on the information, image information at a position where the 3D feature points are projected on the input object image using the parameters for each of the plurality of 3D feature points, and the part image of the 3D feature points The 3D position of each of the plurality of extracted feature points is determined as a three-dimensional position of each of the 3D feature points so that the sum of local likelihoods representing the degree of fitness of the 3D feature points calculated based on the information is maximized. In addition to obtaining the original position, the alignment parameter can be obtained.

  The reference point position determining unit according to the present invention is based on the three-dimensional position of each of the plurality of extracted feature points obtained by the positioning unit and the three-dimensional position of each of the plurality of reference 3D feature points. , Obtaining a plurality of 3D reference points in the three-dimensional shape of the object represented by the input object image corresponding to the plurality of reference 3D reference points, and using the alignment parameter, the plurality of 3D references By projecting points, the positions of a plurality of reference points of the input object image can be obtained.

  A plurality of reference 3D reference points according to the present invention can be set so as to be arranged substantially evenly over the entire object.

  The plurality of reference 3D reference points according to the present invention are back-projected onto the three-dimensional shape of the three-dimensional object model by arranging points arranged at equal intervals on a plane that is substantially opposite the three-dimensional object model. Can be obtained.

  A plurality of reference 3D reference points according to the present invention can be set so that the reference 3D reference points are not arranged at positions where image characteristics of the object are predicted to change in advance.

  Further included is a feature quantity extraction means for extracting a feature quantity for matching with a registered object image from the input object image based on the positions of the plurality of reference points obtained by the reference point position determination means according to the present invention. be able to.

  As described above, according to the reference point position determination apparatus of the present invention, the deviation in at least one of the two-dimensional space or the three-dimensional space between the plurality of extracted feature points extracted from the input object image and the plurality of reference 3D feature points. By projecting a plurality of standard 3D reference points to the input object image using the alignment parameter obtained so that the minimum is determined, the positions of the plurality of reference points of the input object image are obtained, and the position depends on the direction of the object. In addition, it is possible to stably determine a reference position for extracting features from the entire object.

It is the schematic which shows the structure of the entrance / exit management system which concerns on embodiment of this invention. It is a block diagram which shows the structure of the face image authentication apparatus which concerns on embodiment of this invention. It is a schematic diagram of the reference | standard 3D feature point arrange | positioned on the three-dimensional shape of a standard human face. It is a figure which shows the example which has arrange | positioned the reference | standard 3D feature point with respect to the three-dimensional shape of the standard person face which faced the front. It is a figure which shows the example which has arrange | positioned the reference | standard 3D reference point with respect to the three-dimensional shape of the standard person face which faced the front. It is a figure which shows the example which did not arrange | position a reference | standard 3D reference point about the face part where an image characteristic is anticipated to fluctuate. It is a figure which shows the example of the reference | standard 2D feature point which projected the face feature point extracted from the face area image and the reference | standard 3D feature point to the face area image with the alignment parameter. It is a figure which shows the example of the 2D reference point arrange | positioned according to the face direction of the face area image. It is a figure which shows the example of the 3D feature point which is each three-dimensional position of the face feature point of a face area image, and the reference | standard 3D feature point coordinate-transformed using the alignment parameter. It is a figure which shows the example of the perturbation 3D reference point matched with the individual feature point position. It is a figure which shows the example of a reference | standard 3D feature point, a reference | standard 3D reference point, a 3D feature point, and a perturbation 3D reference point. It is a flowchart which shows the operation | movement of the authentication process by the face image authentication apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to an access control system will be described as an example.

<System configuration>
Hereinafter, referring to FIG. 1 schematically showing an installation example of the entrance / exit management system 100 for managing entrance / exit of the room entrance and FIG. 2 showing a schematic configuration of the entrance / exit management system 100 to which the present invention is applied, The configuration of the embodiment of the present invention will be described.

(Access control system 100)
The access management system 100 includes an imaging device 1, an electric lock control device 2, and a face image authentication device 3. In the schematic diagram shown in FIG. 1, the imaging device 1, the electric lock control device 2, and the face image authentication device 3 are arranged in the vicinity of the entrance / exit of the room. As shown in FIG. 1, the imaging apparatus 1 has a shooting direction directed slightly downward on a wall surface or ceiling near the entrance 101 so that the passage leading to the entrance 101 can be taken as a shooting area, and toward the passage side leading to the entrance 101. It is attached in the state of facing. Thereby, the imaging device 1 can image the person 110 heading toward the entrance 101 along the traveling direction 102.

  The face image authentication device 3 manages the entrance / exit of the entrance 101 of the room by controlling the electric lock 104 via the electric lock control device 2. The entrance 101 is provided with a door 103, and the door 103 is provided with an electric lock 104. The electric lock 104 can be locked and unlocked by the face image authentication device 3. The electric lock 104 always locks the door 103, and is unlocked only for a fixed time (for example, 5 seconds) if it can be authenticated that the face image authentication device 3 is permitted to pass. The electric lock 104 is automatically locked when the door is closed and the door is closed and a predetermined time elapses after the door is opened.

  Hereinafter, the face image authentication device 3 will be described in detail with reference to FIG. FIG. 2 is a diagram showing a schematic configuration of the access control system 100 to which the present invention is applied.

(Imaging device 1)
The imaging device 1 is a surveillance camera that captures a predetermined imaging region as shown in FIG. The imaging device 1 is connected to the face image authentication device 3 and outputs the acquired input image to the face image authentication device 3.

(Electric lock control device 2)
The electric lock control device 2 is connected to the face image authentication device 3, and locks or unlocks the electric lock 104 according to a signal from the face image authentication device 3.

(Face image authentication device 3)
The face image authentication device 3 includes a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), peripheral circuits, terminals, and various memories. The face image authentication device 3 authenticates a person's face image captured by the imaging device 1, and If it is authenticated that the person is permitted to pass 101, the control signal of the electric lock 104 is output to the electric lock control device 2. The face image authentication device 3 is functionally composed of a face image acquisition unit 10, an output unit 20, a storage unit 30, and an image processing unit 40. Hereinafter, each part of the face image authentication device 3 will be described in detail.

(Face image acquisition unit 10)
The face image acquisition unit 10 is connected to the imaging device 1, extracts a face area image including a human face from an input image captured by the imaging device 1, and outputs it to the image processing unit 40. When a plurality of human faces are photographed in the input image, face area images are sequentially extracted and output to the image processing unit 40.

  Many face area image extraction methods have been proposed in the past, and known methods may be employed as appropriate. For example, a method of extracting a face image with a filter called a discriminator that has learned, a method of generating a binarized edge image of an input image, and making an elliptical portion that is the shape of the face in the edge image as a face area image Should be adopted.

(Output unit 20)
The output unit 20 is an interface connected to the electric lock control device 2 that is an external connection device and a control circuit thereof. When the output unit 20 receives a signal indicating that the person has been successfully authenticated from the image processing unit 40, the output unit 20 outputs a signal indicating that the authentication has been successful to the connected electric lock control device 2.

(Storage unit 30)
The storage unit 30 includes a semiconductor memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory), or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 30 stores a computer program and various data for controlling the face image authentication device 3, and inputs / outputs such information to / from the image processing unit 40. The various data includes a standard 3D feature point 31, a standard 3D reference point 32, and a registered face image 33 of a registrant permitted to pass the entrance 101.

(Standard 3D feature point 31)
The reference 3D feature points 31 are “right eye”, “left eye”, “nose”, and “mouth” which are organs of a person's face, and “eye corners”, “eyes”, “center of upper eyelids” corresponding to each organ. ”And“ center of the lower eyelid ”, and the like, are positions on a three-dimensional shape of a standard human face.

FIG. 3 is a schematic diagram of the reference 3D feature points 31 arranged on the three-dimensional shape 300 of a standard human face. For example, as shown in FIG. 3, the center of gravity of a standard three-dimensional shape 300 of a human face is set as an origin, and an X axis 301 that is a horizontal axis, a Y axis 302 that is a vertical axis, and a Z axis 303 that is a depth axis are provided. The 3D coordinates (x ₁ , y ₁ , z ₁ ) of the “eye corner”, which is the feature of the “right eye”, are associated with the “right eye” and the “eye corner” as reference parts. 31 is stored in the storage unit 30. A standard three-dimensional shape 300 of a human face is an example of an object model.

  The number of the reference 3D feature points 31 is not particularly limited, but the reference 3D feature points 31 are arranged at positions where characteristic image features can be obtained even in a two-dimensional face image such as an edge or outline of a human face organ. In FIG. 4, reference 3D feature points are arranged with respect to a three-dimensional shape 300 of a standard human face facing the front (the rotation angle with respect to the X axis 301, the Y axis 302, and the Z axis 303 is 0 °). An example is shown. In this example, 20 reference 3D feature points are arranged.

  Note that the standard three-dimensional shape 300 of the human face need only be prepared when the reference 3D feature point 31 is obtained, and need not always be stored in the storage unit 30.

(Standard 3D reference point 32)
The reference 3D reference point 32 is a position of a point that is uniformly and entirely arranged on the three-dimensional shape 300 of the standard human face.

FIG. 5 shows the direction of the X axis 301 with respect to a standard three-dimensional shape 300 of a human face facing the front (the rotation angle with respect to the X axis 301, the Y axis 302, and the Z axis 303 is 0 °). On the plane defined by the direction of the Y-axis 302, that is, on the plane that is substantially opposite to the three-dimensional shape 300 of the standard human face, for example, points that are arranged evenly at intervals of 1 cm and over the entire face, An example of a standard 3D reference point 32 obtained by projecting onto a standard three-dimensional shape 300 of a human face is shown. In this example, 160 standard 3D reference points are arranged. For example, the upper left reference 3D reference point is the first, serial numbers are assigned in the raster scan order toward the lower right, and the three-dimensional coordinates (X _n , Y _n , Z _n ) of the n th reference 3D reference point are given. Is stored as the reference 3D reference point 32 in association with the serial number n. The number of the reference 3D reference points 32 is not particularly limited, but is arranged not only around the facial organ where the reference 3D feature point 31 is arranged, but also in a place where a human facial organ does not exist, such as a cheek. It is desirable to do. For example, the reference 3D feature points 31 may be arranged also at the interpolation points and extrapolation points of the reference 3D feature points 31 around the organ.

  In addition, as shown in FIG. 6, you may make it the structure which does not arrange | position a reference point about the facial region where it is estimated that an image characteristic will change by attachment or detachment of an ornament, or expression change. In the example of FIG. 6, it is assumed that the eyeglasses are attached and detached and the mouth is opened and closed, and no reference points are arranged around the eyes and at the center of the mouth.

(Registered face image 33)
The registered face image 33 is a face image of a person permitted to enter and exit in advance, and is used for authentication with the input face image. One or more face images are stored.

(Image processing unit 40)
The image processing unit 40 is configured by a so-called computer such as a memory, a peripheral circuit thereof, and a microprocessor thereof, and executes various processes on the face image acquired from the face image acquisition unit 10 while referring to the storage unit 30. The result is output to the output unit 20. The image processing unit 40 includes an alignment unit 41, a reference point position determination unit 42, and a face matching unit 43. Each means of the image processing unit 40 is a functional module realized by software operating on a microprocessor.

(Positioning means 41)
The alignment unit 41 aligns the face feature point position of the face area image acquired by the face image acquisition unit 10 with the reference 3D feature point 31 stored in the storage unit 30.

Specifically, the alignment means 41 is arranged so that the face feature point of the face area image and the reference 2D feature point obtained by projecting the reference 3D feature point 31 onto the face area image are sufficiently close to each other. The position of the standard three-dimensional shape 300, the orientation of the face, and the like are adjusted, and at least the translation (T _x , T _y , T _z ) and the rotation angle (R) with respect to the X axis 301, the Y axis 302, and the Z axis 303 _x , _Ry , _Rz ) and a scaling parameter s are obtained as alignment parameters for the face area image.

A general method may be adopted for projecting the reference 3D feature point 31 onto the face area image. For example, in the case of a pinhole camera model, assuming that the three-dimensional coordinates of the nth reference 3D feature point 31 are (x _n , y _n , z _n ), the nth reference 3D feature point 31 is the face region image. The position (x _n ′, y _n ′) projected onto is calculated by using the translation (T _x , T _y , T _z ), the rotation angle (R _x , R _y , R _z ), and the scaling parameter s. 1).

(1)
However,

Here, T represents transposition.

FIG. 7 shows a parameter for aligning the facial feature point 701 extracted from the facial region image 700 and the reference 3D feature point 31 with the facial region image.

The example of the reference | standard 2D feature point 702 projected by FIG. Here, since the face feature point 701 is a feature point extracted from the individual face image, the reference 2D feature point 702 obtained by projecting the reference 3D feature point 31 arranged on the three-dimensional shape 300 of the standard human face. Is not an exact match. Therefore, the alignment parameter

Is obtained as a parameter that minimizes the positional deviation between the face feature point 701 and the reference 2D feature point 702.

  The misregistration may be quantified by various known methods. For example, the distance between the corresponding feature points of the face feature point 701 and the reference 2D feature point 702 is obtained, and the sum of the distances is determined as the misalignment. Also good. Further, the alignment parameter may be obtained so that the positional deviation between the face feature point 701 extracted from the face area image 700 and the reference 3D feature point 31 in the three-dimensional space is minimized. The sum of the distances between the straight line obtained by back projecting the face feature point 701 in the three-dimensional space and the corresponding reference 3D feature point 31 may be used as the positional deviation.

  Various known methods can be used to extract the facial feature points of the face area image. For example, an edge extraction process is performed on the face area image to extract edge pixels having a large luminance difference from surrounding pixels, and the feature amount obtained based on the position and pattern of the edge pixels is the eye, nose, Each face feature point can be extracted by checking whether or not a predetermined condition is satisfied for a part such as a mouth and the like and specifying the position of each part. Further, instead of performing edge extraction processing to extract edge pixels, Gabor transform processing or wavelet transform processing may be performed to extract pixels having large local changes in a plurality of different spatial frequency bands. Furthermore, the face feature points may be extracted by performing template matching between the template corresponding to each part of the face and the face region image 700 to specify the position of each part of the face.

Further, for example, by using a feature point extraction method disclosed in Japanese Patent Application Laid-Open No. 2014-178862, translation (T _x , T _y , T _z ) and a rotation angle ( _Rx , _Ry , _Rz ), not only the scaling parameter s, but also the shape parameters of the three-dimensional shape of the face represented by the face area image are extracted, and the three-dimensional position of each face feature point of the face area image is obtained. .

Specifically, first, a 3D feature point corresponding to a plurality of reference 3D feature points and a part for each 3D feature point, which are arranged on a three-dimensional shape model that can express differences in individual face shapes and face orientation variations. Image information in advance. Further, the initial values of the parameters of the three-dimensional shape model using the translation parameters (T _x , T _y , T _z ), the rotation angles (R _x , R _y , R _z ) and the scaling parameter s, which are alignment parameters. Set.

  For each 3D feature point, refer to the image information at the position where the 3D feature point is projected on the face area image using the shape parameter and orientation of the 3D shape model, and the part image information of the 3D feature point. To calculate the local likelihood representing the degree of matching of the 3D feature points, and obtain the likelihood as the degree of matching of the entire 3D feature points by the sum of the local likelihoods. Since the arrangement of 3D feature points is determined by the shape parameter and orientation of the 3D shape model, the shape parameter, position, and orientation of the 3D shape model are perturbed so that the likelihood is maximized. The arrangement of 3D feature points that are most suitable for the 2D face image is obtained, and the 3D position of each face feature point of the face area image is obtained, and the shape parameters, position, and orientation of the 3D shape model at this time are acquired. To do.

(Reference point position determining means 42)
The reference point position determination unit 42 projects the reference 3D reference point 32 stored in the storage unit 30 onto the face region image using the alignment parameter output from the alignment unit 41, and 2D on the face region image. Determine the position of the reference point.

As for the projection of the standard 3D reference point 32 onto the face area image, a general method may be adopted as in the projection of the standard 3D feature point 31. For example, if the pinhole camera model, the three-dimensional coordinates of the n-th reference 3D reference points 32 (X _n, Y _n, Z _n) When the n-th reference 3D reference points 32 the facial area image The position (X _n ′, Y _n ′) of the 2D reference point projected onto the rotation matrix is calculated by the alignment means 41.

And translation vector

And the scaling parameter s can be expressed by Equation (2).

(2)
The right side of the above equation (2) represents the result of converting the standard 3D reference point 32 into a 3D reference point at the rotation / translation position of the three-dimensional shape model that matches the orientation and size of the face represented by the face area image. Yes. That is, the above equation (2) converts the standard 3D reference point 32 into a 3D reference point in the three-dimensional shape of the face represented by the face area image, and projects the 3D reference point onto the face area image. This represents obtaining the position.

FIG. 8 shows the alignment parameter obtained by the alignment means 41.

An example of a 2D reference point 801 obtained by projecting the standard 3D reference point 32 onto the face area image 700 using the formula (2) is shown. As illustrated in FIG. 8, 2D reference points 801 are arranged in accordance with the face orientation of the face area image 700.

The alignment means 41 uses, for example, a feature point extraction method disclosed in Japanese Patent Application Laid-Open No. 2014-178862, so that translation (T _x , T _y , T _z) that is an alignment parameter of the face area image is used. ), Rotation angle (R _x , R _y , R _z ), and scaling parameter s, as well as the three-dimensional position of each face feature point of the face area image, The position of the 2D reference point fitted to the feature point position of the face area image may be obtained using the three-dimensional position, the shape parameter, the position, and the orientation.

Hereinafter, a procedure for obtaining a 2D reference point fitted to the feature point position of the face area image will be described with reference to the schematic diagrams of FIGS. FIG. 9 shows a 3D feature point 901 that is a three-dimensional position of each face feature point of the face area image, a translation (T _x , T _y , T _z ) and a rotation angle obtained as alignment parameters by the alignment means 41. A reference 3D feature point 31 that has been coordinate-transformed using (R _x , R _y , R _z ) is shown. Here, since the 3D feature point 901 is a feature point extracted from an individual's face image, a position that completely matches the reference 3D feature point 31 arranged on the three-dimensional shape 300 of a standard human face. It will not be.

  Since the reference 3D feature point 31 and the reference 3D reference point 32 are both points arranged on the standard three-dimensional shape 300 of the human face, the reference 3D feature point 31 moves to the position of the 3D feature point 901. Thus, when the standard 3D reference point 32 is displaced using a known warping technique such as thin plate spline transformation, a perturbation 3D reference point 1001 that matches the individual feature point position is obtained as shown in FIG.

  FIG. 11 shows the displacement result of the reference point by warping in which the face direction is the front, that is, the translation and the rotation angle are both set to 0 so that the perturbation of the reference point can be easily understood. In this figure, the arrangement of the standard 3D feature point 31, the standard 3D reference point 32, the 3D feature point 901, and the perturbation 3D reference point 1001 is shown. In FIG. 11, facial organs such as eyes and nose are schematically shown so that the arrangement of the feature points and reference points is easy to understand, but in actuality, the reference point position determination means 42 uses the standard 3D feature points 31 and the standard 3D. Only point information of the reference point 32, the 3D feature point 901, and the perturbation 3D reference point 1001 is handled.

  When the extraction results of the reference 3D feature point 31 and the 3D feature point 901 shown in FIG. 11 are compared, the distance between the eyebrows and the eyes is slightly narrower and the nose is positioned slightly above the standard three-dimensional shape of the human face. In addition, it can be seen that there is an individual feature that the mouth is also slightly small, but the warping method reflects the individual feature by warping the reference 3D feature point 31 to move to the position of the 3D feature point 901. It is shown that a perturbed 3D reference point 1001 is obtained.

As for the projection of the perturbation 3D reference point 1001 onto the face area image, a general method may be adopted in the same manner as the projection of the standard 3D feature point 31. For example, in the case of a pinhole camera model, the three-dimensional coordinates of the n-th standard 3D reference point 32 in the above equation (2) are replaced with (X _n , Y _n , Z _n ), and the n-th perturbation 3D reference is referred. Using the three-dimensional coordinates (X _{n ″} , Y _{n ″} , Z _{n ″} ) of the point 1001, a rotation matrix

And translation vector

The position of the 2D reference point obtained by projecting the nth perturbation 3D reference point 1001 onto the face area image by using the shape parameter obtained from the shape parameter, rotation and translation of the three-dimensional shape model instead of the scaling parameter s. (X _n ′, Y _n ′) can be obtained.

(Face matching means 43)
The face collating unit 43 collates the face area image extracted by the face image acquiring unit 10 with all the registered face images 33 stored in the storage unit 30.

  Hereinafter, the matching process in the face matching unit 43 will be described in detail. The face collating unit 43 uses the corresponding reference points for the 2D reference point 801 on the face area image 700 determined by the reference point position determining unit 42 and the 2D reference point of the registered face image stored in the storage unit 30. Find the similarity of. The similarity between reference points is calculated from the feature vector obtained from a local region image around the reference point. As for the method of obtaining the feature vector from the local region image and calculating the degree of similarity, for example, various known methods such as using the luminance and color difference of each pixel in the local region can be used. Description is omitted. Then, the face matching unit 43 selects a registered face image that maximizes the sum of the similarities between the corresponding reference points. If the sum of the similarities is equal to or greater than the authentication threshold, the face matching image 43 appears in the face area image 700. It is determined that the existing face is the face of the registrant corresponding to the registered face image.

  The image processing unit 40 is stored in the storage unit 30 by the processes of the alignment unit 41 and the reference point position determination unit 42 in the same manner as the 2D reference point 801 is determined from the face area image 700. 2D reference points for all registered face images 33 are determined. Alternatively, the image processing unit 40 may determine a reference point for each registered face image by the same method as that for determining the 2D reference point 801 from the face area image 700 every time the face area image is collated. Good.

  If the face collating means 43 determines that the face is the registrant's face, it outputs a signal indicating successful authentication to the output unit 20 and causes the electric lock control device 2 to output a signal for performing unlock control.

<Operation of face image authentication device>
Hereinafter, the operation of the authentication process by the face image authentication apparatus 3 to which the present invention is applied will be described with reference to the flowchart shown in FIG. The operation described below is executed every time one input image is acquired. Further, an example will be described in which 2D reference points for all registered face images 33 stored in the storage unit 30 are determined in advance and stored in the storage unit 30.

  First, the face image acquisition unit 10 of the face image authentication device 3 acquires an input image captured by the imaging device 1 (step S101). Then, the face image acquisition unit 10 extracts a face area image from the input face image (step S102). The face image acquisition unit 10 determines whether or not one or more face area images have been extracted (step S103). If no face area has been extracted, the authentication process ends without performing the subsequent processes. To do. On the other hand, when one or more face area images are extracted, the face image acquisition unit 10 shifts the processing to step S104.

  The following steps S104 to S109 are performed for each face area image extracted by the face image acquisition unit 10.

The alignment means 41 is based on the position of the feature point on the face area image, and the alignment parameter of the reference 3D feature point 31

Is obtained (step S104). Next, the reference point position determining unit 42 outputs the alignment parameter output from the alignment unit 41.

The standard 3D reference point 32 is projected onto the face area image using to determine the position of the 2D reference point (step S105).

  Next, the face collating unit 43 uses the local region image around the 2D reference point determined by the reference point position determining unit 42 and the local region image around the 2D reference point of all the registered face images stored in the storage unit 30. Each feature vector is obtained to calculate the similarity (step S106), and the registered face image with the highest sum of the similarities of the local region images is selected (step S107). If the sum of the similarities of the local area images in the selected registered face image is equal to or greater than the authentication threshold (step S108), the face of the person in the face area image is the face of the registrant corresponding to the registered face image. And a signal indicating successful authentication is output to the output unit 20 (step S109). When a signal indicating successful authentication is output to the output unit 20, a signal for performing unlock control on the electric lock control device 2 is output from the output unit 20. On the other hand, when the sum of the similarities of the local area images in the selected registered face image is less than the authentication threshold, the face matching unit 43 does not perform any particular processing.

  When the processes of steps S104 to S109 are completed for all face area images, the image processing unit 40 ends a series of steps.

  As described above, the face image authentication apparatus according to the embodiment of the present invention is a two-dimensional space of a plurality of extracted feature points extracted from a face area image obtained from an input image and a plurality of reference 3D feature points. A registration parameter is obtained so that a positional deviation in at least one of the three-dimensional spaces is minimized, and a plurality of reference 3D reference points are projected onto the face area image using the registration parameter, so that a plurality of reference of the face area image is obtained. By obtaining the position of the point, it is possible to stably determine the reference position for extracting the feature from the entire face area image regardless of the face direction.

  Further, the reference point for determining the local region position for calculating the feature vector of the face region image can be determined according to the face direction of the face region image. Furthermore, the reference points can be arranged not only around the organs such as the eyes and nose and mouth but also on the skin such as the cheeks and the forehead according to the face direction. By obtaining a feature vector from the local region image around the reference point determined according to the embodiment of the present invention and collating the input face image and the registered face image with the similarity of the feature vector, even a face image other than the front direction It is possible to evaluate the degree of similarity not only in the vicinity of organs such as the eyes and nose and mouth but also in skin parts such as the cheeks and forehead, so that authentication can be performed with high accuracy.

<Modification>
The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. For example, in the present embodiment, the face image acquisition unit 10 processes the input image captured by the imaging device 1 and outputs the processed image to the image processing unit 40. However, the face image acquisition unit 10 acquires the input image from a medium such as a hard disk, May be extracted and output to the image processing unit 40.

  In this embodiment, since the authentication result is used to pass or not, the authentication result is output to the electric lock control device 2. However, the authentication result is transmitted via a communication line such as a general public line or a mobile phone line. May be output to an external device such as a monitoring center device. In addition, a lamp for displaying the authentication result may be provided in the face image authentication device 3 without outputting it to the external device, and the authentication result may be notified to the passerby.

  In the present embodiment, the reference 3D feature points 31 include the right and left eyebrows center for the eyebrows, the eyes, the corners of the eyes, the upper eyelid center, and the lower eyelid center for each of the left eye and the right eye, and the nose point, the nose point, The left and right nose sides and the lower nose side, and the upper lip center upper end, lower lip center lower end, and left and right mouth corner points of the mouth are stored in 20 locations, but fewer or more feature points are set than in this example. May be.

In addition, the alignment means 41 performs alignment parameter

In addition, when the detailed 3D feature point 901 is obtained, the reference point position determination unit 42 perturbs the position of the reference 3D reference point 32 based on the positional relationship between the reference 3D feature point 31 and the 3D feature point 901. The perturbed 3D reference point 1001 may be obtained and the 2D reference point may be determined by projecting the perturbed 3D reference point 1001 onto the face area image.

  In this embodiment, the orthogonal coordinate system is adopted as the coordinate system representing the position, rotation, and translation of the reference 3D feature point 31, the reference 3D reference point 32, the 3D feature point 901, and the perturbation 3D reference point 1001. If the position, rotation, and translation of the 3D feature point 31, the standard 3D reference point 32, the 3D feature point 901, and the perturbation 3D reference point 1001 can be expressed, various coordinate systems can be employed.

In the present embodiment, the alignment unit 41 uses the alignment parameter.

In the example described above, the shape parameter, position and orientation of the three-dimensional shape model and the detailed 3D feature point 901 are obtained. However, the present invention is not limited to this. Alignment parameters

The feature parameters, position and orientation of the three-dimensional shape model and detailed 3D feature points 901 are obtained using the feature point extraction method disclosed in Japanese Patent Application Laid-Open No. 2014-178862. Also good. Furthermore, when the orientation of the face is known in advance, only the shape parameters and positions of the three-dimensional shape model and detailed 3D feature points 901 may be obtained. For example, this is a case where the face orientation is known in advance.

  Further, the face collating means 43 selects a registered face image that maximizes the similarity between corresponding reference points, and the face shown in the face area image 700 is the face of the registrant corresponding to the registered face image. However, the present invention is not limited to this. For example, the face collating unit 43 selects a registered face image that maximizes the sum of the similarities between corresponding reference points and corresponding feature points, and the face reflected in the face area image 700 becomes the registered face image. You may make it determine whether it is a corresponding registrant's face.

  Moreover, although the case where this invention is applied to a face image authentication apparatus was demonstrated to the example, it is not limited to this. For example, the present invention may be applied to a device that estimates the state of the face represented by the input face image. In this case, the face state may be estimated from the feature amount of the 2D reference point determined for the input face image.

  Further, the case where the 2D reference point of the input face image is determined has been described as an example, but the present invention is not limited to this, and the 2D reference point of the image representing the object having a shape other than the face is determined. May be. For example, a 2D reference point of an image representing a car may be determined.

  As described above, those skilled in the art can make various changes in accordance with the embodiment to be implemented within the scope of the present invention.

3 Face Image Authentication Device 31 Standard 3D Feature Point 32 Standard 3D Reference Point 40 Image Processing Unit 41 Positioning Unit 42 Reference Point Position Determination Unit 43 Face Matching Unit 100 Access Control System 700 Face Area Image 701 Face Feature Point 702 Standard 2D Feature Point 801 2D reference point 901 3D feature point 1001 Perturbation 3D reference point

Claims (4)

A reference point position determination device that determines the position of each of a plurality of reference points in an input object image obtained by imaging an object,
A plurality of standard 3D feature points representing the three-dimensional positions of the plurality of feature points in the three-dimensional object model, and a plurality of standard 3D references representing the three-dimensional positions of the plurality of reference points in the three-dimensional object model. Storage means for storing points in advance;
The cubic of the object represented by the input object image so that the deviation in at least one of the two-dimensional space and the three-dimensional space between the plurality of extracted feature points extracted from the input object image and the plurality of reference 3D feature points is minimized. and alignment means asking you to three-dimensional position of each of Rutotomoni the plurality of extracted feature points determined parameters for correcting the original shape orientation and shape of the alignment parameters,
Corresponding to the plurality of reference 3D reference points based on the three-dimensional position of each of the plurality of extracted feature points obtained by the alignment means and the three-dimensional position of each of the plurality of reference 3D feature points. , Obtaining a plurality of 3D reference points in the three-dimensional shape of the object represented by the input object image, projecting the plurality of 3D reference points onto the input object image using the alignment parameter, and Reference point position determining means for obtaining the position of each of the plurality of reference points;
A reference point position determining device including: The alignment means includes
Based on the plurality of 3D feature points corresponding to the plurality of reference 3D feature points and the part image information of each of the plurality of 3D feature points arranged on a predetermined three-dimensional shape model, the plurality of the plurality of 3D feature points For each 3D feature point, it is calculated based on the image information at the position where the 3D feature point is projected on the input object image using the parameters and the part image information of the 3D feature point. Obtaining the three-dimensional position of each of the 3D feature points as the three-dimensional position of each of the plurality of extracted feature points so that the sum of local likelihoods representing the fitness of the 3D feature points is maximized, and the alignment The reference point position determining apparatus according to claim 1 , wherein the parameter is obtained. A reference point position determination device that determines the position of each of a plurality of reference points in an input object image obtained by imaging an object,
A plurality of standard 3D feature points representing the three-dimensional positions of the plurality of feature points in the three-dimensional object model, and a plurality of standard 3D references representing the three-dimensional positions of the plurality of reference points in the three-dimensional object model. Storage means for storing points in advance;
The cubic of the object represented by the input object image so that the deviation in at least one of the two-dimensional space and the three-dimensional space between the plurality of extracted feature points extracted from the input object image and the plurality of reference 3D feature points is minimized. Alignment means for obtaining a parameter for correcting at least one of the orientation and shape of the original shape as an alignment parameter;
Reference point position determining means for projecting the plurality of reference 3D reference points onto the input object image using the alignment parameter to determine the position of each of the plurality of reference points of the input object image;
Only including,
The reference point position determination device in which the plurality of reference 3D reference points are set so that the reference 3D reference points are not arranged at positions where the image characteristics of the object are expected to fluctuate in advance . The feature amount extracting unit further extracts a feature amount for collating with a registered object image from the input object image based on the positions of the plurality of reference points obtained by the reference point position determining unit. The reference point position determination apparatus according to claim 3 .