JP4577771B2 - Face image recognition device - Google Patents

Description

  The present invention relates to a face image recognition device, and more particularly to a face image recognition device that can recognize a face image close to a human subjective judgment result and can be used for searching for a person with a similar face image.

  In recent years, research and development of face image recognition by computers has been actively conducted. Conventionally, several recognition techniques have been proposed for personal authentication using face images. The inventor of the present invention has proposed an image recognition apparatus capable of performing high-precision personal authentication in the following patent documents (prior application) 1 and 2. This uses recognition technology based on the subspace method. When registering a face dictionary in advance, a plurality of face images of each person are captured, common components of the captured face images are extracted, and dictionary data is extracted. As a result, it is possible to absorb the effects of fluctuations in posture caused by the inclination of the face and lighting fluctuations caused by environmental differences.

  FIG. 7 is a block diagram of a conventional image recognition apparatus proposed in the following patent documents (prior application) 1 and 2. This image recognition device is used to verify whether a person is the person from the image obtained by projecting a person's face or to identify who the person is, Then, it is comprised from the dictionary registration part 71 which registers dictionary data previously, and the recognition process part 72 which collates and identifies a person from an input image.

The dictionary registration unit 71 creates dictionary data for registration in the database 73. In creating the dictionary data, first, one person having the identification information ID is photographed, and a photographed image of the person is input. The normalization processing unit 74 outputs a normalized image x _ID obtained by normalizing the input image with respect to position, area, size, luminance level, and the like. Such normalized image x _ID creating plural for the same person.

The dictionary data creation unit 75 obtains an autocorrelation matrix from a plurality of input normalized images x _ID , extracts its eigenvectors, and creates a subspace Φ _ID spanned by them. The dictionary data A is obtained by adding the identification information ID of the person to the partial space Φ _ID . The database 73 stores the dictionary data A created by the dictionary data creation unit 75. The database 73 also stores other person's dictionary data B, C,.

  After the dictionary data A, B, C,... Are accumulated in the database 73 as described above, the face of an unspecified person is photographed, and one image obtained thereby is input to the recognition processing unit 72. Then, the normalization processing unit 76 normalizes the input image with respect to position, area, size, brightness, and the like, and outputs a normalized image x, as in dictionary registration. The normalized image x is input to the correlation value calculation unit 77.

  The correlation value calculation unit 77 compares the normalized image x input from the normalization processing unit 76 with the registered images in the dictionary data A, B, C,. (Correlation value) S is calculated.

In the case of the collation process, the identification information ID _{S of the} person to be collated is designated, and the normalized image x input from the normalization processing unit 76 and the dictionary of the person of the identification information ID _S stored in the database 73 The similarity S is obtained by comparing with the data. In the case of the identification process, the normalized image x input from the normalization processing unit 76 and the registered images in the dictionary data A, B, C,... By comparison, the person identification information ID _r indicating the maximum similarity S is obtained.

In the case of collation processing, the similarity S is input to the threshold value determination unit 78, and whether or not the person in the input image is the person with the identification information ID _S depending on whether or not the similarity S is greater than or equal to the threshold T (OKorNG) is determined. In the case of the identification process, the person identification information ID _r indicating the maximum similarity S, and the similarity S if necessary are output.
Japanese Patent Application No. 2004-235281 (prior application) Japanese Patent Application No. 2004-337465 (prior application)

  When determining whether or not a person's face is similar, it is considered that a human is not necessarily paying attention to the whole face but paying attention to specific parts such as eyes, nose and mouth. For example, if all parts of the eyes, nose, mouth, etc. are not similar, only the eyes are very similar, or only the nose is very similar and other parts are slightly different Often, the subjective judgment is that both faces are similar.

  However, the recognition technology used in the field of personal authentication objectively determines whether or not people are similar using the distance in the space for the entire face image. When used for image retrieval or the like, the result may be different from a result subjectively determined by a human.

  Facial image recognition can be applied to biometrics authentication using facial images, as well as to entertainment that automatically determines who is the person on the camera, and how much the person is similar. In any application, the recognition result of the image recognition apparatus can be given reliability by bringing the result of face image recognition closer to the human subjective determination result.

  When considering application to entertainment such as searching and presenting facial images of similar celebrities, the degree of interest may differ depending on whether or not a recognition result close to the result of subjective judgment is obtained. Therefore, it is particularly preferable that the result of facial image recognition be close to the human subjective judgment result.

  An object of the present invention is to provide a face image recognition apparatus capable of bringing a face image recognition result close to a human subjective determination result.

In order to solve the above problems, the present invention obtains a difference image of a pair of subjectively similar face images prepared in advance, calculates a principal component analysis of the difference image, and derives an eigenvector, and a registration Kaoga projected image to the difference space spanned is derived eigenvectors by pre Symbol differential space creating unit and the vector on the interpolation space the registered image and the registered image is a vector difference between the projected component in the difference space a registered face image creation unit that derives a projection and the input face image and input face image input face image image difference space spanned by eigenvectors derived by pre Symbol differential space creating unit is projected to the difference spatial components along with deriving a vector of the interpolation space is the vector difference of the registered face image and the input face image by using the vector of the interpolation space derived by the vector and the registered face image creation unit in the interpolation space There is a first feature in that provided an authentication processing unit for obtaining the similarity score.

Further, in the present invention, the authentication processing unit is derived by a vector on an interpolation space that is a vector difference between an input face image and a component of the input face image projected on the difference space, and the registered face image creation unit. The second feature is that the similarity between the registered face image and the input face image is obtained by measuring the similarity between vectors on the interpolation space.

  Further, in the present invention, the registered face image creation unit derives a vector on an interpolation space for each of a plurality of registered face images of one person, and the authentication processing unit is derived by the registered face image creation unit. The similarity between the registered face image and the input face image is obtained by measuring the similarity between the principal component space spanned by the vector on the interpolation space and the vector on the interpolation space derived by the registered face image creation unit. There are three features.

  According to the first feature of the present invention, a component on a difference space spanned by an eigenvector derived by principal component analysis of a difference image, that is, a component that is not used when humans subjectively perform similarity determination is removed. Since the similarity between the registered face image and the input face image is obtained using the component in the interpolation space, that is, the component that contains a lot of components that are important when humans subjectively judge the similarity. It is possible to bring the result of the calculation of the image and the search result of the similar face image close to the human subjective judgment result. Therefore, it is possible to give reliability to a recognition result in an image recognition apparatus that performs authentication and search of a person using a face image. In addition, it can be effectively applied to entertainment.

  Further, according to the second feature, it is only necessary to derive a vector on the interpolation space for one registered face image in the registered face image creation process, so that dictionary data can be easily registered.

  Furthermore, according to the third feature, since a principal component space spanned by a vector on an interpolation space derived for each of a plurality of registered face images of one person is used, illumination variation or Even if there is a face movement or the like, the similarity between the registered face image and the input face image can be accurately obtained without being affected by them.

  The present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram showing an embodiment of a face image recognition apparatus according to the present invention. The face image recognition device according to the present embodiment is roughly composed of a difference space creation unit 1, a registered face image creation unit 2, and an authentication processing unit 3 when roughly classified by function.

First, the difference space creation unit 1 will be described. Here, similar face image pairs g ^a ₁ and g ^b ₁ , g ^a ₂ and g ^b ₂ ,..., G ^a _NN and g ^b _NN of persons who are subjectively similar are prepared. There may be one or more similar face image pairs. The difference image creation unit 1-1 derives a difference image between each pair, and creates a difference image (vector) d _i on the image space represented by Expression (1).

Next, principal component analysis is performed on the difference image set, and eigenvectors e ₁ , e ₂ ,..., E _MM are derived by extracting those common components. Specifically, first, the difference image d _i created by the difference image creation unit 1-1 is input to the autocorrelation matrix creation unit 1-2, and the autocorrelation matrix R is created by Expression (2). In the formula (2), T represents transposition.

Next, the autocorrelation matrix R is input to the eigenvector deriving unit 1-3 to derive eigenvectors e ₁ , e ₂ ,..., E _MM . The eigenvectors e ₁ , e ₂ ,..., E _MM represented by the expression (3) span the principal component space (referred to as difference space) Ψ of the difference image.

FIG. 2 is an explanatory diagram of the differential image derivation in the present invention. Similar face image pairs g ^a ₁ and g ^b ₁ , g ^a ₂ and g ^b ₂ ,..., G ^a _NN and g ^b _NN of a person judged to be subjectively similar are prepared, and their similarities A difference image between the face image pairs is derived. The difference image g ^a _i -g ^b _i derived in this way can be said to be a component that is not used when a human subjectively makes a similarity determination.

FIG. 3 is an explanatory diagram of derivation of eigenvectors e ₁ , e ₂ ,..., E _MM and difference space in the present invention. The difference image g ^a _i -g ^b _i between each pair is represented in the image space. The autocorrelation matrix creating unit 1-2 and the eigenvector deriving unit 1-3 perform principal component analysis on the difference image set, extract common components thereof, and perform eigenvectors e ₁ , e _{2 ,.} Is derived. The eigenvectors e ₁ , e ₂ ,..., E _MM span a difference space Ψ.

Next, the registered face image creation unit 2 will be described. The registration face image creation section 2 a plurality of registered face images _{x j (j = 1,2, ···} , L) inputs a. These registered face images _xj are images obtained by normalizing the face area images of each person.

The difference space projection unit 2-1 of the registered face image creating unit 2 projects the input registered face image x _j onto the difference space Ψ created by the difference space creating unit 1, and the vector x _j ′ on the difference space is projected. To derive.

Complement the space vector generation unit 2-2, to create a vector Δx _j on the complementary space by subtracting the vector x _{j 'in} the difference space from the registered face image x _j. The database (DB) 4 stores the vector Δx _j on the complementary space created by the complementary space vector creating unit 2-2 as dictionary data. As dictionary data to be stored in the database 4, preferably storing face images and names of the person linked to the vector [Delta] x _j on complement.

FIG. 4 is an explanatory diagram of vector creation on the complementary space in the present invention. If the face image is x, first, the face image x is projected onto the difference space Ψ to derive a vector x ′ on the difference space Ψ. Next, a vector Δx _j on the complementary space is created by subtracting the vector x ′ from the face image x.

  A vector x ′ on the difference space Ψ derived by projecting the face image x onto the difference space Ψ is a component that is not used when a human subjectively makes a similarity determination in the face image x. Therefore, the vector Δx on the complementary space is a vector that includes a large amount of components that are important when humans subjectively make similarity determinations when this component is absorbed.

Next, the authentication processing unit 3 will be described. After the vector Δx _j in the complementary space for the registered face image x _j is accumulated in the database 4, the face image y of the person to be recognized is input to the recognition processing unit 3. The input face image y is an image obtained by normalizing the face area image of the person to be recognized.

  The difference space projection unit 3-1 derives a vector y ′ on the difference space Ψ for the input face image y. In addition, the interpolation vector creation unit 3-2 creates a vector Δy on the complementary space. These are the same as the difference space projection unit 2-1 and the complementary space vector creation unit 2-2 in the registered face image creation unit 2. A vector y ′ on the difference space Ψ is expressed by Expression (4), and a vector Δy on the complementary space is expressed by Expression (5).

The similarity measuring unit 3-3 compares the vector Δy on the complementary space input from the interpolation vector creating unit 3-2 with the vector Δx _j on the complementary space stored in the database 4, and compares the similarity S _j Measure.

Similarity ranking determining unit 3-4, sorts the similarities S _j in descending order, the largest person similarity S _j, or similarity S _j is the face image and name of few persons in descending order from the database 4 Extracted and sent out as a similar face image search result.

Similarity measurement can be performed by various methods. In the above embodiment, the measurement of similarity by the first method of measuring the similarity S _j of the vector [Delta] x _j and the vector [Delta] y. As shown in FIG. 5, this method can be realized by obtaining cos θ as the similarity S _j where θ is the angle formed between the vectors Δx _j and Δy, and cos θ is expressed by equation (6). in so that, as the molecular inner product of the vector [Delta] x _j and [Delta] y, it can be determined by calculation that the product of the vector [Delta] x _j, the [Delta] y respectively sized and molecules. In this case, the similarity S _j is −1 ≦ S _j ≦ 1, so that the similarity is closer to 1, and the similarity is closer to −1.

The degree of similarity can also be measured by the second method shown below. This method measures a similarity S _j between a principal component space created from a vector Δx _jk (k = 1, 2,..., N) on a plurality of N complementary spaces and a vector Δy for one person. As shown in FIG. 6, when the angle formed between the principal component space of the vector Δx _jk and Δy is θ, the similarity S _j is expressed as cos ² θ expressed by the equation (7). It can be realized by seeking. In this case, the similarity S _j is 0 ≦ S _j ≦ 1, and the closer to 1, the more similar, and the closer to 0, the dissimilar.

This method requires component analysis of a vector Δx _jk (k = 1, 2,..., N) on the complementary space for each person and generation of the principal component space. The principal component space of the vector Δx _jk on the complementary space for each person is the complementary space for each person from face images x _jp (p = 1, 2, N) obtained by photographing N faces of each person. The above vector Δx _jk is calculated, an autocorrelation matrix R (or covariance matrix C) is calculated from these vectors Δx _jk , eigenvectors of the calculated matrix are derived, and M (M ≦ N eigenvectors _{u 1,} u 2) of, ..., removed _{u M,} the eigenvectors _{u 1,} u 2, ..., can be generated by obtaining the space spanned by _{u M.}

  Since the present invention provides a face image recognition result close to human subjective judgment, it can be effectively applied to entertainment. For example, a facility that provides entertainment is equipped with a face image recognition device according to the present invention together with a camera and a display, the face of a visitor is photographed with the camera, and a face image of a celebrity who is similar to the person is taken out of the database and displayed. Or how much they are similar. Moreover, if the similarity for each region of the face image is measured, the similarity for each region can be presented in an expression such as what percentage. Furthermore, both face images and composite face images can be displayed on a display or output by a printer.

It is a functional block diagram which shows one Embodiment of the face image recognition apparatus which concerns on this invention. It is explanatory drawing of the difference image derivation in this invention. It is explanatory drawing of derivation | leading-out of the eigenvector and difference space in this invention. It is explanatory drawing of the vector creation on the complement space in this invention. It is explanatory drawing of the similarity measurement in this invention. It is other explanatory drawing of the similarity measurement in this invention. It is a block block diagram of the conventional image recognition apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Difference space creation part, 1-1 ... Difference image creation part, 1-2 ... Auto correlation matrix creation part, 1-3 ... Eigen vector calculation part, 2 ... Registration face image creation Units, 2-1, 3-1 ... difference space projection unit, 2-2, 3-2 ... complementary space vector creation unit, 3 ... authentication processing unit, 3-3 ... similarity measurement 3-4... Similarity determining unit 4, 73... Database 71. Dictionary registration unit 72. Recognition processing unit 74.76 Normalization processing unit 75. ..Dictionary data creation unit, 77 ... Correlation value calculation unit

Claims (3)

A difference space creation unit that obtains a differential image of a pair of subjectively similar facial images prepared in advance and derives an eigenvector by performing principal component analysis on the differential image;
Projecting the difference space eigenvectors derived registration Kaoga image by pre Symbol differential space creating unit spanned, vectors on the interpolation space the registered image and the registered image is a vector difference between the projected component in the difference space A registered face image creation unit for deriving
The projected input face image and input face image input face image image difference space spanned by eigenvectors derived by pre Symbol differential space creating unit is a vector difference between the projected component to the difference space on the interpolation space An authentication processing unit is provided that derives a vector and obtains a similarity between the registered face image and the input face image using the vector on the interpolation space and the vector on the interpolation space derived by the registered face image creation unit. A facial image recognition apparatus characterized by the above. The authentication processing unit includes a vector on an interpolation space that is a vector difference between an input face image and a component of the input face image projected on the difference space, and a vector on the interpolation space derived by the registered face image creation unit. The face image recognition apparatus according to claim 1, wherein the similarity between the registered face image and the input face image is obtained by measuring the similarity. The registered face image creation unit derives a vector on an interpolation space for each of a plurality of registered face images of one person,
The authentication processing unit measures a similarity between a principal component space spanned by a vector on the interpolation space derived by the registered face image creation unit and a vector on the interpolation space derived by the registered face image creation unit. The face image recognition apparatus according to claim 1, wherein a similarity between the registered face image and the input face image is obtained.

Images