JP5514599B2 - 2D / 3D combined verification device - Google Patents

Description

  The present invention relates to a 2D / 3D combined verification apparatus that combines a two-dimensional verification (2D verification) and a three-dimensional verification (3D verification) to verify an object.

  In recent years, in the fields that require personal authentication, such as access control to computer rooms and important machine rooms, and access control to computer terminals and bank financial terminals, biometric information has been used instead of conventional PINs and ID cards. A system for personal authentication based on the above has been developed.

  Personal authentication systems using biometric information are attracting attention because they do not require memory or possession and have a low risk of theft. In personal authentication using biometric information, fingerprints, irises, faces, palm prints, handwriting, and the like are used. Among them, personal authentication using a face is attracting attention because of its high convenience and acceptability.

  For example, as a face authentication method used for entrance / exit management or the like, 2D face authentication using a 2D image (2D face image) taken with a camera or the like, and a 3D image (using a 3D scanner, stereo vision, etc.) 3D face authentication using (3D face image). In the case of 2D face authentication, the authentication performance decreases due to changes in brightness and face orientation, but robust authentication is possible against partial changes in facial expressions and glasses. On the other hand, in the case of 3D face authentication, although the authentication performance deteriorates due to a change in facial expression, authentication that is robust to changes in brightness and face orientation is possible. As described above, 2D face authentication and 3D face authentication each have advantages and disadvantages.

  Therefore, for example, Patent Document 1 proposes a 2D / 3D combined authentication apparatus that performs authentication by combining 2D face authentication and 3D face authentication. In this 2D / 3D composite authentication apparatus, both two-dimensional information (2D face image) and three-dimensional information (3D face image) of an authentication object are acquired, and authentication is performed by adjusting the weights of both. Specifically, the similarity between the three-dimensional information of the authentication object (verification object) and the three-dimensional information of the comparison object (registration object) is calculated as the first similarity, and the two-dimensional of the authentication object The similarity between the information and the two-dimensional information of the comparison object is calculated as the second similarity, the first similarity and the second similarity are synthesized using the adjusted weighting factor, and the authentication object And the total similarity with the comparison object are calculated, and authentication determination is performed based on the calculated total similarity. Thereby, 2D face authentication and 3D face authentication are combined, and the authentication performance is improved.

JP 2007-58393 A JP 10-132534 A (Patent No. 3347608) JP 2007-164669 A JP-A-9-277184

Introduction to computer image processing (Nippon Industrial Technology Center, published by Soken Publishing Co., Ltd., pages 44-45) P. J. Besl and N. D. McKay: "A method for registration of 3-D shapes", IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256 (Feb. 1992) X. Lu and A. K. Jain: "Matching 2.5D face scans to 3D models", IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 31-43 (Jan. 2006)

  However, in the above-described conventional 2D / 3D composite authentication device, it is necessary to always obtain the similarity between the two-dimensional information and the similarity between the three-dimensional information, and particularly, the processing amount for obtaining the similarity of the three-dimensional information is large. There was a problem that authentication took time. That is, in the conventional 2D / 3D composite authentication apparatus, both the 2D face image matching and the 3D face image matching are substantially always performed, so that the calculation processing amount for the 2D face image matching is increased. There is a problem in that the amount of calculation processing for collating 3D face images is added and the collating speed cannot be increased.

  The present invention has been made in order to solve such problems. The object of the present invention is to achieve both improvement in collation performance and increase in collation speed by combining 2D collation and 3D collation. An object of the present invention is to provide a 2D / 3D collation device that can be used.

In order to achieve such an object, the present invention extracts a region to be collated from at least one of an image captured from the first direction and an image captured from the second direction. 3D collation image creating means for creating a 3D image of a collation object in the region as a 3D collation image, 2D collation image creating means for creating a 2D image of the collation object in the collation target region as a 2D collation image, and the 2D 2D collation means for collating the 2D collation image created by the collation image creation means with two-dimensional images of a plurality of registration objects stored in advance as 2D registration images, and 2D collation by the two-dimensional collation means When all the matching results between the image and the 2D registered image are not consistent, the 3D matching image created by the 3D matching image creating unit is preliminarily set as the 3D registered image. And a three-dimensional collating means for collating the 3-dimensional images of a plurality of registered objects stored in each individual, 3D match image created by the 3D match image creation means is operable to determine whether or not facing forward, front When it is determined that it is not facing, a front determination unit that immediately executes a comparison between the 3D verification image and the 3D registered image by the three-dimensional verification unit is provided.

According to the present invention, it is determined whether or not the 3D collation image created by the 3D collation image creating unit is facing the front. If it is not determined that the 3D collation image is not facing the front, that is, it is determined that the front is facing the front. In the case where it is done, the 2D collation image and the 2D registered image are collated (2D collation). If a 2D registered image that matches the 2D collation image is found by this 2D collation, a collation result indicating “match” is obtained at that time. If the 2D registered image that matches the 2D collated image is not found, the 3D collated image and the 3D registered image are collated (3D collation) assuming that the 2D collation cannot be performed. On the other hand, when it is determined that it is not facing the front, the 3D collation image and the 3D registered image are collated immediately (3D collation).

  In the present invention, the 2D collation image may be created as a two-dimensional image when the 3D collation image (a three-dimensional image of the collation target object in the collation target region) is viewed from the front. You may make it produce as a two-dimensional image of the collation target object in the collation target area | region extracted from the image imaged from the direction and the image imaged from the 2nd direction.

Also, in the present invention, information indicating the characteristics of the verification target obtained from the 3D verification image created by the 3D verification image creation means or the characteristics of the verification target obtained from the 2D verification image created by the 2D verification image creation means 3D collation images created by 3D collation image creation means, such as narrowing down a 3D image to be used as a 3D registration image from a plurality of stored three-dimensional images of registration objects based on information indicating Based on the information indicating the characteristics of the verification target obtained from the information or the information indicating the characteristics of the verification target obtained from the 2D verification image created by the 2D verification image creation means, A two-dimensional image to be used as a 2D registered image may be narrowed down from the two-dimensional images.

  In the present invention, information (for example, the distance from the camera to the face) that defines the actual size of the comparison target area obtained by the 3D collation image creating means is obtained as the regulation information, and the obtained regulation is obtained. You may make it normalize the image size of the 2D collation image created by the 2D collation image creation means based on the information. In this case, the two-dimensional collation unit collates the 2D collation image normalized by the normalization unit with the 2D registered image.

According to the present invention, it is determined whether or not the 3D collation image created by the 3D collation image creating unit is facing the front. When it is determined that the 3D collation image is not facing the front, While checking with a registered image is performed immediately, when it is not determined that it is not facing the front (when it is determined that it is facing the front), a matching between a 2D matching image and a plurality of 2D registered images In the case where all the matching results between the 2D matching image and the 2D registered image do not match, the 3D matching image and the plurality of 3D registered images are checked. immediately verification result is obtained if, 3D collation image is assumed that only 3D verification if it is unable to determine the case and 2D collation is determined not facing the front is performed, 2D irradiation And it is possible to achieve both faster improvement and matching the speed of the matching performance due to complex with 3D matching.

It is a block block diagram which shows one Embodiment of 2D / 3D composite collation apparatus which concerns on this invention. It is a flowchart of the 1st example (Embodiment 1) of the collation process which the control part in this 2D / 3D compound collation apparatus performs. 3 is a flowchart of a first example of collation processing following FIG. 2. It is a flowchart of the 1st example of the collation process following FIG. It is a figure explaining a mode that 3D collation face image and 2D collation face image are produced in the 1st example of this collation processing. It is a figure which shows the example of the coordinate of the face area | region calculated | required in the 1st example of this collation process, and the coordinate of a pupil (2D pupil coordinate, 3D pupil coordinate). It is a figure explaining the creation example of the 2D collation face image from the collation object area | region in the image of a left camera, or the image of a right camera. It is a figure explaining collation for every local area by the phase only correlation method of 2D collation face image and 2D registration face image. It is a figure explaining another example of collation for every local area by the phase only correlation method of 2D collation face image and 2D registration face image. 3 is a functional block diagram according to the first embodiment. FIG. 3 is a functional block diagram in a case where a 2D collation face image is created from an image of the left camera in Embodiment 1. FIG. It is a flowchart of the 2nd example (Embodiment 2) of the collation process which a control part performs. It is a flowchart of the 2nd example of the collation process following FIG. It is a flowchart of the 2nd example of the collation process following FIG. 12 and FIG. FIG. 6 is a functional block diagram of a second embodiment. FIG. 10 is a functional block diagram when a 2D collation face image is created from an image of a left camera in the second embodiment.

Hereinafter, the present invention will be described in detail with reference to the drawings. In addition, in the following description, what is not included in the scope of the right of the present invention is described as an embodiment, but here, it will be described as an embodiment. FIG. 1 is a block diagram showing an embodiment of a 2D / 3D combined verification apparatus according to the present invention.

  In FIG. 1, 10 is a first CCD camera, 11 is a second CCD camera, 12 is a liquid crystal display (LCD), 20 is a processing unit, and the processing unit 20 includes a control unit 20-1 having a CPU. ROM 20-2, RAM 20-3, hard disk (HD) 20-4, frame memory (FM) 20-5, external connection unit (I / F) 20-6, and Fourier transform unit (FFT) 20 -7, and the ROM 20-2 stores a 2D / 3D combined verification program as a program unique to the present embodiment. The HD 20-4 stores two-dimensional images and three-dimensional images of a plurality of registration objects (human faces) as 2D registered face images and 3D registered face images. The 2D registered face image and the 3D registered face image will be described later.

  In the 2D / 3D combined verification apparatus, the CCD cameras 10 and 11 are installed with a predetermined distance L1. That is, the CCD cameras 10 and 11 are arranged side by side in the horizontal direction with the distance between the lenses 10-1 and 11-1 being L1. In the figure, for easy understanding, the 2D / 3D combined verification device is a diagram seen from above, and the object (human face) is a diagram seen from the side.

[Verification of object: Embodiment 1]
In this 2D / 3D combined verification apparatus, when the verification target is a human face M1, the verification of the face M1 is performed as follows. 2 to 4 show flowcharts of a first example (embodiment 1) of collation processing executed by the control unit 20-1 in accordance with the 2D / 3D composite collation program stored in the ROM 20-2.

[Image capture]
The user stands in front of the CCD cameras 10 and 11. Then, the control unit 20-1 recognizes that the face M1 has entered the imaging areas of the CCD cameras 10 and 11, and captures the face S1 from the CCD camera 10 via the frame memory 20-5 (S1). FIG. 5A: an image captured from the first direction) is captured as an image of the left camera, and an image S2 capturing the face M1 from the CCD camera 11 (FIG. 5B: an image captured from the second direction). ) As a right camera image (FIG. 2: Step S101).

[Face detection, pupil detection]
Then, the control unit 20-1 performs face detection and pupil detection by image processing such as a separability filter on the captured image S1 of the left camera, and coordinates of the face region and left and right pupil coordinates (2D pupil). Coordinates) is obtained (step S102). FIG. 6A shows an example of the coordinates of the face area (face area (upper left), face area (lower right)) at this time. FIG. 6B shows an example of the left and right pupil coordinates (2D pupil coordinates (left), 2D pupil coordinates (right)) at this time. In this example, face detection and pupil detection are performed on the left camera image S1, but face detection and pupil detection may be performed on the right camera image S2.

[Create 3D verification face image]
Next, the control unit 20-1 sets the face area obtained in step S102 as the verification target area AR1, and performs 3D verification on the three-dimensional image of the face M1 in the verification target area AR1 from the left camera image S1 and the right camera image S2. A face image S3 (FIG. 5C) is created (step S103). The 3D collation face image S3 is created by a method (a method based on the principle of triangulation) shown in Patent Document 2 for which the applicant has already obtained a patent. In this method, the phase-only correlation method is used for associating stereo images, and face 3D data (actually measured values) can be acquired with high accuracy.

[Acquire 3D pupil coordinates]
And the control part 20-1 calculates | requires the coordinate of the right and left pupil in 3D collation face image S3 as 3D pupil coordinate from the 2D pupil coordinate calculated | required by step S102 (step S104). FIG. 6C shows an example of the left and right pupil coordinates (3D pupil coordinates (left), 3D pupil coordinates (right)) at this time. In addition, you may make it obtain | require this 3D pupil coordinate immediately by 3D measurement from 2D pupil coordinate, before obtaining 3D collation face image S3.

[3D alignment (pupil ICP)]
And the control part 20-1 uses the reference | standard template arrange | positioned on the basis of the position and attitude | position of a standard human pupil as a three-dimensional pattern of an attention part (for example, refer patent document 3), ICP (Iterative Closest Point) The 3D matching face image S3 is aligned by an algorithm (for example, see Patent Document 4) (step S105). The alignment based on the position of the 0 pupil of the reference template is referred to as a pupil ICP.

  In the ICP algorithm, when the data of the reference template is M and the data of the 3D matching face image S3 is M ′, the position and orientation of the 3D matching face image S3 are as follows (1) to (4). Matched with reference template.

(1) For all the points mi included in M, the closest point mi ′ is searched from M ′, and the set of obtained mi and mi ′ is set as a set of corresponding points.
(2) Based on the current correspondence relationship, the optimum motion pattern (rotation R, translation t) between the two M-dimensional pattern data M and M ′ is estimated. Specifically, the translation t is obtained from the 3D coordinates of the pupil. The rotation R is estimated by the least square method.
(3) M ′ is coordinate-transformed using R and t obtained in (2) above.
(4) The above (1) to (3) are repeated until the change in R and t becomes sufficiently small and converges with respect to M and M ′ after coordinate transformation.

  The controller 20-1 aligns the 3D matching face image S3 with the pupil ICP in this manner, and then uses the 3D matching face image S3 that has been aligned as a 3D matching face image 3G0 (FIG. 5 (d)). Are stored in the HD 20-4 (step S106). Further, a two-dimensional face image obtained by viewing the 3D collation face image 3G0 from the front is acquired as a 2D collation face image 2G0 (FIG. 5E) and stored in the HD 20-4 (step S107).

  In this example, the 2D face image obtained by viewing the 3D matching face image 3G0 from the front is acquired as the 2D matching face image 2G0. However, as shown in FIG. 7, the matching target area of the image S1 of the left camera A two-dimensional image of the face M1 in AR1 may be acquired as the 2D collation face image 2G0. In addition, the verification target area AR1 is extracted from the image S2 of the right camera, and the two-dimensional image of the face M1 in the verification target area AR1 extracted from the image S2 of the right camera is acquired as the 2D verification face image 2G0. Also good.

[2D verification]
Next, the control unit 20-1 acquires the number of 2D registered face images stored in the HD 20-4 as N1max (FIG. 3: Step S108), and sets N1 (initial value 0) to N1 = N1 + 1 (Step S108). S109) After confirming that N1 ≦ N1max (YES in step S110), the N1th 2D registered face image is read from the HD 20-4 (step S111). In this case, if the 2D registered face images stored in the HD 20-4 are 2G1 to 2GN1max, N1 = 1st 2D registered face image 2G1 is read.

  Note that the 2D registered face images 2G1 to 2GN1max stored in the HD 20-4 at this time are two-dimensional face images at the time of registration obtained in the same manner as the 2D collation face image 2G0. However, when creating the 2D registered face images 2G1 to 2GN1max, the registered object (human face) M1 is imaged with the posture corrected with the distance from the cameras 10 and 11 as a specified distance. In this example, the 2D registered face images 2G1 to 2GN1max are obtained as two-dimensional face images when 3D registered face images 3G1 to GN1max described later are viewed from the front.

  Then, the control unit 20-1 reads the 2D matching face image 2G0 stored in the HD 20-4 (step S112), and reads the read 2D matching face image 2G0 and the 2D registered face image 2G1 read in step S111. Collation is performed (step S113). The collation between the 2D collation face image 2G0 and the 2D registered face image 2G1 is performed by dividing the 2D collation face image 2G0 into local regions, and collating the corresponding local regions in the 2D registered face image 2G1 for each of the divided local regions. Is performed by the phase-only correlation method (POC).

[Verification of 2D verification face image and 2D registered face image by phase only correlation method (2D verification)]
The collation between the 2D collation face image 2G0 and the 2D registered face image 2G1 in step S113 is performed as follows. The control unit 20-1 divides the 2D verification face image 2G0 (FIG. 8A) into m × n local regions 2G0 (i, j) (FIG. 8B). Then, i = 1 and j = 1, and the image data of the local region 2G0 (1,1) of the 2D matching face image 2G0 is sent to the Fourier transform unit 20-7 to perform two-dimensional discrete Fourier transform. Thereby, the image data of the local region 2G0 (1,1) becomes Fourier image data 2F0 (1,1). The two-dimensional discrete Fourier transform is described in, for example, Non-Patent Document 1 and the like, and detailed description thereof is omitted here.

  Next, the control unit 20-1 divides the 2D registered face image 2G1 (FIG. 8B) into m × n local regions 2G1 (i, j) (FIG. 8D). Then, i = 1 and j = 1, and the image data of the local region 2G1 (1,1) of the 2D registered face image 2G1 is sent to the Fourier transform unit 20-7 to perform two-dimensional discrete Fourier transform. Thereby, the image data of the local region 2G1 (1, 1) becomes Fourier image data 2F1 (1, 1).

  And the control part 20-1 synthesize | combines Fourier image data 2F0 (1, 1) and Fourier image data 2F1 (1, 1), and obtains synthetic Fourier image data.

  Here, when the Fourier image data 2F0 is set to A · exp (jθ) and the Fourier image data 2F1 is set to B · exp (jφ), the composite Fourier image data has a complex conjugate of the Fourier image data 2F1 to the Fourier image data 2F0. It is expressed by A · B · exp (j (θ−φ)) obtained by multiplication. However, A, B, θ, and φ are functions of the spatial frequency (Fourier) space (u, v).

And A · B · exp (j (θ−φ)) is
A · B · exp (j (θ−φ)) = A · B · cos (θ−φ) + j · A · B · sin (θ−φ) (1)
As A · exp (jθ) = α ₁ + jβ ₁ and B · exp (jφ) = α ₂ + jβ ₂ , A = (α ₁ ² + β ₁ ² ) ^1/2 , B = (α ₂ ² + Β ₂ ² ) ^1/2 , θ = tan ⁻¹ (β ₁ / α ₁ ), φ = tan ⁻¹ (β ₂ / α ₂ ). By calculating the equation (1), synthesized Fourier image data is obtained.

A · B · exp (j (θ−φ)) = A · B · exp (jθ) · exp (−jφ) = A · exp (jθ) · B · exp (−jφ) = (α ₁ + jβ ₁ ) · (α ₂ −jβ ₂ ) = (α ₁ · α ₂ + β ₁ · β ₂ ) + j (α ₂ · β ₁ −α ₁ · β ₂ ) .

  And the control part 20-1 performs an amplitude suppression process, after acquiring synthetic | combination Fourier image data in this way. In this embodiment, all amplitudes are set to 1 as amplitude suppression processing. That is, the amplitude A · B of A · B · exp (j (θ−φ)), which is the arithmetic expression of the synthesized Fourier image data, is set to 1, and only the phase is set. Note that log processing or √ processing may be performed as the amplitude suppression processing. If only the phase is used, there are the advantage that the amount of calculation can be reduced and the data can be reduced as compared with the log processing and the √ processing.

  Next, the control unit 20-1 sends the combined Fourier image data having only the phase to the Fourier transform unit 20-7, and again performs two-dimensional discrete Fourier transform. The synthesized Fourier image data subjected to the two-dimensional discrete Fourier transform has an amplitude of 1 in the frequency space, but basically, the image of the local region 2G0 (1,1) of the 2D matching face image 2G0 The image of the local area 2G1 (1,1) of the 2D registered face image 2G1 can be considered as convolved image data, and the image of the local area 2G0 (1,1) of the 2D collation face image 2G0 and the 2D registered face image. It represents the correlation with the image of the local region 2G1 (1, 1) of 2G1.

  Then, the control unit 20-1 takes in the synthesized Fourier image data subjected to the two-dimensional discrete Fourier transform, and each pixel in a predetermined correlation component area (all areas in this embodiment) from the synthesized Fourier image data. Is scanned as the correlation peak Pa1 (FIG. 8 (e)).

  In this case, there is no image data that matches the image of the local region 2G1 (1,1) of the 2D registered face image 2G1 in the image of the local region 2G0 (1,1) of the 2D verification face image 2G0, or a matching image. Even if there is data, it is a background, so the correlation peak Pa1 appears as a small one.

  Next, i = 2, j = 1, and the correlation peak Pa1 in the local region 2G0 (1,2) of the 2D matching face image 2G0 and the local region 2G1 (1,2) of the 2D registered face image 2G1 is extracted. Similar processing is repeated until i = n and j = m. And the control part 20-1 averages correlation peak Pa1 in each local area | region obtained in this way to make a collation score, compares this collation score with a predetermined threshold value, and a collation score is a threshold value. If it is above, the collation result “match” is output, and if the collation score does not satisfy the threshold, the collation result “mismatch” is output.

  If a collation result “match” is output (YES in step S114), the control unit 20-1 sets “authentication OK” (step S115). This authentication result is displayed on the liquid crystal display device 12. Moreover, it outputs as an unlocking command etc. via the external connection part 20-6.

  On the other hand, if a collation result of “mismatch” is output (NO in step S114), the control unit 20-1 sets N1 = N1 + 1 = 2 (step S109), and performs the 2D collation face image in the same manner as described above. 2G0 and 2D registered face image 2G2 (2GN1) are collated (steps S110 to S113). This collation between the 2D collation face image 2G0 and the 2D registered face image 2GN1 is repeated while increasing N1 by 1 until a collation result of “match” is obtained.

  Since the control unit 20-1 obtains a collation result of “mismatch” for all of the 2D registered face images 2G1 to 2GN1max, N1 counted up in step S109 becomes N1> N1max (NO in step S110). After exiting the processing loop of steps S109 to S114, the process proceeds to step S116 (FIG. 4). That is, when the collation results of the 2D collation face image 2G0 and the 2D registered face images 2G1 to 2GN1max are all inconsistent, the control unit 20-1 determines that the 2D collation cannot be performed, and the processing after step S116 is performed. Next, the process proceeds to 3D verification described below.

  In this example, in the 2D collation in step S113, the 2D collation face image 2G0 is divided into local regions, and each of the divided local regions is collated with the corresponding local region in the 2D registered face image 2G1. However, as shown in FIG. 9, the 2D registered face image 2G1 is divided in the horizontal direction, and the respective divided regions 2G1 (j) and local regions 2G0 (i, j) obtained thereby are collated. May be. Further, it may be divided in the vertical direction instead of in the horizontal direction, and collation with each divided area may be performed in the same manner. In this example, the local region 2G0 (i, j) is cut out from the 2D matching face image 2G0 without overlapping, but may be cut out while shifting the image little by little.

[3D verification]
In step S116, the control unit 20-1 acquires the number of 3D registered face images stored in the HD 20-4 as N2max. Then, N2 (initial value 0) is set to N2 = N2 + 1 (step S117). After confirming that N2 ≦ N2max (YES in step S118), the N2th 3D registered face image is read from the HD 20-4 (step S119). ). In this case, assuming that the 3D registered face image stored in the HD 20-4 is 3G1 to 3GN2max, N2 = 1st 3D registered face image 3G1 is read.

  Note that the 3D registered face images 3G1 to GN1max stored in the HD 20-4 at this time are three-dimensional face images at the time of registration obtained in the same manner as the 3D matching face image 3G0. However, when the 3D registered face images 3G1 to GN1max are created, the registered object (human face) M1 is imaged with the posture corrected with the distance from the cameras 10 and 11 as a specified distance. The 2D registered face images 2G1 to 2GN1max described above are obtained as two-dimensional face images when the 3D registered face images 3G1 to GN1max are viewed from the front.

  Then, the control unit 20-1 reads the 3D matching face image 3G0 stored in the HD 20-4 (step S120), and reads the read 3D matching face image 3G0 and the 3D registered face image 3G1 read in step S119. Collation is performed (step S121). The 3D collation face image 3G0 and the 3D registered face image 3G1 are collated using an ICP algorithm. The three-dimensional matching using the ICP algorithm is described in, for example, Non-Patent Documents 2 and 3 and the like, so detailed description thereof is omitted here. If the distance between the two data becomes sufficiently small, it is considered that the authentication is successful.

  Here, if the collation result “match” is output (YES in step S122), the control unit 20-1 sets “authentication OK” (step S123). This authentication result is displayed on the liquid crystal display device 12. Moreover, it outputs as an unlocking command etc. via the external connection part 20-6.

  On the other hand, if a collation result of “mismatch” is output (NO in step S122), the control unit 20-1 sets N2 = N2 + 1 = 2 (step S117), and performs the 3D collation face image in the same manner as described above. 3G0 and 3D registered face image 3G2 (3GN2) are collated (steps S118 to S122). The collation between the 3D collation face image 3G0 and the 3D registered face image 3GN2 is repeated while increasing N2 by one until a collation result of “match” is obtained.

  Since the control unit 20-1 obtains a collation result of “mismatch” for all of the 3D registered face images 3G1 to GN2max, N2 counted up in step S117 becomes N2> N2max (NO in step S118). Through the processing loop of steps S117 to S122, “authentication NG” is set (step S124). This authentication result is displayed on the liquid crystal display device 12. Then, the control unit 20-1 returns to Step S101 and restarts the 2D / 3D collation processing from the image capture again.

  In the first embodiment, the 2D collation face image 2G0 and the 2D registered face image 2GN1 are collated (2D collation), and a 2D registered face image 2GN1 that matches the 2D collation face image 2G0 is found by the 2D collation. For example, a result of “authentication OK” is obtained at that time. Thereby, the collation speed can be increased. If the 2D registered face image 2GN1 that matches the 2D verified face image 2G0 is not found, the 3D verified face image 3G0 and the 3D registered face image 2GN2 are verified (3D verified) as those that cannot be determined by the 2D verification. Is called. Thereby, the improvement of collation performance is ensured.

  In this way, in the first embodiment, it is possible to achieve both improvement in collation performance and increase in collation speed by combining 2D collation and 3D collation. For example, in the case of this embodiment, the processing time from image capture to authentication OK result output is about 0.7 seconds for 2D verification OK and about 1.2 seconds for 3D verification OK. It is possible to achieve speed.

  Further, in this embodiment, since high-precision 3D matching face image 3G0 is acquired by high-precision stereo vision based on the phase-only correlation method, it is sufficient to simply perform alignment by the ICP algorithm at the time of 3D matching. Authentication performance is obtained. However, the ICP algorithm is a method of performing registration by repeatedly associating the nearest neighbors between 3D data. For high-density 3D data, excessive processing is required for nearest neighbor search, so 1 to N Not suitable for authentication. Therefore, in this embodiment, attention is paid to the fact that, in a normal face authentication system, the user faces the camera in most cases. First, 2D matching with a relatively small processing amount is performed, and 2D matching is performed. Only when the determination cannot be made, 3D verification is performed. As a result, the chance of 3D collation requiring a large amount of arithmetic processing is reduced, and the collation speed when viewed as a 2D / 3D combined collation apparatus as a whole is increased.

  FIG. 10 shows a functional block diagram of the first embodiment. In the figure, 31 is a 3D collation face image creation unit, 32 is a 2D collation face image creation unit, 33 is a two-dimensional collation unit, 34 is a three-dimensional collation unit, 35 is a 2D registration face image storage unit, and 36 is a 3D registration face. An image storage unit.

  In this functional block diagram, the 3D verification face image creation unit 31 extracts a verification target area AR1 from at least one of the left image S1 and the right image S2 of the camera that captured the verification target (human face) M1, and performs this verification. A three-dimensional image of the verification object M1 in the target area AR1 is created as a 3D verification face image. The 2D matching face image creation unit 32 creates a 2D image obtained by viewing the 3D matching face image created by the 3D matching face image creation unit 31 from the front as a 2D matching face image.

  The two-dimensional collation unit 33 collates the 2D collation face image created by the 2D collation face image creation unit 32 with a plurality of 2D registration face images stored in the 2D registration face image storage unit 35, and compares each of them. If there is a 2D registered face image that matches the face image, a result of “authentication OK” is immediately obtained. Further, the two-dimensional matching unit 33 sends a 3D matching start command to the three-dimensional matching unit 34 when all the matching results between the 2D matching face image and the 2D registered face image do not match.

  Upon receiving a 3D matching start command from the two-dimensional matching unit 33, the three-dimensional matching unit 34 is stored in the 3D matching face image created by the 3D matching face image creating unit 31 and the 3D registered face image storage unit 36. A plurality of registered 3D face images are compared with each other, and a result of “authentication OK” / “authentication NG” is obtained.

  FIG. 11 shows a functional block diagram in the case where the 2D verification face image 2G0 is created from the image S1 of the left camera in the first embodiment. In this case, the 2D matching face image creation unit 32 creates a 2D matching face image from the two-dimensional image of the face M1 in the matching target area AR1 of the image S1 of the left camera, and sends it to the two-dimensional matching unit 33. The 2D registered face image storage unit 35 stores a 2D registered face image at the time of registration created in the same manner as the 2D collated face image created by the 2D collated face image creating unit 32.

[Verification of object: Embodiment 2]
FIGS. 12 to 14 show flowcharts of a second example (embodiment 2) of collation processing executed by the control unit 20-1 in accordance with the 2D / 3D composite collation program stored in the ROM 20-2.

  Also in the second embodiment, the control unit 20-1 takes in the images S1 and S2 of the left and right cameras (FIGS. 5A and 5B) from the CCD cameras 10 and 11 as in the first embodiment. (FIG. 12: Step S201) Face coordinates and left and right pupil coordinates (2D pupil coordinates) are obtained by performing face detection and pupil detection on the image S1 of the left camera (Step S202). A 3D collation face image S3 (FIG. 5C) of the collation target area (face area) is created from the images S1 and S2 (step S203), 3D pupil coordinates are obtained from the 2D pupil coordinates (step S204), and the pupil ICP The 3D matching face image S3 is aligned by (Step S205), and the aligned 3D matching face image S3 is stored in the HD 20-4 as the 3D matching face image 3G0 (FIG. 5 (d)) (Step S2). 6).

[Refinement of 3D registered face images]
In the second embodiment, when the control unit 20-1 creates the 3D matching face image 3G0, the pupil distance, face length, pupil size, and distance from the camera to the face from the created 3D matching face image 3G0. (L2 (actual measurement value) shown in FIG. 1) is obtained (step S207). In this case, the distance L2 from the camera to the face is calculated from the 3D pupil coordinates.

  Then, the control unit 20-1 classifies the 3D registered face images stored in the HD 20-4 based on the inter-pupil distance, the face length, and the pupil size obtained in step S207, and generates a 3D matching face image 3G0. As similar face image groups, the 3D registered face images in the HD 20-4 are narrowed down to about 10 (step S208). If there is no narrowed-down 3D registered face image (NO in step S209), “authentication NG” is immediately set (step S210). If there is a narrowed 3D registered face image (YES in step S209), the front determination is performed on the 3D matching face image 3G0 (step S211).

  In this example, the inter-pupil distance, face length, and pupil size in the 3D matching face image 3G0 are used to narrow down the 3D registered face image used for 3D matching. That is, from the 3D matching face image 3G0. The 3D registered face image used for 3D matching is narrowed down based on the characteristics of the obtained face M1, but as will be described later, information indicating the characteristics of the face M1 is obtained from the 2D matching face image 2G0. You may make it narrow down the 3D registration face image used in the case of 3D collation from the information which shows the characteristic.

[Front Judgment]
The front determination in step S211 is performed as follows. The control unit 20-1 extracts the closest 3D registered face image from the narrowed down 3D registered face images, and in the X, Y, and Z axis directions between the 3D registered face image and the 3D matching face image 3G0. Posture differences Δθ, Δρ, Δψ are obtained, and the posture differences Δθ, Δρ, Δψ are compared with predetermined values.

  In this case, if the posture differences Δθ, Δρ, and Δψ are all smaller than the predetermined value, it is determined that the 3D matching face image 3G0 faces the front. If any one of the posture differences Δθ, Δρ, Δψ is larger than the predetermined value, it is determined that the 3D matching face image 3G0 is not facing the front.

  In this example, in the front determination in step S211, the closest 3D registered face image is extracted from the narrowed 3D registered face images, and the X between the 3D registered face image and the 3D matching face image 3G0 is extracted. , Y, and Z-axis orientation differences Δθ, Δρ, and Δψ are obtained, but the X, Y, and Z-axis inclinations θ, ρ, and ψ of the 3D verification face image 3G0 are obtained as differences from the reference orientation. The front determination may be performed based on the posture differences θ, ρ, and ψ.

  If the control unit 20-1 determines that the 3D matching face image 3G0 is facing the front (YES in step S211), the control unit 20-1 determines that the 3D matching face image 3G0 is an image suitable for 2D matching, and step S212 ( FIG. 13) As the subsequent processing, the process proceeds to 2D verification described below. On the other hand, if it is determined that the 3D verification face image 3G0 is not facing the front (NO in step S211), it is determined that the 3D verification face image 3G0 is an image that is not suitable for 2D verification, and step S226 (FIG. 14). As a subsequent process, the process proceeds to 3D verification described below.

[2D verification]
[Image size normalization]
In step S212, the control unit 20-1 reads the 3D matching face image 3G0 from the HD 20-4, and converts the two-dimensional face image obtained by viewing the read 3D matching face image 3G0 from the front to the 2D matching face image 2G0 (FIG. 5 ( e)). Then, the image size (face image size) of the 2D matching face image 2G0 is normalized based on the distance L2 from the camera to the face obtained in step S207 (step S213).

  That is, based on the ratio between the actual distance L2 from the camera to the face obtained in step S207 (information defining the actual size of the verification target area AR1) and the specified distance L2 from the camera to the face at the time of registration. The face image size of the 2D verification face image 2G0 is matched with the face image size of the 2D registered face image. In this case, the face image size is increased if the actual distance L2 from the camera to the face is long, and the face image size is decreased if the actual distance L2 from the camera to the face is short.

[Refine 2D registered face image]
Next, the control unit 20-1 obtains the inter-pupil distance, face length, and pupil size in the normalized 2D matching face image 2G0 (step S214), and the obtained inter-pupil distance, face length, pupil Based on the size, the 2D registered face images stored in the HD 20-4 are classified, and the 2D registered face images in the HD 20-4 are narrowed down to about 10 as a face image group similar to the 2D matching face image 2G0 ( Step S215). If there is no narrowed-down 2D registered face image (NO in step S216), “authentication NG” is immediately set (step S217). If there is a narrowed 2D registered face image (YES in step S216), the process proceeds to step S218.

  In this example, the 2D collation face image used in 2D collation is narrowed down using the inter-pupil distance, face length, and pupil size in the normalized 2D collation face image 2G0. The 2D registered face image used for 2D matching is narrowed down based on the feature of the face M1 obtained from the face image 2G0. Information indicating the feature of the face M1 is obtained from the 3D matching face image 3G0, and the feature of the face M1 is obtained. The 2D registered face image to be used for the 2D collation may be narrowed down from the information indicating.

  In this example, the face image size of the 2D matching face image 2G0 is normalized based on the actual distance from the camera to the face (distance from the camera to the 3D pupil coordinates), but the 3D matching face image 3G0 is used. The coordinates of the center of gravity of the face in the image may be obtained as the center of gravity 3D coordinates of the face, and the face image size of the 2D matching face image 2G0 may be normalized based on the distance from the camera to the center of gravity 3D coordinates. Further, the face image size of the 2D matching face image 2G0 may be normalized based on measured values such as the inter-pupil distance, face length, and pupil size obtained from the 3D matching face image 3G0.

[Verification with narrowed 2D registered face image]
In step S218, the control unit 20-1 acquires the number of 2D registered face images narrowed down in step S215 as N1max. Then, N1 (initial value 0) is set to N1 = N1 + 1 (step S219), and after confirming that N1 ≦ N1max (YES in step S220), the N1th narrowed 2D registered face image is read from the HD 20-4. (Step S221). In this case, if the narrowed 2D registered face images stored in the HD 20-4 are 2G1 to 2GN1max, N1 = 1st 2D registered face image 2G1 is read.

  Then, the control unit 20-1 reads the 2D matching face image 2G0 stored in the HD 20-4 (step S221), and reads the read 2D matching face image 2G0 and the 2D registered face image 2G1 read in step S221. Collation is performed (step S223). The collation between the 2D collation face image 2G0 and the 2D registered face image 2G1 is performed by dividing the 2D collation face image 2G0 into local regions in the same manner as in the first embodiment, and for each divided local region, the 2D registered face image The matching with the corresponding local region in 2G1 is performed by performing phase-only correlation (POC).

  If a collation result “match” is output (YES in step S224), the control unit 20-1 sets “authentication OK” (step S225). This authentication result is displayed on the liquid crystal display device 12. Moreover, it outputs as an unlocking command etc. via the external connection part 20-6.

  On the other hand, if a collation result of “mismatch” is output (NO in step S224), the control unit 20-1 sets N1 = N1 + 1 = 2 (step S219), and performs the 2D collation face image in the same manner as described above. 2G0 and 2D registered face image 2G2 (2GN1) are collated (steps S220 to S223). This collation between the 2D collation face image 2G0 and the 2D registered face image 2GN1 is repeated while increasing N1 by 1 until a collation result of “match” is obtained.

  When the collation result “mismatch” is obtained for all of the narrowed-down 2D registered face images 2G1 to 2GN1max, the control unit 20-1 obtains N1> N1max counted up in step S219 (in step S220). NO), exits the processing loop of steps S219 to S224, and proceeds to step S226 (FIG. 14). That is, when all the matching results of the 2D matching face image 2G0 and the narrowed 2D registered face images 2G1 to 2GN1max do not match, the control unit 20-1 determines that it cannot be determined by the 2D matching, and after step S226 The process proceeds to the 3D verification described below.

[3D verification]
[Attitude adjustment]
In step S226, the control unit 20-1 reads the 3D verification face image 3G0 stored in the HD 20-4, and performs posture alignment of the read 3D verification face image 3G0.

  In this example, the closest 3D registered face image is extracted from the 3D registered face images narrowed down in step S208, and the X, Y, and Z axis directions between the 3D registered face image and the 3D matching face image 3G0 are extracted. Posture differences Δθ, Δρ, and Δψ are obtained, and the posture of the 3D verification face image 3G0 is corrected so that the differences Δθ, Δρ, and Δψ are all zero.

  Note that the inclinations θ, ρ, and ψ in the X, Y, and Z-axis directions of the 3D matching face image 3G0 are obtained as differences from the reference posture, and the three-dimensional matching is performed so that the posture differences θ, ρ, and ψ are all zero. The posture of the face image 3G0 may be corrected.

[Verification with narrowed 3D registered face image]
Next, the control unit 20-1 acquires the number of 3D registered face images narrowed down in step S208 as N2max (step S227). Then, N2 (initial value 0) is set to N2 = N2 + 1 (step S228), and after confirming that N2 ≦ N2max (YES in step S229), the N2th narrowed 3D registered face image is read from the HD 20-4. (Step S230). In this case, if the narrowed-down 3D registered face images stored in the HD 20-4 are 3G1 to GN2max, N2 = 1st 3D registered face image 3G1 is read.

  Then, the control unit 20-1 reads the 3D matching face image 3G0 stored in the HD 20-4 (step S231), and reads the read 3D matching face image 3G0 and the 3D registered face image 3G1 read in step S230. Collation is performed (step S232). The 3D collation face image 3G0 and the 3D registered face image 3G1 are collated using an ICP algorithm.

  Here, if the collation result “match” is output (YES in step S233), the control unit 20-1 sets “authentication OK” (step S234). This authentication result is displayed on the liquid crystal display device 12. Moreover, it outputs as an unlocking command etc. via the external connection part 20-6.

  On the other hand, if a collation result of “mismatch” is output (NO in step S233), the control unit 20-1 sets N2 = N2 + 1 = 2 (step S228), and performs the 3D collation face image in the same manner as described above. 3G0 and 3D registered face image 3G2 (3GN2) are collated (steps S229 to S232). The collation between the 3D collation face image 3G0 and the 3D registered face image 3GN2 is repeated while increasing N2 by one until a collation result of “match” is obtained.

  When the collation result “mismatch” is obtained for all of the narrowed down 3D registered face images 3G1 to GN2max, the control unit 20-1 obtains N2> N2max counted up in step S228 (in step S229). NO), the process loop of steps S228 to S233 is exited and “authentication NG” is set (step S235). This authentication result is displayed on the liquid crystal display device 12. Then, the control unit 20-1 returns to Step S201 and restarts the 2D / 3D collation processing from the image capture again.

  In the second embodiment, since the 2D registered face image used for 2D collation is narrowed down in step S215, and the 3D registered face image used for 3D collation is narrowed down in step S208, it is narrowed down. 2D matching or 3D matching is performed only with the registered face image, and the matching speed is increased.

  In the second embodiment, since the face image size of the 2D matching face image 2G0 is normalized based on the actual distance from the camera to the face in step S213, the face M1 from the camera at the time of registration and matching is used. The face image sizes of the 2D matching face image 2G0 and the 2D registered face image 2GN1 match even if the positions up to are shifted, and the matching accuracy of 2D matching is improved.

  In the second embodiment, the front determination is performed on the 3D matching face image 3G0 in step S211, and the 2D matching is performed only when the 3D matching face image 3G0 faces the front. The probability of “authentication OK” in the verification increases, and the verification speed increases. Also, if it is determined that all of the 2D verifications are mismatched, 3D verification is performed, so that the verification performance is improved. As a result, as in the first embodiment, it is possible to achieve both improvement in collation performance and increase in collation speed by combining 2D collation and 3D collation.

  In the second embodiment, when the 3D verification face image 3G0 is not facing the front, 2D verification is not performed and 3D verification is performed immediately. As a result, 2D verification that is likely to be “authentication NG” does not waste processing time, and 3D verification that is likely to be “authentication OK” is immediately performed, thereby increasing the verification speed. To do.

  In the second embodiment, when it is determined that all are inconsistent in 2D collation, the process proceeds to 3D collation. However, the process may not be proceeded to 3D collation and may be immediately set as “authentication NG”. That is, a method may be used in which a front determination is performed on the 3D verification face image 3G0, and only 2D verification is performed if facing the front, and only 3D verification is performed if not facing the front.

  FIG. 15 shows a functional block diagram of the second embodiment. In the figure, 41 is a 3D matching face image creation unit, 42 is a front determination unit, 43 is a 2D matching face image creation unit, 44 is a normalization unit, 45 is a 2D registered face image narrowing unit, and 46 is a two-dimensional matching unit. 47 is a 3D registered face image narrowing unit, 48 is a three-dimensional matching unit, 49 is a 2D registered face image storage unit, 50 is a 3D registered face image storage unit, and 51 is a regulation information acquisition unit.

  In this functional block diagram, the 3D verification face image creation unit 41 extracts a verification target area AR1 from at least one of the left image S1 and the right image S2 of the camera that captured the verification target (human face) M1, and performs this verification. A three-dimensional image of the verification object M1 in the target area AR1 is created as a 3D verification face image.

  The 3D registered face image narrowing-down unit 47 stores the 3D registered face image storage unit 50 in the 3D registered face image storage unit 50 based on the information indicating the characteristics of the matching target M1 obtained from the 3D matching face image created by the 3D matching face image creating unit 41. A 3D registered face image used for 3D matching is narrowed down from a plurality of stored 3D registered face images.

  The front determination unit 42 determines whether or not the 3D verification face image created by the 3D verification face image creation unit 41 faces the front, and if the 3D verification face image faces the front, the 2D verification face image creation unit A 3D collation face image is sent to 43, and if the 3D collation face image does not face the front, a 3D collation start command is sent to the three-dimensional collation unit 48.

  When the 3D collation face image is sent from the front determination unit 42, the 2D collation face image creation unit 43 creates a 2D image obtained by viewing the 3D collation face image from the front as a 2D collation face image. The regulation information acquisition unit 51 acquires information (in this example, the actual distance from the camera to the face) that defines the actual size of the verification target area obtained by the 3D verification face image creation unit 41 as the regulation information. The normalization unit 44 normalizes the image size (face image size) of the 2D verification face image created by the 2D verification face image creation unit 43 based on the regulation information acquired by the regulation information acquisition unit 51.

  The 2D registered face image narrowing unit 45 is stored in the 2D registered face image storage unit 49 based on information indicating the characteristics of the verification target M1 obtained from the 2D verification face image normalized by the normalization unit 44. The 2D registered face images used for the 2D collation are narrowed down from a plurality of 2D registered face images.

  The two-dimensional collation unit 46 includes the 2D collation face image normalized by the normalization unit 44, the 2D registration face image narrowed down by the 2D registration face image narrowing unit 45 stored in the 2D registration face image storage unit 49, and Are identified, and if there is a 2D registered face image that matches the 2D collation face image, a result of “authentication OK” is immediately obtained. Also, the two-dimensional matching unit 46 sends a 3D matching start command to the three-dimensional matching unit 48 when all the matching results between the 2D matching face image and the 2D registered face image do not match.

  The 3D collation unit 48 receives a 3D collation start command from the 2D collation unit 46 or the front determination unit 42, and the 3D collation face image created by the 3D collation face image creation unit 41 and the 3D registered face image storage unit 50, the 3D registered face image narrowed down by the 3D registered face image narrowing unit 50 is collated with each individual, and the result of “authentication OK” / “authentication NG” is obtained.

  FIG. 16 shows a functional block diagram when a 2D collation face image 2G0 is created from the left camera image S1 in the second embodiment. In this case, the front determination unit 42 determines whether or not the 3D verification face image created by the 3D verification face image generation unit 41 faces the front, and if the 3D verification face image faces the front, the 2D verification face A command for creating a 2D collation face image is sent to the image creation unit 43. In response to this creation command, the 2D matching face image creation unit 43 creates a 2D matching face image from the two-dimensional image of the face M1 in the matching target area AR1 of the image S1 of the left camera, and sends it to the normalization unit 44. The 2D registered face image storage unit 35 stores a 2D registered face image at the time of registration created in the same manner as the 2D collated face image created by the 2D collated face image creating unit 32.

  The 2D / 3D combined verification apparatus of the present invention is not limited to a system for managing entrance / exit, but can be used in a system for authenticating various objects (for example, image check such as printing, solder board check, etc.). is there.

  DESCRIPTION OF SYMBOLS 10, 11 ... CCD camera, 12 ... Liquid crystal display device (LCD), 20 ... Processing part, 20-1 ... Control part, 20-2 ... ROM, 20-3 ... RAM, 20-4 ... Hard disk (HD), 20 -5 ... frame memory (FM), 20-6 ... external connection part (I / F), 20-7 ... Fourier transform part (FFT), 10-1, 11-1 ... lens, 31 ... 3D verification face image creation 32 ... 2D collation face image creation unit, 33 ... 2D collation unit, 34 ... 3D collation unit, 35 ... 2D registration face image storage unit, 36 ... 3D registration face image storage unit, 41 ... 3D collation face image creation , 42... Front determination unit, 43... 2D collation face image creation unit, 44... Normalization unit, 45... 2D registration face image narrowing unit, 46. 48 ... 3D matching unit, 49 ... 2D registered face image storage unit, 50 ... 3D registration Face image storing unit, 51 ... defining information acquisition unit.

Claims (7)

A region to be verified is extracted from at least one of an image captured from the first direction and an image captured from the second direction, and a 3D image of the verification target in the verification target region is extracted. 3D verification image creation means for creating an image;
2D collation image creating means for creating a 2D image of the collation object in the collation target region as a 2D collation image;
2D collation means for collating each 2D collation image created by the 2D collation image creation means and 2D images of a plurality of registration objects stored in advance as 2D registration images;
When the collation result between the 2D collation image and the 2D registration image by the two-dimensional collation unit does not match, the 3D collation image created by the 3D collation image creation unit and a plurality of 3D registration images stored in advance are stored. Three-dimensional matching means for matching a three-dimensional image of a registration object with each other ;
It is determined whether or not the 3D collation image created by the 3D collation image creating unit is facing the front. If it is determined that the 3D collation image is not facing the front, A 2D / 3D combined verification apparatus, comprising: a front determination unit that immediately executes verification. In the 2D / 3D combined verification apparatus according to claim 1,
Information indicating the characteristics of the verification target obtained from the 3D verification image created by the 3D verification image creation means or the characteristics of the verification target obtained from the 2D verification image created by the 2D verification image creation means 3D registered image narrowing means for narrowing down a 3D image to be used as the 3D registered image from among the stored 3D images of a plurality of registered objects based on information Compound verification device. In the 2D / 3D combined verification apparatus according to any one of claims 1 and 2,
Information indicating the characteristics of the verification target obtained from the 3D verification image created by the 3D verification image creation means or the characteristics of the verification target obtained from the 2D verification image created by the 2D verification image creation means 2D / 3D characterized by comprising 2D / 3D image narrowing means for narrowing down a 2D image to be used as the 2D registered image from the stored two-dimensional images of the plurality of registered objects based on the information Compound verification device. The 2D / 3D combined verification apparatus according to any one of claims 1 to 3,
Regulation information acquisition means for acquiring, as regulation information, information that defines the actual size of the collation target area obtained by the 3D collation image creation means;
Normalizing means for normalizing the image size of the 2D collation image created by the 2D collation image creation means based on the regulation information acquired by the regulation information acquisition means;
The two-dimensional matching means includes
A 2D / 3D combined collation apparatus that collates a 2D collation image normalized by the normalization unit and a 2D registered image . A region to be verified is extracted from at least one of an image captured from the first direction and an image captured from the second direction, and a 3D image of the verification target in the verification target region is extracted. 3D verification image creation means for creating an image;
2D collation image creating means for creating a 2D image of the collation object in the collation target region as a 2D collation image;
2D collation means for collating each 2D collation image created by the 2D collation image creation means and 2D images of a plurality of registration objects stored in advance as 2D registration images;
3D collating means for collating the 3D collated image created by the 3D collated image creating means with the 3D images of a plurality of registration objects stored in advance as 3D registered images;
It is determined whether or not the 3D collation image created by the 3D collation image creating unit is facing the front. When it is determined that the 3D collation image is facing the front, the 2D collation image and the 2D registered image by the two-dimensional collation unit are determined. A front determination unit that executes collation, and determines that the 3D collation unit performs collation between the 3D collation image and the 3D registration image when it is determined that it is not facing the front;
2D · 3D composite collating apparatus comprising: a. In the 2D / 3D combined verification apparatus according to any one of claims 1 to 5,
The 2D collation image creating means includes:
A 2D / 3D combined collation apparatus, wherein a 2D image obtained by viewing the 3D collation image created by the 3D collation image creating means from the front is created as a 2D collation image . In the 2D / 3D combined verification apparatus according to any one of claims 1 to 5 ,
The 2D collation image creating means includes:
A two-dimensional image of the verification target in a verification target region extracted from at least one of an image captured from the first direction and an image captured from the second direction is created as a 2D verification image. 2D / 3D combined verification device.

Images