JP5226570B2 - Image detection apparatus, image detection method, and image detection program - Google Patents

Description

  The present invention relates to an image detection apparatus, an image detection method, and an image detection program that can detect an inclination angle of an object in an image.

  Taking a face image detection device as an example, conventionally, a face image including a person's face is detected from an input image obtained by capturing the subject's person, and identity verification is performed by comparing the face image with a pre-registered face image. An image detection device used for a face matching device to perform is known (for example, see Patent Document 1).

  This image detection apparatus evaluates the face-likeness of the image based on the characteristic shape of the face, and determines that the face is a face image if the evaluation value exceeds a threshold value.

  Further, as a technique for correcting the inclination of the face of the subject in the input image, each image in the search area scanned on the input image is rotated 15 degrees between 0 degrees and 345 degrees, and each rotated search is performed. A technique for evaluating the facial appearance of an image in an area and correcting the tilt of the input image based on the angle at which the evaluation value is maximized is known (see, for example, Patent Document 2).

JP 2007-34723 A Japanese Patent No. 417779

  In the face collation device, if the face of the subject is tilted in either the left or right direction in the input image for registration, if the input image is registered, the accuracy of personal authentication may be reduced during the collation.

  This is because the tilt is not always the same at the next verification. In order to cope with both left and right inclinations, the registered image is preferably an upright image. Therefore, in the face matching device, when the face of the subject in the input image is tilted, it is desirable to perform image processing on the input image to correct the tilt and register the face in an upright state.

  Even if the face is registered with the face upright, if the tilt angle of the face between the input image and the registered image is different, the accuracy of the personal authentication may be similarly reduced. It is desirable to correct the tilt and collate.

  However, in the technique described in Patent Document 1, it is not known whether the face is inclined. In the technique described in Patent Document 2, it is necessary to rotate all search areas in the input image. In addition, if the face image is not detected from all the search areas rotated by 15 degrees between 0 degrees and 345 degrees, the angle that maximizes the evaluation value of the faceness cannot be calculated. There is a problem that it takes time to detect the tilt angle. Also, the finally obtained inclination angle is in units of 15 degrees.

  The present invention has been made in view of the above, and an object of the present invention is to provide an image detection apparatus and an image detection method that can accurately and quickly detect the tilt angle of an object in an input image.

Therefore, in the present invention, an image detection apparatus that detects a target image of a target object existing in the input image and a tilt of the target object, and when detecting a target image including the target object from the input image , A first evaluation means for calculating an evaluation value representing the object likeness as the angle of the object is closer to a predetermined angle; and an evaluation value calculated by the first evaluation means is greater than a predetermined first threshold value. First object image detection means for determining that an object has been detected in the input image, rotation image generation means for generating a rotation image by rotating the object image to a predetermined rotation angle, and the rotation with respect to a predetermined angle to evaluate the angle of the object in the image, a second evaluation unit angle of the object is calculated the predetermined angle larger evaluation value representing a more object likeliness close, evaluation calculated by the second evaluation means Previous value A second target image detecting means for determining that a target object is detected in the input image when the target value is larger than a predetermined second threshold value which is a value larger than the first threshold value; and an input image in which the target object is detected On the other hand, the generation of the rotation image by the rotation image generation means, the calculation of the evaluation value representing the object likeness by the second evaluation means, and the detection of the object by the second target image detection means are repeatedly executed. Evaluation for generating an evaluation function that approximates the calculated evaluation value and the corresponding rotation angle by a quadratic or higher function for at least three rotation images having different rotation angles from which the object is detected. image, characterized in that it comprises a function generating means and the inclination angle determination means for determining the tilt angle based on the rotation angle evaluation value becomes the maximum value calculated by using the evaluation function It was to provide a detection apparatus. As a result, it is not necessary to generate an image having the same tilt angle as the desired angle, and a precise rotation angle can be calculated with a small number of rotated images, which enables accurate and high-speed tilt angle detection processing. In particular, when detecting an object image, the object can be detected even from an input image having a large rotation angle and a small object, and even if an object is erroneously detected from an input image that does not include the object, the object is judged correctly at the time of evaluation. Can do.

  In the image detection apparatus according to the aspect of the invention, the evaluation function may be a cubic or higher order function, and the inclination angle determination unit may set the evaluation value calculated by the evaluation function to the rotation angle at which the evaluation value is a maximum value and a maximum value. The tilt angle of the object in the input image is determined based on the input image. This makes it possible to calculate a more precise tilt angle by approximating with a higher-order function.

  Further, the present invention provides the image detection apparatus, based on the tilt angle of the target object in the input image determined by the tilt angle determination unit, so that the tilt angle of the target object becomes a predetermined tilt angle. Rotation correction means for rotating the input image is provided. Thereby, the input image can be corrected to a predetermined angle. As a result, for example, when an image corrected to a predetermined angle is registered or the image is collated, the tilt angle of the input image is matched with the tilt angle of the registered image, and accurate collation can be performed. .

Further, the present invention is an image detection method for detecting the target image of the target object existing in the input image and the tilt of the target object, and when detecting the target image including the target object from the input image , A first evaluation step for calculating an evaluation value representing the object likeness as the angle of the object is closer to a predetermined angle, and an evaluation value calculated by the first evaluation step is greater than a predetermined first threshold value. A first target image detecting step for determining that an object is detected in the input image, a rotated image generating step for generating a rotated image by rotating the target image to a predetermined rotation angle, and the rotation with respect to a predetermined angle to evaluate the angle of the object in the image, a second evaluation step angle of the object is calculated the predetermined angle evaluation value representing a more object likeliness close large, the second evaluation step A second target image detection step for determining that an object has been detected in the input image when the calculated evaluation value is greater than a predetermined second threshold value that is greater than the first threshold value; For the input image in which an object has been detected, generation of the rotated image by the rotated image generation step, calculation of an evaluation value representing the likelihood of the object by the second evaluation step, and object by the second target image detection step The evaluation values calculated and the corresponding rotation angles are approximated by a quadratic or higher function for at least three rotation images having different rotation angles from which the object is detected. inclination determining an evaluation function generation step of generating an evaluation function, the inclination angle based on the rotation angle evaluation value becomes the maximum value is calculated using the evaluation function It was to provide an image detection method characterized by having an angle determining step. Accordingly, it is not necessary to generate an image having the same tilt angle as the desired angle, and a precise rotation angle can be calculated with a small number of rotation images, so that high-speed processing is possible. In particular, when detecting an object image, the object can be detected even from an input image having a large rotation angle and a small object, and even if an object is erroneously detected from an input image that does not include the object, the object is judged correctly at the time of evaluation. Can do.

Further, the present invention is an image detection program for detecting the target image of the target object existing in the input image and the tilt of the target object, and when detecting the target image including the target object from the input image , A first evaluation procedure for calculating an evaluation value representing an object likeness as the angle of the object is closer to a predetermined angle, and an evaluation value calculated by the first evaluation procedure is greater than a predetermined first threshold value. A first target image detection procedure for determining that an object has been detected in the input image, a rotated image generation procedure for generating a rotated image by rotating the target image to a predetermined rotation angle, and the rotation with respect to a predetermined angle to evaluate the angle of the object in the image, a second evaluation procedure angle of the object is calculated the predetermined angle evaluation value representing a more object likeliness near large, evaluation calculated by the second evaluation procedure value A second target image detection procedure for determining that an object has been detected in the input image when greater than a predetermined second threshold value that is greater than the first threshold value; and an input in which the object is detected For the image, the rotation image generation by the rotation image generation procedure, the calculation of the evaluation value indicating the object likeness by the second evaluation procedure, and the detection of the object by the second target image detection procedure are repeatedly executed. Generating an evaluation function that approximates the calculated evaluation value and the corresponding rotation angle by a quadratic or higher function with respect to at least three rotation images having different rotation angles from which the object is detected. be an evaluation function generation procedure, characterized in that it has a tilt angle determination procedure for determining the tilt angle based on the rotation angle evaluation value becomes the maximum value calculated by using the evaluation function It was to provide an image detection program. Accordingly, it is not necessary to generate an image having the same tilt angle as the desired angle, and a precise rotation angle can be calculated with a small number of rotation images, so that high-speed processing is possible. In particular, when detecting an object image, the object can be detected even from an input image with a large rotation angle and a small object-likeness. Can do.

  ADVANTAGE OF THE INVENTION According to this invention, the inclination angle detection of the target object in an input image can be performed accurately and at high speed.

FIG. 1 is an explanatory diagram showing a flow of image processing from detection of a face image from an input image. FIG. 2 is an explanatory diagram when division areas are set for an upright face and an inclined face. FIG. 3 is an explanatory diagram illustrating an example of an image processing procedure when detecting the tilt angle of the face image detected from the input image. FIG. 4 is an explanatory diagram illustrating an example of an evaluation function obtained by approximating the evaluation result related to the reliability of the face image by a quadratic function. FIG. 5 is a functional block diagram showing the face image detection apparatus according to the present embodiment. FIG. 6 is a flowchart illustrating processing executed by the control unit of the face image detection apparatus according to the present embodiment. FIG. 7 is a flowchart illustrating processing executed by the control unit of the face image detection apparatus according to the present embodiment.

  Exemplary embodiments of an image detection apparatus, an image detection method, and an image detection program according to the present invention will be described below with reference to the accompanying drawings.

  In the present embodiment, a case where the present invention is applied to a face image detection apparatus that detects a face image of a subject (person) included in an input image as a target image will be described as an example. Note that the target image in the present invention is not limited to a face image, and may be an image of another part of a person such as a fingerprint or a palm print or an arbitrary object as long as the image has characteristics. .

  The face image detection apparatus according to the present embodiment detects, for example, a face image including the face of the subject from the input image that has been input, detects the tilt angle of the detected face image, and performs tilt correction. As a registered image, or a device for collating a detected face image with a registered image.

  Here, first, a procedure for detecting a face image by the face image detecting apparatus according to the present embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram showing a flow of image processing from detection of a face image from an input image. Here, a case where the subject's face is tilted 16 degrees to the right will be described as an example.

  As shown in FIG. 1, in the face image detection device, when an input image 10 that is a target for detecting whether or not a face image is included is input, first, a plurality of different reduction ratios are input from the input image 10. Reduced images 11a to 11c are generated.

  Here, the case where the reduced images 11a to 11c are generated will be described. However, when the size of the face image included in the input image 10 is predicted to be smaller than a predetermined size, a plurality of enlargement rates are different. An enlarged image of is generated.

  When the size of the face image included in the input image 10 cannot be predicted, both the reduced images 11a to 11c and the enlarged image are generated. This is because templates 12a to 12c described below can be applied to images. Then, it is determined by linear discriminant analysis whether or not each image includes images of facial parts of the right eye, left eye, nose and mouth.

  Specifically, the face image detection device calculates the edge intensity and the edge direction for each pixel by applying a Gabor filter to the pixel value of each pixel of the image.

  Then, the face image detection device selects the square templates 12a to 12c centering on the pixel to be detected of the face part image, and the predetermined number of pixels around the pixel to be detected. The information on the edge intensity and the edge direction in the pixel is acquired.

  Here, the face image detection device detects pixels corresponding to the points at the four corners of the rectangle and pixels corresponding to the midpoints of the sides of the rectangle as the pixels around the pixels to be detected in the face part image. To do.

  Then, the face image detection apparatus acquires edge intensity and edge direction information for a total of nine pixels together with the pixels to be detected. It should be noted that the templates 12a to 12c to be used are machine-learned so that a face portion pixel with little illumination variation is selected. A plurality of templates 12a to 12c having different vertical and horizontal lengths are shown.

  Thereafter, the face image detection device determines whether or not the image part corresponding to the detection target pixel of the face part image is a face part image using the acquired edge intensity and edge direction information of the nine pixels. Perform linear discriminant analysis.

  At this time, the face image detection apparatus calculates the discrimination score (P) by inputting the values of the edge strength and the edge direction into a linear discriminant that discriminates whether or not the image is a face part image, and the discrimination score (P). The above determination is performed based on the value of.

  When each face part image is detected by the above processing, the face image detection device determines a representative point 13 representing the position of the face image based on the detected position of each face part image. Specifically, the face image detection device has a direction and a distance (number of pixels) in which the representative point 13 of the face image exists, as viewed from the detection target pixels in which the face part images are detected using the templates 12a to 12c. Set in advance.

  Then, the face image detection device calculates the distribution of the positions of the representative points 13 of the face image from the position of each face part image based on the setting information of the direction and distance, and obtains the peak of the distribution by calculating the peak of the distribution. The representative point 13 is determined.

  When the face representative point 13 is determined, the face image detection apparatus divides the image by a predetermined number of pixels around the representative point 13 and divides the image into nine divided regions 14. Specifically, the face image detection device determines the range of each divided region 14 so that the four face part images of the right eye, the left eye, the nose, and the mouth are included in the upper left, upper right, center, and lower center divided regions 14, respectively. Set.

  When the range of each divided region 14 is set in this way, when the image is a face image, the face part image is present at a predetermined position and should not be present at other positions. When an image that is not a face part image is erroneously detected as a face part image, it can be determined that the image is not a face image, and the accuracy of face image detection can be improved.

  Then, the face image detection device calculates the total value of the discrimination scores of the four face part images of the right eye, the left eye, the nose, and the mouth located in each divided region 14, and determines whether the image is a face or a face other than the total value. A 36-dimensional (9 regions × 4 face parts) feature quantity for determination is generated.

  Thereafter, the face image detection apparatus performs 36-dimensional linear discriminant analysis that determines whether the image is a face image or a non-face image using the generated feature amount. Specifically, the face image detection apparatus inputs a feature amount into a linear discriminant, calculates a discrimination score (F) that is an evaluation value representing the likelihood of a face, and the discrimination score (F) is greater than a predetermined threshold value. Is larger, it is determined that the image is a face image.

  FIG. 2 is an explanatory diagram when division areas are set for an upright face and an inclined face. As shown in FIG. 2, it can be seen that the position of the face part differs between the upright face (a) and the inclined face (b). Therefore, the feature amount calculated from each divided region 14 is different between an upright face (a) and an inclined face (b). In the face image detection apparatus according to the present embodiment, since the linear discriminant is set with the upright face (a) as a standard, the discrimination score (F) of the upright face (a) is the inclined face (b). A value higher than the discrimination score (F) is calculated.

  Note that the standard face angle is not limited to being upright, but may be other angles. In the present embodiment, the angle at which the face stands upright is a predetermined angle.

  Here, when a face image is detected from the input image, if the threshold value to be compared with the discrimination score (F) is excessively increased, the face image is included in the region due to the inclination of the face image in the input image. Therefore, there is a risk of erroneously determining that the face image is not included.

  In contrast, when detecting a face image from an input image, the face image detection apparatus according to the present embodiment roughens the face image using a relatively low first threshold that allows the position of the face image to be specified. Therefore, even if the face image in the input image is slightly inclined, the face image can be detected by specifying the position of the face image.

  In the present embodiment, the above-described face image detection device is used, but another face image detection device whose evaluation value changes when the face angle changes may be used. For example, Patent Document 2 describes a method in which the evaluation value of the facialness is calculated to be larger as the inclination of the face of the input image is closer to the learned inclination of the face image.

  In the present invention, a predetermined angle assumed by such a known face image detection device is used as a reference, and how much the face (target image) is inclined with respect to the predetermined angle is determined. And the inclination of the face in the image is obtained from the inclination with respect to this. The face-likeness (object-likeness) here is detected as a reference face image (reference image) using a general face matching technique in addition to the evaluation value at the time of detection such as the discrimination score. The similarity obtained by collating with an image can be used.

  Next, a procedure for detecting a tilt angle of a face image by the face image detection apparatus of the present embodiment will be described with reference to FIGS. FIG. 3 is an explanatory diagram showing an example of an image processing procedure when detecting the tilt angle of the face image detected from the input image, and FIG. 4 is an evaluation obtained by approximating the evaluation result related to the reliability of the face image with a quadratic function. It is explanatory drawing which shows an example of a function.

  As shown in FIG. 3A, when the face image detection device detects a face image tilted 16 degrees to the right with respect to the vertically upward direction, the face image detection device generates a rotated image obtained by rotating the face image by a predetermined angle. The center of rotation when rotating the image is preferably the center point of the image. Further, it may be determined based on the position of the face part. In the present embodiment, the rotation angle for rotating the face image detected from the input image clockwise will be described as a positive angle, and the rotation angle for rotating counterclockwise will be described as a negative angle.

  Here, the face image detection device first generates a rotated image obtained by rotating the face image detected from the input image by −20 degrees as shown in FIG.

Then, the face image detection device evaluates the face image and calculates a search score for the rotated image rotated by −20 degrees . The search score is a degree indicating the face likeness of the face image detected by detecting the face image from the rotated image. Here, the discrimination score (F) is used as the search score, but other calculation means for calculating a lower value for an image with a tilted face may be used.

  At this time, the face image detection device detects the face image from the rotated image by performing image processing similar to the image processing from the input image shown in FIG. 1 until the face image is detected, on the rotated image. However, when detecting a face image from the rotated image, the face image detection device finely detects the face image using a second threshold value higher than the first threshold value.

  This second threshold is a value sufficient to determine that the detected image is a face. Thus, even if an image that is not a face is detected, the first threshold value can be excluded.

  Then, the face image detection apparatus detects the four face part images of the right eye, the left eye, the nose, and the mouth that are located in each divided region 14 (see FIG. 1) of the face image as in the case of detecting the face image from the input image. The total value of the discrimination score (P) is calculated, and a 36-dimensional (9 regions × 4 face parts) feature quantity for determining whether the image is a face or a face other than the face is generated from the total value.

  After that, the face image detection apparatus inputs the generated feature amount into a predetermined linear discriminant, performs 36-dimensional linear discriminant analysis for determining whether the image is a face image or a non-face image, and determines a discriminant score (F). calculate.

  Hereinafter, the 36-dimensional linear discriminant analysis for determining whether an image is a face image or a non-face image by inputting a feature amount generated from the rotated image into a linear discriminant is referred to as face search. The discrimination score (F) obtained as a result of the face search is called a search score.

  Subsequently, as shown in FIGS. 3C to 3G, the face image detection device generates a rotated image obtained by sequentially rotating the rotated image shown in FIG. 3B to the right by 5 degrees. Then, a face search is performed on the rotated image to calculate a search score. Thereafter, the face image detection device sequentially generates the rotated image and calculates the search score until the face image detected from the input image (see FIG. 3A) is tilted to +20 degrees to the right.

  In particular, the face image detection apparatus according to the present embodiment generates all rotated images obtained by rotating the face image detected from the input image every 5 degrees from −20 degrees to +20 degrees every time an input image is input. Instead of detecting the tilt angle of the face image by ending the rotation of the face image when the estimation of the tilt angle of the input image succeeds in the process of sequentially calculating the search score of the rotated image at each rotation angle. The processing time required is shortened.

  Specifically, as shown in FIG. 4, the face image detection apparatus evaluates the transition of the search score, which is the evaluation result of the face image in the rotated image at each rotation angle, by quadratic function approximation in the order of the rotation angle of the face image. Generate a function.

  Then, the face image detection apparatus determines the rotation angle corresponding to the search score that is the maximum value of the generated evaluation function as the tilt angle of the input image.

  Here, the face image detection device calculates each search score by rotating the face image discontinuously such as −20 degrees, −15 degrees, −10 degrees, and 0 degrees. By generating a continuous evaluation function by approximating a typical search score to a quadratic function, it is possible to estimate a search score at a rotation angle at which the search score is not calculated.

  As a result, the face image detection apparatus does not calculate the search score of the rotated image at the rotation angle between −20 degrees, −15 degrees, and −10 degrees, but the evaluation function shown in FIG. , It can be determined that the state in which the face image is rotated by −16 degrees is the upright state of the face image. That is, the face image detection apparatus can estimate that the input image is inclined +16 degrees.

  In this embodiment, in order to increase the reliability of the maximum value of the evaluation function, after the determination of the maximum value of the evaluation function, rotation image generation and face search are performed at each rotation angle of −5 degrees and 0 degrees. A search score is calculated and an evaluation function based on a total of five search scores is generated. If the maximum value of the evaluation function can be determined, the search score is calculated when at least three search scores are calculated. The calculation of may be terminated.

  As described above, when the face image detection apparatus according to the present embodiment determines the tilt angle of the input image based on the evaluation function, the calculation of the search score ends at that point. Therefore, it is not necessary to generate every rotation image obtained by rotating the face image in increments of 5 degrees from −20 degrees to +20 degrees, so that the processing time required for detecting the tilt angle of the input angle can be shortened.

  In this embodiment, the evaluation function is a quadratic function. However, a higher-order function may be used if the order is 2 or more. In this case, the rotation angle that becomes the maximum value of the evaluation function and becomes the maximum value within the interval from −20 degrees to +20 degrees is determined as the inclination angle of the input image.

  Further, in the present embodiment, as an example of the range related to the rotation angle of the face image, the range of −20 degrees to +20 degrees is illustrated, but the rotation angle range is not limited to −20 degrees to +20 degrees. A general adult can appropriately change the angle within a range in which the head can tilt left and right, and the step size of the rotation angle of the rotated image can be arbitrarily changed. Further, the order of calculating the search score is not fixed from -20 degrees. For example, it can be set in any order such as 0 degrees, +5 degrees, -5 degrees, +10 degrees, and the like.

  As described above, when the rotation angle of the face image is set within a range in which a general adult can tilt the head to the left and right, the search score is not calculated by rotating the face image greatly. Nevertheless, since sufficient face search can be performed for face authentication, the processing time required for detecting the tilt angle of the face image can be reduced without degrading the tilt angle detection accuracy of the face image input for registration. It can be shortened.

  Thereafter, the face image detection device rotates the face image by the inclination angle determined for the face image, and performs rotation correction so that the face stands upright. Then, when the input image is an image input for registration, the face image detection apparatus stores the face image that has been subjected to the rotation correction of the tilt angle as a registered image and registers it.

  On the other hand, when the input image is an image input for verification, the face image detection device collates the face image that has been subjected to the rotation correction of the tilt angle and the registered image registered in a predetermined storage unit. The collation result is output.

  Next, the functional configuration of the face image detection apparatus according to the present embodiment will be described with reference to FIG. FIG. 5 is a functional block diagram showing the face image detection apparatus according to the present embodiment. In the figure, only components necessary for explaining the features of the face image detection apparatus are shown, and descriptions of general components are omitted.

  As shown in FIG. 5, the face image detection apparatus includes an input unit 20, a display unit 21, a control unit 22, a storage unit 23, and an operation unit 24.

  The input unit 20 is an imaging device that captures an image of a subject, and outputs the captured input image to a reduced / enlarged image generation unit 31 described later. When the subject is imaged for verification, the input unit 20 outputs the captured input image to the below-described collation unit 44 in addition to the below-described reduced / enlarged image generation unit 31 and rotation correction unit 43. The display unit 21 is a display device such as a display that displays a collation result or the like by a collation unit 44 described later.

  The control unit 22 is a processing unit that performs overall control of the operation of the entire face image detection apparatus. The storage unit 23 is a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive).

  The information stored in the storage unit 23 will be described in detail later. In the storage unit 23, the control unit 22 detects a face image from the input image and rotationally corrects the tilt angle of the input image. The face image detection program 51 to be executed is stored.

  The control unit 22 is physically configured by a CPU (Central Processing Unit). This CPU reads the face image detection program 51 from a ROM (Read Only Memory) area in the storage unit 23, and A reduced / enlarged image generation unit 31, a Gabor feature image generation unit 32, a template selection unit 33, a learning processing unit 34, and a face part image detection unit 35 which are activated by executing using a RAM (Random Access Memory) area as a work area. , Face image representative point calculation unit 36, face image detection feature value generation unit 37, face image detection unit 38, rotation image generation unit 40, evaluation function generation unit 41, angle determination unit 42, rotation correction unit 43, and collation unit 44 It has.

  When the input image 10 is input, the reduced / enlarged image generation unit 31 generates a plurality of reduced images 11a to 11c obtained by reducing the input image 10 at different reduction rates and a plurality of enlarged images enlarged at different enlargement rates. Part.

  The reduced / enlarged image generation unit 31 generates a plurality of reduced images 11a to 11c and enlarged images by performing linear interpolation. Here, whether to reduce or enlarge the input image 10 is determined by the relationship between the size of the templates 12 a to 12 c to be used and the predicted size of the face image included in the input image 10.

  The Gabor feature image generation unit 32 is a processing unit that applies a Gabor filter to each image generated by the reduced / enlarged image generation unit 31 to generate a Gabor feature image.

  The Gabor feature image generation unit 32 selects a 7-pixel square area from the image, applies a Gabor filter to four directions every 90 degrees of the area, generates a Gabor feature image, and detects a face part image. To the unit 35. Note that the size of 7 pixels square is set assuming that the right eye and the left eye of the image are about 12 pixels apart when the size of the face image is 24 pixels square.

のように表される。また、jは虚数単位であり、θは方向（角度）であり、λは波長であり、σ₁，σ₂はスケールであり、ｋは９０度ごとの４つの方向である。 Here, the Gabor filter

It is expressed as Further, j is an imaginary unit, θ is a direction (angle), λ is a wavelength, σ ₁ and σ ₂ are scales, and k is four directions every 90 degrees.

The Gabor feature image generation unit 32 calculates values of the real part g _k ^real and the imaginary part g _k ^imag of the Gabor filter, and uses the five-dimensional element V ₁ shown below for each pixel in the image from these values. generating a Gabor feature image by calculating a characteristic quantity composed of ~V _5.

のように表される。この１次元目の要素V₁は、画像に含まれるエッジの強度を表す量である。 Here, the first dimension element V ₁ of the feature amount is

It is expressed as The first dimension element V ₁ is a quantity representing the strength of the edge included in the image.

のように表される。この２次元目から５次元目までの要素は、画像に含まれるエッジの方向を表す量である。 The elements V ₂ to V ₅ from the second dimension to the fifth dimension are

It is expressed as The elements from the second dimension to the fifth dimension are quantities representing the direction of the edge included in the image.

  When detecting the face part image, the template selection unit 33 selects a plurality of templates 12 a to 12 c suitable for the image from the template information stored in the storage unit 23 using the result learned by the learning processing unit 34. It is a processing unit. The template selection unit 33 outputs the selected templates 12a to 12c to the face part image detection unit 35.

  The learning processing unit 34 is a processing unit that learns which template 12a to 12c is suitable for an image when detecting a face part image. Specifically, the learning processing unit 34 uses a boosting algorithm to learn which of the templates 12a to 12c can detect pixels and the like with little illumination variation.

  The face part image detection unit 35 is a detection unit that detects a face part image using the templates 12 a to 12 c selected by the template selection unit 33.

  The face part image detection unit 35 selects nine pixels of the image using the first templates 12a to 12c, and the image is detected based on the edge intensity and edge direction information of the Gabor feature image corresponding to these pixels. A linear discriminant analysis is performed to discriminate whether or not the image is a face part image.

のように表される。ここで、a₀，a_iは係数であり、ｗ_iはテンプレート１２ａ〜１２ｃにより選択された９点における５次元の特徴量V₁〜V₅の値が代入される変数であり、iは１から４５（９点×５次元）までの値をとる。 The linear discriminant used in this linear discriminant analysis is

It is expressed as Here, a ₀ and a _i are coefficients, w _i is a variable into which the values of the five-dimensional feature values V _{1 to} V ₅ at the nine points selected by the templates 12a to 12c are substituted, and i is 1 To 45 (9 points × 5 dimensions).

The coefficient a _i of the linear discriminant is calculated in advance so that it can be appropriately determined whether or not the image is a face part image. The greater the value of z (discrimination score (P)) when the values of the five-dimensional feature values V _{1 to} V ₅ at 9 points are substituted into this linear discriminant, the higher the probability that the image is a face part image. Determined.

  Then, the face part image detection unit 35 calculates a value obtained by multiplying the weight of the used templates 12a to 12c and the discrimination score (P) for each pixel that is a detection target of the face part image, and sets it as a collation value. When the collation value is smaller than a predetermined threshold, it is determined that the image is not a face part image.

  On the other hand, when the collation value is larger than a predetermined threshold, the face part image detection unit 35 calculates the discrimination score (P) using the next templates 12a to 12c, and determines the weight of the used templates 12a to 12c. A value obtained by multiplying the discrimination score (P) is added to the collation value to set as a new collation value. Again, when the collation value becomes smaller than the predetermined threshold, the face part image detection unit 35 determines that the image is not a face part image.

  The face image representative point calculation unit 36 is a processing unit that calculates the position of the representative point 13 representing the position of the face image from the position of each face part image detected by the face part image detection unit 35. The face image representative point calculation unit 36 is located at a distance of how many pixels in which direction the representative point 13 of the face image is seen from the detection target pixels when the face part images are detected using the templates 12a to 12c. The setting information related to the representative point for which k is set is acquired.

  Then, the face image representative point calculation unit 36 calculates the distribution of the positions of the representative points 13 of the face image based on the information related to the representative points and the information of the positions of the face part images, and a predetermined threshold value. The position of the representative point 13 of the face image is determined by obtaining the above points.

  The face image detection feature value generation unit 37 is a processing unit that generates a feature value used when detecting a face image from an image by linear discriminant analysis. The face image detection feature quantity generation unit 37 divides the image into nine divided regions 14 by dividing the image by a predetermined number of pixels around the representative point 13 calculated by the face image representative point calculation unit 36.

  Then, the face image detection feature value generation unit 37 positions the detection target pixel that detects the face part image as the total value of the matching values for each face part image obtained by applying each of the templates 12a to 12c. Calculated for each divided region 14 and generates 36-dimensional (9 regions x 4 facial parts: right eye, left eye, nose, mouth) feature values for determining whether the image is a face image or a non-face image from the total value Then, the image is output to the face image detection unit 38.

  The face image detection unit 38 is a processing unit that detects a face image from an image by performing 36-dimensional linear discriminant analysis using the 36-dimensional feature value input from the face image detection feature-value generating unit 37. Further, the face image detection unit 38 functions as an evaluation unit that evaluates the face likeness (object likeness) of the face image.

Note that the linear discriminant used in this linear discriminant analysis is the same as the equation (4). However, in this case, a ₀ and a _i are coefficients calculated in advance so that it can be properly determined whether or not the image is a face image, and w _i is a variable into which each value of the 36-dimensional feature value is substituted. Yes, i takes a value from 1 to 36.

  The face image detection unit 38 calculates a search score (value of z in the linear discriminant) by substituting a 36-dimensional feature amount into the linear discriminant, and detects a face image by comparing with a predetermined threshold value. When the search score is smaller than the threshold value, it is determined that the face image is not a face image and is not detected. Then, the face image detection unit 38 outputs the detected face image to the rotation image generation unit 40 and outputs the search score to the evaluation function generation unit 41.

  The rotation image generation unit 40 generates a rotation image obtained by rotating the face image input from the face image detection unit 38 by a predetermined angle, and outputs the rotation image to the face part image detection unit 35.

  The evaluation function generating unit 41 is an evaluation function (for example, FIG. 4) that approximates a quadratic function in order of the rotation angle of the face image in the rotated image corresponding to each search score for each search score transition input from the face image detecting unit 38. A processing unit that generates and outputs to the angle determination unit 42.

  The angle determination unit 42 detects a local maximum value in the evaluation function based on the evaluation function input from the evaluation function generation unit 41, and based on the rotation angle of the rotated image corresponding to the local maximum value, the inclination angle of the input image It is a processing part which determines. Specifically, the rotation angle when the evaluation function takes the maximum value is determined as the tilt angle.

  When the angle determination unit 42 succeeds in determining the tilt angle of the input image, the angle determination unit 42 outputs a signal indicating the fact to the control unit 22 and outputs the determined tilt angle of the input image to the rotation correction unit 43.

  The rotation correction unit 43 is a processing unit that rotationally corrects the tilt angle of the input image input from the input unit 20 by the amount of the tilt angle input from the angle determination unit 42. When the input image subjected to the rotation correction is a registration input image, the rotation correction unit 43 outputs the input image after the rotation correction to the storage unit 23 as a registered image and stores it.

  On the other hand, when the input image that has been subjected to the rotation correction of the tilt angle is an input image for verification, the rotation correction unit 43 outputs the input image after the rotation correction to the verification unit 44.

  The collation unit 44 is a processing unit that collates the collation input image input from the rotation correction unit 43 with the registered image stored in the storage unit 23, and outputs the collation result to the display unit 21 for display. is there.

  The storage unit 23 stores a face image detection program 51, template information 52, a face part image detection linear discriminant 53, a face image detection linear discriminant 54, a registered image 55, and the like.

  The face image detection program 51 is read by the CPU of the control unit 22 so that the control unit 22 performs the face image detection process shown in FIG. 1 and the input image inclination angle detection process shown in FIGS. This is an information processing program that is executed.

  The template information 52 is information on the templates 12a to 12c used when detecting the face part image. The template information 52 is information on the relative positions of the eight pixels detected when the face part image is detected with respect to the detection target pixel of the face part image, and information on the weights of the templates 12a to 12c.

  The face part image detection linear discriminant 53 is information of a linear discriminant used when detecting a face part image. The face image detection linear discriminant 54 is linear discriminant information used when detecting a face image. The registered image 55 is image information including a face image obtained by performing rotation correction on the input image for registration by the rotation correction unit 43.

  The operation unit 24 is an operation switch that is operated to register an input image input by the user of the face image detection device as a registered image in the face image detection device.

  Next, processing performed when the CPU included in the control unit 22 of the face image detection apparatus executes the face image detection program 51 will be described with reference to FIGS. 6 and 7. 6 and 7 are flowcharts illustrating processing executed by the control unit of the face image detection apparatus according to the present embodiment.

  When the execution of the face image detection program 51 is started, the CPU first determines whether or not an input image has been received from the input unit 20 (step S101) and determines that the input image has been received, as shown in FIG. If so (step S101: Yes), the process proceeds to step S102. On the other hand, when determining that the input image is not received (step S101: No), the CPU ends the process.

  In step S102, the CPU generates a plurality of reduced images 11a to 11c having different reduction ratios from the input image confirmed to be accepted in step S101, and moves the process to step S103.

  Here, the case where the reduced / enlarged image generation unit 23 generates the reduced images 11a to 11c is described as an example, but the reduced / enlarged image generation unit 23 generates an enlarged image or the reduced images 11a to 11c. When both 11c and the enlarged image are generated, the same processing as described below is performed.

  Subsequently, the CPU applies a plurality of templates 12a to 12c to a plurality of reduced images 11a to 11c having different reduction ratios, calculates a collation value by linear discriminant analysis, and stores each of the right eye, left eye, nose, and mouth. Processing for detecting a face part image is performed (step S103).

  Then, the CPU calculates the position of the representative point 13 of the face image from the position of the face part image detected in step S103 (step S104), and then inputs the input image with a predetermined number of pixels around the calculated representative point 13 as a center. By dividing the image, the image is divided into nine divided regions 14, and the total value of the collation values for each face part image is calculated for each divided region 14 in which the detection target pixel that detected the face part image is located. A 36-dimensional feature amount for determining whether the image is a face image or a non-face image is generated (step S105).

  Subsequently, the CPU executes a face image detection process for detecting a face image from the input image by linear discriminant analysis using a 36-dimensional feature amount (step S106), and then executes a face image tilt angle correction process. Then, rotation correction of the tilt angle of the input image is performed (step S107), and the process proceeds to step S108. The tilt correction processing executed by the CPU will be specifically described later with reference to FIG.

  Then, in step S108, the CPU determines whether or not a registration operation has been performed by the operation unit 24 with respect to the received input image. If the CPU determines that the registration operation has been performed (step S108: Yes), step S107 is performed. In step S109, the input image subjected to the rotation correction is registered in the storage unit 23 as the registered image 55 and registered (step S109).

  On the other hand, if the CPU determines that the registration operation has not been performed in step S108 (step S108: No), the CPU displays the input image subjected to the rotation correction in step S107 and the registered image 55 stored in the storage unit 23. After collation (step S110), the collation result is displayed on the display unit 21 (step S111), and the process is terminated.

  Next, the tilt angle correction process executed in step S107 will be described with reference to FIG. When the tilt correction process is started, the CPU first generates a rotated image obtained by rotating the face image detected from the input image by −20 degrees as shown in FIG. 7 (step S201).

  Subsequently, the CPU performs a face search on the generated rotated image, performs face image evaluation to calculate a search score (step S202), and generates an evaluation function by approximating the search score to a quadratic function (step S202). Step S203). However, since the quadratic function cannot be approximated when the search score is two or less, the process proceeds to the next step.

  Thereafter, the CPU determines whether or not the maximum value exists in the generated evaluation function (step S204). If it is determined that the maximum value exists (step S204: Yes), the process proceeds to step S205. On the other hand, if the CPU determines that there is no maximum value in the evaluation function (step S204: No), the process proceeds to step S206. If the quadratic function cannot be approximated in step S203, the process proceeds to step S206.

  In step S206, the CPU determines whether or not the rotation angle of the rotated image is +20 with respect to the face image detected from the input image, and determines that it is +20 degrees (step S206: Yes), the process ends. On the other hand, if the CPU determines that the rotation angle is not +20 degrees (step S206: No), the process proceeds to step S207.

  In step S207, the CPU generates a rotated image obtained by rotating the rotated image generated in step S201 by +5 degrees, and then moves the process to step S202. Then, the CPU sequentially repeats the processes of step S202, step S203, step S204, step S206, and step S207 until it is determined that the maximum value exists in the evaluation function (step S204: Yes).

  In step S207, an image rotated by +5 degrees is generated based on the previously generated rotated image. However, the face image detected from the input image may be rotated by a corresponding angle to generate a rotated image. Good.

  If the CPU determines that there is a maximum value in the evaluation function (step S204: Yes), the process proceeds to step S205, where only the rotation angle of the rotation image corresponding to the maximum value of the evaluation function is detected from the input image. Then, the tilt angle for rotating the face image is corrected, the tilt angle correction process (step S107) is terminated, and the process proceeds to step S108 shown in FIG.

  As described above, in the face image detection apparatus according to the present embodiment, a face image including the face of the subject is detected from the face images existing in the input image, and a predetermined angle is set around the center of the face image. A rotated image is generated, and the face image is evaluated for each face image detected from each rotated image, and a search score is calculated.

  Then, in this face image detection device, an evaluation function is generated by approximating the transition of the search score of the face image by a quadratic function in the order of the rotation angle of the face image, and based on the rotation angle corresponding to the maximum value of the evaluation value in the evaluation function. To determine the tilt angle of the input image.

  For this reason, in this face image detection device, the detection of the tilt angle of the input image can be terminated when the maximum value appears in the evaluation function, and it is not necessary to evaluate the face image at all preset rotation angles. The processing time required for detecting the tilt angle of the input image can be shortened.

  Further, the face image detection device performs evaluation when the face image in the rotated image is rotated by a predetermined angle and calculates discontinuous evaluation values, and then changes in these discontinuous evaluation values. Is approximated by a quadratic function, a transition state of continuous evaluation values can be estimated.

  As a result, in this face image detection device, the estimation accuracy of the tilt angle of the input image is reduced even when the face image is not evaluated for the rotated image having the rotation angle that matches the tilt angle of the input image. Can be prevented.

  In addition, since the face image detection device can determine that the face is not a face when the transition of the evaluation value monotonously increases or protrudes downward in the generated evaluation function, the face image detection device can easily detect a face and a face other than the face. Can be separated.

  This can be used for detecting an inclination of an image that can detect a predetermined feature in advance. For example, printing, sticking a sticker, attaching a part, detecting the inclination of a processed shape, etc.

DESCRIPTION OF SYMBOLS 10 Input image 11a, 11b, 11c Reduced image 12a, 12b, 12c Template 13 Representative point 14 Division area 20 Input part 21 Display part 22 Control part 23 Storage part 31 Reduced enlarged image generation part 32 Gabor feature image generation part 33 Template selection part 34 Learning processing unit 35 Face part image detection unit 36 Face image representative point calculation unit 37 Face image detection feature amount generation unit 38 Face image detection unit 40 Rotation image generation unit 41 Evaluation function generation unit 42 Angle determination unit 43 Rotation correction unit 44 Collation unit 51 Face image detection program 52 Template information 53 Linear discriminant for face part image detection 54 Linear discriminant for face image detection 55 Registered image

Claims (5)

An image detection device that detects a target image of a target existing in an input image and a tilt of the target,
When detecting a target image including the target object from the input image, a first evaluation unit that calculates an evaluation value representing the target object as the angle of the target object is closer to a predetermined angle;
First target image detection means for determining that an object has been detected in the input image when the evaluation value calculated by the first evaluation means is greater than a predetermined first threshold ;
Rotation image generation means for generating a rotation image by rotating the target image at a predetermined rotation angle;
A second evaluation unit that evaluates an angle of the object in the rotated image with respect to a predetermined angle, and calculates an evaluation value that represents the object likeness as the angle of the object is closer to the predetermined angle;
A second target image for determining that an object has been detected in the input image when the evaluation value calculated by the second evaluation means is greater than a predetermined second threshold value that is greater than the first threshold value; Detection means;
For the input image in which the object is detected, the rotation image generation unit generates the rotation image, the second evaluation unit calculates the evaluation value indicating the object likeness, and the second object image detection unit The detection of the object is repeatedly executed, and the evaluation value calculated for each of the at least three rotation images having different rotation angles from which the object is detected and the corresponding rotation angle are expressed by a quadratic or higher function. An evaluation function generating means for generating an evaluation function approximated by
An inclination angle determining means for determining an inclination angle based on the rotation angle at which an evaluation value calculated using the evaluation function is a maximum value ;
An image detection apparatus comprising: The evaluation function is a function of third order or higher,
The inclination angle determination means determines an inclination angle of an object in the input image based on the rotation angle at which an evaluation value calculated by the evaluation function is a maximum value and a maximum value. The image detection apparatus described.   Rotation correction means for rotating the input image based on the inclination angle of the object in the input image determined by the inclination angle determination means so that the inclination angle of the object becomes a predetermined inclination angle. The image detection apparatus according to claim 1, wherein: An image detection method for detecting an object image of an object existing in an input image and an inclination of the object,
When detecting a target image including the target object from the input image, a first evaluation step of calculating an evaluation value representing the target object as the angle of the target object is closer to a predetermined angle;
A first target image detection step for determining that an object has been detected in the input image when the evaluation value calculated in the first evaluation step is greater than a predetermined first threshold ;
A rotation image generation step of generating a rotation image by rotating the target image at a predetermined rotation angle;
A second evaluation step of evaluating an angle of the object in the rotated image with respect to a predetermined angle, and calculating an evaluation value representing the object likeness as the angle of the object is closer to the predetermined angle;
A second target image that determines that an object has been detected in the input image when the evaluation value calculated in the second evaluation step is greater than a predetermined second threshold value that is greater than the first threshold value; A detection step;
With respect to the input image in which the object is detected, the rotation image is generated by the rotation image generation step, the evaluation value representing the object likeness is calculated by the second evaluation step, and the second object image detection step is performed. The detection of the object is repeatedly executed, and the evaluation value calculated for each of the at least three rotation images having different rotation angles from which the object is detected and the corresponding rotation angle are expressed by a quadratic or higher function. An evaluation function generation step for generating an evaluation function approximated by
An inclination angle determining step of determining an inclination angle based on the rotation angle at which an evaluation value calculated using the evaluation function is a maximum value ;
An image detection method characterized by comprising: An image detection program for detecting an object image of an object existing in an input image and an inclination of the object,
When detecting a target image including the target object from the input image, a first evaluation procedure for calculating a larger evaluation value representing the target object as the angle of the target object is closer to a predetermined angle;
A first target image detection procedure for determining that an object has been detected in the input image when the evaluation value calculated by the first evaluation procedure is greater than a predetermined first threshold ;
A rotation image generation procedure for generating a rotation image by rotating the target image to a predetermined rotation angle;
A second evaluation procedure for evaluating an angle of the object in the rotated image with respect to a predetermined angle, and calculating an evaluation value representing the object likeness as the angle of the object is closer to the predetermined angle;
A second target image for determining that an object has been detected in the input image when the evaluation value calculated by the second evaluation procedure is greater than a predetermined second threshold value that is greater than the first threshold value; Detection procedure;
With respect to the input image in which the object is detected, the rotation image is generated by the rotation image generation procedure, the evaluation value representing the object likeness is calculated by the second evaluation procedure, and the second object image detection procedure is used. The detection of the object is repeatedly executed, and the evaluation value calculated for each of the at least three rotation images having different rotation angles from which the object is detected and the corresponding rotation angle are expressed by a quadratic or higher function. An evaluation function generation procedure for generating an evaluation function approximated by
An inclination angle determination procedure for determining an inclination angle based on the rotation angle at which an evaluation value calculated using the evaluation function becomes a maximum value ;
An image detection program comprising:

Images