JP4146751B2 - Face identification device, face identification method, and face identification program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a face identification device, a face identification method, and a face identification program for identifying the face of a target person who is an imaging target.
More specifically, based on a captured image obtained by capturing an imaged person as an imaging target, the imaging target is classified into a predetermined category, for example, an adult and a child, and the predetermined processing prepared for each classification is performed. The present invention relates to a face identification apparatus, a face identification method, and a face identification program for acquiring face data to be imaged by obtaining an image and identifying the face of a target person based on the face data.
[0002]
[Prior art]
In recent years, many methods have been proposed for recognizing a target person's face from an image obtained by capturing the target person with an imaging means such as a CCD camera.
Examples of this type of conventional technique include the following.
[0003]
[Patent Document 1]
JP 10-232934 A (paragraphs [0028]-[0032], FIG. 4)
[0004]
[Patent Document 2]
Japanese Patent No. 2872776 (Column 5, Column 6, FIG. 4)
[0005]
[Problems to be solved by the invention]
In the case of the inventions described in Patent Document 1 and Patent Document 2, a face area (face area) is extracted from a captured image obtained by capturing the target person, and eyes and mouths included in the extracted face area, etc. A normalization process is performed on the face area on the basis of the feature points and a normalized face area image (normalized face image) is created. Then, the face of the target person is recognized and identified by comparing and collating the normalized face image with a dictionary image prepared in advance.
[0006]
The normalization processing performed in Patent Document 1 and Patent Document 2 is performed according to the same reference regardless of the positional relationship, size, and the like of each part of the face such as the eyes and mouth. Therefore, when the target person is an adult and a child, the positional relationship and size of each part of the face are different between the adult and the child, so if normalization processing is performed according to the same standard, In the normalized face image obtained by the normalization process, there is a problem that part of the face of the target person is out of frame or part of the face part is displayed distorted in the normalized face image was there.
Therefore, in the identification of the face of the target person in the conventional invention, due to a defect in the obtained normalized face image, comparison / matching between the normalized face image and the prepared dictionary image does not work well, and the target person There was a problem that the face recognition rate decreased.
[0007]
[Means for Solving the Problems]
The present invention relates to a face identification device including identification means for identifying a face of a target person based on a captured image of the target person captured by a camera. The face identification device includes target information acquisition means for acquiring height information and face information of the target person based on contour information and color area information of the target person generated from the captured image, and based on the height information. , Target person classifying means for classifying the target person by category, In the cut-out image obtained by cutting out the face area included in the face information, an eye peripheral area including two pupils is set so that a line segment connecting the two pupils included in the eye peripheral area is horizontal. A cutout image creation unit that rotates the cutout image to form a cutout image for normalization processing; Referring to the classification result input from the target person classification means, Correction Normalization processing unit that normalizes cut-out images for normalization processing When And normalizing the cut-out image for normalization processing, and then creating face data creation means for creating face data of the target person, and face data created by the face data creation means And a face data registering means for registering to.
[0008]
According to the face identification device having such a configuration, the height information and face information of the target person are acquired from the captured image based on the contour information and color area information of the target person created from the captured image of the target person. Then, based on the height information and the face information, the category classification of which category the target person belongs to is prepared. Subsequently, the target person's face data is created after normalizing the target person's face information based on conditions set in advance for each classification. Then, the created face data is registered in the storage means, and the created face data is compared with the face data group stored in the storage means, and the target person stores the face data. The target person is identified as to which of the persons stored in the means corresponds.
[0009]
In this way, face information is processed under the conditions prepared for each category classification, and face data is created.For example, if the target person belongs to the category classification of adults, the category of children under the conditions for adults If it belongs to the classification, the face information of the target person is created after the face information is normalized under the conditions for children, so that faces of eyes, nose, mouth, etc., like adults and children Even if the positional relationship and the size of each part are different, it is possible to prevent a problem from occurring in the face data obtained as a result of the processing.
[0010]
The classification of the target person into each category in the target information acquisition means of the face identification device of the present invention includes not only the height information of the target person but also the head height of the target person, the eye height on the face of the target person, and This is preferably performed using at least one of the proportions of the eyes in the face of the target person.
Thereby, since the classification of the target person into the categories is performed using at least one clear standard, the classification of the target person into each category is more clearly performed.
[0011]
Here, with the method according to the present invention, when face data is created after performing normalization processing of face information under the conditions prepared for each category, the position accuracy of the created face data is excellent, The face data in each category is a set of similar faces.
Therefore, when the eigenspace method is used for the face identification of the target person in the face identification means of the face identification device of the present invention, the position accuracy of the created face data is used in the face identification using the eigenspace method. Is a factor that greatly affects the accuracy of face identification. As a result, the accuracy of the face identification is improved as a result of improving the positional accuracy of the face data.
In addition, if the eigenspace is created from a set of similar faces, the face can be appropriately classified by the higher eigenvector, so that the accuracy of face identification is improved.
[0012]
Furthermore, in the face identification device according to the present invention, the registration of the normalized face data to the storage means performed by the face data registration means is in a state where adults and children can be identified, and more preferably, the target person classification means It is preferable to be performed in a separate database according to the classification of adults and children classified in.
Thereby, the face data group composed of the face data stored in the storage means is distinguished into a data group in which the face data for adults is collected and a data group in which the face data for children are collected.
Therefore, when the target person is an adult, the created face data is created as an adult data group, and when the target person is a child, the created face data is created as a child data group. Since the face data can be compared and collated separately, the search time required for recognizing the target person's face can be shortened.
[0013]
The face data registering unit is configured to detect the face data when face data matching or similar to the face data created by the face data creating unit does not exist in the face data group stored in the storage unit. May be stored in the information recording means.
[0014]
As a result, only the face data of the target person whose face data is not registered in the storage means is sequentially stored, so that duplication of the face data stored in the storage means is prevented and the comparison / collation face data is compared. The amount of information becomes abundant.
Therefore, when face data is created from the captured image of the target person imaged by the camera, and a lot of face data is stored when comparing the created face data and the face data group stored in the storage means, Since it is registered in the means, the probability that the face of the target person can be identified is improved.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Here, first, the configuration of a face image identification system including a face image identification device according to the present invention will be described with reference to FIGS. 1 to 11, and then the operation of the face image identification system will be described with reference to FIGS. The description will be given with reference.
[0016]
(Configuration of face identification system A)
First, an overall configuration of a face identification system A including a face identification device 4 according to the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the overall configuration of the face identification system A.
[0017]
As shown in FIG. 1, the face identification system A analyzes two cameras 1 (1a, 1b) that capture a target person (not shown) and an image (captured image) captured by the camera 1. The captured image analysis device 2 that generates various types of information, the contour extraction device 3 that extracts the contour of the target person based on the various types of information generated by the captured image analysis device 2, and the various types of information generated by the captured image analysis device 2 And a face identification device 4 for identifying the face of the target person based on the contour (outline information) of the target person extracted by the contour extraction device 3. Hereinafter, the camera 1, the captured image analysis device 2, the contour extraction device 3, and the face identification device 4 will be described in order.
[0018]
(Camera 1)
The camera 1 (1a, 1b) is a color CCD camera, and the right camera 1a and the left camera 1b are arranged side by side by a distance B on the left and right. Here, the right camera 1a is used as a reference camera. Images captured by the cameras 1a and 1b (captured images) are stored in a frame grabber (not shown) for each frame, and then input in synchronization with the captured image analysis apparatus 2.
[0019]
The images (captured images) by the cameras 1a and 1b are input to the captured image analysis apparatus 2 after performing calibration processing and rectification processing by a correction device (not shown) and correcting the images. When the camera 1 is a moving camera and the background in the captured image changes, the captured image is corrected by the camera movement amount based on the camera movement amount such as pan and tilt for each captured image.
[0020]
(Captured image analysis device 2)
The captured image analysis apparatus 2 is an apparatus that analyzes images (captured images) input from the cameras 1a and 1b and generates “distance information”, “motion information”, “edge information”, and “skin color area information”. Yes (see FIG. 1).
[0021]
FIG. 2 is a block diagram showing the configuration of the captured image analysis device 2 and the contour extraction device 3 included in the face identification system A shown in FIG. As illustrated in FIG. 2, the captured image analysis apparatus 2 includes a distance information generation unit 21 that generates “distance information”, a motion information generation unit 22 that generates “motion information”, and an edge that generates “edge information”. The information generation unit 23 includes a skin color region information generation unit 24 that generates “skin color region information”.
[0022]
(Distance information generator 21)
The distance information generation unit 21 detects the distance from the camera 1 for each pixel based on the parallax between the two captured images captured by the cameras 1a and 1b at the same time. Specifically, the parallax is obtained from the first captured image captured by the camera 1a as the reference camera and the second captured image captured by the camera 1b using the block correlation method, and the trigonometric method is obtained from the parallax. Is used to determine the distance from the camera 1 to the object imaged by each pixel. Then, the obtained distance is associated with each pixel of the first captured image, and a distance image D1 (see FIG. 3A) in which the distance is represented by a pixel value is generated. This distance image D1 becomes distance information. In the example of FIG. 3A, the target person C exists at the same distance.
[0023]
In the block correlation method, the first captured image and the second captured image are compared with the same block (for example, 16 × 16 pixels) having a specific size, and the first captured image and the second captured image are compared. This is a method for detecting parallax by examining how many pixels the subject in the block is shifted.
[0024]
(Motion information generating unit 22)
The motion information generation unit 22 is a reference camera. right The motion of the target person is detected based on the difference between the “captured image (t)” at time t captured in time series by the camera 1a and the “captured image (t + 1)” at time t + 1. Specifically, the difference between the “captured image (t)” and the “captured image (t + 1)” is taken to examine the displacement of each pixel. Then, a displacement vector is obtained based on the examined displacement, and a difference image D2 (see FIG. 3B) in which the obtained displacement vector is represented by a pixel value is generated. This difference image D2 becomes motion information. In the example of FIG. 3B, a motion is detected on the left arm of the target person C.
[0025]
(Edge information generation unit 23)
The edge information generation unit 23 is a reference camera. right Based on the shading information or color information of each pixel in the image (captured image) captured by the camera 1a, an edge image is generated by extracting the edge existing in the captured image. Specifically, based on the luminance of each pixel in the captured image, a portion where the luminance greatly changes is detected as an edge, and an edge image D3 (see FIG. 3C) including only the edge is generated. This edge image D3 becomes edge information.
[0026]
For example, the Sobel operator is multiplied for each pixel, and a line segment having a predetermined difference from the adjacent line segment is detected as an edge (horizontal edge or vertical edge) in units of rows or columns. Note that the Sobel operator is an example of a coefficient row having a weighting factor for pixels in the vicinity of a certain pixel.
[0027]
(Skin color area information generation unit 24)
The skin color area information generation unit 24 is a reference camera. right From the image (captured image) captured by the camera 1a, the skin color area of the target person existing in the captured image is extracted. Specifically, the RGB values of all the pixels in the captured image are converted into an HLS space consisting of hue, brightness, and saturation, and pixels whose hue, brightness, and saturation are within a preset threshold range are converted to a flesh-colored region. (See FIG. 3D). In the example of FIG. 3D, the face of the target person C is extracted as the skin color region R1, and the hand is extracted as the skin color region R2. The skin color areas R1 and R2 serve as skin color area information.
[0028]
The “distance information (distance image D1)”, “motion information (difference image D2)”, and “edge information (edge image D3)” generated by the captured image analysis device 2 are input to the contour extraction device 3. Further, “distance information (distance image D1)” and “skin color area information (skin color areas R1, R2)” generated by the captured image analysis apparatus 2 are input to the face identification apparatus 4.
[0029]
(Outline extraction device 3)
The contour extraction device 3 is based on the “distance information (distance image D1)”, “motion information (difference image D2)”, and “edge information (edge image D3)” generated by the captured image analysis device 2. It is an apparatus which extracts the outline of (refer FIG. 1).
[0030]
As illustrated in FIG. 2, the contour extraction device 3 generates a target distance setting unit 31 that sets a “target distance” that is a distance where the target person exists, and a “target distance image” based on the “target distance”. From the target distance image generating unit 32, the target region setting unit 33 for setting the “target region” in “in the target distance image”, and the contour extracting unit 34 for extracting “the contour of the target person” from “in the target region” It is configured.
[0031]
(Target distance setting unit 31)
The target distance setting unit 31 includes a distance image D1 (see FIG. 3A) generated by the captured image analysis device 2, and a difference image D2 (see FIG. 3B) generated by the motion information generation unit 22. Based on the above, a “target distance” that is a distance where the target person exists is set. Specifically, the pixels having the same pixel value in the distance image D1 are grouped (pixel group), and the pixel values of the pixel group in the difference image D2 are accumulated. Then, it is considered that a moving object with the largest amount of movement, that is, a target person exists in an area where the cumulative value of pixel values is larger than a predetermined value and is closest to the camera 1, and the distance is set as a target. The distance is used (see FIG. 4A). In the example of FIG. 4A, the target distance is set to 2.2 m. The target distance set by the target distance setting unit 31 is input to the target distance image generation unit 32.
[0032]
(Target distance image generation unit 32)
The target distance image generation unit 32 refers to the distance image D1 generated by the captured image analysis device 2 (see FIG. 3A), and applies the pixel existing in the target distance ± αm set by the target distance setting unit 31. A “target distance image D4” is generated by extracting corresponding pixels from the edge image D3 (see FIG. 3C). Specifically, a pixel corresponding to the target distance ± αm input from the target distance setting unit 31 in the distance image D1 is obtained. Then, only the obtained pixels are extracted from the edge image D3 (see FIG. 3C) generated by the edge information generation unit 23, and the target distance image D4 (see FIG. 4B) is generated. Therefore, the target distance image D4 is an image representing the target person existing at the target distance with an edge. In the example of FIG. 3C, α is set to 0.5 m. The target distance image D4 generated by the target distance image generation unit 32 is input to the target region setting unit 33 and the contour extraction unit 34.
[0033]
(Target area setting unit 33)
The target area setting unit 33 sets the “target area T” in the target distance image D4 generated by the target distance image generating unit 32. Specifically, a histogram H in which the pixel values in the vertical direction of the target distance image D4 are accumulated is generated, and the position where the frequency in the histogram H is maximum is specified as the center position in the horizontal direction of the target person C (FIG. 5). (See (a)). Then, a range of a specific size (for example, 0.5 m) is set as the target region T on the left and right of the specified center position (see FIG. 5B). The vertical range of the target region T is set to a specific size (for example, 2 m). When setting the target area T, the setting range of the target area T is corrected with reference to camera parameters such as the tilt angle of the camera 1 and the height of the camera 1. The target area T set by the target area setting unit 33 is input to the contour extraction unit 34.
[0034]
(Outline extraction unit 34)
The contour extraction unit 34 is a target generated by the target distance image generation unit 32. distance In the image D4 (see FIG. 4B), the contour of the target person is extracted from the target area T set by the target area setting unit 33. Specifically, the contour O of the target person C is extracted using a technique called Snakes (see FIG. 5C). Snakes is a technique for extracting the contour of an object by using a dynamic contour model consisting of a closed curve called a snake and contracting and deforming the closed curve so as to minimize a predefined energy. The contour O of the target person C extracted by the contour extraction unit 34 is input to the face identification device 4 as “contour information” (see FIG. 1).
[0035]
(Face identification device 4)
FIG. 6 is a block diagram showing a configuration of the face identification device 4 included in the face identification system A shown in FIG.
The face identification device 4 identifies the target person based on the contour information input from the contour extraction device 3 and the skin color area information and distance information input from the captured image analysis device 2.
[0036]
As shown in FIGS. 1 and 6, the face identification device 4 includes a target information acquisition unit 41, a target person classification unit 42, a face data creation unit 43, a face data registration unit 44, a face identification unit 45, Storage means 46.
[0037]
Registered in the storage means 46 are an adult DB 46a, a child DB 46b, and adult and child eigenvectors (not shown) that define eigenspaces generated for each adult and child. .
In the adult DB 46a, coefficients for a plurality of persons are registered when each individual adult is represented by a linear sum of eigenvectors for adults.
In the child DB 46b, a coefficient for each individual child is expressed for a plurality of persons when the individual child is represented by a linear sum of eigenvectors for the child.
Each of eigenvectors for adults and children has a high contribution ratio among eigenvectors obtained by performing principal component analysis on a plurality of normalized face images having sufficient variation.
[0038]
(Target information acquisition means 41)
The target information acquisition unit 41 acquires the height information and face information of the target person based on the contour information input from the contour extraction device 3 and the skin color area information and distance information input from the captured image analysis device 2. It is.
[0039]
Specifically, as illustrated in FIG. 7, the target information acquisition unit 41 is the first among the skin color regions R1,..., Rn of the target person C included in the skin color region information input from the captured image analysis device 2. The skin color area (R1 in the figure) having a large area is regarded as the face of the target person C, and the area A of the skin color area R1 (hereinafter referred to as the face area) regarded as the face is obtained from the skin color area information.
Subsequently, the target information acquisition unit 41 extracts a part related to the skin color region R1 regarded as the face region from the skin color region information and sets it as face region information. From the face region information and the area A, the target person C Is created (see FIG. 8).
[0040]
Further, the target information acquisition unit 41 is located at the position closest to the face area considered to be the face of the target person C among the outlines O of the target person C included in the contour information input from the contour extraction device 3. The end portion at the highest position is regarded as the top portion P of the target person C, and the top position information indicating the position of the top portion P of the target person C is acquired.
And the target information acquisition means 41 acquires the target distance which shows the separation distance from the camera 1 to the target person C from the distance information input from the captured image analysis apparatus 2, and acquires the acquired target distance and the top position information. Then, the height information of the target person is created (see FIG. 8).
[0041]
The target information acquisition unit 41 outputs the created height information and face information to the target person classification unit 42 and outputs only the face information to the face data creation unit 43.
[0042]
(Target person classification means 42)
The target person classification means 42 determines whether the target person C imaged by the camera 1 is an adult or a child based on the height information and face information input from the target information acquisition means 41, Alternatively, the target person C is classified into the category of children.
[0043]
As shown in FIG. 6, the target person classification unit 42 includes a height recognition unit 47, a head-and-body recognition unit 48, an eye position recognition unit 49, an eye ratio recognition unit 50, and an adult / child determination unit 51. It is comprised including.
[0044]
The height recognition unit 47 represents the height of the target person in numerical form. The height recognizing unit 47 obtains the height of the target person C, who is the subject of imaging, based on the height information input from the target information acquisition means 41, and uses the obtained value as height data indicating the height. Output to the child determination unit 51
[0045]
Specifically, as shown in FIG. 7, the height recognition unit 47 determines the height of the top portion P (P ′) of the target person C (C ′) from the target distance and the top position information included in the height information. The position H (H ′) is obtained, and the height position H (H ′) of the crown P (P ′) is regarded as the height of the target person C (C ′). Then, the height position H (H ′) regarded as the height of the target person is output to the adult / child determination unit 51 as height data.
[0046]
The head / body recognizing unit 48 represents the head / body of the target person in numerical form. The head / body recognition unit 48 obtains the head / body of the target person who is the imaging target by quantifying based on the height information and the face information input from the target information obtaining unit 41, and the obtained value indicates the head / head indicating the head / body. The data is output to the adult / child determination unit 51 as body data.
[0047]
Specifically, as shown in FIG. 9, the head-and-body recognizing unit 48 determines the length L (L ′) from the top P (P ′) of the target person C (C ′) to the lower end of the face area, This is obtained from the top position information indicating the position of the top P (P ′) of the target person C (C ′) and the face area information.
Then, the obtained length L (L ′) is regarded as the length in the height direction of the head of the target person C (C ′) (hereinafter referred to as the head length), and this head length L (L ') And the target person C (C') obtained from the height information Height position Ratio with H (H ') ( Height position H (H ′) / head length L (L ′)) is obtained. The obtained value is output to the adult / child determination unit 51 as head-and-body data.
[0048]
The eye position recognition unit 49 represents the position of the eyes in the face area of the target person in numerical form. The eye position recognizing unit 49 obtains the eye position in the face area of the target person C numerically based on the face information input from the target information obtaining unit 41, and uses the obtained value as eye position data. And output to the adult / child determination unit 51.
[0049]
Specifically, as illustrated in FIG. 10A, the eye position recognition unit 49 determines the skin color area (face area) that is regarded as the face of the target person C from the face area information included in the face information. The length L1 in the height direction of R1 is obtained. Then, the eye position recognition unit 49 Skin color area (face area) Two black circles (pupils) are searched from R1, and an eye peripheral region including the two searched black circles (pupils) is set. Then, the black circle (pupil) is accurately extracted by the circular edge filter from the set eye peripheral region, and from the center in the height direction of the extracted black circle (pupil), Skin color area (face area) A length L2 to the lower end of R1 is obtained from the face area information.
Subsequently, the eye position recognition unit 49 obtains a ratio (L2 / L1) between the lengths L1 and L2, and outputs the obtained value to the adult / child determination unit 51 as eye position data.
[0050]
The eye ratio recognition unit 50 Skin color area (face area) In R1, the target person C pupil Area E is concerned Skin color area (face area) It expresses how much of R1 occupies by quantification. The eye position recognizing unit 49 obtains the ratio of the eyes in the face area of the target person C numerically based on the face information input from the target information acquisition unit 41, and uses the obtained value as eye ratio data. And output to the adult / child determination unit 51.
[0051]
Specifically, as shown in FIG. 10 (b), the eye ratio recognizing unit 50 uses the face information input from the target information acquiring unit 41 to surface Get the product A.
Then, the eye rate recognition unit 50 Skin color area (face area) Two black circles (pupils) are searched from R1, and an eye peripheral region including the two searched black circles (pupils) is set. Then, a black circle (pupil) is accurately extracted from the set eye peripheral region by a circular edge filter, and an area (pupil area) of the extracted black circle (pupil) is obtained as a pupil area E. Here, since a human face usually has two eyes, the pupil area E is obtained by the sum of the areas of two black circles (El, Er) (where El is the area of the left pupil toward the left). , Er is the area of the right pupil toward the right).
[0052]
Subsequently, the eye ratio recognition unit 50 includes the pupil area E, surface Ratio to product A (pupil area E / surface The product A) is obtained, and the obtained value is output to the adult / child determination unit 51 as eye ratio data.
[0053]
The adult / child determination unit 51 includes the height data of the target person input from the height recognition unit 47, the head / body data of the target person input from the head / body recognition unit 48, and the eye input from the eye position recognition unit 49. The target person C is classified as an adult or a child using at least one of height data and eye ratio data input from the eye ratio recognition unit 50.
[0054]
When the target person C is classified using the height data of the target person C input from the height recognition unit 47, the adult / child determination unit 51 compares the height data of the target person C with a predetermined threshold. When the height data of the target person C is larger than the threshold value, it is determined that the target person C is an adult. On the other hand, when the height data of the target person C is equal to or less than the threshold value, it is determined that the target person is a child.
Then, a classification result indicating whether the target person C is an adult or a child is output to the face data creation unit 43.
[0055]
In the case of the present embodiment, the threshold is set to 120 cm corresponding to the height of an average second grader to third grader.
Therefore, if the height data of the target person C is larger than 120, the target person C is classified as an adult, and if the height data of the target person C is 120 or less, the target person C is classified as a child.
[0056]
When performing classification of the target person using the head / body data of the target person input from the head / body recognition unit 48, the adult / child determination unit 51 compares the head / body data of the target person with a predetermined threshold. If the head-body data of the target person is larger than the threshold value, it is determined that the target person is an adult. On the other hand, if the head / body data of the target person is equal to or less than the threshold value, it is determined that the target person is a child.
Then, a classification result indicating whether the target person is an adult or a child is output to the face data creation unit 43.
[0057]
In the case of the present embodiment, the threshold value is set to 6 heads and bodies corresponding to the average heads of second graders to third graders.
Therefore, if the head / body data of the target person C is larger than 6, the target person C is classified as an adult, and if the head / body of the target person C is 6 or less, the target person C is classified as a child.
[0058]
When classifying the target person C using the eye position data input from the eye position recognizing unit 49, the adult / child determining unit 51 sets the eye position data of the target person C, a predetermined threshold value, and the like. When the eye position data of the target person C is larger than the threshold value, it is determined that the target person C is an adult. On the other hand, if the eye position data of the target person C is equal to or less than the threshold value, it is determined that the target person C is a child.
Then, a classification result indicating whether the target person C is an adult or a child is output to the face data creation unit 43.
[0059]
In the case of the present embodiment, the threshold value is set to 60% corresponding to the eye height of the average second grader to third grader.
Therefore, if the eye position data of the target person C is larger than 0.6, the target person C is classified as an adult, and if the eye position data of the target person C is 0.6 or less, the target person C is a child. are categorized.
[0060]
When the target person C is classified using the eye ratio data input from the eye ratio recognition unit 50, the adult / child determination unit 51 sets the eye ratio data of the target person C, a predetermined threshold value, and the like. If the ratio data of the eyes of the target person C is larger than the threshold value, it is determined that the target person C is a child. On the other hand, if the eye ratio data of the target person C is equal to or less than the threshold value, it is determined that the target person C is an adult.
Then, a classification result indicating whether the target person C is an adult or a child is output to the face data creation unit 43.
[0061]
In the case of the present embodiment, the threshold is set to 2%, which corresponds to the ratio of the average second grader to third grader eye.
Therefore, if the eye ratio data of the target person C is larger than 0.02, the target person C is classified as a child, and if the eye ratio data of the target person C is 0.02 or less, the target person C is an adult. are categorized.
[0062]
As described above, the adult / child determination unit 51 determines whether the target person is an adult or a child using at least one of the methods described above.
[0063]
Here, when performing classification of adults / children by simultaneously using a plurality of methods described above, different classification results may be obtained depending on the type of data used for determination.
In this case, the adult / child determination unit 51 may adopt a configuration in which the appearance frequency of the obtained adult / child classification results is used as the classification result.
[0064]
In the adult / child determination unit 51, the weights taking into account the accumulated amount of data output from the respective recognition units 47, 48, 49, 50, the dispersion rate and accuracy of the obtained values, and the like are assigned to the respective recognition units 47, It is also possible to adopt a configuration in which adults / children are determined based on the results obtained by weighting performed on each of the data output from 48, 49, 50.
[0065]
(Face data creation means 43)
The face data creation means 43 creates a normalized face image of the target person C by performing normalization processing of the face information input from the target information acquisition means 41 under conditions according to the classification result in the target person classification means 42. Then, feature parameters representing the face of the target person are obtained by feature extraction of the created normalized face image.
[0066]
As shown in FIG. 6, the face data creation unit 43 includes a cut-out image creation unit 52, a normalization processing unit 53, and a feature parameter generation unit 54.
[0067]
The cut-out image creation unit 52 cuts out the face area of the target person C from the face information input from the target information acquisition unit 41.
Specifically, as shown in FIG. 11, a skin color area (face area) regarded as the face of the target person is cut out from the face area information included in the face information to create a cut-out image. Then, in this cut-out image, two black circles (pupils) are searched, and an eye peripheral area including the two searched black circles (pupils) is set.
[0068]
Then, the cutout image is rotated so that the line segment connecting the two black circles (pupils) included in the eye peripheral region is horizontal, the inclination of the face of the target person C in the cutout image is corrected, and normalization is performed. A cut-out image for processing is formed.
Then, the cutout image creating unit 52 outputs the cutout image for normalization processing to the normalization processing unit 53.
[0069]
The normalization processing unit 53 normalizes the cut-out image to create a normalized face image of the target person's face area.
[0070]
Specifically, as illustrated in FIG. 11, the normalization processing unit 53 displays the classification result input from the target person classification unit 42, that is, the classification result indicating whether the target person is an adult or a child. The normalized face image is created by normalizing the clipped image for normalization processing input from the clipped image creating unit 52 under the conditions set in advance for each classification result.
[0071]
The cut-out image normalization process will be described below for the case where the target person is an adult and the case where the target person is a child.
[0072]
<When the target person is an adult>
When the target person is an adult, the normalization processing unit 53 normalizes the cut-out image under conditions for adults, and creates a normalized face image composed of a 64 × 64 pixel region.
At this time, the normalization processing unit 53 further adjusts the width direction of one pupil of the target person so that the position of the target person's pupil is 3/4 of the height direction of the area of 64 × 64 pixels. The face area of the target person is enlarged / reduced so as to be expressed by 35 pixels between the center part α of the other pupil and the center part β in the width direction of the other pupil to create a normalized face image .
[0073]
<When the target person is a child>
When the target person is a child, the normalization processing unit 53 normalizes the cut-out image under the conditions for children and creates a normalized face image composed of a 64 × 64 pixel area.
At this time, the normalization processing unit 53 further increases the width direction of one pupil of the target person so that the position of the pupil of the target person becomes a position 2/3 of the height direction of the area of 64 × 64 pixels. The face area is enlarged / reduced so that a 40-pixel area is formed between the center part α of the other pupil and the center part β in the width direction of the other pupil, thereby creating a normalized face image.
[0074]
Then, the face data creation unit 43 outputs the normalized face image obtained by the normalization process to the feature parameter generation unit 54.
[0075]
The feature parameter generation unit 54 represents the normalized face image input from the normalization processing unit 53 as a vector, performs feature extraction processing by taking the inner product of this vector and each eigenvector in the storage means 46, and performs the target person The characteristic parameters of the are calculated.
[0076]
Specifically, the feature parameter generation unit 54 obtains the normalized facial image feature parameters (feature parameters of the target person's face region) while performing normalization so that face identification by pattern matching can be performed quickly. Information compression of the face image is performed, and the obtained characteristic parameters are output to the face data registration unit 44 and the face identification unit 45, respectively.
[0077]
(Face data registration means 44)
The face data registration unit 44 registers the feature parameter input from the face data creation unit 43 in the storage unit 46.
[0078]
Specifically, the face data registration unit 44 refers to the classification result input from the target person classification unit 42, and when the classification result that the target person C is an adult is input, the feature parameter is The data is stored in the adult area (adult DB 46a) of the storage means 46.
On the other hand, when the classification result that the target person C is a child is input, the characteristic parameter is stored in the child area (child DB 46b) of the storage means 46.
That is, the face data registration unit 44 stores the created feature parameter in the storage unit 46 in a state where the adult and child can be identified.
At this time, the face data registration unit 44 includes name information indicating who the target person C is and face information that is information related to the area of the face of the target person C in the captured image of the target person C. It is stored in the storage means 46 together with the characteristic parameters.
[0079]
Note that the storage (registration) of the characteristic parameters in the storage unit 46 is not limited to this configuration as long as it is performed in a state where adults and children can be identified. For example, an adult / child identification field is provided and stored in the same area (DB) of the storage means 46, and a configuration in which adult / child identification can be performed based on data input to the adult / child identification field is also selected. Is possible.
[0080]
In this embodiment, in order to reduce the amount of data, the characteristic parameter of the target person C is stored in the storage unit 46. However, the normalized face image of the target person C and the captured image of the target person C are stored. Can be stored in the storage means 46 as it is.
[0081]
(Face identification means 45)
The face identification means 45 refers to the storage means 46 based on the feature parameters input from the face data creation means 43, and from the face data creation means 43 in the group of persons whose feature parameters are registered in the storage means 46. A person having a feature parameter closest to the inputted feature parameter is searched.
[0082]
Specifically, the face identifying unit 45 refers to the classification result input from the target person classifying unit 42, and when the classification result that the target person C is an adult is input, For the area for use (adult DB 46 a), comparison / collation with the feature parameter input from the face data creation means 43 is performed.
On the other hand, when the classification result indicating that the target person C is a child is input, the face identifying unit 45 is input from the face data creating unit 43 for the child area (child DB 46b) of the storage unit 46. Compare and collate with the selected feature parameter.
[0083]
Here, the search for the face data of the target person C finally created from the captured image of the target person C, that is, the person having the feature parameter closest to the feature parameter is performed by using the feature parameter generated by the feature parameter generation unit 54. , A search is made in the storage means 46 for a difference between the characteristic parameters of each person stored in the storage means 46 that is equal to or less than a certain threshold value, and a feature parameter that satisfies this condition matches a feature parameter of face data Treated as data.
[0084]
Then, when there is face data that matches the characteristic parameter in the storage means 46, the face identifying means 45 refers to the name information of the face data and recognizes who the target person is. Output the result.
On the other hand, if there is no face data that matches the feature parameters, a recognition result indicating that the target person is not registered is output.
[0085]
Next, processing performed in the face identification device 4 will be described in detail with reference to the block diagram shown in FIG. 1 and the flowchart shown in FIG.
[0086]
The captured image analysis apparatus 2 shown in FIG. 1 captures the target person C, and distance information and skin color area information are input from the captured image analysis apparatus 2 and contour information is input from the contour extraction apparatus 3 to the face identification apparatus 4, respectively. Then, the target information acquisition unit 41 acquires the height information and face information of the target person C from the input distance information, skin color area information, and contour information (step S1). Then, the acquired height information and face information are output to the target person classification means 42, and only the face information is output to the face data creation means 43.
[0087]
When the height information and the face information are input to the target person classifying unit 42, each part of the target person classifying unit 42 (height recognition unit 47, head-body recognition unit 48, eye position recognition unit 49, and eye ratio recognition unit 50). ) Creates classification data (height data, head-and-body data, eye position data, and eye ratio data) for classifying the target person as an adult or a child (step S2).
[0088]
Then, the adult / child determination unit 51 of the target person classifying unit 42 determines whether the target person C is an adult or a child, and classifies the target person as an adult or a child (step S3).
Subsequently, a classification result indicating whether the person is an adult or a child is output to the face data creation unit 43.
[0089]
When the classification result is input to the face data creation means 43, the face data creation means 43 confirms whether the target person C is an adult or a child (step S4).
[0090]
When the target person C is an adult, the face data creation unit 43 creates a cut-out image from the face information input from the target information acquisition unit 41, and normalizes the cut-out image under conditions for adults. Processing is performed to create a normalized face image, and face data is created from the normalized face image (step S5).
Then, the generated face data is output to the face data registration unit 44.
[0091]
When the face data is input to the face data registration means 44, the face data registration means 44 registers the face data in the adult database (adult DB 46a) of the storage means 46 (step S6).
[0092]
On the other hand, when the target person C is a child, the face data creation unit 43 creates a cut-out image of the target person C from the face information input from the target information acquisition unit 41, and applies the child data to the cut-out image. The normalized face image is created by performing the normalization process under the conditions for use, and the face data is created from the normalized face image (step S7).
Then, the generated face data is output to the face data registration unit 44.
[0093]
When the face data is input to the face data registration means 44, the face data registration means 44 registers the face data in the child database (child DB 46b) in the storage means 46 (step S8).
[0094]
(Operation of face identification system A)
Next, the operation of the face identification system A will be described with reference to the block diagram showing the overall configuration of the face identification system A shown in FIG. 1 and the flowcharts shown in FIGS. 13 and 14.
[0095]
<Captured image analysis step>
Referring to the flowchart shown in FIG. 13, in captured image analysis apparatus 2, when captured images are input from cameras 1 a and 1 b (step S <b> 101), distance information that is distance information from the captured images in distance information generation unit 21. D1 (see FIG. 3A) is generated (step S102), and the motion information generating unit 22 generates a difference image D2 (see FIG. 3B) as motion information from the captured image (step S103). Further, the edge information generation unit 23 generates an edge image D3 (see FIG. 3C) as edge information from the captured image (step S104), and the skin color region information generation unit 24 uses the skin color region information from the captured image. A certain skin color region R1, R2 (see FIG. 3D) is extracted (step S105).
[0096]
<Outline extraction step>
Continuing with reference to the flowchart shown in FIG. 14, in the contour extracting device 3, first, in the target distance setting unit 31, the distance at which the target person exists from the distance image D <b> 1 and the difference image D <b> 2 generated in steps S <b> 102 and S <b> 103. The target distance is set (step S106). Subsequently, the target distance image generation unit 32 generates a target distance image D4 (see FIG. 4B) obtained by extracting pixels existing at the target distance set in step S106 from the edge image D3 generated in step S104. (Step S107).
[0097]
Next, the target area setting unit 33 sets the target area T (see FIG. 5B) in the target distance image D4 generated in step S107 (step S108). Then, the contour extraction unit 34 extracts the contour O (see FIG. 5C) of the target person from the target region T set in step S108 (step S109).
[0098]
<Target information acquisition step>
Still referring to the flowchart shown in FIG. 14, in the face identification device 4, the target information acquisition unit 41 extracts the distance information generated in step S <b> 102, the skin color area information extracted in step S <b> 105, and extracted in step 109. Target person circle Guo O Thus, the height information and face information of the target person are acquired (step S110).
[0099]
<Target person classification step>
Next, the target person classification unit 42 generates target person classification data from the height information and face information of the target person acquired in step S110. Then, by comparing the generated classification data with a predetermined threshold, it is determined whether the target person is an adult or a child, and the target person is classified as an adult or a child (step S112).
[0100]
<Face data creation step>
The face data creation unit 43 generates a cut-out image of the target person from the face information generated in step S110. Then, referring to the classification result in step S112, normalization is performed on the generated cut-out image under the condition of adult when the target person is classified as an adult, and under the condition of child when the target person is classified as a child. Processing is performed to create a normalized face image, and face data is generated from the normalized face image (step S113).
[0101]
<Face data registration step>
The face data registration means 44 refers to the classification result in step S112. When the target person is classified as an adult, it is stored in the adult DB 46a in the storage means 46 when the target person is classified as a child. The face data is registered in the child DB 46b in the means 46 (step S114).
[0102]
<Face identification step>
The face identification unit 45 refers to the classification result in step S112, and when the target person is classified as an adult, refers to the adult DB 46a in the storage unit 46 based on the face data generated in step S113, Search for face data that matches the face data. On the other hand, when the target person is classified as a child, the child DB 46b in the storage means 46 is referred to based on the face data generated in step S113, and the face data matching the face data is searched. Then, the search result is output (step S115).
[0103]
【The invention's effect】
According to the face identification device of the present invention, the height information and face information of the target person are acquired from the captured image based on the contour information and color area information of the target person created from the captured image of the target person. Based on the information and the face information, a classification into an adult and a child is performed regarding whether the target person is an adult or a child.
Subsequently, the target person's face data is created after normalizing the target person's face information based on conditions set in advance for each classification. Then, the created face data is registered in the storage means, and the created face data is compared with the face data group stored in the storage means, and the target person stores the face data. The target person is identified as to which of the persons stored in the means corresponds.
[0104]
In this way, face information is processed under the conditions prepared for each classification of adults and children, and face data is created, that is, if the target person is an adult, it is an adult condition, and if it is a child, it is a child Since the face data of the target person is created after the face information is normalized under the conditions for use, the positional relationship and size of each part of the face, such as the eyes, nose, and mouth, like adults and children Even if the lengths are different, the face data obtained as a result of the processing is prevented from being defective. Therefore, the recognition rate of the target person's face is improved.
[0105]
Here, when face data is created after performing normalization processing of face information under the conditions prepared for each category by the method according to the present invention, the position accuracy of the created face data is excellent.
Therefore, when the eigenspace method is used for the face identification of the target person in the face identification means of the face identification device of the present invention, the position of the created face data in the case of face identification using the eigenspace method Since accuracy is a factor that greatly affects the accuracy of face identification, the accuracy of face identification is improved as a result of improving the accuracy of the position of face data.
In other words, even an object having similar characteristics such as a human face can be accurately identified.
[0106]
Furthermore, in the face identification device according to the present invention, the registration of the face data in the storage means performed by the face data registration means is performed in a separate database in a state where adults and children can be identified.
Thereby, the face data group composed of the face data stored in the storage means is distinguished into a data group in which the face data for adults is collected and a data group in which the face data for children are collected. When the target person is an adult, the created face data is the adult data group, and when the target person is a child, the created face data is the child data group. Since the data can be compared and verified separately, the search time required for recognizing the target person's face can be reduced.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall configuration of a face identification system A. FIG.
FIG. 2 is a block diagram showing a configuration of a captured image analysis device 2 and a contour extraction device 3 included in the face identification system A shown in FIG.
3A is a diagram illustrating a distance image D1, FIG. 3B is a difference image D2, FIG. 3C is an edge image D3, and FIG. 3D is a diagram illustrating skin color regions R1 and R2.
FIG. 4 is a diagram for explaining a method of setting a target distance.
FIG. 5 is a diagram for explaining a method for setting a target region T and a method for extracting a contour O of a target person C from within the target region T;
6 is a block diagram showing a configuration of a face identification device 4 included in the face identification system A shown in FIG. 1. FIG.
FIG. 7 is a diagram for explaining a method for generating height data of a target person.
FIG. 8 is an explanatory diagram illustrating a configuration of height information and face information generated by a target information acquisition unit.
FIG. 9 is a diagram for explaining a method for generating head / body data of a target person;
FIG. 10A is a diagram for explaining a method of generating eye height data of a target person. (B) is a figure for demonstrating the production | generation method of the ratio data of a target person's eyes.
FIG. 11 is an explanatory diagram for explaining a flow from generation of face data of a target person to generation of face data (normalized face image).
FIG. 12 Face Identification device 5 is a flowchart for explaining processing in FIG.
FIG. 13 Face identification 10 is a flowchart for explaining processing in system A.
FIG. 14 Face identification 10 is a flowchart for explaining processing in system A.
[Explanation of symbols]
A face identification system
1 Camera
2 Captured image analyzer
3 Contour extraction device
4 Face recognition device
41 Target information acquisition means
42 Target person classification means
43 Face data creation means
44 Face data registration means
45 Face identification means
46 Memory means
47 Height Recognition Department
48 Head and Body Recognition
49 Eye position recognition unit
50th rate recognition unit
51 Adult / Child Judgment Department
52 Cutout image creation unit
53 Normalization processing part

Claims (5)

A face identification device comprising identification means for identifying a face of a target person based on a captured image of the target person imaged by a camera,
Target information acquisition means for acquiring height information and face information of the target person based on contour information and color area information of the target person generated from the captured image;
A target person classification means for classifying the target person into categories based on the height information;
In the cut-out image obtained by cutting out the face area included in the face information, an eye peripheral area including two pupils is set so that a line segment connecting the two pupils included in the eye peripheral area is horizontal. A cutout image creation unit that rotates the cutout image to form a cutout image for normalization processing, and a classification result input from the target person classification unit, and a normalization processing unit that processes normalize the cutout image for KiTadashi-normalized treatment, in terms of the cut-out image for the normalization process and normalization process, the face data generated face data of the target person Creating means;
Face data registration means for registering the face data created by the face data creation means in a storage means;
A face identification device comprising: The target person classification means, the normalization processing unit classifies the target person as an adult or a child based on the height information and the face information,
The normalization processing unit refers to the classification result input from the target person classification unit, and normalizes the cut-out image for normalization processing under the conditions for adults when the target person is an adult, If the target person is a child, normalize the cut-out image for normalization under the conditions for children,
The face data registering unit is configured so that the face data created by the face data creating unit can be distinguished from an adult or a child. If the target person is an adult, the target person is stored in an adult database. In the child database,
With reference to the classification result input from the target person classification means, if the target person is an adult, the adult database is compared and compared with the face data input from the face data creation means, 2. The apparatus according to claim 1, further comprising: a face identifying unit that compares and collates the child data with the face data input from the face data creating unit when the target person is a child. Face identification device. The normalization processing unit increases the position in the height direction of the line segment connecting the two pupils when the target person is an adult and the target person is a child. The face identification device according to claim 1, wherein normalization processing is performed. A method for identifying a face of a target person based on a captured image of the target person imaged by a camera,
A target information acquisition step for acquiring height information and face information of the target person based on contour information and color area information of the target person generated from the captured image;
A target person classification step of classifying the target person into categories based on the height information;
In the cut-out image obtained by cutting out the face area included in the face information, an eye peripheral area including two pupils is set so that a line segment connecting the two pupils included in the eye peripheral area is horizontal. the cut-out image is rotated by image creation section cut out cut-out image for the normalization process is formed, with reference to the classification result in said subject person classification step, in the classification result separately preset condition, before KiTadashi Tadashi A normalization processing unit that normalizes the cut-out image for normalization processing, normalizes the cut-out image for normalization processing, and then creates a face data creation step for creating face data of the target person;
A face data registration step of registering the face data created in the face data creation step in a storage means;
A face identification step of comparing / collating the face data created in the face data creation step with the face data registered in the storage means;
A face identification method characterized by comprising: In order to identify the target person's face based on the captured image of the target person captured by the camera,
Target information acquisition means for acquiring height information and face information of the target person based on contour information and color area information of the target person generated from the captured image;
A target person classification means for classifying the target person into categories based on the height information;
In the cut-out image obtained by cutting out the face area included in the face information, an eye peripheral area including two pupils is set so that a line segment connecting the two pupils included in the eye peripheral area is horizontal. A cutout image creation unit that rotates the cutout image to form a cutout image for normalization processing, and a classification result input from the target person classification unit, and a normalization processing unit that processes normalize the cutout image for KiTadashi-normalized treatment, in terms of the cut-out image for the normalization process and normalization process, the face data generated face data of the target person Creation means,
Face data registration means for registering the face data created by the face data creation means in a storage means;
Face identification means for comparing and collating face data created by the face data creation means with face data registered in the storage means;
A face identification device program that is made to function as:

Images