JP6355307B2 - Face matching device, face matching method, and face matching program - Google Patents

Description

  According to the present invention, registration data including a face image of a person registered in advance or a feature amount of the face image, a face image of a person to be collated (hereinafter referred to as “verification target person”), or the face image of the face image The present invention relates to a face collation apparatus, a face collation method, and a face collation program that collate input data composed of feature amounts.

  Conventionally, in order to determine whether or not the person to be collated matches a person registered in advance, the registered data including the face image of the person registered in advance or the feature amount of the face image, and the person to be collated 2. Description of the Related Art A face matching technique that performs a matching process with a face image or input data composed of a feature amount of the face image is known.

  For example, Patent Documents 1, 2, and 3 disclose techniques for calculating a distance between a local area of input data and a corresponding local area of registered data, and performing face matching by integrating the calculated distances.

  Further, in Patent Document 4, when two images are compared, a Mahalanobis distance is calculated using a variance value for a difference vector between images calculated in advance based on a contrast index and an asymmetry index. A technique for determining the degree is disclosed.

JP 2008-251039 A Japanese Patent No. 4803214 Japanese Patent No. 4161659 Japanese Patent No. 3903783

  However, the above-described conventional technique has a problem in that the accuracy of face matching may be reduced due to the influence of glasses, hair, a mask, and the like. For example, the area around the eyes is a place where personal features are likely to appear, and is effective as a feature point for face matching. However, when wearing glasses, the image around the eyes changes, and the matching rate of face matching is greatly reduced. Similarly, hairstyles, masks, wrinkles, and the like may change the feature amount of feature points and cause a reduction in matching accuracy. As described above, noise data is a factor that changes the facial feature amount in an image and reduces the matching accuracy of face matching.

  Therefore, how to dynamically and adaptively reduce the influence of noise data on the face matching process is an important issue. Note that noise data may be generated in both input data and registered data, and noise data is generated in either one of them, thereby causing a reduction in collation accuracy.

  The present invention has been made to solve the above-described problems of the prior art, and is capable of dynamically and adaptively reducing the influence of noise data on face matching, a face matching device, a face matching method, and face matching. The purpose is to provide a program.

In order to solve the above-described problems and achieve the object, the present invention performs a collation process on input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a previously registered face image or its feature quantity. Influence reference data that is a face collation device and serves as a reference when calculating the degree of influence that a local area of a face image has on collation accuracy using learning data including a plurality of face images from which noise data due to wear is removed Based on the influence reference data generation means for generating the influence reference data and the influence reference data calculated by the influence reference data generation means, the degree of influence on the collation accuracy in each local region of the input data and / or the registered data is calculated. An influence level calculation means, and a local pattern in the vicinity of each local area of the input data and a local pattern in the vicinity of the corresponding local area of the registered data The input data based on the similarity calculating means for calculating the degree of similarity, the similarity of the local pattern in the vicinity of each local area calculated by the similarity calculating means, and the influence calculated by the influence calculating means. And a collation processing means for collating the registration data.
Moreover, the present invention is characterized in that, in the above-mentioned invention, the wearing item is at least one of glasses, a mask, hair, or a heel.

Further, in the above invention, before Symbol effect reference data calculation means, an average feature amount of the local area of the training data including a plurality of face images by removing noise data by the wearable as the effect reference data It is characterized by calculating.

  Further, the present invention is the above invention, wherein the influence calculation means is a distance between a feature quantity of each local area of the input data and / or the registration data and an average feature quantity of a corresponding local area of the influence reference data. And a distance evaluation value indicating that there is a high possibility that the larger the calculated distance value is, the lower the collation accuracy is.

  In the above invention, the present invention further includes similarity weight calculation means for calculating a similarity weight for the similarity of local patterns in the vicinity of each local region according to the influence degree, and the matching processing means includes the similarity processing means. The input data and the registered data are collated by weighting the similarity of the local pattern in the vicinity of each local region using the similarity weight calculated by the degree weight calculating means.

  Further, in the present invention according to the above invention, the similarity weight calculating unit may calculate the distance evaluation value of the average feature amount in each local region for both the input data and the registered data. , The similarity weight is calculated using the distance evaluation value corresponding to the larger distance for each local region.

In addition, the present invention is a face matching method for matching input data consisting of a face image of a person to be matched or a feature amount thereof with registered data consisting of a face image registered in advance or a feature amount thereof. Using the learning data including the plurality of face images from which the data is removed, the influence reference data generating step for generating the influence reference data serving as a reference when calculating the degree of influence of the local region of the face image on the matching accuracy; Based on the influence reference data calculated by the influence reference data generation step, an influence degree calculating step for calculating an influence degree on collation accuracy in each local region of the input data and / or the registered data, and each of the input data Similarity calculating the similarity between the local pattern near the local area and the local pattern near the corresponding local area of the registered data The input data and the registration data are calculated based on the calculation step, the similarity of the local pattern in the vicinity of each local region calculated by the similarity calculation step, and the influence calculated by the influence calculation step. And a verification processing step for performing verification processing.

Further, the present invention provides a collation object person of the face image or registered data and the collation processing for the face collation program input data consisting of the feature quantity consisting of pre-registered face image or a feature amount, the noise due to wear thereof Using the learning data including a plurality of face images from which the data has been removed, the influence reference data generation procedure for generating the influence reference data serving as a reference when calculating the degree of influence of the local region of the face image on the matching accuracy; Based on the influence reference data calculated by the influence reference data generation procedure, an influence degree calculation procedure for calculating an influence degree on the matching accuracy in each local region of the input data and / or the registered data, and each of the input data Similarity calculation for calculating the similarity between a local pattern near a local area and a local pattern near the corresponding local area of the registered data The input data and the registered data are collated based on the order, the similarity of the local pattern in the vicinity of each local region calculated by the similarity calculation procedure, and the influence calculated by the influence calculation procedure It is characterized by causing a computer to execute a collation processing procedure to be processed.
In addition, the present invention is a face collation device that collates input data consisting of a face image of a person to be collated or a feature amount thereof with registered data consisting of a face image registered in advance or a feature amount thereof. Based on the influence reference data, which is generated by using learning data including a plurality of face images from which noise data has been removed, and which serves as a reference when calculating the degree of influence that the local area of the face image has on matching accuracy, and And / or an influence degree calculating means for calculating an influence degree on matching accuracy in each local area of the registered data, a local pattern in the vicinity of each local area of the input data, and a local pattern in the vicinity of the corresponding local area of the registered data A similarity calculation means for calculating a similarity between the local patterns in the vicinity of each local region calculated by the similarity calculation means Based on the degree of influence calculated by the influence degree calculation means, characterized in that a matching processing means for matching process and said input data the registration data.
The present invention is also a face collation program for collating input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a face image or its feature quantity registered in advance, Based on the influence reference data, which is generated by using learning data including a plurality of face images from which noise data has been removed, and which serves as a reference when calculating the degree of influence that the local area of the face image has on matching accuracy, and And / or an influence calculation procedure for calculating an influence on the matching accuracy in each local area of the registered data, a local pattern in the vicinity of each local area of the input data, and a local pattern in the vicinity of the corresponding local area of the registered data, A similarity calculation procedure for calculating the similarity between the two, and a local pattern in the vicinity of each local region calculated by the similarity calculation procedure And similarity, based on the degree of influence calculated by the influence degree calculation procedure, characterized in that to perform the matching procedure for matching process and the registration data and the input data to the computer.

According to the present invention, it is possible to reduce the effect on the face collation by the noise data dynamically and adaptively.

FIG. 1 is an explanatory diagram for explaining the concept of face matching processing by the face matching device. FIG. 2 is a block diagram showing the configuration of the face matching device. FIG. 3 is a block diagram of the non-variable space generation unit shown in FIG. FIG. 4 is an explanatory diagram for explaining details of registration and verification. FIG. 5 is an explanatory diagram for explaining the non-variable space data and the registration data. FIG. 6 is a flowchart illustrating a processing procedure related to learning of the face collation apparatus. FIG. 7 is a flowchart showing a processing procedure related to registration of the face collation apparatus. FIG. 8 is a flowchart showing a processing procedure related to the collation of the face collation apparatus. FIG. 9 is an explanatory diagram when each functional unit of the face collation apparatus is realized by a program.

  Exemplary embodiments of a face matching apparatus, a face matching method, and a face matching program according to the present invention will be described below with reference to the accompanying drawings. In the following embodiment, a case where the influence of glasses, hair, a mask, and the like is reduced as noise data will be described. However, the present invention can be similarly applied to a case where the influence of other noise data is reduced.

  First, the concept of the face matching process of the face matching device 10 according to the present embodiment will be described. FIG. 1 is an explanatory diagram for explaining the concept of face collation processing by the face collation apparatus 10. The face matching device 10 shown in the figure does not treat glasses, hair, masks, etc. as “part of the face” as in the conventional face matching device, but treats it as noise data that is not part of the face. Face matching is performed while reducing the influence of data. A feature amount A shown in the figure is calculated from a plurality of local regions in the verification target region, and is used for calculating the similarity between registered data and input data.

  Specifically, the face collation apparatus 10 performs a learning process using learning data prior to a registration process for registering a registered image that is a face of the person to be recognized as registration data and a collation process using input data. . The learning data of the face image without fluctuation is used for the learning data. A face image without fluctuation is a face image that does not include noise data such as glasses, hair, and masks in the verification target region.

  The face matching device 10 forms a space without fluctuation using learning data that uses a standard face that is a model without fluctuation. Here, the space without variation is a set of feature amounts B of each local region of the face created using learning data, which is a set of face images without variation. Here, the feature amount B may be the image pattern itself, or may be obtained by applying a partial filter to the image. Details of the embodiment of the feature amount B will be described later. The face collation apparatus 10 generates non-variable space data 13a using the learning data. The non-variable spatial data 13a is data used as a reference when reducing the influence of noise data during face matching.

  Since the space without fluctuation indicated by the non-variable space data 13a does not include data regarding glasses, hair, masks, etc., as an example, the average of the feature amount B as a space without fluctuation is the registered data and the input data. Can be used as a measure of how much the "fluctuates". Specifically, when noise data such as glasses or hair is included in the registration data or input data, the feature amount B of the local region affected by the noise data is the feature amount of the space without variation. It will be far from the average of B.

  In this way, if the non-variable spatial data 13a is generated in the learning stage, the face matching device 10 receives the input of the registration data and calculates the feature quantity A and the feature quantity B in a plurality of local regions of the registration data. .

  Then, using the non-variable space data 13a generated from the learning data, a distance evaluation value indicating how close the feature value B of each local region is to the average of the corresponding feature values B of the space without fluctuation is calculated. In other words, how much data each local area of the input data and registration data is like a face (is less likely to be affected by noise data) is calculated. Here, the distance evaluation value is increased as the distance value is smaller, that is, the possibility of being affected by the noise data is lower, and the distance evaluation value is decreased as the distance is increased. FIG. 1 shows a state in which the face collation apparatus 10 has calculated the feature amount A data 13c and the distance evaluation value data 13d with respect to a non-fluctuating space for n local areas (n is a natural number) of registered data. ing. Such a local region can be set using various conventional techniques.

  Next, when the face collation apparatus 10 receives input data composed of face image data of the person to be collated, the face collation apparatus 10 calculates a feature amount A and a feature amount B of each local region from the input data. In addition, the face matching device 10 uses the non-variable space data 13a generated from the learning data to correspond to the distance evaluation value indicating how close the feature amount B of each local region of the input data is to the space without variation. Calculate for each local region. FIG. 1 shows a state in which feature amount A data and distance evaluation value data in n local regions of input data are calculated.

  Then, the face matching device 10 calculates the similarity weight in each local region using the distance evaluation value data 13d of the registration data and the distance evaluation value data of the input data. The similarity weight is set to a smaller value as the distance evaluation value is smaller (as the possibility of being influenced by noise data is higher).

  When the similarity weight is calculated in this way, the face matching device 10 obtains the similarity of the feature quantity A of the n local regions corresponding to the registered data and the input data, and the local region is calculated based on the obtained similarity. The weighting similarity is calculated by applying the similarity weight in, and the registered data and the input data are collated using the weighting similarity. For example, if the sum of the weighted similarities is equal to or greater than a predetermined threshold, it is determined that they are the same person, and if the sum is less than the threshold, it is determined that they are different persons.

  As described above, the face collation apparatus 10 evaluates the influence of noise data such as glasses, hair, and mask on the face image, and generates the non-variable space data 13a for reducing the influence from the learning data. Then, a distance evaluation value is calculated from the average of the feature amount B of the local area of the registered data and the input data and the feature amount B of the corresponding local area of the non-variable space data 13a, and weighting according to the magnitude of the distance evaluation value A matching process using the similarity is performed.

  The non-variable spatial data 13a indicates the presence of noise data with respect to the registration data and the input data. For this reason, when noise data such as glasses, hair, and masks exist in the input data, the distance evaluation value with the space without fluctuation becomes small, so the weight of similarity is lowered and the influence of the noise data is reduced. Can be reduced.

  Next, the configuration of the face matching device 10 shown in FIG. 1 will be described. FIG. 2 is a block diagram showing a configuration of the face matching device 10 shown in FIG. As illustrated in FIG. 2, the face collation apparatus 10 includes an image input receiving unit 11, a display operation unit 12, a storage unit 13, and a control unit 14.

  The image input receiving unit 11 is an interface that receives a face image as learning data, a face image as registration data, and a face image as input data from a camera, a scanner, or the like. The display operation unit 12 is an interface such as a touch panel display, a keyboard, and a mouse.

  The storage unit 13 is a storage device including a flash memory, a hard disk device, and the like, and stores the non-variable space data 13a and the registration data 13b.

  The non-variable space data 13a is generated and stored in the storage unit 13 when learning data is input. The registration data 13b is data including the feature amount A data 13c of each local area of the registration data and the distance evaluation value data 13d indicating the proximity to the space without fluctuation, and is generated when the registration data is input. It is stored in the storage unit 13.

  The control unit 14 is a control unit that controls the face collating apparatus 10 as a whole. The control unit 14 includes a non-variable space generation unit 14a, a feature amount calculation unit 14b, a distance evaluation value calculation unit 14c, a similarity calculation unit 14d, a similarity weight calculation unit 14e, and a matching processing unit 14f.

  When the image input receiving unit 11 receives the learning data, the non-changeable space generating unit 14a generates the variable space data 13a that is data obtained by calculating the average of the feature amounts B of the local regions from the entire learning data. The processing unit is stored in the storage unit 13.

  When the image input receiving unit 11 receives registration data or input data, the feature amount calculating unit 14b receives the feature amount A data 13c for calculating similarity for each local region from the received data and the similarity of the local region. It is a processing unit that calculates a feature quantity B for weight calculation. The feature amount calculation unit 14b calculates the feature for calculating the similarity weight of each local region in m dimensions (m is a natural number).

  The distance evaluation value calculation unit 14c is a processing unit that calculates the degree of proximity to a space without fluctuation in each local region. Specifically, the degree of closeness is calculated for each local region using the feature amount B output from the feature amount calculation unit 14b and the non-variable space data 13b.

  The similarity calculation unit 14d is a processing unit that calculates the similarity of corresponding local regions between registered data and input data. The similarity weight calculation unit 14e calculates the similarity weight in each local region using the distance evaluation value calculated by the distance evaluation value calculation unit 14c. The distance evaluation value decreases as the difference between the feature quantity B and the non-variable space data 13a increases, and data such as glasses and masks are included in the input data. The similarity weight calculation unit 14e sets the similarity weight to a smaller value. To do.

  The matching processing unit 14f calculates a weighted similarity by multiplying the similarity calculated by the similarity calculation unit 14d by the similarity weight calculated by the similarity weight calculation unit 14e, and uses the weighted similarity. It is a processing unit that performs a matching process between registered data and input data.

  Next, the generation of the non-variable space data 13a will be further described. As described above, a set of face images having no variation such as glasses, hair, and masks is input to the face matching device 10 as learning data. The non-variable space generation unit 14a generates, as an example, non-variable space data 13a obtained by averaging the feature values B of the local regions as a non-fluctuating space based on the learning data.

  FIG. 3 is a block diagram of the non-variable space generation unit 14a shown in FIG. The non-variable space generation unit 14a includes a feature amount B calculation unit 41 and an average feature amount B calculation unit 42 therein. When learning data is received from the image receiving unit 11, the feature amount B calculating unit 41 calculates the feature amount B of the set local region. Then, the feature amount B of the local region of all the learning data is calculated and output to the average feature amount B calculation unit 42.

  The average feature amount B calculation unit 42 totals the feature amounts for each local region and calculates the average. The average feature amount B calculation unit 42 stores the calculated feature amount for each local region in the storage unit 13 as non-variable space data 13a.

  Next, details of registration and verification will be described. FIG. 4 is an explanatory diagram for explaining details of registration and verification. As illustrated in FIG. 4, the feature amount calculation unit 14b stores the feature amount A for calculating the similarity of the local region from the registration data in the storage unit 13 as the feature amount A data 13c of the registration data 13b. The feature amount B for similarity weight calculation is output to the distance evaluation value calculation unit 14c. Further, the feature amount calculation unit 14b outputs the feature amount A for calculating the similarity of the local region from the input data to the similarity calculation unit 14d, and calculates the feature amount B for calculating the similarity weight of the local region as a distance evaluation value. To the unit 14c.

  The distance evaluation value calculation unit 14c calculates a distance evaluation value indicating the proximity of the space without variation from the feature amount B for similarity weight calculation output from the feature amount calculation unit 14b and the non-variable space data 13a. . When the distance evaluation value is calculated from the registered data, the distance evaluation value calculation unit 14c stores the calculated distance evaluation value in the storage unit 13 as the distance evaluation value data 13d. When the distance evaluation value is calculated in the same manner from the input data, the distance evaluation value calculation unit 14c outputs the calculated distance evaluation value to the similarity weight calculation unit 14e.

Here, calculation of the distance evaluation value will be described. When the distance evaluation value calculation unit 14c calculates the distance evaluation value, the m-dimensional feature quantity B of each local region of the non-variable space data 13a, the registration data, and the input data is used as an element of the m-term column vector. Therefore, a difference vector between the m-term column vector of the non-variable spatial data 13a and the m-term column vector of the registration data and the input data is obtained. Then, the length of the difference vector is calculated. Then, using the length xj (j = 1,..., N) of the difference vector, a distance evaluation value dj indicating the proximity to the space without fluctuation is calculated by the following equation.
dj = exp (−a (xj∧2)) (1)
Here, a (> 0) is constant. ∧ represents a power.

  Equation (1) indicates that the smaller the difference vector, that is, the closer to a space with no fluctuation, the larger the value of dj, and the larger the difference vector, the smaller the value of dj, the farther away the space without fluctuation. Show. That is, dj expresses the degree of proximity to a space without fluctuation.

  The similarity calculation unit 14d calculates the similarity between the input data and the registration data for each local region and outputs the calculation to the matching processing unit 14f.

  When the distance evaluation value dj is output from the input data by the distance evaluation value calculation unit 14c, the similarity weight calculation unit 14e is the smaller of the distance evaluation value data 13d of the registration data 13b and the distance evaluation value data of the input data. Is selected, the similarity weight for each local region is calculated, and output to the matching processing unit 14f.

Here, the calculation of the similarity weight will be described. Using the distance evaluation value dj obtained by the distance evaluation value calculation unit 14c, the similarity weight calculation unit 14e calculates the similarity weight wj of each local region using the following equation.
wj = dj∧b (2)
Here, b (> 0) is constant, and ∧ represents a power. Finally, the weights are normalized so that the sum is 1.

  In the equation (2), the distance evaluation value dj increases as the distance from the space without fluctuation becomes closer, and thus the calculated weight increases. Conversely, when the distance evaluation value dj is small, the weight is small, and in this case, the influence of glasses, hair, a mask, etc. is reduced.

  The matching processing unit 14f calculates a weighted similarity by multiplying the similarity calculated by the similarity calculation unit 14d by the similarity weight calculated by the similarity weight calculation unit 14e, and uses the weighted similarity. A verification process is performed between the registered data and the input data. Specifically, the matching processing unit 14f calculates the sum of the weighted similarities, and determines that the registered data and the input data are the same person when the sum of the weighted similarities is equal to or greater than a threshold value.

  Next, the non-variable space data 13a and the registration data 13b will be described. FIG. 5 is an explanatory diagram for explaining the non-variable space data 13a and the registration data 13b. FIG. 5A shows the non-variable space data 13a, and FIG. 5B shows the registration data 13b. As shown in FIG. 5A, the non-variable spatial data 13a stores m-dimensional average feature values B of each local region calculated using learning data having no variation. Specifically, the average feature amount B of the local region P1 of the non-variable space data 13a is B01, the average feature amount B of the local region P2 is B02, and the average feature amount B of the local region Pn is B0n.

  The registration data 13b illustrated in FIG. 5B includes a registration ID, a name, a feature amount A and a distance evaluation value regarding n local regions. The registration ID is information for uniquely identifying registration data of a registered person. The name is the name of the person corresponding to the registration data. In addition, not only a name but arbitrary information regarding a person can be included in registration data.

  In FIG. 5B, there is registration data corresponding to the face image of the person whose registration ID “001” is the name “ABC”, the feature amount A of the local region P1 is A11, and the distance evaluation value of the local region P1 is Nr11, the feature amount A of the local region P2 is A12, the distance evaluation value in the local region P2 is Nr12, the feature amount A of the local region Pn is A1n, and the distance evaluation value in the local region Pn is Nr1n. It shows that there is.

  Similarly, the registration data corresponding to the face image of the person whose registration ID “002” is the name “EFG”, the feature amount A of the local region P1 is A21, and the distance evaluation value in the local region P1 is Nr21. The feature amount A of the local region P2 is A22, the distance evaluation value in the local region P2 is Nr22, the feature amount A of the local region Pn is A2n, and the distance evaluation value in the local region Pn is Nr2n. Show.

  Next, a processing procedure related to learning of the face matching device 10 will be described. FIG. 6 is a flowchart showing a processing procedure related to learning of the face matching device 10.

  First, the image input receiving unit 11 receives learning data (step S101). The feature amount B calculation unit 41 of the non-variable space generation unit 14a calculates the feature amount B of each local region (step S102) and outputs it to the average feature amount B calculation unit 42.

  When the calculation of the feature amount B for each local region is completed for all the learning data, the average feature amount B calculation unit 42 calculates the average by totaling the feature amount B for each local region (step S103). Then, the face matching device 10 stores the calculated average feature amount B as the non-variable space data 13a in the storage unit 13 (step S104), and ends the process.

  Next, a processing procedure related to registration of the face matching device 10 will be described. FIG. 7 is a flowchart showing a processing procedure related to registration of the face collation apparatus 10.

  First, the image input reception unit 11 receives registration data (step S201). The feature amount calculation unit 14b calculates the feature amount A and the feature amount B of the local region from the registered data (step S202). The distance evaluation value calculation unit 14c reads the non-variable spatial data 13a from the storage unit 13 (step S203), and calculates a distance evaluation value for each local region of the registered data using the equation (1) (step S204). The face matching device 10 stores the feature amount A calculated in step S202 and the distance evaluation value calculated in step S204 in the storage unit 13 as registration data 13b (step S205), and ends the process.

  Next, a processing procedure related to collation of the face collation apparatus 10 will be described. FIG. 8 is a flowchart showing a processing procedure related to the matching of the face matching device 10.

  First, the image input receiving unit 11 receives input data (step S301). The feature amount calculation unit 14b calculates a feature amount A and a feature amount B for each local region from the input data (step S302). The distance evaluation value calculation unit 14c reads the non-variable space data 13a from the storage unit 13 (step S303), and calculates the distance evaluation value of each local region (step S304).

  Thereafter, the face collation apparatus 10 selects the registration data 13b to be collated with the input data (Step S305). For each local region, the similarity weight calculation unit 14e compares the distance evaluation value of the selected registered data 13b with the distance evaluation value of the input data, and selects the smaller distance evaluation value (step S306). The similarity weight calculation unit 14e calculates the similarity weight of each local region based on the selected distance evaluation value (step S307).

  The similarity calculation unit 14d calculates the similarity between the feature amount A of the selected registration data 13b and the feature amount A of the input data for each local region (step S308). The matching processing unit 14f calculates the weighted similarity by multiplying the similarity calculated in step S308 by the similarity weight calculated in step S307, and totals the weighted similarities (step S309).

  The collation processing unit 14f determines that the selected registration data 13b and the input data belong to the same person if the total weighted similarity (total similarity) is equal to or greater than the threshold (step S310; Yes). Then, information (name, etc.) relating to the selected registration data 13b is output as a collation result (step S313), and the process is terminated.

  If the total similarity is less than the threshold value (step S310; No), the matching processing unit 14f determines whether all the registered data 13b have been selected (step S311).

  If unselected registration data 13b remains (step S311; No), the collation processing unit 14f selects registration data 134b to be collated with input data from the unselected registration data 13b (step S305). If all the registered data 13b have been selected (step S311; Yes), the collation processing unit 14f outputs “no corresponding person” as a collation result (S312), and ends the process.

  Next, the case where each function part of the face collation apparatus 10 shown in FIG. 2 is implement | achieved by a program is demonstrated. FIG. 9 is an explanatory diagram when each functional unit of the face collation apparatus 10 shown in FIG. 2 is realized by a program. As shown in the figure, the face collation apparatus 10 shown in FIG. 2 has a configuration in which a CPU 111 (including a main memory), a flash memory 112, a display unit 113, and an operation reception unit 114 are connected by a bus 119.

  Here, the face matching program 130 is stored in the flash memory 112, and the CPU 111 reads out the face matching program 130 from the flash memory 112 and expands it in a main memory (not shown), and executes the process of the face matching program 130. To do.

  Specifically, the face collation program 130 includes a non-variable space generation process 120a, a feature amount calculation process 120b, a distance evaluation value calculation process 120c, a similarity calculation process 120d, a similarity weight calculation process 120e, and a verification processing process 120f. Generated.

  The non-variable space generation process 120a corresponds to the non-variable space generation unit 14a illustrated in FIG. 2, the feature amount calculation process 120b corresponds to the feature amount calculation unit 14b illustrated in FIG. 2, and the distance evaluation value calculation process 120c includes 2 corresponds to the distance evaluation value calculation unit 14c shown in FIG. 2, the similarity calculation process 120d corresponds to the similarity calculation unit 14d shown in FIG. 2, and the similarity weight calculation process 120e shows the similarity weight shown in FIG. Corresponding to the calculation unit 14e, the collation process 120f corresponds to the collation processing unit 14f illustrated in FIG.

  As described above, the face collation apparatus 10 according to the present embodiment generates the non-variable space data 13a serving as a reference for detecting the presence of glasses, hair, a mask, and the like in the face image from the learning data, and the registration data and input A distance evaluation value indicating the closeness of the data to the non-fluctuating space in each local region is calculated from the non-variable space data 13a, and collation processing is performed based on the weighted similarity according to the distance evaluation value. For this reason, even if glasses, hair, a mask, etc. exist in input data, the face collation apparatus 10 can reduce an influence dynamically and adaptively.

  In the above embodiment, when the total similarity between any registered data and the input data is equal to or greater than the threshold value, the process is terminated and the total similarity with any registered data is also obtained. Although notification is performed even when the threshold value is less than the threshold value, it is possible to appropriately set how to output the collation result.

  For example, the total similarity may be calculated for all registered data, and information regarding registered data that maximizes the total similarity may be output as a matching result. Further, all registered data whose total similarity exceeds a threshold value may be output as a matching result. Further, when the total similarity with any registered data is less than the threshold value, the notification need not be performed.

  Further, in the above-described embodiment, the case where the matching is performed based on the total similarity obtained by summing all the weighted similarities weighted by the distance evaluation value has been described. However, the similarity of a local region having a large distance evaluation value is described. The verification may be performed selectively.

  In the above embodiment, the case where the weight is calculated by applying the calculation formula unconditionally from the distance evaluation value has been described. However, the distance evaluation value may increase even when wearing glasses or the like. It is also possible to perform weighting learning for glasses, hair, masks, etc., and prepare and apply weight calculation formulas according to the wear item and the wearing part.

  In the above embodiment, the distance evaluation value is obtained for both the input data and the registration data, and one of them is used for calculating the weight for the similarity. However, the distance evaluation value of the input data and the registration data are described. The weight may be calculated using both of the distance evaluation values. Alternatively, the distance evaluation value may be calculated for only one of the input data and the registered data, and the weight for the similarity may be calculated.

  As described above, the face matching device, the face matching method, and the face matching program according to the present invention are suitable for dynamically and adaptively reducing the influence of noise data such as glasses, hair, and masks on face matching. Yes.

DESCRIPTION OF SYMBOLS 10 Face collation apparatus 11 Image input reception part 12 Display operation part 13 Memory | storage part 13a Unchangeable space data 13b Registration data 13c Feature-value A data 13d Distance evaluation value data 14 Control part 14a Non-change-space generating part 14b Feature-value calculation part 14c Distance Evaluation value calculation unit 14d Similarity calculation unit 14e Similarity weight calculation unit 14f Collation processing unit 41 Feature amount B calculation unit 42 Average feature amount B calculation unit

Claims (10)

A face collation device for collating input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a face image registered beforehand or its feature quantity,
Generation of influence reference data that is used as reference for calculating the degree of influence of the local area of the face image on the matching accuracy using learning data including multiple face images from which noise data due to wear is removed Means,
Based on the influence reference data calculated by the influence reference data generation means, an influence degree calculation means for calculating an influence degree on the matching accuracy in each local region of the input data and / or the registration data;
Similarity calculating means for calculating a similarity between a local pattern in the vicinity of each local region of the input data and a local pattern in the vicinity of the corresponding local region of the registration data;
Collation for collating the input data with the registered data based on the similarity of the local pattern in the vicinity of each local area calculated by the similarity calculation means and the influence degree calculated by the influence calculation means A face matching device comprising: processing means.   The face matching apparatus according to claim 1, wherein the wearing item is at least one of glasses, a mask, hair, and a heel. Before SL Effect reference data calculation means, according to claim 1, characterized in that to calculate the average feature amount of the local area of the training data including a plurality of face images by removing noise data by the wearable as the effect reference data Or the face collation apparatus of 2. The influence calculation means calculates a distance between a feature amount of each local region of the input data and / or the registration data and an average feature amount of a corresponding local region of the influence reference data, and the calculated distance value is The face collation apparatus according to claim 1, 2, or 3 , wherein a distance evaluation value indicating that the larger the value is, the higher the possibility that the collation accuracy is likely to be reduced is calculated as the degree of influence. Further comprising a similarity weight calculating means for calculating a similarity weight for the similarity of the local pattern in the vicinity of each local region according to the influence level,
The collation processing unit performs weighting on the similarity of the local pattern in the vicinity of each local region using the similarity weight calculated by the similarity weight calculation unit, and collates the input data with the registered data. The face collation device according to claim 1, wherein The similarity weight calculating unit calculates the distance evaluation value between each of the local regions and the average feature amount in each local region for each of the input data and the registered data. The face matching device according to claim 5 , wherein the similarity weight is calculated using a distance evaluation value corresponding to a larger distance. A face collation method for collating input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a face image registered beforehand or its feature quantity,
Generation of influence reference data that is used as reference for calculating the degree of influence of the local area of the face image on the matching accuracy using learning data including multiple face images from which noise data due to wear is removed Steps,
Based on the influence reference data calculated by the influence reference data generation step, an influence degree calculating step for calculating an influence degree on the matching accuracy in each local region of the input data and / or the registration data;
A similarity calculation step of calculating a similarity between a local pattern in the vicinity of each local region of the input data and a local pattern in the vicinity of a corresponding local region of the registration data;
Collation for collating the input data and the registered data based on the similarity of the local pattern in the vicinity of each local area calculated by the similarity calculation step and the influence calculated by the influence calculation step A face matching method comprising: processing steps. A face collation program for collating input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a face image registered beforehand or its feature quantity,
Generation of influence reference data that is used as reference for calculating the degree of influence of the local area of the face image on the matching accuracy using learning data including multiple face images from which noise data due to wear is removed Procedure and
Based on the influence reference data calculated by the influence reference data generation procedure, an influence calculation procedure for calculating the influence on the matching accuracy in each local region of the input data and / or the registration data;
A similarity calculation procedure for calculating a similarity between a local pattern in the vicinity of each local area of the input data and a local pattern in the vicinity of the corresponding local area of the registration data;
Collation for collating the input data with the registered data based on the similarity of the local pattern in the vicinity of each local area calculated by the similarity calculation procedure and the influence calculated by the influence calculation procedure A face matching program that causes a computer to execute a processing procedure.   A face collation device for collating input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a face image registered beforehand or its feature quantity,
  Based on the impact criteria data that is used as a reference when calculating the degree of influence that the local area of the face image has on the matching accuracy, generated using learning data that includes a plurality of face images from which noise data due to wear has been removed in advance, An influence degree calculating means for calculating an influence degree on collation accuracy in each local region of the input data and / or the registration data;
  Similarity calculating means for calculating a similarity between a local pattern in the vicinity of each local region of the input data and a local pattern in the vicinity of the corresponding local region of the registration data;
  Collation for collating the input data with the registered data based on the similarity of the local pattern in the vicinity of each local area calculated by the similarity calculation means and the influence degree calculated by the influence calculation means Processing means and
  A face collation apparatus comprising:   A face collation program for collating input data consisting of a face image of a person to be collated or its feature quantity with registered data consisting of a face image registered beforehand or its feature quantity,
  Based on the impact criteria data that is used as a reference when calculating the degree of influence that the local area of the face image has on the matching accuracy, generated using learning data that includes a plurality of face images from which noise data due to wear has been removed in advance, An impact calculation procedure for calculating the impact on the matching accuracy in each local region of the input data and / or the registration data;
  A similarity calculation procedure for calculating a similarity between a local pattern in the vicinity of each local area of the input data and a local pattern in the vicinity of the corresponding local area of the registration data;
Collation for collating the input data with the registered data based on the similarity of the local pattern in the vicinity of each local area calculated by the similarity calculation procedure and the influence calculated by the influence calculation procedure Procedure and
  A face collation program for causing a computer to execute.

Images