JP7833664B2 - Facial recognition system and method for updating registered facial images - Google Patents

Description

This invention relates to a facial recognition system that compares a facial image of a person to be recognized, obtained by photographing the person, with a registered facial image for each user, and to a method for updating a registered facial image in the facial recognition system using an information processing device.

In recent years, facial recognition technology has become widespread for purposes such as access control to identify individuals. This facial recognition technology identifies individuals by comparing their facial image, obtained by taking a photograph, with their registered facial image. However, such facial recognition systems have a problem: as time passes after the facial image is registered, and the person's face changes due to aging, etc., the matching accuracy decreases.

Therefore, to address the problem of reduced matching accuracy due to outdated registered facial images, a technique has been developed (see Patent Document 1) that involves accumulating facial images of authenticated individuals obtained during authentication, and then, at appropriate intervals, selecting a suitable facial image from the accumulated images and replacing older images with it. This technique automates the administrator's update process, thereby reducing the administrator's workload.

Japanese Patent Publication No. 2002-259980

In facial recognition systems, one-to-many (1-to-N) facial matching is performed, comparing one person to a large number of registered users. However, in large-scale systems, the processing load for facial matching becomes significant due to the large number of registered users. Therefore, to reduce this processing load, the system is operated using one registered facial image per user.

On the other hand, with conventional technology, there is a possibility that inappropriate facial images may be selected during the update process. In particular, in systems that operate with one registered facial image per user, the selection of an inappropriate facial image significantly reduces the reliability of subsequent facial matching. Therefore, it is desirable for administrators to visually check and select appropriate facial images. In this regard, it is desirable to minimize the administrator's workload as much as possible.

Therefore, the main objective of the present invention is to provide a facial recognition system and a method for updating registered facial images that can reduce the burden on administrators by enabling them to efficiently select appropriate facial images from the facial images of authenticated individuals through visual confirmation.

The present invention relates to a facial recognition system that compares a facial image of a person to be recognized, obtained by photographing the person to be recognized, with a registered facial image for each user, comprising : a facial recognition machine that photographs the person to be recognized and obtains the facial image of the person to be recognized; and a server device that stores the registered facial images for each user and performs a comparison process between the facial image of the person to be recognized and the registered facial image, wherein the server device stores the facial image of the person to be recognized obtained from the facial recognition machine , and also stores authentication log management information that associates the facial image of the person to be recognized with the authentication result and the user related to the authentication event that occurred in the facial recognition machine , and performs a processing procedure The administrator device displays a screen allowing the administrator to specify the item, extracts users with a low score tendency who are subject to update according to the processing conditions from among the users who have successfully authenticated based on the matching score included in the authentication log management information, extracts candidate face images from the authenticated user face images of the extracted users according to the processing conditions, displays a screen on the administrator device showing the extracted candidate face images to the administrator and allowing the administrator to select a candidate face image, and performs an update process in which the selected candidate face image replaces the current registered face image as the new registered face image.

Furthermore, the registered face image update method of the present invention is a method for updating a registered face image using an information processing device in a face recognition system that compares a face image of a person to be recognized obtained by photographing the person to be recognized with a registered face image for each user, wherein the registered face image update method of the present invention is configured to store the face images of the person to be recognized obtained from a face recognition device, store authentication log management information that associates the face images of the person to be recognized with the authentication results and users related to authentication events that occurred at the face recognition device, display a screen on the administrator device that allows the administrator to specify processing conditions, extract users with a tendency towards low scores that are to be updated according to the processing conditions from among users who have been successfully authenticated based on the matching score included in the authentication log management information, extract candidate face images from among the face images of the person to be recognized according to the processing conditions, present a plurality of the extracted candidate face images to the administrator and display a screen on the administrator device that allows the administrator to select a candidate face image, and perform an update process that replaces the current registered face image with the selected candidate face image as the new registered face image.

According to the present invention, based on authentication log management information that associates the authenticated person's facial image, authentication result, and user with the authentication event that occurred in the facial recognition device , candidate facial images that are candidates for new registered facial images of users with a tendency towards low scores are presented to the administrator, allowing the administrator to visually confirm the candidate facial images. Furthermore, the target user and authenticated person's facial images are narrowed down according to the processing conditions specified by the administrator. This makes it possible for the administrator to efficiently select an appropriate facial image as a new registered facial image from the authenticated person's facial images through visual confirmation, thereby reducing the burden on the administrator's update work.

Overall configuration diagram of the facial recognition system according to this embodiment. Diagram illustrating the contents of the authentication log management table processed by facial recognition server 2. Block diagram showing the schematic configuration of Checker 1 and Face Recognition Server 2. Diagram illustrating the overview of the processing performed by the facial recognition server 2. Diagram illustrating the management screen displayed on administrator terminal 3. Diagram illustrating the filtering settings screen displayed on administrator terminal 3. This is an explanatory diagram showing the candidate face image selection screen displayed on the administrator terminal 3. This diagram illustrates the enlarged registered face image screen and the enlarged candidate face image screen displayed on the administrator terminal 3.

The first invention made to solve the aforementioned problems is a facial recognition system that compares a facial image of a person to be recognized obtained by photographing the person to be recognized with a registered facial image for each user, comprising: a facial recognition machine that photographs the person to be recognized and obtains the facial image of the person to be recognized; and a server device that stores the registered facial images for each user and performs a comparison process between the facial image of the person to be recognized and the registered facial image, wherein the server device stores the facial image of the person to be recognized obtained from the facial recognition machine and also stores authentication log management information that associates the facial image of the person to be recognized with the authentication result and the user related to the authentication event that occurred in the facial recognition machine. The system displays a screen on the administrator device allowing the administrator to specify processing conditions, extracts users with a low score tendency who are subject to updating according to the processing conditions from among the users who have successfully been authenticated based on the matching score included in the authentication log management information, extracts candidate face images from among the authenticated user face images of the extracted users according to the processing conditions, displays a screen on the administrator device showing the extracted candidate face images to the administrator and allowing the administrator to select a candidate face image, and performs an update process in which the selected candidate face image replaces the current registered face image with the new registered face image.

According to this system, based on authentication log management information that associates authenticated user facial images, authentication results, and users related to authentication events that occur in the facial recognition device , candidate facial images that could be used as new registration facial images for users with a tendency towards low scores are presented to the administrator, allowing the administrator to visually review the candidate facial images. Furthermore, the target user and authenticated user facial images are narrowed down according to the processing conditions specified by the administrator. This allows the administrator to efficiently select an appropriate facial image from the authenticated user facial images as a new registration facial image through visual review, thereby reducing the burden on the administrator's update work.

Furthermore, the second invention is configured such that the server device displays a period specification section on the administrator's device screen, where the administrator specifies the target period, and performs aggregation targeting authentication events included in the target period specified by the administrator.

According to this, it is possible to narrow down the target authentication events by the date and time the authentication event occurred.

Furthermore, the third invention is configured such that the server device displays a threshold specification unit on the screen of the administrator device, which allows the administrator to specify a threshold value for extraction judgment when extracting target users based on the matching score , and extracts users whose matching score trend value is less than the extraction judgment threshold value specified by the administrator as target users.

According to this, by extracting users with low matching score trends (low-score authenticated users) as targets, it becomes possible to perform update work only on users who truly need updating.

Furthermore, the fourth invention is configured such that the server device displays a value that is a predetermined value greater than the matching judgment threshold as the initial value of the extraction judgment threshold on the screen of the administrator device.

According to this, the burden on administrators can be reduced.

Furthermore, the fifth invention is configured such that the server device displays a number specification unit on the screen of the administrator device for the administrator to specify the number of candidate face images, extracts the number of candidate face images specified by the administrator, and displays the extracted multiple candidate face images side by side on the screen of the administrator device.

According to this, administrators can compare multiple candidate facial images and select the most suitable one as the new registered image.

Furthermore, the sixth invention is configured such that, when extracting candidate face images, the server device determines whether the target face image corresponds to a spoofed image detected during face recognition, and excludes face images that correspond to spoofed images from the extraction of candidate face images.

According to this, it is possible to prevent fake facial images from being selected as registered facial images.

Furthermore, the seventh invention is configured such that, when extracting candidate face images, the server device determines whether the target face image meets the image quality requirements for face matching, and excludes face images that do not meet the image quality requirements from the extraction of candidate face images.

According to this, it is possible to avoid extracting face images that do not meet the image quality requirements, i.e., face images that are clearly unsuitable as registered face images, as candidate face images.

Furthermore, the eighth invention relates to a facial recognition system that compares a facial image of a person to be recognized obtained by photographing the person to be recognized with a registered facial image for each user, and provides a method for updating the registered facial image using an information processing device, wherein the system stores the facial image of the person to be recognized obtained from a facial recognition device, stores authentication log management information that associates the facial image of the person to be recognized with the authentication result and the user related to the authentication event that occurred at the facial recognition device, displays a screen on the administrator device that allows the administrator to specify processing conditions, extracts users with a tendency towards low scores that are subject to updating according to the processing conditions from among users who have successfully been authenticated based on the matching score included in the authentication log management information, extracts candidate facial images from among the facial images of the person to be recognized according to the processing conditions, presents a plurality of the extracted candidate facial images to the administrator and displays a screen on the administrator device that allows the administrator to select a candidate facial image, and performs an update process that replaces the current registered facial image with the selected candidate facial image as the new registered facial image.

According to this, similar to the first invention, it is possible to efficiently perform the task of selecting the appropriate facial image from the facial images of the authenticated person by visual confirmation, thereby reducing the burden on the administrator's update work.

The embodiments of the present invention will be described below with reference to the drawings.

Figure 1 is an overall configuration diagram of the facial recognition system according to this embodiment.

This facial recognition system comprises a checker 1 (facial recognition device), a facial recognition server 2 (server device), and an administrator terminal 3. The checker 1, facial recognition server 2, and administrator terminal 3 (administrator device) are interconnected via a network.

Checker 1 captures the face of the person undergoing facial recognition, obtains the facial image, and transmits it to the facial recognition server 2. Checker 1 also receives the authentication result from the facial recognition server 2 and performs an action corresponding to that result. For example, if Checker 1 is used to restrict entry to pre-registered individuals, it might perform actions such as opening the gate upon successful authentication.

The facial recognition server 2 performs facial recognition based on the facial image of the person to be recognized received from the checker 1, and transmits the recognition result to the checker 1. At this time, the facial recognition server 2 compares the facial image of the person to be recognized obtained from the checker 1 with the registered facial image of each person registered in the registrant database. If the matching score exceeds a predetermined threshold, the server identifies the person in the registered facial image as the person to be recognized. If multiple people are found with matching scores exceeding the threshold, the server identifies the person with the highest matching score as the person to be recognized.

The administrator terminal 3 is operated by the administrator. By accessing the facial recognition server 2 from the administrator terminal 3, a management screen related to the facial recognition server 2 is displayed. In this embodiment, in particular, a management screen (update operation screen) related to updating registered facial images registered in the registrant database is displayed. On this management screen, for the same person, the registered facial image and multiple candidate facial images extracted from the authenticated person's facial image acquired during facial recognition are displayed on the administrator terminal 3. The administrator visually confirms and selects the optimal candidate facial image. The facial recognition server 2 then performs a process (update) to replace the current registered facial image with the candidate facial image selected by the administrator as the new registered facial image.

In this embodiment, the registered user database is configured to be built on the facial recognition server 2. However, the registered user database may also be configured to be built on a separate database server from the facial recognition server 2.

Next, we will explain the authentication log management table processed by the facial recognition server 2. Figure 2 is an explanatory diagram showing the contents of the authentication log management table.

The facial recognition server 2 manages an authentication log management table (authentication log management information). This table registers authentication log information for each authentication event, where the person to be authenticated is photographed by checker 1 and the facial recognition process is executed by facial recognition server 2. Each time an authentication event occurs, the authentication log information related to that event is added to the authentication log management table.

Furthermore, during facial recognition, a one-to-many (1:N) facial matching process is performed, meaning a one-to-one facial matching between one facial image of the person to be recognized and one registered user is repeated for each registered user. Authentication log information is created for each of these one-to-N facial matchings. In other words, one authentication log is generated (output) for each one-to-N facial matching.

The authentication log information for each authentication event includes the date and time of authentication, the checker 1 that performed the authentication, the authenticated person's facial image ID, the authentication result registrant ID, the authentication result, and the matching score obtained during authentication.

Here, the authentication log information and the authenticated person's facial image are linked using the authenticated person's facial image ID included in the authentication log information. In this embodiment, since multiple authenticated person facial images are acquired and face matching is performed for one person's authentication, multiple authentication logs are generated for one person's authentication, and an authenticated person's facial image is linked to each log.

Furthermore, the authentication result included in the authentication log information indicates whether authentication was successful or not. Specifically, "OK" means that at least one person was successfully matched (the match score exceeded a predetermined threshold), i.e., the person being authenticated could be identified. "NG" means that no person was successfully matched, i.e., the person being authenticated could not be identified.

Furthermore, in this embodiment, the checker 1 performs a process to detect impersonation, that is, the act of impersonating someone else to obtain authentication by displaying another person's facial image on the screen of a smartphone or tablet device and holding it up to the checker 1, or by holding another person's facial photograph up to the checker 1. If impersonation is detected, the result will be "NG" even if the matching score exceeds a predetermined threshold.

Next, we will describe the schematic configuration of Checker 1 and Face Recognition Server 2. Figure 3 is a block diagram showing the schematic configuration of Checker 1 and Face Recognition Server 2. Figure 4 is an explanatory diagram showing an overview of the processing performed by Face Recognition Server 2.

Checker 1 comprises a camera 11, a communication unit 12, a storage unit 13, and a processor 14.

Camera 11 captures the face of the person undergoing facial recognition.

The communication unit 12 communicates with the facial recognition server 2 via the network.

The memory unit 13 stores programs and other data executed by the processor 14.

The processor 14 performs various processes related to facial recognition by executing the program stored in the memory unit 13. In this embodiment, the processor 14 performs processes such as impersonation detection and acquisition of the authenticated person's facial image.

In the impersonation detection process, processor 14 detects impersonation, that is, the act of impersonating someone else to obtain authentication by displaying another person's face image on the screen of a smartphone or tablet device and holding it over checker 1, or by holding another person's face photograph over checker 1. This impersonation detection process, for example, detects facial contours based on changes in the image (frame) captured by camera 11 to determine whether the subject is a real face or a face displayed on a screen.

In the process of acquiring the authenticated person's face image, the processor 14 detects the person's (authenticated person's) face from the image captured by the camera 11 and extracts the face area from the image to acquire the authenticated person's face image. The acquired authenticated person's face image, along with the impersonation detection result, is transmitted from the communication unit 12 to the face authentication server 2. This process is performed, for example, five times per second. Therefore, if, for example, a person is in front of the checker for two seconds (within the camera's field of view for two seconds), the process will be performed ten times.

The facial recognition server 2 comprises a communication unit 21, a storage unit 22, and a processor 23.

The communication unit 21 communicates with the checker 1 and the administrator terminal 3 via the network.

The storage unit 22 stores programs executed by the processor 23. The storage unit 22 also stores the authentication log management table (see Figure 2), the authenticated user's facial image, and registration information from the registrant database. The registrant database contains attribute information and registered facial images for each user.

Furthermore, the authentication log information for each authentication event registered in the authentication log management table is deleted after a predetermined period (e.g., 6 months) has elapsed since the occurrence of the authentication event. This predetermined period can be set to any desired period from the settings screen (not shown). The authenticated user's facial image and the registered facial image are stored in the storage unit 22 in an encrypted state. While only one registered facial image per user is required in the registrant database, it is also possible to register two or more registered facial images.

The processor 23 performs various processes related to facial recognition by executing the program stored in the memory unit 22. In this embodiment, the processor 23 performs facial recognition processing, authentication log management processing, matching score aggregation processing, target person extraction processing, facial image filtering processing, candidate facial image extraction processing, and registered facial image update processing.

In the facial recognition process, the processor 23 performs facial recognition based on the facial image of the person to be recognized received from the checker 1 and the results of the impersonation detection. At this time, the facial feature information extracted from the facial image of the person to be recognized is compared with the facial feature information of each registered person registered in the registered person database to calculate a matching score. This matching score is then compared to a predetermined threshold (e.g., 0.75) to determine whether the person to be recognized is the same person as any of the registered persons (facial matching process). If the matching score is equal to or greater than the threshold, the facial matching is determined to be successful, and the person to be recognized is identified. On the other hand, if the matching score is less than the threshold, the facial matching is determined to have failed. Furthermore, if impersonation is detected, the facial recognition is determined to have failed.

In this embodiment, the checker 1 transmits the facial image of the person to be authenticated to the facial recognition server 2, and the facial recognition server 2 extracts facial feature information from the facial image. However, the checker 1 may extract the facial feature information from the facial image of the person to be authenticated and transmit the facial feature information from the checker 1 to the facial recognition server 2.

Furthermore, in this embodiment, multiple facial images of the person being authenticated (for example, five per second) are acquired during the authentication process. In the facial matching process, the facial feature information of each of these multiple facial images is compared with the facial feature information of each registered user. A matching score is calculated for each of the multiple facial images to identify the person being authenticated.

In the authentication log management process, processor 23 manages the authentication log management table (authentication log management information) (see Figure 2). This authentication log management table registers authentication log information for each authentication event, where the checker 1 photographs the person to be authenticated and the facial authentication process is performed by the facial authentication server 2. Each time an authentication event occurs, processor 23 adds the authentication log information related to that event to the authentication log management table.

In the matching score aggregation process, the processor 23 aggregates authentication scores (matching scores) for each user (authenticated person) and checker 1 based on the authentication log management table. In this embodiment, the average value of the matching scores (average score) is calculated. Furthermore, this matching score aggregation process is performed only on authentication events where authentication was successful.

Furthermore, in the matching score aggregation process, authentication events that meet predetermined extraction criteria are extracted as targets for aggregation. In this embodiment, aggregation is performed on authentication events included in the target period specified as the extraction criterion. The target period is specified by the administrator.

Furthermore, in this embodiment, the extraction criterion is set to aggregate data for users whose number of authentication events (number of authentications) within the target period is equal to or greater than a predetermined threshold (for example, 10 times per checker unit). The number of authentications is aggregated for each user, regardless of the authentication result. The threshold for the number of authentications is specified by the administrator. This excludes users whose number of authentications falls below the threshold from the update target. Therefore, it is possible to avoid updating users for whom the update is not very meaningful due to a small sample size.

In the target user extraction process, the processor 23 extracts users who are eligible for face image updates (low-score authenticated users) based on the average score obtained in the matching score aggregation process. Specifically, it extracts users whose average score is below a predetermined threshold (e.g., 0.8). The extraction threshold, i.e., the threshold for the average score, is specified by the administrator. The extraction threshold is greater than the matching threshold (e.g., 0.75). Note that in the matching score aggregation process, matching score trend values such as the median score or the most frequent score value may be obtained in addition to the average score. In this way, by identifying users whose faces have changed due to aging or other factors over time since image registration and who tend to have low scores, it becomes possible to perform update processing targeting users who require updates.

Furthermore, in the target user selection process, if the number of users whose average score falls below a threshold exceeds a predetermined number (e.g., 10 users), the system will select users to fit within that predetermined number. In this case, users should be selected in order from the user with the lowest average score, up to the specified number. The number of users to be selected is specified by the administrator.

In the face image filtering process, the processor 23 filters all face images (authenticated face images for each authentication event) associated with the authentication logs of users (low-score authenticated users) extracted in the target user extraction process, excluding inappropriate face images. In this embodiment, filtering is performed based on image quality and impersonation.

In the image quality filtering process, the processor 23 determines whether the subject's face image satisfies a predetermined image quality standard (quality determination process), and subject face images that do not meet the quality standard are excluded. Specifically, for example, face images with excessive facial expressions such as smiles are determined to not meet the image quality standard. While a low matching score and frequent matching failures often occur when a subject's face image does not meet the image quality standard, matching may still succeed. In such cases, subject face images that do not meet the image quality standard may be extracted as candidate face images.

Furthermore, in the filtering for impersonation, the processor 23 determines whether a face image is impersonated or not based on the results of impersonation detection during face recognition (impersonation detection process), and impersonated face images are excluded from the target. Note that in the case of impersonation, face recognition fails as a result of impersonation detection, but the matching score in face matching is high, so face matching itself is often successful. In this case, it is stored as an authentication event where face matching was successful. Therefore, filtering is necessary to prevent impersonated face images from being extracted as candidate face images.

In the candidate face image extraction process, the processor 23 extracts a predetermined number of candidate face images (e.g., 10 images) from the face images narrowed down by the face image filtering process, for each user (low-score authenticated user) whose face image is to be updated. The number of images to be extracted is specified by the administrator. At this time, the face images to be extracted are those associated with successful authentication events among the authentication events of low-score authenticated users.

Furthermore, in this embodiment, the number of face images extracted are selected as candidate face images, starting with those that have the highest suitability as registered face images. Specifically, the number of face images extracted are selected as candidate face images, starting with those with the highest matching scores. Also, the number of face images extracted are selected as candidate face images, starting with those with the highest image quality.

In the registered face image update process, the processor 23 replaces the current registered face image with a candidate face image selected by the administrator from among the multiple candidate face images obtained in the candidate face image extraction process. In this embodiment, the multiple candidate face images obtained in the candidate face image extraction process are displayed on the administrator terminal 3, and the administrator visually confirms and selects the optimal candidate face image.

Next, we will explain the management screen displayed on administrator terminal 3. Figure 5 is an explanatory diagram showing the management screen.

This management screen includes an authentication log file selection section 31, a registered face image file selection section 32, and an import button 33.

In the authentication log file selection unit 31, when the user operates the selection button 34, a selection screen (not shown) for authentication log files (including authentication log management information) is displayed. When the user selects an authentication log file on this selection screen, the file name of the selected authentication log file is displayed in the imported file display unit 35. Similarly, in the registered face image file selection unit 32, when the user operates the selection button 36, a selection screen (not shown) for registered face image files is displayed. When the user selects a registered face image file on this selection screen, the file name of the selected registered face image file is displayed in the imported file display unit 37.

When the user selects an authentication log file in the authentication log file selection unit 31 and a registered face image file in the registered face image file selection unit 32, and then operates the import button 33, the selected authentication log file, the authenticated person's face image associated with that authentication log file, and the selected registered face image file are loaded. Note that the authentication log file and the authenticated person's face image are stored in an encrypted state, so decryption is performed during loading.

Furthermore, this management screen is equipped with a first mode selection button 41, a second mode selection button 42, and a third mode selection button 43. When the administrator operates the first mode selection button 41, this management screen, specifically the screen displaying authentication score statistics for users, is displayed. When the administrator operates the second mode selection button 42, a screen displaying authentication score statistics for Checker 1 is displayed. Finally, when the administrator operates the third mode selection button 43, a screen displaying authentication log information is displayed.

Furthermore, this management screen is equipped with a first extraction condition setting unit 45, a second extraction condition setting unit 46, and a third extraction condition setting unit 47.

The first extraction condition setting unit 45 sets the extraction conditions (processing conditions) related to the matching score aggregation process. This first extraction condition setting unit 45 is equipped with an extraction button 51, a target period specification unit 52, and an authentication count threshold specification unit 53.

In the target period specification section 52, the administrator can specify the start date and time of the target period, and the end date and time of the target period.

The authentication count threshold specification unit 53 allows the administrator to specify a threshold (minimum number of authentications) related to the number of authentications.

When the administrator specifies the target period in the target period specification unit 52 and the authentication count threshold in the authentication count threshold specification unit 53, and then operates the extract button 51, the extraction process related to the matching score aggregation process is executed. Authentication events included in the target period are extracted, and users whose authentication count exceeds the threshold are further extracted. Based on the extraction results, the list in the statistical information display unit 61 is then updated.

The second extraction condition setting unit 46 sets the extraction conditions (processing conditions) related to the target person extraction process. This second extraction condition setting unit 46 is equipped with an extraction button 54, a score threshold specification unit 55, and an extraction number specification unit 56.

The score threshold specification unit 55 allows the administrator to specify thresholds (extraction judgment thresholds) related to the matching score trend values (e.g., mean, median, and most frequent score values) used when extracting users who are eligible for face image updates (low-score authenticated users). These extraction judgment thresholds are higher than the thresholds used to determine the success or failure of face matching (matching judgment thresholds).

In this embodiment, the upper limit is set to a value (e.g., 0.80) that is greater than the matching judgment threshold (e.g., 0.75) by a predetermined value (e.g., 0.05). This upper limit is displayed as the initial value in the score threshold specification unit 55. Furthermore, the extraction judgment threshold can be selected within the range from the matching judgment threshold to the upper limit. Specifically, a pull-down menu displays threshold options (e.g., 0.80, 0.79, 0.78, 0.77, 0.76).

In the extraction number specification unit 56, the administrator can specify the number of users (low-score certified users) to be extracted.

The administrator specifies the threshold for extraction using the score threshold specification unit 55, and also specifies the number of users to be extracted using the extraction number specification unit 56. When the administrator operates the extraction button 54, the target user extraction process is executed. Users whose matching score trend value is below the threshold are extracted, and then the number of users selected is extracted starting from the user with the lowest matching score trend value (worst score N). Based on the extraction specification results, the list in the statistical information display unit 61 is updated.

The third extraction condition setting unit 47 sets the extraction conditions (processing conditions) related to the candidate face image extraction process. This third extraction condition setting unit 47 is equipped with an extraction button 57 and a face image extraction number specification unit 58. The face image extraction number specification unit 58 allows the administrator to specify the number of candidate face images to be extracted (for example, 10 images).

After the administrator specifies the number of images to extract using the image extraction quantity specification unit 58, and then operates the extraction button 57, the candidate face image extraction process is executed. Face images matching the specified number are extracted as candidate face images, starting with those with the highest suitability (matching score or image quality). Based on the extraction results, the list in the statistical information display unit 61 is then updated.

Furthermore, the statistical information display unit 61 displays a list of the aggregated matching scores for all checkers 1, as well as the aggregated matching scores for each individual checker 1. This list displays the number of face matches, and the minimum, maximum, and average matching scores for each user.

Furthermore, the statistical information display unit 61 is equipped with vertical and horizontal scrolling operation units 62 and 63 (scroll buttons and scroll bars). By operating these scrolling operation units 62 and 63, the administrator can shift the display range of the list vertically and horizontally.

Furthermore, this management screen includes a settings button 65. When the administrator operates this settings button 65, the system transitions to the filtering settings screen (see Figure 6).

Furthermore, this management screen is equipped with an execution button 66. When the administrator operates this execution button 66, the system transitions to the candidate face image selection screen (see Figure 7).

Next, we will explain the filtering settings screen displayed on administrator terminal 3. Figure 6 is an explanatory diagram showing the filtering settings screen.

This filtering settings screen allows administrators to specify whether or not filtering (narrowing down) is necessary for each item related to image quality, and whether or not filtering is necessary for impersonation. Specifically, the filtering settings screen has checkbox 71 for each item related to image quality, and checkbox 72 for impersonation.

Image quality filtering allows administrators to specify whether filtering is necessary for the following items: wearing sunglasses, wearing a mask, multiple faces in the image, faces being too small, faces being blurred or distorted, backlighting, overexposure, wearing a hat, excessive facial expressions such as smiles, eyes being obscured by hair, eyes being closed, facial features being obscured by hands, etc., and faces not facing forward (sideways, upward, downward).

This filtering settings screen includes a settings button 73. When an administrator presses this button 73, the settings information for face image filtering is registered based on the inputs in checkboxes 71 and 72.

Next, we will explain the candidate face image selection screen displayed on administrator terminal 3. Figure 7 is an explanatory diagram showing the candidate face image selection screen.

This candidate face image selection screen includes a registered face image display unit 81 and a candidate face image display unit 82. The registered face image display unit 81 displays the currently registered face image. The candidate face image display unit 82 displays a predetermined number (for example, 10) of extracted candidate face images side by side. This allows the administrator to visually confirm the suitability of the candidate face images by comparing them with the currently registered face image and select a candidate face image to be the new registered face image.

Furthermore, the registered face image display unit 81 is equipped with a forward button 83a and a back button 83b. When the administrator operates these forward and back buttons 83a and 83b, they can change the users whose images are being viewed among the extracted users (low-score authenticated users), and display each user's registered face image one by one according to the extraction order.

Furthermore, this candidate face image selection screen is equipped with a number display unit 84 and a file name display unit 85 related to the registered face image. The number display unit 84 shows the number of the currently displayed user and the total number of extracted users (score worst N) as a fraction. For example, 9/10 indicates that the user is the 9th out of the 10 extracted users.

Furthermore, when the administrator performs an operation to enlarge the registered face image (for example, by double-clicking on the image) on the registered face image display unit 81, the system transitions to the enlarged registered image screen (see Figure 8(A)).

Furthermore, the candidate face image display unit 82 is provided with radio buttons 86 for each candidate face image. The administrator can select a candidate face image by operating these radio buttons 86. When a candidate face image is selected, it is highlighted. For example, the display frame of the candidate face image is highlighted.

Furthermore, when the administrator performs an operation to enlarge a candidate image (for example, by double-clicking on the image) on the candidate face image display unit 82, the system transitions to the enlarged candidate image screen (see Figure 8(B)).

Furthermore, this candidate face image selection screen is equipped with a number display unit 87 and a management information display unit 88 related to the candidate face images. The number display unit 87 shows the number of the candidate face image selected in the candidate face image display unit 82 and the total number of extracted candidate face images as a fraction. For example, 1/10 indicates that it is the first candidate face image out of the 10 extracted images. The management information display unit 88 displays management information related to the selected candidate face image. Specifically, it displays the authentication log ID, authentication event ID, the person ID of the person being authenticated, the authentication date and time, the authentication result, and the matching score.

Furthermore, this candidate face image selection screen is provided with a file save button 89. When the administrator selects a candidate face image on the candidate face image display unit 82 and operates the file save button 89, the selected candidate face image and its associated management information (attribute information) are saved to a batch registration file. Once the selection of candidate images for the extracted users (users with the worst score N) is complete, the administrator performs a batch registration operation. This triggers an update process based on the candidate images and management information for each person saved in the batch registration file. In other words, each of the candidate images saved in the batch registration file is replaced with a new registered image, replacing the current registered image.

In this embodiment, one registered face image is registered for each user, so one registered face image is displayed on the registered face image display unit 81. However, multiple registered face images may be registered for each user, in which case multiple registered face images will be displayed side-by-side on the registered face image display unit 81. Furthermore, the administrator may be allowed to select multiple candidate face images from the multiple candidate face images displayed on the candidate face image display unit 82, up to a maximum of the number of registered face images.

Furthermore, among the candidate face images displayed on the candidate face image display unit 82, those not selected by the administrator are assumed to be unsuitable as registered face images. Therefore, they may be deleted or excluded from being extracted as candidate face images in the next update.

Next, we will explain the registered face image enlargement screen and the candidate face image enlargement screen displayed on the administrator terminal 3. Figure 8 is an explanatory diagram showing the registered face image enlargement screen and the candidate face image enlargement screen.

As shown in Figure 8(A), the enlarged registered face image screen is provided with an enlarged registered face image display unit 91. This enlarged registered face image display unit 91 displays the registered face image in an enlarged format. Furthermore, this enlarged registered face image screen is provided with a number display unit 92 and a file name display unit 93, similar to the number display unit 84 and file name display unit 85 in the candidate face image selection screen (see Figure 7).

As shown in Figure 8(B), the candidate face image enlargement screen is provided with an enlarged candidate face image display unit 95. This enlarged candidate face image display unit 95 displays the candidate face images in an enlarged format. Similar to the number display unit 87 and management information display unit 88 in the candidate face image selection screen (see Figure 7), a number display unit 96 and a management information display unit 97 are provided.

As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to modified, replaced, added, or omitted embodiments. Furthermore, it is possible to combine the components described in the above embodiments to create new embodiments.

The facial recognition system and registered facial image update method according to the present invention have the effect of reducing the burden on administrators on update work by enabling administrators to efficiently select appropriate facial images from the facial images of those to be recognized through visual confirmation. They are useful as a facial recognition system that compares a facial image of a person to be recognized obtained by photographing that person with a registered facial image for each user, and as a registered facial image update method in a facial recognition system that updates the registered facial image using an information processing device.

1. Checker (facial recognition device)
2. Facial Recognition Server (Server Device)
3. Administrator terminal (administrator device)
11 Camera 12 Communication unit 13 Storage unit 14 Processor 21 Communication unit 22 Storage unit 23 Processor 45 First extraction condition setting unit 46 Second extraction condition setting unit 47 Third extraction condition setting unit 51 Extraction button 52 Target period specification unit (period specification unit)
53 Authentication count threshold specification unit 54 Extraction button 55 Score threshold specification unit (threshold specification unit)
56. Number of people to extract (specify number of images) 57. Extraction button 58. Number of face images to extract (specify number of images)
61 Statistical information display unit 65 Setting button 66 Execute button 81 Registered face image display unit 82 Candidate face image display unit

Claims (8)

A facial recognition system that compares a facial image of a person to be authenticated, obtained by taking a photograph of the person to be authenticated, with a registered facial image for each user,
A facial recognition device that photographs a person to be authenticated and acquires a facial image of the person to be authenticated,
The system includes a server device that stores the registered facial images for each user and performs a matching process between the authenticated user's facial image and the registered facial images,
The server device is
The system stores the facial image of the person being authenticated obtained from the facial recognition device , and also stores authentication log management information that associates the facial image of the person being authenticated with the authentication result and the user related to the authentication event that occurred at the facial recognition device.
A screen is displayed on the administrator device allowing the administrator to specify the processing conditions.
Based on the matching score included in the authentication log management information, users with a tendency towards low scores who are subject to updating according to the processing conditions are extracted from among the users who successfully authenticated .
From the authenticated facial images of the extracted users, candidate facial images are extracted according to the processing conditions.
The system displays a screen on the administrator device that presents the extracted candidate face images to the administrator and allows the administrator to select a candidate face image.
A facial recognition system characterized by performing an update process in which the selected candidate facial image replaces the current registered facial image with a new registered facial image. The server device is
The period specified by the administrator is displayed on the screen of the administrator device,
The facial recognition system according to claim 1, characterized in that it aggregates data based on authentication events included in the aforementioned period specified by the administrator. The server device is
The threshold specification section, which allows the administrator to specify the threshold for extraction when extracting targets based on the aforementioned matching score, is displayed on the screen of the administrator device.
The facial recognition system according to claim 1, characterized in that it extracts users as targets whose matching score trend value is less than the threshold value for extraction determination specified by the administrator. The server device is
The facial recognition system according to claim 3, characterized in that a value that is a predetermined value greater than the matching judgment threshold is displayed on the screen of the administrator device as the initial value of the extraction judgment threshold. The server device is
The administrator specifies the number of candidate facial images, and this number specification section is displayed on the screen of the administrator device.
The administrator extracts a specified number of candidate facial images.
The facial recognition system according to claim 1, characterized in that it displays a plurality of extracted candidate facial images side by side on the screen of the administrator device. The server device is
The facial recognition system according to claim 1, characterized in that, when extracting candidate facial images, it determines whether the target facial image is a spoof that would be detected during facial recognition, and excludes facial images that are spoofs from the extraction of candidate facial images. The server device is
The facial recognition system according to claim 1, characterized in that, when extracting candidate facial images, it determines whether the target facial image satisfies the image quality requirements for facial matching, and excludes facial images that do not meet the image quality requirements from the extraction of candidate facial images. A facial recognition system that compares a facial image of a person to be recognized obtained by photographing the person to be recognized with a registered facial image for each user, wherein the registered facial image is updated by an information processing device,
The system stores the facial images of the person being authenticated obtained from the facial recognition device , and also stores authentication log management information that associates the facial images of the person being authenticated with the authentication results and users related to the authentication events that occurred at the facial recognition device.
A screen is displayed on the administrator device allowing the administrator to specify the processing conditions.
Based on the matching score included in the authentication log management information, users with a tendency towards low scores who are subject to updating according to the processing conditions are extracted from among the users who successfully authenticated .
From the authenticated facial images of the extracted users, candidate facial images are extracted according to the processing conditions.
The system displays a screen on the administrator device that presents the extracted candidate face images to the administrator and allows the administrator to select a candidate face image.
A method for updating a registered face image, characterized by performing an update process that replaces the current registered face image with the selected candidate face image as a new registered face image.