JP7528733B2 - Authentication system, authentication device, authentication method, and program - Google Patents

Description

The present invention relates to authentication technology, and in particular to authentication technology suitable for entrance and exit management.

Patent document 1 discloses an entrance/exit management device that can manage entrance/exit without compromising the convenience of entering and exiting a facility.

This entrance/exit management device detects people who plan to enter or exit from images captured by a camera installed to capture the area in front of the entrance/exit door, and sets the number of people who plan to enter or exit as the number of people who plan to enter or exit. In addition, multiple people who plan to enter or exit who are located within a specified interpersonal distance are treated as an entrance/exit candidate group, and the number of people who plan to enter or exit who belong to this entrance/exit candidate group is set as the number of people who plan to enter or exit. Next, when a member of the entrance/exit candidate group enters or exits, people who plan to enter or exit are detected from the image captured by the camera immediately afterwards, and the number of people who plan to enter or exit is set as the number of people who have not yet entered or exited. Then, if the value obtained by subtracting the number of entrance/exit candidates from the number of people who plan to enter or exit is greater than the number of people who have not yet entered or exited, it is determined that unauthorized entry or exit, so-called tailgating, has occurred. This makes it possible to detect the occurrence of tailgating without compromising convenience when entering or exiting, thereby improving security.

JP 2010-211510 A

However, the entrance/exit management device described in Patent Document 1 can detect the occurrence of tailgating, but cannot prevent it from happening in advance. In addition, it does not take into consideration entrance/exit management for areas that require various security levels, such as research facilities and server rooms where only pre-registered people are allowed to enter, or hospitals and nursing homes where people who are not pre-registered (e.g., patients or people receiving care) may be accompanied by people.

The present invention was made in consideration of the above circumstances, and its purpose is to provide authentication technology suitable for managing entrance and exit to areas requiring various security levels.

In order to solve the above problem, in the present invention, facial images are detected from images captured by a camera installed to capture an area in front of an entrance or exit, and for each facial image, features of the facial image are extracted and the size and coordinates of the facial image are identified. Then, for each facial image, facial recognition processing is performed using the features of the facial image, and if facial recognition is successful, it is evaluated whether the size of the facial image is equal to or larger than a preregistered validity area, and whether the coordinates of the facial image are located within a preregistered validity coordinate range. On the other hand, if facial recognition is unsuccessful, it is evaluated whether the size of the facial image is equal to or larger than the validity area. Then, it is determined whether the facial recognition results and the evaluation results of the size and coordinates of each facial image match a preregistered judgment pattern, and if they match, an entry/exit action linked to this judgment pattern is executed.

For example, the present invention provides
An authentication system for authenticating users in a predetermined area, comprising:
An electronic lock installed on an entrance or exit door of the specified area;
A camera installed to capture an image of an area in front of an entrance or exit door of the specified area;
A guidance output device installed at an entrance or exit door of the predetermined area;
an authentication device that authenticates a user who is attempting to enter or leave the specified area by using an image captured by the camera;
The authentication device includes:
a face image detection means for detecting a face image from an image captured by the camera and extracting a feature amount of the face image together with information capable of identifying a size and coordinates of the face image;
a face authentication information storage means for storing face authentication information of users who are permitted to use the predetermined area;
a face authentication means for performing face authentication for each face image detected from the image captured by the camera by the face image detection means, based on a feature amount extracted from the face image and face authentication information stored in the face authentication information storage means;
a validation information storage means for storing a validation area, which is the size of a face image to be recognized as a user of the predetermined area, and a validation coordinate range, which is a coordinate range;
an evaluation means for evaluating, for each face image detected by the face image detection means from the captured image of the camera, if the authentication result by the face authentication means is face authentication success, whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means, and evaluating whether or not the coordinates of the face image are within the validation coordinate range stored in the validation information storage means, if the authentication result by the face authentication means is face authentication failure, evaluating whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means;
a determination pattern storage means for storing, as determination patterns, patterns of the number of successful face authentications, the number of unsuccessful face authentications, the number of successful face authentications of face images equal to or larger than the valid area, the number of unsuccessful face authentications of face images equal to or larger than the valid area, and the number of successful face authentications of face images within the valid coordinate range, linked to actions for the electronic lock and the guidance output device;
The system further includes an action control means for searching the judgment pattern storage means for a judgment pattern that matches the face authentication result by the face authentication means and the evaluation result by the evaluation means for each face image detected from the image captured by the camera by the face image detection means, and for controlling the electronic lock and the guidance output device to execute the action linked to the searched judgment pattern.

In the present invention, face recognition processing is performed for each face image detected from an image captured by a camera installed to capture an image in front of an entrance or exit. If face recognition is successful, it is evaluated whether the size of the face image is equal to or larger than the valid area, and whether the coordinates of the face image are located within the valid coordinate range. If face recognition is unsuccessful, it is evaluated whether the size of the face image is equal to or larger than the valid area. Then, an entry/exit action is executed that is linked to a judgment pattern that matches the face recognition result and the evaluation results of the size and coordinates of each face image.

Therefore, according to the present invention, by changing the activation area, activation coordinate range, and judgment pattern, it can be applied to entrance and exit management of areas requiring various security levels.

FIG. 1 is a schematic diagram of an authentication system according to an embodiment of the present invention. FIG. 2 is a schematic functional configuration diagram of the authentication device 1. As shown in FIG. FIG. 3 is a diagram showing an example of registered contents in the validation information storage unit 104. As shown in FIG. FIG. 4 is a diagram showing an example of registered contents in the determination pattern storage unit 106. As shown in FIG. 5A and 5B are diagrams showing examples of images 30 captured by the camera 3 for explaining the determination patterns. 6A and 6B are diagrams showing examples of images 30 captured by the camera 3 for explaining the determination patterns. FIG. 7 is a diagram showing an example of registered contents in the application information storage unit 107. As shown in FIG. FIG. 8 is a flow diagram for explaining the operation of authentication device 1. As shown in FIG. FIG. 9 is a flow diagram for explaining the application pattern search process S16 shown in FIG.

Below, one embodiment of the present invention will be described with reference to the drawings.

Figure 1 is a schematic diagram of an authentication system according to this embodiment.

Here, an example is shown in which the authentication system according to this embodiment is applied to entrance management for each of multiple areas 5-1 to 5-n (hereinafter also simply referred to as areas 5) in a facility that has these areas.

As shown in the figure, the authentication system according to this embodiment includes an authentication device 1, autolocks 2-1 to 2-n (hereinafter simply referred to as autolocks 2) installed at the entrance doors of areas 5-1 to 5-n that require various security levels, such as entrances, reception rooms, conference rooms, offices, laboratories, server rooms, and entrances to nursing care facilities, cameras 3-1 to 3-n (hereinafter simply referred to as cameras 3) installed to capture images of the area in front of the entrances to areas 5-1 to 5-n, and guidance output devices 4-1 to 4-n (hereinafter simply referred to as guidance output devices 4) installed at the entrances to areas 5-1 to 5-n that output guidance.

Here, the autolock 2, camera 3, and guidance output device 4 are connected to the authentication device 1 via a wired network and/or a wireless network (not shown). Note that the autolock 2 and camera 3 can be existing autolocks and surveillance cameras equipped with network communication functions. The guidance output device 4 can be an existing network terminal equipped with a display and/or speaker.

The authentication device 1 uses the facial image contained in the image captured by the camera 3 to authenticate users attempting to enter the areas 5-1 to 5-n in which the camera 3 is installed, thereby managing entry to the areas 5-1 to 5-n.

Figure 2 is a schematic functional configuration diagram of the authentication device 1.

As shown in the figure, the authentication device 1 includes a network communication unit 100, a face image detection unit 101, a face authentication information storage unit 102, a face authentication unit 103, an activation information storage unit 104, an evaluation unit 105, a judgment pattern storage unit 106, an application information storage unit 107, an application pattern search unit 108, an action control unit 109, an unlock command transmission unit 110, and a guidance output command transmission unit 111.

The network communication unit 100 is an interface for communicating with the autolock 2, the camera 3, and the guidance output device 4 via a wired network and/or a wireless network (not shown).

The face image detection unit 101 detects all face images contained in the captured images received from the camera 3 via the network communication unit 100, and for each detected face image, extracts the features of the face image and also extracts the coordinates of the pixel located in the lower left corner of the face image (hereinafter referred to as the lower left corner coordinates) and the coordinates of the pixel located in the upper right corner (hereinafter referred to as the upper right corner coordinates).

The facial authentication information storage unit 102 stores, for each camera ID, which is the identification information of the camera 3, facial authentication information (feature values of facial images) of each user who is permitted to enter the area 5 in which the camera 3 is installed.

For each face image detected by the face image detection unit 101 from the captured image of the camera 3, the face authentication unit 103 performs face authentication using the feature amount extracted from this face image and the face authentication information storage unit 102. Specifically, face authentication information that matches the feature amount extracted from this face image is searched for among the face authentication information that is linked to the camera ID of this camera 3 and stored in the face authentication information storage unit 102. If matching face authentication information is detected, it is determined that face authentication is successful, and if matching information is not detected, it is determined that face authentication is not successful.

The validation information storage unit 104 stores, for each camera 3, the size (hereinafter referred to as the validation area) and coordinate range (hereinafter referred to as the validation coordinate range) of a face image to be recognized from an image captured by the camera 3 as a user attempting to enter the area 5 in which the camera 3 is installed.

Figure 3 is a schematic diagram showing an example of the registered contents of the activation information storage unit 104.

As shown in the figure, the activation information storage unit 104 stores records 1040 of the activation area and activation coordinate range (hereinafter, these are also referred to as activation information) for each camera 3. The activation information record 1040 has a field 1041 in which the camera ID of the camera 3 is registered, a field 1042 in which the type of area 5 in which the camera 3 is installed is registered, a field 1043 in which the activation area is registered, and a field 1044 in which the activation coordinate range is registered. Here, the activation area is expressed using the number of pixels, and the activation coordinate range is expressed using the coordinates of the lower left corner and the upper right corner of the face image. Note that FIG. 3 illustrates a case where the size of the captured image of the camera 3 is 640×480 pixels, and when the activation coordinate range is (000,000) or (640,480), it indicates that the entire captured image of the camera 3 is the activation coordinate range.

For each face image detected by the face image detection unit 101 from the image captured by the camera 3, the evaluation unit 105 determines the size of the face image (the number of pixels in the rectangular area specified by the lower left and upper right corner coordinates) and the center coordinates (the center coordinates of the line connecting the lower left and upper right corner coordinates) based on the lower left and upper right corner coordinates extracted from the face image. Then, using the size and center coordinates of the face image and the validation information storage unit 104, it evaluates whether or not the face image should be recognized as a user attempting to enter the area 5 in which the camera 3 is installed.

Specifically, for a face image for which the authentication result by the face authentication unit 103 is successful, it is evaluated whether the size of the face image is equal to or larger than the activation area associated with the camera ID of the camera 3 that captured the face image and stored in the activation information storage unit 104, and it is also evaluated whether the center coordinates of the face image are within the activation coordinate range associated with the camera ID and stored in the activation information storage unit 104. In addition, for a face image for which the authentication result by the face authentication unit 103 is unsuccessful, it is evaluated whether the size of the face image is equal to or larger than the activation area associated with the camera ID of the camera 3 that captured the face image and stored in the activation information storage unit 104.

The judgment pattern storage unit 106 stores multiple patterns of the authentication results by the face authentication unit 103 and the evaluation results by the evaluation unit 105 as judgment patterns, together with actions for the autolock 2 and the guidance output device 4.

Figure 4 is a schematic diagram showing an example of the contents registered in the judgment pattern storage unit 106.

As shown in the figure, the judgment pattern memory unit 106 stores a judgment pattern record 1060 for each judgment pattern. The record 1060 of the judgment pattern has a field 1061 in which a pattern ID, which is identification information of the judgment pattern, is registered; a field 1062 in which the number of successful face authentications in the face authentication unit 103 for each face image detected from the captured image of the camera 3 is registered; a field 1063 in which the number of unsuccessful face authentications in the face authentication unit 103 for each of these face images is registered; a field 1064 in which the number of successful face authentications in the face authentication unit 103 for each of these face images evaluated by the evaluation unit 105 to be equal to or larger than the valid area is registered; a field 1065 in which the number of unsuccessful face authentications in the face authentication unit 103 for each of these face images evaluated by the evaluation unit 105 to be equal to or larger than the valid area is registered; a field 1066 in which the number of successful face authentications in the face authentication unit 103 for each of these face images evaluated by the evaluation unit 105 to be within the valid coordinate range is registered; and a field 1067 in which actions for the autolock 2 and the guidance output device 4 are registered.

Figures 5(A), 5(B), 6(A), and 6(B) are diagrams showing examples of an image 30 captured by camera 3 to explain a judgment pattern. Here, the area 31 surrounded by a dashed line indicates the face image detected from the image 30 captured by camera 3, the area 32 surrounded by a dotted line indicates the size of the activation area (compared to the face image 31), and the area 33 surrounded by a dashed line indicates the activation coordinate range.

In the example shown in FIG. 5(A), the entire captured image 30 is set to the valid coordinate range 33. In this example, only one face image 31 is detected from the captured image 30, the size of this face image 31 is equal to or larger than the valid area 32, and its center coordinates are within the valid coordinate range 33.

If the facial recognition result of this facial image 31 is successful, the number of successful facial recognitions will be "1", the number of unsuccessful facial recognitions will be "0", the number of successful facial recognitions above the valid area will be "1", the number of unsuccessful facial recognitions above the valid area will be "0", and the number of successful facial recognitions within the valid coordinate range will be "1", which respectively match the judgment pattern of record 1060 in which pattern ID "A" is registered in field 1061 and the judgment pattern of record 1060 in which pattern ID "F" is registered in field 1061.

On the other hand, if the facial recognition result of this facial image 31 is unsuccessful, the number of successful facial recognitions will be "0", the number of unsuccessful facial recognitions will be "1", the number of successful facial recognitions above the valid area will be "0", the number of unsuccessful facial recognitions above the valid area will be "1", and the number of successful facial recognitions within the valid coordinate range will be "0", which matches the determination pattern of record 1060 in which pattern ID "E" is registered in field 1061.

The judgment pattern with pattern ID "A" is suitable, for example, for allowing multiple employees to enter an employee entrance or a conference room, provided that there are no third parties other than employees. The judgment pattern with pattern ID "F" is suitable for areas 5 that require a high level of security, such as laboratories, where users are required to enter alone. The judgment pattern with pattern ID "E" denies entry to unauthorized users only, and can be used in areas 5 with various security levels.

In the example shown in FIG. 5(B), the entire captured image 30 is set to the valid coordinate range 33. In this example, two face images 31 are detected from the captured image 30, and the size of both of these face images 31 is equal to or larger than the valid area 32, and both of their center coordinates are within the valid coordinate range 33.

When the facial recognition results of both of these face images 31 are successful, the number of successful face recognitions is "2", the number of unsuccessful face recognitions is "0", the number of successful face recognitions above the valid area is "2", the number of unsuccessful face recognitions above the valid area is "0", and the number of successful face recognitions within the valid coordinate range is "2", which match the judgment pattern of record 1060 in which pattern ID "A" is registered in field 1061 and the judgment pattern of record 1060 in which pattern ID "G" is registered in field 1061. Also, when the facial recognition result of one face image 31 is successful and the facial recognition result of the other face image 31 is unsuccessful, the number of successful face recognitions is "1", the number of unsuccessful face recognitions is "1", the number of successful face recognitions above the valid area is "1", the number of unsuccessful face recognitions above the valid area is "1", the number of unsuccessful face recognitions above the valid area is "1", and the number of successful face recognitions within the valid coordinate range is "1", which match the judgment pattern of record 1060 in which pattern ID "C" is registered in field 1061.

The judgment pattern with pattern ID "G" is suitable for areas 5 that require a high level of security, such as server rooms and archive rooms, where entry of authorized staff is permitted on the condition that they are accompanied by a person in charge who is also authorized to enter. The judgment pattern with pattern ID "C" is suitable for areas such as reception rooms where authorized staff are permitted to enter accompanied by a third party other than staff who is not authorized to enter.

In the example shown in FIG. 6(A), the entire captured image 30 is set to the valid coordinate range 33. In this example, two face images 31 are detected from the captured image 30, and the center coordinates of these face images 31 are both within the valid coordinate range 33, but one of them is equal to or larger than the valid area 32, while the other is smaller than the valid area 32.

If the facial authentication result of a facial image 31 with a valid area of 32 or more is successful, and the facial authentication result of a facial image 31 with a valid area less than 32 is unsuccessful, the number of successful facial authentications will be "1", the number of unsuccessful facial authentications will be "1", the number of successful facial authentications equal to or greater than the valid area will be "1", the number of unsuccessful facial authentications equal to or greater than the valid area will be "0", and the number of successful facial authentications within the valid coordinate range will be "1", which matches the judgment pattern of record 1060 in which pattern ID "B" is registered in field 1061.

The judgment pattern with pattern ID "B" is applied when there is a possibility of tailgating, and entry is restricted even for users who are permitted to enter area 5.

In the example shown in FIG. 6(B), the valid coordinate range 33 is set in the upper center of the captured image 30. In this example, two face images 31 are detected from the captured image 30, and the sizes and center coordinates of these face images 31 are such that one is less than the valid area 32 and within the valid coordinate range 33, and the other is equal to or greater than the valid area 32 and outside the valid coordinate range 33.

If the facial authentication result of the facial image 31 that is less than the valid area 32 and within the valid coordinate range 33 is successful, and the facial authentication result of the facial image 31 that is greater than or equal to the valid area 32 and outside the valid coordinate range 33 is unsuccessful, the number of successful facial authentications will be "1", the number of unsuccessful facial authentications will be "1", the number of successful facial authentications greater than or equal to the valid area will be "0", the number of unsuccessful facial authentications greater than or equal to the valid area will be "1", and the number of successful facial authentications within the valid coordinate range will be "1", which matches the judgment pattern of record 1060 in which pattern ID "D" is registered in field 1061.

The judgment pattern with pattern ID "D" is suitable for a case where a caregiver who is permitted to enter a nursing care facility, etc., accompanies a wheelchair-bound person requiring care who is not permitted to enter the facility.

Note that depending on the security level of the area 5 to which it is applied, the same judgment pattern may be associated with different actions. In such cases, different pattern IDs are assigned to the judgment patterns.

Let's return to Figure 2 to continue the explanation.

The application information storage unit 107 stores application information for each camera 3, the application information including the judgment pattern to be applied to the area 5 in which the camera 3 is installed.

Figure 7 is a schematic diagram showing an example of registered contents in the application information storage unit 107.

As shown in the figure, the application information storage unit 107 stores an application information record 1070 for each camera 3. The application information record 1070 has a field 1071 in which the camera ID of the camera 3 is registered, a field 1072 in which the entry permission pattern for the area 5 in which the camera 3 is installed is registered, a field 1073 in which the pattern ID of at least one judgment pattern to be applied to this area 5 is registered together with the priority order, and fields 1074 and 1075 in which the autolock ID, which is identification information for the autolock 2 installed in this area 5, and the guidance output device ID, which is identification information for the guidance output device 4, are registered, respectively.

When multiple pattern IDs are registered in field 1073, care must be taken to avoid inconsistencies, such as registering patterns with different pattern IDs but the same judgment pattern content (different associated actions) in the same field 1073.

The application pattern search unit 108 refers to the judgment pattern storage unit 106 and searches for a judgment pattern (hereinafter referred to as an application pattern) that matches the face authentication results by the face authentication unit 103 and the evaluation results by the evaluation unit 105 for each face image detected by the face image detection unit 101 from the image captured by the camera 3 from among the judgment patterns whose pattern IDs are registered in the application information records 1070 linked to this camera 3 and stored in the application information storage unit 107.

The action control unit 109 identifies an action for the autolock 2 and guidance output device 4 from the judgment pattern record 1060 linked to the pattern ID of the application pattern searched by the application pattern search unit 108 and stored in the judgment pattern storage unit 106, and identifies the autolock 2 and guidance output device 4 installed in the same area 5 as the camera 3 from the application information record 1070 linked to the camera ID of the camera 3 that transmitted the captured image from which the face image was detected by the face image detection unit 101 and stored in the application information storage unit 107. Then, it controls the release command transmission unit 110 and the guidance output command transmission unit 111 to execute the identified action for the identified autolock 2 and guidance output device 4.

The unlock command sending unit 110 sends an unlock command to the autolock 2 according to the instructions of the action control unit 109.

The guidance output command transmission unit 111 transmits various guidance output commands to the guidance output device 4 according to instructions from the action control unit 109.

The functional configuration of the above-mentioned authentication device 1 may be realized in hardware using an integrated logic IC such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), or may be realized in software using a computer such as a DSP (Digital Signal Processor). Alternatively, it may be realized by at least one general-purpose computer such as a PC (Personal Computer) that includes a CPU (Central Processing Unit), memory, an auxiliary storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), and a communication interface such as a NIC (Network Interface Card) or a wireless LAN (Local Area Network) adapter, and the CPU loads a specific program from the auxiliary storage device into the memory and executes it.

Figure 8 is a flow diagram for explaining the operation of authentication device 1.

This flow starts when the face image detection unit 101 monitors the captured images sent from each camera 3 via the network communication unit 100 and detects a face image continuously for a predetermined period of time (e.g., 2 seconds) from the captured images of any of the cameras 3.

First, the face image detection unit 101 extracts the features, lower left corner coordinates, and upper right corner coordinates of each face image detected continuously for a predetermined period of time from the images captured by the camera 3 (S10). Then, the features, lower left corner coordinates, and upper right corner coordinates of each face image are passed to the face authentication unit 103 together with the camera ID of the camera 3 that captured the image and that was added to the captured image.

The face authentication unit 103 then performs face authentication on the features of each face image using face authentication information that is associated with the camera ID of the original camera 3 and stored in the face authentication information storage unit 102 (S11). Then, the face authentication result, the lower left corner coordinates, and the upper right corner coordinates of each face image are passed to the evaluation unit 105 together with the camera ID of the original camera 3.

In response, the evaluation unit 105 uses the coordinates of the lower left corner and the coordinates of the upper right corner of each face image to identify the size of each face image (the number of pixels in the rectangular area identified by the coordinates of the lower left corner and the coordinates of the upper right corner) (S12). Then, for each face image, it is evaluated whether or not the size is equal to or larger than the activation area registered in the record 1040 of the activation information that is linked to the camera ID of the camera 3 that captured the image and stored in the activation information storage unit 104 (S13).

The evaluation unit 105 also uses the lower left and upper right corner coordinates of each face image for which the face authentication result is successful to identify the center coordinates of each of these face images (the center coordinates of the line connecting the lower left and upper right corner coordinates) (S14). Then, for each face image for which the face authentication result is successful, it is evaluated whether or not the center coordinates are within the activation coordinate range registered in the activation information record 1040 linked to the camera ID of the original camera 3 and stored in the activation information storage unit 104 (S15).

Next, the evaluation unit 105 passes the face authentication results for each face image, the evaluation results for whether or not the area is equal to or larger than the effective area of each face image, and the evaluation results for whether or not the face authentication results are within the effective coordinate range of each face image for which face authentication is successful, together with the camera ID of the original camera 3, to the application pattern search unit 108. In response to this, the application pattern search unit 108 performs an application pattern search process described below, and searches for an application pattern from among the determination patterns whose pattern IDs are registered in the application information records 1070 linked to the camera ID of the original camera 3 and stored in the application information storage unit 107 (S16).

Next, if the application pattern search unit 108 fails to search for an application pattern (NO in S17), it ends this flow. On the other hand, if the application pattern search unit 108 is successful in searching for an application pattern (YES in S17), it identifies actions for the autolock 2 and the guidance output device 4 from the judgment pattern record 1060 linked to the pattern ID of this application pattern and stored in the judgment pattern storage unit 106 (S18). It also identifies the autolock ID and guidance output device ID from the application information record 1070 linked to the camera ID of the shooting source camera 3 and stored in the application information storage unit 107 (S19). Then, the application pattern search unit 108 passes the actions for the autolock 2 and the guidance output device 4 to the action control unit 109 together with the autolock ID and guidance output device ID.

In response to this, the action control unit 109 controls the release command sending unit 110 and the guidance output command sending unit 111 to execute a predetermined action on the autolock 2 and guidance output device 4 identified by the autolock ID and guidance output device ID (S20).

Specifically, if the action for autolock 2 is "unlock," the action control unit 109 notifies the unlock command sending unit 110 of an instruction to send an unlock command accompanied by the autolock ID, and if the action for autolock 2 is "reject unlock," the action control unit 109 does not notify the unlock command sending unit 110 of an instruction to send an unlock command accompanied by the autolock ID. When the unlock command sending unit 110 receives an instruction to send an unlock command accompanied by the autolock ID from the action control unit 109, it sends an unlock command to the autolock 2 identified by the autolock ID via the network communication unit 100, causing the autolock 2 to be unlocked.

The action control unit 109 also notifies the guidance output command sending unit 111 of the guidance type specified in the action for the guidance output device 4, together with the guidance output device ID. In response to this, the guidance output command sending unit 111 sends an output command for the specified guidance type to the guidance output device 4 identified by this guidance output device ID via the network communication unit 100, causing the guidance output device 4 to output guidance of this guidance type.

For example, if the guidance type is "Entry permitted", an entry permission message such as "The door will open. Please enter" is displayed and/or output by voice from the guidance output device 4. If the guidance type is "Entry permitted, caution against accompanying persons", an entry permission message such as "The door will open. Please enter with your accompanying persons" is displayed and/or output by voice from the guidance output device 4, encouraging entry with accompanying persons. If the guidance type is "Entry denied", an entry refusal message such as "Entry denied. The door will not open" is displayed and/or output by voice from the guidance output device 4. If the guidance type is "Entry denied, caution against tailgating", an entry refusal message such as "Entry denied. Be careful of persons who may enter illegally" is displayed and/or output by voice from the guidance output device 4, encouraging caution against tailgating.

Figure 9 is a flow diagram for explaining the application pattern search process S16 shown in Figure 8.

First, the application pattern search unit 108 counts the number of successful face recognitions, the number of unsuccessful face recognitions, the number of successful face recognitions that are equal to or greater than the valid area, the number of unsuccessful face recognitions that are equal to or greater than the valid area, and the number of successful face recognitions within the valid coordinate range, based on the face recognition results for each face image received from the evaluation unit 105 together with the camera ID of the camera 3 that captured the image, the evaluation results of whether or not the area is equal to or greater than the valid area for each face image, and the evaluation results of whether or not the face recognition results are within the valid coordinate range for each face image for which face recognition is successful (S160).

Next, the application pattern search unit 108 sets the counter value N to 1 (S161). Then, it identifies the pattern ID of the judgment pattern of priority order N from the application information record 1070 that is linked to the camera ID of the shooting source camera 3 received from the evaluation unit 105 and stored in the application information storage unit 107, and further identifies the judgment pattern of priority order N from the record 1060 that is linked to the identified pattern ID and stored in the judgment pattern storage unit 106 (S162).

Next, the application pattern search unit 108 determines whether the counting results of S160 (the number of successful face recognitions, the number of unsuccessful face recognitions, the number of successful face recognitions that are greater than or equal to the valid area, the number of unsuccessful face recognitions that are greater than or equal to the valid area, and the number of successful face recognitions within the valid coordinate range) match the determination pattern of priority order N (S163).

If the counting result of S160 matches the judgment pattern of priority order N (YES in S163), the application pattern search unit 108 detects this judgment pattern as an application pattern (S164). On the other hand, if the counting result of S160 does not match the judgment pattern of priority order N (NO in S163), the application pattern search unit 108 increments the counter value N by one (S165) and determines whether or not the pattern ID of the judgment pattern of priority order N is registered in the application information record 1070 linked to the camera ID of the shooting source camera received from the evaluation unit 105 and stored in the application information storage unit 107 (S166). If the pattern ID of the judgment pattern of priority order N is registered (YES in S166), the process returns to S162, and if it is not registered (NO in S166), the process ends without detecting it as an application pattern.

The above describes one embodiment of the present invention.

In this embodiment, face recognition processing is performed for each face image detected from the image captured by camera 3 installed to capture the area in front of the entrance to area 5. If face recognition is successful, it is evaluated whether the size of the face image is equal to or larger than the valid area, and whether the center coordinates of the face image are located within the valid coordinate range. If face recognition is unsuccessful, it is evaluated whether the size of the face image is equal to or larger than the valid area. Then, an action linked to a judgment pattern that matches the face recognition result, size, and center coordinate evaluation result of each face image is executed on auto lock 2 and guidance output device 4 installed in area 5.

Therefore, according to this embodiment, by changing the activation area, activation coordinate range, and judgment pattern, it can be applied to entrance management to area 5, where various security levels are required.

For example, in this embodiment, the determination pattern of pattern ID "A" in which the number of successful face authentications is 1 or more, the number of unsuccessful face authentications is 0, the number of successful face authentications for face images with an area larger than the valid area is 1 or more, the number of unsuccessful face authentications for face images with an area larger than the valid area is 0, and the number of successful face authentications for face images within the valid coordinate range is 1 or more is stored in association with the unlocking of the autolock 2 and the output of an entry permission message from the guidance output device 4. Therefore, for example, the authentication system according to this embodiment can be used in an area 5 in an employee entrance, conference room, etc., where multiple employees are allowed to enter at the same time, provided that there are no third parties other than employees.

In addition, in this embodiment, the judgment pattern of pattern ID "F" in which the number of successful face authentications is 1, the number of unsuccessful face authentications is 0, the number of successful face authentications for face images larger than the valid area is 1, the number of unsuccessful face authentications for face images larger than the valid area is 0, and the number of successful face authentications for face images within the valid coordinate range is 1 is stored in association with the unlocking of the autolock 2 and the output of an entry permission message from the guidance output device 4, so that the authentication system according to this embodiment can be used in areas 5 that require a high level of security, such as laboratories, and require users to enter alone.

In addition, in this embodiment, the determination patterns of pattern ID "G" in which the number of successful face authentications is 2 or more, the number of unsuccessful face authentications is 0, the number of successful face authentications for face images with an area larger than the valid area is 2 or more, the number of unsuccessful face authentications for face images with an area larger than the valid area is 0, and the number of successful face authentications for face images within the valid coordinate range is 2 or more are stored in association with the unlocking of the autolock 2 and the output of an entry permission message from the guidance output device 4. Therefore, the authentication system according to this embodiment can be used in areas 5 that require a high level of security, such as server rooms and archive rooms, where entry is permitted for authorized staff members on the condition that they are accompanied by a person in charge who is authorized to enter the area.

In addition, in this embodiment, the judgment pattern of pattern ID "B" in which the number of successful face recognitions is 1 or more, the number of unsuccessful face recognitions is 1 or more, the number of successful face recognitions for face images larger than the valid area is 1 or more, the number of unsuccessful face recognitions for face images larger than the valid area is 0, and the number of successful face recognitions for face images within the valid coordinate range is 1 or more is stored in association with the refusal to unlock the autolock 2 and the output of an entry refusal message that warns against tailgating. Therefore, when there is a possibility of so-called tailgating, the authentication system according to this embodiment can be used in area 5, where entry is restricted even for users who are permitted to enter area 5.

In addition, in this embodiment, the determination pattern of pattern ID "C" in which the number of successful face authentications is 1 or more, the number of unsuccessful face authentications is 1 or more, the number of successful face authentications of face images larger than the valid area is 1 or more, the number of unsuccessful face authentications of face images larger than the valid area is 1 or more, and the number of successful face authentications of face images within the valid coordinate range is 1 or more is stored in association with the unlocking of the autolock 2 and the output of an entry permission message from the guidance output device 4 that encourages entry with a companion, so that the authentication system according to this embodiment can be used in areas 5 such as reception rooms where staff members who are permitted to enter are permitted to enter accompanied by third parties other than staff members who are not permitted to enter.

In addition, in this embodiment, the judgment pattern of pattern ID "D" in which the number of successful face authentications is 1, the number of unsuccessful face authentications is 1, the number of successful face authentications for face images larger than the valid area is 0, the number of unsuccessful face authentications for face images larger than the valid area is 1, and the number of successful face authentications for face images within the valid coordinate range is 1 is stored in association with the unlocking of the autolock 2 and the output of an entry permission message from the guidance output device 4 that encourages entry with a companion. Therefore, in a care facility or the like, the authentication system according to this embodiment can be used in area 5 where a caregiver who is permitted to enter is permitted to enter accompanied by a wheelchair-bound person who is not permitted to enter.

The present invention is not limited to the above-described embodiment, and many variations are possible within the scope of the invention.

For example, in the above embodiment, the face image detection unit 101 monitors the captured images sent from each camera 3 via the network communication unit 100, and when a face image is continuously detected from the captured images of any of the cameras 3 for a predetermined time, the flow shown in FIG. 8 is executed to search for an application pattern, and an action linked to the application pattern is executed for the auto lock 2 and the guidance output device 4 installed in the same area 5 as the camera 3 that transmitted the captured images. However, the present invention is not limited to this. Each time a face image is continuously detected from the captured images of the camera 3 for a predetermined time, S10 to S17 of the flow shown in FIG. 8 are executed to search for an application pattern, and when the same application pattern is detected from the captured images of the same camera 3 a predetermined number of times in succession, S18 to S20 of the flow shown in FIG. 8 are executed for this application pattern, and the action linked to the application pattern may be executed for the auto lock 2 and the guidance output device 4 installed in the same area 5 as the camera 3 that transmitted the captured images. In this way, the accuracy of the search for the application pattern can be improved, and the desired action can be executed more correctly for the auto lock 2 and the guidance output device 4.

In the above embodiment, a subfield may be provided for each applicable time period in field 1073 of application information storage unit 107, and at least one pattern ID of a judgment pattern may be registered in each subfield together with a priority order. In step S162 of the flow shown in FIG. 9, the pattern ID of a judgment pattern with priority order N registered in a subfield of an applicable time period including the current time may be identified from field 1073 of record 1070 of application information stored in application information storage unit 107 in association with the camera ID of the shooting source camera received from evaluation unit 105, and the judgment pattern with priority order N of record 1060 stored in judgment pattern storage unit 106 in association with the identified pattern ID may be identified. In this way, the security level to be applied during the day and at night may be changed even in the same area 5.

In the above embodiment, an example is shown in which the authentication system according to the present embodiment is applied to the entry management of each area 5 in a facility with multiple areas 5, but the present invention can be widely applied to the entry and/or exit management of area 5. When the present invention is applied to the exit management of area 5, the autolock 2 is installed at the door at the exit of area 5, the camera 3 is installed so as to capture an image in front of the exit of area 5, and the guidance output device 4 is installed at the exit of area 5.

1: Authentication device 2-1 to 2-n: Autolock 3-1 to 3-n: Camera 4-1 to 4-n: Guidance output device 5-1 to 5-n: Area 100: Network communication unit 101: Face image detection unit 102: Face authentication information storage unit 103: Face authentication unit 104: Activation information storage unit 105: Evaluation unit 106: Determination pattern storage unit 107: Application information storage unit 108: Application pattern search unit 109: Action control unit 110: Release command transmission unit 111: Guidance output command transmission unit

Claims (13)

An authentication system for authenticating users in a predetermined area, comprising:
An electronic lock installed on an entrance or exit door of the specified area;
A camera installed to capture an image of an area in front of an entrance or exit door of the specified area;
A guidance output device installed at an entrance or exit door of the predetermined area;
an authentication device that authenticates a user who is attempting to enter or leave the specified area by using an image captured by the camera;
The authentication device includes:
a face image detection means for detecting a face image from an image captured by the camera and extracting a feature amount of the face image together with information capable of identifying a size and coordinates of the face image;
a face authentication information storage means for storing face authentication information of users who are permitted to use the predetermined area;
a face authentication means for performing face authentication for each face image detected from the image captured by the camera by the face image detection means, based on a feature amount extracted from the face image and face authentication information stored in the face authentication information storage means;
a validation information storage means for storing a validation area, which is the size of a face image to be recognized as a user of the predetermined area, and a validation coordinate range, which is a coordinate range;
an evaluation means for evaluating, for each face image detected by the face image detection means from the captured image of the camera, if the authentication result by the face authentication means is face authentication success, whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means, and evaluating whether or not the coordinates of the face image are within the validation coordinate range stored in the validation information storage means, if the authentication result by the face authentication means is face authentication failure, evaluating whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means;
a determination pattern storage means for storing, as determination patterns, patterns of the number of successful face authentications, the number of unsuccessful face authentications, the number of successful face authentications of face images equal to or larger than the valid area, the number of unsuccessful face authentications of face images equal to or larger than the valid area, and the number of successful face authentications of face images within the valid coordinate range, linked to actions for the electronic lock and the guidance output device;
an action control means for searching the judgment pattern storage means for a judgment pattern that matches the face authentication result by the face authentication means and the evaluation result by the evaluation means for each face image detected from the image captured by the camera by the face image detection means, and for controlling the electronic lock and the guidance output device to execute the action linked to the searched judgment pattern. 2. The authentication system according to claim 1,
The judgment pattern storage means includes:
an authentication system characterized in that the judgment patterns, which set the number of successful face authentications to 1 or more, the number of unsuccessful face authentications to 0, the number of successful face authentications for face images larger than the valid area to 1 or more, the number of unsuccessful face authentications for face images larger than the valid area to 0, and the number of successful face authentications for face images within the valid coordinate range to 1 or more, are stored in association with the action including unlocking the electronic lock. 2. The authentication system according to claim 1,
The judgment pattern storage means includes:
an authentication system characterized in that the judgment patterns, which set the number of successful face authentications to 1, the number of unsuccessful face authentications to 0, the number of successful face authentications for face images larger than the valid area to 1, the number of unsuccessful face authentications for face images larger than the valid area to 0, and the number of successful face authentications for face images within the valid coordinate range to 1, are stored in association with the action including unlocking the electronic lock. 2. The authentication system according to claim 1,
The judgment pattern storage means includes:
an authentication system characterized in that the judgment patterns, which set the number of successful face authentications to 2 or more, the number of unsuccessful face authentications to 0, the number of successful face authentications for face images larger than the valid area to 2 or more, the number of unsuccessful face authentications for face images larger than the valid area to 0, and the number of successful face authentications for face images within the valid coordinate range to 2 or more, are stored in association with the action including unlocking the electronic lock. 2. The authentication system according to claim 1,
The judgment pattern storage means includes:
an authentication system characterized in that the judgment patterns, which include a number of successful face authentications of 1 or more, a number of unsuccessful face authentications of 1 or more, a number of successful face authentications of 1 or more for face images larger than the valid area, a number of unsuccessful face authentications of 0 for face images larger than the valid area, and a number of successful face authentications of 1 or more for face images within the valid coordinate range, are stored in association with the actions, which include a refusal to unlock the electronic lock and an output from the guidance output device of guidance to the effect that someone is attempting to enter or leave the room illegally. 2. The authentication system according to claim 1,
The judgment pattern storage means includes:
an authentication system characterized in that the judgment patterns, which include a number of successful face authentications of 1 or more, a number of unsuccessful face authentications of 1 or more, a number of successful face authentications of 1 or more for face images larger than the valid area, a number of unsuccessful face authentications of 1 or more for face images larger than the valid area, and a number of successful face authentications of 1 or more for face images within the valid coordinate range, are stored in association with the actions including unlocking the electronic lock and outputting guidance from the guidance output device to the effect that someone is attempting to enter or leave the room together. 2. The authentication system according to claim 1,
The judgment pattern storage means includes:
an authentication system characterized in that the judgment patterns, which set the number of successful face authentications to 1, the number of unsuccessful face authentications to 1, the number of successful face authentications for face images larger than the valid area to 0, the number of unsuccessful face authentications for face images larger than the valid area to 1, and the number of successful face authentications for face images within the valid coordinate range to 1, are stored in association with the actions including unlocking the electronic lock and outputting guidance from the guidance output device to the effect that someone is attempting to enter or leave the room together. An authentication system according to any one of claims 1 to 7,
The action control means
An authentication system characterized by controlling the electronic lock and the guidance output device to execute the action linked to the judgment pattern when the judgment pattern searched for from the judgment pattern storage means is identical a predetermined number of times in succession for each of a plurality of images captured by the camera. An authentication system according to any one of claims 1 to 8,
The judgment pattern storage means includes:
the determination pattern associated with the action for the electronic lock and the guidance output device is stored together with an applicable time period;
The action control means
An authentication system characterized by searching for a judgment pattern that matches the face authentication result by the face authentication means and the evaluation result by the evaluation means for each face image detected from the image captured by the camera by the face image detection means from the judgment patterns stored in the judgment pattern storage means in association with an applicable time period including the current time, and controlling the electronic lock and the guidance output device to execute the action linked to the searched judgment pattern. An authentication system according to any one of claims 1 to 9,
There are a plurality of the predetermined areas,
The electronic lock is installed at an entrance or exit door of each of the predetermined areas,
The camera is installed for each of the predetermined areas so as to capture an image of an area in front of an entrance or exit door of the predetermined area,
the guidance output device is installed at an entrance or an exit of each of the predetermined areas,
the face authentication information storage means stores, for each of the predetermined areas, face authentication information of users who are permitted to use the predetermined areas;
the validation information storage means stores the validation area and the validation coordinate range for each of the predetermined areas,
The judgment pattern storage means includes:
for each of the predetermined areas, the number of successful face authentications, the number of unsuccessful face authentications, the number of successful face authentications in which the face image is equal to or larger than the valid area, the number of unsuccessful face authentications in which the face image is equal to or larger than the valid area, and the number of successful face authentications in which the face image is within the valid coordinate range are stored as the determination pattern,
The face authentication means includes:
performing face authentication for each face image detected by the face image detection means from the captured image of the camera, based on a feature amount extracted from the face image and face authentication information stored in the face authentication information storage means in association with the predetermined area in which the camera is installed;
The evaluation means includes:
for each face image detected by the face image detection means from the captured image of the camera, if the authentication result by the face authentication means is face authentication success, evaluate whether or not the size of the face image is equal to or larger than the validation area associated with the specified area in which the camera is installed and stored in the validation information storage means, and evaluate whether or not the coordinates of the face image are within the validation coordinate range associated with the specified area and stored in the validation information storage means; if the authentication result by the face authentication means is face authentication failure, evaluate whether or not the size of the face image is equal to or larger than the validation area associated with the specified area in which the camera is installed and stored in the validation information storage means;
The action control means
An authentication system characterized by searching for a judgment pattern that matches the face authentication result by the face authentication means and the evaluation result by the evaluation means for each face image detected from the image captured by the camera by the face image detection means from among the judgment patterns stored in the judgment pattern storage means in correspondence with the specified area in which the camera is installed, and controlling the electronic lock and the guidance output device to execute the action linked to the searched judgment pattern. An authentication device for authenticating a user in a predetermined area,
a face image detection means for detecting a face image from an image captured by a camera installed so as to capture an image of an area in front of an entrance or exit door of the specified area, and extracting a feature amount of the face image together with information capable of identifying a size and coordinates of the face image;
a face authentication information storage means for storing face authentication information of users who are permitted to use the predetermined area;
a face authentication means for performing face authentication for each face image detected from the image captured by the camera by the face image detection means, based on a feature amount extracted from the face image and face authentication information stored in the face authentication information storage means;
a validation information storage means for storing a validation area, which is the size of a face image to be recognized as a user of the predetermined area, and a validation coordinate range, which is a coordinate range;
an evaluation means for evaluating, for each face image detected by the face image detection means from the captured image of the camera, if the authentication result by the face authentication means is face authentication success, whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means, and evaluating whether or not the coordinates of the face image are within the validation coordinate range stored in the validation information storage means, if the authentication result by the face authentication means is face authentication failure, evaluating whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means;
a determination pattern storage means for storing, as determination patterns, patterns of the number of successful face authentications, the number of unsuccessful face authentications, the number of successful face authentications of face images equal to or larger than the valid area, the number of unsuccessful face authentications of face images equal to or larger than the valid area, and the number of successful face authentications of face images within the valid coordinate range, linked to actions for electronic locks and guidance output devices installed at the entrance or exit doors of the specified area;
an action control means for searching the judgment pattern storage means for the judgment pattern that matches the face authentication result by the face authentication means and the evaluation result by the evaluation means for each face image detected from the image captured by the camera by the face image detection means, and for controlling the electronic lock and the guidance output device to execute the action linked to the searched judgment pattern. An authentication method for authenticating a user in a predetermined area using an authentication device, comprising:
The authentication device includes:
detecting a face image from an image captured by a camera installed to capture an image of an area in front of an entrance or exit door of the specified area, and extracting a feature amount of the face image together with information capable of identifying a size and coordinates of the face image;
performing face authentication for each face image detected from the captured image of the camera, based on a feature amount extracted from the face image and face authentication information of a user who is permitted to use the predetermined area and which is registered in advance;
For each face image detected from the image captured by the camera, if the authentication result of the face authentication is successful, evaluate whether or not the size of the face image is equal to or larger than a valid area, which is the size of a face image to be recognized as a user of the specified area, and evaluate whether or not the coordinates of the face image are within a valid coordinate range, which is the coordinate range of a face image to be recognized as a user of the specified area; if the authentication result of the face authentication is unsuccessful, evaluate whether or not the size of the face image is equal to or larger than the valid area;
an authentication method comprising: searching for a judgment pattern that matches the authentication result of the face authentication for each face image detected from an image captured by the camera, the evaluation result of the valid area, and the evaluation result of the valid coordinate range when the face authentication is successful, from among patterns of the number of successful face authentications, the number of unsuccessful face authentications, the number of successful face authentications for face images larger than the valid area, the number of unsuccessful face authentications for face images larger than the valid area, and the number of successful face authentications for face images within the valid coordinate range, which are pre-registered as judgment patterns; and controlling an electronic lock and a guidance output device installed at an entrance or exit door of the specified area to execute an action that is pre-registered in association with the searched judgment pattern. A program for causing a computer to function as an authentication device for authenticating a user in a predetermined area,
The program is
The computer,
a face image detection means for detecting a face image from an image captured by a camera installed so as to capture an image of an area in front of an entrance or exit door of the specified area, and extracting a feature amount of the face image together with information capable of identifying a size and coordinates of the face image;
a face authentication information storage means for storing face authentication information of users permitted to use the predetermined area;
a face authentication means for performing face authentication for each face image detected by the face image detection means from the captured image of the camera, based on a feature amount extracted from the face image and face authentication information stored in the face authentication information storage means;
a validation information storage means for storing a validation area, which is the size of a face image to be recognized as a user of the predetermined area, and a validation coordinate range, which is a coordinate range;
an evaluation means for evaluating, for each face image detected by the face image detection means from the captured image of the camera, if the authentication result by the face authentication means is face authentication success, whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means, and evaluating whether or not the coordinates of the face image are within the validation coordinate range stored in the validation information storage means, if the authentication result by the face authentication means is face authentication failure, evaluating whether or not the size of the face image is equal to or larger than the validation area stored in the validation information storage means;
a determination pattern storage means for storing, as determination patterns, patterns of the number of successful face authentications, the number of unsuccessful face authentications, the number of successful face authentications of face images equal to or larger than the valid area, the number of unsuccessful face authentications of face images equal to or larger than the valid area, and the number of successful face authentications of face images within the valid coordinate range, linked to actions for electronic locks and guidance output devices installed at the entrance or exit doors of the specified area; and
A program characterized by functioning as an action control means that searches the judgment pattern storage means for the judgment pattern that matches the face authentication results by the face authentication means and the evaluation results by the evaluation means for each face image detected from the image captured by the camera by the face image detection means, and controls the electronic lock and the guidance output device to execute the action linked to the searched judgment pattern.

Images