JP7714906B2 - Object recognition device and control method for object recognition device - Google Patents

Description

The present invention relates to an object recognition device.

Facial recognition systems have long been available that recognize a user's face and determine whether the user has the authority to perform a specific action. For example, facial recognition systems are used as devices that authorize entry and exit to high-security areas, or devices that issue permission to use specific devices.

Patent Document 1 discloses technology that enables face recognition to be performed correctly even when the user is wearing a mask or has changed their hairstyle. By prioritizing the features of certain parts and reducing the features of other parts, faces can be recognized correctly even if there are differences from the data registered in the database.

Patent Document 2 is an invention that performs facial recognition when a captured image of a user's face differs from the image registered in a database due to a mask, sunglasses, hat, etc. As in the example above, when there is a difference between the captured image and the registered image in the database, rather than immediately failing facial recognition, the technology divides the captured image into multiple areas, notifies the user of the areas where matching failed, and encourages re-recognition, thereby successfully achieving facial recognition.

Patent document 3 is an invention related to digital cameras, and is a technology that allows for subject recognition to be prioritized by defining subject priorities for each shooting mode, and for optical adjustments to be made quickly to suit the subject.

Patent document 4 describes an invention that reduces the size of a biometric pattern image, such as a vein pattern, by dividing the image into multiple regions, recognizing similar patterns in each region, and storing a number corresponding to the pattern for each region.

Patent document 5 describes an invention that performs facial recognition using a low-resolution image, and if the images are determined to be identical, performs facial recognition using a portion of a higher-resolution image, thereby gradually improving the accuracy of facial recognition while maintaining speed.

Japanese Patent Application Laid-Open No. 2007-257221 Japanese Patent Application Laid-Open No. 2019-168929 Japanese Patent Application Laid-Open No. 2007-068149 JP 2011-065315 A JP 2012-238059 A

However, with conventional technologies such as those described above, facial recognition takes a long time due to the size of the facial features that need to be matched or the large number of targets for facial recognition. This prevents users from moving on to their tasks quickly, causing stress for the user.

With conventional algorithms, processing can be sped up by lowering the resolution of the captured image, but the low resolution can sometimes result in false recognition. In addition to facial recognition, there is also a demand for object recognition.

Therefore, one aspect of the present invention aims to realize a device that can perform object recognition at high speed while maintaining recognition accuracy.

In order to solve the above problems, an object recognition device according to one aspect of the present invention comprises an image acquisition unit that acquires an image of a target object; an image extraction unit that creates a plurality of partial captured images by extracting images of a predetermined plurality of portions of the target object from the captured image; a matching unit that matches the partial captured images with a plurality of partial registered images that are extracted from a predetermined plurality of portions of a registered object to be recognized and are stored in a memory unit, in order according to the priorities associated with the partial registered images; and a recognition output unit that outputs a recognition result of the target object when the matching performed by the matching unit satisfies a predetermined condition.

With the above configuration, it is possible to determine whether a target object is a registered object by repeatedly matching a partial captured image with a partial registered image in the order of priority associated with the partial registered image until a predetermined condition is met. Since the recognition result is output when the predetermined condition is met, it is possible to output the recognition result of the target object quickly.

The matching unit may perform matching by calculating the similarity between the partial registration image and the partial captured image and comparing it with a predetermined threshold.

With the above configuration, the partial registration image can be matched with the partial captured image by comparing the similarity with a threshold value.

The matching unit may perform matching in an order according to the priority, and when it determines that the partial registration image and the partial captured image corresponding to the partial registration image substantially match, the recognition output unit may output the recognition result of the target object.

With the above configuration, when the matching unit determines that there is a partial captured image that substantially matches a partial registered image, it can output a recognition result that the target object is a registered object.

The storage unit may store the partial registered images and the priorities of the partial registered images for multiple registered objects, and the matching unit may perform matching against the partial registered images of all registered objects at a certain priority, and then perform matching against the partial registered images of all registered objects at a next priority.

With the above configuration, partial captured images are compared for all registered objects in order of priority, and when a substantial match is determined, a recognition result can be output indicating that the target object is a registered object.

The matching unit may perform matching in an order according to the priority, and when it determines that the partial registration image and the partial captured image corresponding to the partial registration image do not substantially match, the recognition output unit may output the recognition result of the target object.

With the above configuration, when the matching unit determines that there is a partial captured image that does not substantially match the partial registered image, it can output a recognition result that the target object is not a registered object.

The storage unit stores the partial registration images and the priorities of the partial registration images for multiple target objects, and the matching unit performs matching against the partial registration images of all the registered objects at a certain priority, and then performs matching against the partial registration images of all the registered objects at a next priority. If it is determined that the partial registration image and the partial captured image corresponding to that partial registration image do not substantially match, it is not necessary to perform matching for the registered object at the next priority.

With the above configuration, partial captured images are compared in order of priority for all registered objects, and when it is determined that none of the registered objects substantially match, a recognition result can be output indicating that the target object is not a registered object.

The matching unit may further include a priority setting unit that stores the similarity calculation results as a similarity history and sets the priority of the partial registration image based on the similarity history.

With the above configuration, the priority of partial registered images can be automatically set according to the similarity history. Therefore, by setting the priority of partial registered images according to the similarity trend, it is possible to more quickly output a recognition result indicating whether the target object is a registered object.

The system may further include a priority setting unit that sets the priority of the partial registration image based on user input.

With the above configuration, the priority of partial registration images can be set arbitrarily based on user input. This allows you to change the priority of areas you want to intentionally lower or raise.

The target object may be a person's face.

With the above configuration, for example, the object may be a person's face, and can be used for early face recognition.

An object recognition device according to another aspect includes an image acquisition step of acquiring an image of a target object; an image extraction step of creating a plurality of partial captured images by extracting images of a plurality of predetermined portions of the target object from the captured image; a matching step of matching the partial captured images with a plurality of partial registered images, which are extracted images of a plurality of predetermined portions of a registered object to be recognized and are stored in a storage unit, in an order according to the priorities associated with the partial registered images; and a recognition output step of outputting a recognition result of the target object when the matching in the matching unit satisfies a predetermined condition.

The object recognition device according to each aspect of the present invention may be realized by a computer. In this case, the object recognition program for the object recognition device, which causes the computer to operate as each unit (software element) of the object recognition device, thereby realizing the object recognition device on the computer, and the computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention.

According to one aspect of the present invention, a partial captured image extracted from a captured image is matched with a pre-registered partial registered image using a priority associated with the partial registered image, and recognition can be terminated early when the matching result satisfies a predetermined condition.

1 is a block diagram showing the configuration of a main part of a face recognition system according to a first embodiment. 3A and 3B are model diagrams showing a captured image and a partial captured image according to the first embodiment; 10 is a flowchart showing a flow of registering a partial registration image. 10 is a flowchart showing the flow of an operation in Operation Example 1 according to the first embodiment. 10 is a flowchart showing the flow of operations in an operation example 2 according to the first embodiment. 10 is a flowchart showing the flow of operations in Operation Example 3 according to the first embodiment. 10 is a flowchart showing the flow of operations in Operation Example 4 according to the first embodiment. FIG. 10 is a block diagram showing the configuration of the main parts of a face recognition system according to a second embodiment. 10A and 10B are model diagrams showing a captured image and a partial captured image according to a third embodiment.

[Embodiment 1]
Hereinafter, an embodiment according to one aspect of the present invention (hereinafter also referred to as "the present embodiment") will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and description thereof will not be repeated.

1 is a block diagram showing the configuration of the main parts of a face recognition system 100 according to embodiment 1. The face recognition system 100 includes a camera 20 that captures an image including a face, a face recognition device (object recognition device) 10 that performs face recognition on the captured image, and an electromagnetic lock 30 controlled by the face recognition device 10.

The face recognition device 10 extracts a partial captured image from the captured image that corresponds to a pre-registered partial registration image, and performs matching by comparing the partial captured image with the partial registration image. Here, multiple partial registration images are prepared, and a priority is assigned to each partial registration image. Matching for each partial registration image is performed in order according to priority.

The face recognition device 10 has two main modes: a mode that prioritizes the speed of person recognition, in which the person being recognized is a registered person, and a mode that prioritizes the speed of other person recognition, in which the person being recognized is not a registered person. In the mode that prioritizes the speed of person recognition, once a pair of a set of partial captured images and a partial registered image are recognized as the same image, the face is recognized as being substantially the same. In contrast, in the mode that prioritizes the speed of other person recognition, once a pair of a set of partial captured images and a partial registered image are recognized as being different images, the face is recognized as not being substantially the same. If the faces are recognized as being substantially the same, the face recognition device 10 releases the electromagnetic lock 30.

In other words, the face recognition device 10 performs matching in order of priority, and can output the face recognition results once the recognition results for one set of partial captured images and partial registration images are obtained. Therefore, the recognition results can be output earlier than when face recognition is performed on the entire captured image.

§2. Configuration example (Configuration of face recognition device)
The configuration of a face recognition system 100 will be described with reference to Fig. 1. The face recognition device 10 includes an image acquisition unit 11, a storage unit 12, an image extraction unit 13, a matching unit 14, and a recognition output unit 15.

Camera 20 is a camera that captures an image including the face of a target for facial recognition. Camera 20 outputs the captured image to image acquisition unit 11.

The electromagnetic lock 30 is an electronic lock controlled by the facial recognition device 10. The electromagnetic lock 30 is not limited to an electronic lock, but may be any program-controlled device or program, and unless it is unlocked, the functions of the device or program cannot be used.

For example, if an electromagnetic lock 30 is installed on a door, the door cannot be opened and someone cannot enter unless the electromagnetic lock 30 is unlocked by facial recognition. Furthermore, in an electronic device with a built-in camera 20, such as a smartphone, the smartphone will not accept operations unless the electromagnetic lock 30 is unlocked (disabled) by facial recognition.

The image acquisition unit 11 inputs the captured image. The image acquisition unit 11 outputs the input captured image to the image extraction unit 13.

The memory unit 12 stores multiple partial registration images for each registered person who is the target of face recognition. In addition to each partial registration image, the memory unit 12 also stores a priority, which determines the order in which face recognition is performed on the partial registration images. Note that the partial registration image and priority information may be stored in a memory unit external to the face recognition device 10, and the face recognition device 10 may read the partial registration image and priority information via various communication means.

The image extraction unit 13 extracts portions of the input captured image that correspond to the partial registration images, and sets these as partial captured images. The image extraction unit 13 outputs the extracted multiple partial captured images to the matching unit 14.

The matching unit 14 compares the input partial captured image with the partial registered image and calculates a numerical value called the degree of similarity, which indicates the degree of similarity between the two. The matching unit 14 then compares the similarity with a predetermined threshold to determine whether the images are substantially the same. The matching unit 14 also matches a partial registered image of a certain priority with the partial captured image corresponding to that partial registered image, and, if necessary, matches the partial registered image of the next priority with the partial captured image corresponding to that partial registered image. After performing the matching required for face recognition and confirming the matching result, the matching unit 14 outputs the matching result to the recognition output unit 15.

The recognition output unit 15 receives the input matching result and determines whether to issue a signal to unlock the electromagnetic lock 30. If the matching result indicates that the person being recognized is a registered person, the recognition output unit 15 unlocks the electromagnetic lock 30. On the other hand, if the matching result indicates that the person being recognized is not a registered person, the recognition output unit 15 does not unlock the electromagnetic lock 30.

(Extraction of Partial Captured Images from Captured Images)
2 is a model diagram showing a captured image 111 and partial captured images 112 according to the first embodiment. As shown in FIG. 2, the captured image 111 is an image that captures the entire face of a subject who is the target of face recognition. The partial captured images 112 are, for example, partial images 112a to 112e of five captured images 111. Each partial captured image is an image that partially captures a part that represents a characteristic part of the face.

For example, partial captured image 112a is an image of both eyes, partial captured image 112b is an image of the mouth, partial captured image 112c is an image of the hair, partial captured image 112d is an image of the nose, and partial captured image 112e is an image of the forehead. The ranges of partial captured images 112 (partial captured images 112a to 112e are collectively referred to as partial captured images 112) may overlap with each other. For example, in the example of Figure 2, partial captured image 112a and partial captured image 112e both include the eyebrows.

The partial captured image 112 is extracted from the captured image 111 by the image extraction unit 13, and the extracted range is the range corresponding to the partial registered image.

That is, the image extraction unit 13 extracts partial captured images 112 that correspond to the partial registration images from the captured image 111. At this time, in order to extract the partial captured images 112, pattern matching may be performed on the captured image 111 to derive the range of the partial captured images 112.

Furthermore, the method of extracting the partial captured image 112 is not limited to this method. The position of the partial area to be extracted from the captured image 111 may be determined in advance, and the corresponding partial area may be extracted to create the partial captured image 112.

(How to register a partial image)
3 is a flowchart showing the flow of registering partial registration images. In order to perform face recognition, partial registration images obtained by dividing a face image into parts must be registered in advance.

In S11, the camera 20 captures a facial image of the person to be registered. In S12, the captured facial image is extracted by facial features. Facial features include the eyes, mouth, nose, eyebrows, forehead, and hair, and an image may be extracted that captures each feature as well as the surrounding area. This task is performed by the user operating the facial recognition device 10, or automatically by the facial recognition device 10 according to predetermined rules.

In S13, each divided part is registered as a partial registration image in the memory unit 12. In S14, a priority, which indicates the order in which face recognition will be performed, is assigned to each partial registration image. Here, the priority may be assigned manually by the user, or automatically by the face recognition device 10 in accordance with predetermined rules.

§3. Operational Examples Below, we will explain four types of operational examples with concrete examples.

(Operation example 1: Prioritizing the speed of personal identification for one registered user)
4 is a flowchart showing the flow of operations in Operation Example 1 according to Embodiment 1. Operation Example 1 is an operation in which, when the partial registration image to be subjected to face recognition is a face image of one person, priority is given to the speed of recognizing the person who registered the partial registration image. For simplicity of explanation, the explanation will be given assuming that there are five partial registration images.

In S21, the camera 20 captures a facial image, and the image acquisition unit 11 acquires the facial image as a captured image.

In S22, the recognition output unit 15 performs a matching loop to match the partial registration images for each body part, from i=1 to i=5. Here, i is a natural number indicating the priority of the partial registration image.

In S23, the image extraction unit 13 extracts the partial captured image corresponding to the partial registered image with the i-th priority.

In S24, the matching unit 14 compares the partial captured image with the partial registered image with the i-th priority and performs matching. If the partial captured image and the partial registered image substantially match, the process proceeds to S25. If the partial captured image and the partial registered image do not substantially match, the process proceeds to S26.

In S25, the matching unit 14 recognizes that the subject is the same person as the individual registered person because the partial captured image and the partial registered image substantially match, and exits the matching loop. The recognition output unit 15 then unlocks the electromagnetic lock 30.

In S26, if i = 5, the matching unit 14 exits the matching loop or increments i to i = i + 1 and moves on to the next matching loop.

In S27, the recognition output unit 15 recognizes that the target person is not the same person as the registered person, and therefore the target person is different from the registered person. Therefore, the recognition output unit 15 does not unlock the electromagnetic lock 30.

The above processing allows for early recognition that the subject is the same person as the registered person as soon as it is determined that one partial captured image substantially matches a partial registered image. Furthermore, to ensure security, it is preferable to set a stricter similarity threshold for determining that a partial captured image and a partial registered image substantially match each other. Setting a stricter threshold reduces the possibility that a subject other than the registered person will be recognized as the same person through facial recognition.

Here, in S24, a condition branch is performed depending on whether one partial captured image substantially matches a partial registered image. If the condition is met (Yes in S24), the electromagnetic lock 30 is released (S25). However, it is not necessary to release the electromagnetic lock 30 based on the recognition result of one partial captured image. In other words, if a predetermined number of partial captured images substantially match the partial registered images, the subject may be recognized as the same person as the individually registered person. Security is improved by increasing the number of partial registered images required to determine that they are the same person. Here, it is preferable to set the number of partial registered images required to determine that they are the same person to be fewer than the number of partial registered images stored in the memory unit 12. This allows the matching result to be output more quickly than when matching with all partial registered images.

An actual example of Operation Example 1 is suitable for use when performing facial recognition on a device that is essentially used only by registered users. For example, facial recognition can be used for user recognition on electronic devices such as personally owned smartphones.

(Operation Example 2: Prioritizing recognition speed for one registered user)
5 is a flowchart showing the flow of operations in Operation Example 2 according to Embodiment 1. Operation Example 2 is an operation in which, when the partial registration image to be subjected to face recognition is a face image of one person, priority is given to the speed of other person recognition, which recognizes that the person is not the registrant of the partial registration image. For simplicity of explanation, the explanation will be given assuming that there are five partial registration images.

The processing steps S21 to S23 are the same as those in Operation Example 1, so a detailed explanation will be omitted.

In S34, the matching unit 14 compares the partial captured image with the partial registered image with the i-th priority and performs matching. If the partial captured image and the partial registered image substantially match, the process proceeds to S36. If the partial captured image and the partial registered image do not substantially match, the process proceeds to S35.

In S35, the matching unit 14 determines that the target person is a different person from the individually registered person because the partially captured image and the partially registered image do not substantially match, and exits the matching loop. Thereafter, the recognition output unit 15 does not unlock the electromagnetic lock 30.

If i = 5 in S36, the matching unit 14 exits the matching loop and proceeds to S37, or the matching unit 14 increments i to i = i + 1 and moves on to the next matching loop.

In S37, the recognition output unit 15 recognizes that the subject is the same person as the registered person because the partial captured images corresponding to all partial registration images substantially match. Therefore, the recognition output unit 15 unlocks the electromagnetic lock 30.

The above processing makes it possible to quickly recognize that the subject is not the same person as the enrolled person as soon as it is determined that one partial captured image does not substantially match the partial registered image. Furthermore, in order to prevent the subject from being mistakenly determined to be a non-enrolled person when the subject is a registered person, it is preferable to set a loose similarity threshold for determining that the partial captured image and the partial registered image match. Setting the threshold loosely may result in a subject who is not a registered person being determined to be a substantial match when comparing parts, but by using a large number of parts in the matching loop, it is possible to ultimately output the non-enrolled subject as a person different from the enrolled person.

Here, in S34, a condition branch is performed depending on whether one partial captured image substantially matches a partial registered image. If the condition is not met (No in S34), the recognition process is terminated. However, the process does not necessarily have to terminate with the recognition result based on one partial captured image. In other words, if a predetermined number of partial captured images do not match the partial registered images, it may be determined that the subject is not the same person as the registered person. Increasing the number of partial registered images required can reduce erroneous determinations. Here, it is preferable that the number of partial registered images required to determine that the subjects are not the same person is less than the number of partial registered images stored in the memory unit 12. This allows the matching result to be output more quickly than when matching is performed with all partial registered images.

An actual example of Operation Example 2 is its use in preventing unauthorized use of a device that is primarily intended for use only by registered users. For example, it can be used to improve security for unlocking a device using facial recognition when an electronic device such as a smartphone owned by an individual is lost.

(Operation Example 3: Prioritizing speed of personal identification when multiple users are registered)
6 is a flowchart showing the flow of operations in Operation Example 3 according to Embodiment 1. Operation Example 3 is an operation in which, when the partial registration image to be subjected to face recognition contains face images of multiple people, priority is given to the speed of identifying the person in the partial registration image. For simplicity of explanation, the multiple registered people will be assumed to be three people, and there will be five partial registration images for each registered person.

In S41, the camera 20 captures a facial image, and the image acquisition unit 11 acquires the facial image as a captured image.

In S42, the matching unit 14 performs a matching loop to match the partial registration images for each body part, from i=1 to i=5. Here, i is a natural number indicating the priority of the partial registration image.

In S43, the matching unit 14 performs a registrant loop to match each registrant from j=1 to j=3, where j is a natural number representing the registrant.

In S44, the image extraction unit 13 extracts a partial captured image corresponding to the partial registered image of the i-th priority for the j-th registrant.

In S45, the matching unit 14 compares the partial captured image with the partial registered image of the i-th priority for the j-th registrant and performs matching. If the partial captured image and the partial registered image substantially match, the process proceeds to S46. If the partial captured image and the partial registered image do not substantially match, the process proceeds to S47.

In S46, the matching unit 14 recognizes that the target person is the same person as one of the registered people because the partial captured image and the partial registered image substantially match, and exits the matching loop. The recognition output unit 15 then unlocks the electromagnetic lock 30.

In S47, if j = 3, the matching unit 14 exits the registrant loop or increments j to j = j + 1 and moves on to the next registrant loop.

In S48, if i = 5, the matching unit 14 exits the matching loop or increments i to i = i + 1 and moves on to the next matching loop.

In S49, the recognition output unit 15 recognizes that the target person is a different person from the multiple registered people because the target person does not substantially match any of the multiple registered people. Therefore, the recognition output unit 15 does not unlock the electromagnetic lock 30.

According to the above processing, when it is determined that one partial captured image substantially matches the partial registration image of one of the multiple registered individuals, the target person is recognized as one of the registered individuals and facial recognition is terminated. This allows for early facial recognition of the target person. Furthermore, to ensure security, it is preferable to set a stricter similarity threshold for determining that a partial captured image and a partial registration image substantially match each other. Setting a stricter threshold reduces the possibility that a target person who is not one of the multiple registered individuals will be recognized as the same person as one of the multiple registered individuals through facial recognition.

An actual example of Operation Example 3 is a device that is unlikely to match targets other than registered individuals, and is suitable for use in facial recognition with a device used by multiple registered individuals. For example, it could be used at an entrance gate where only registered individuals are allowed entry, where only authorized individuals are allowed entry.

(Operation Example 4: Prioritizing Recognition Speed of Others When Multiple Registered Users Are Enrolled)
7 is a flowchart showing the flow of operations in Operation Example 4 according to the first embodiment. Operation Example 4 is an operation in which, when the partial registration image to be subjected to face recognition contains face images of multiple people, priority is given to the speed of recognizing other people excluding the person in the partial registration image. For simplicity of explanation, the multiple registered people will be assumed to be three people, and each registered person will have five partial registration images.

The processing steps S41 to S44 are the same as those in Operation Example 3, so a detailed explanation will be omitted.

In S55, the matching unit 14 compares the partial captured image with the partial registered image of the i-th priority for the j-th registrant and performs matching. If the partial captured image and the partial registered image do not substantially match, the process proceeds to S56. If the partial captured image and the partial registered image substantially match, the process proceeds to S57.

In S56, the matching unit 14 determines that the target person is not the jth registrant. Therefore, in subsequent processing, the jth registrant is excluded from candidates, and subsequent matching is not performed.

In S57, if j = 3, the matching unit 14 exits the registrant loop and proceeds to S58, or increments j to j = j + 1 and moves on to the next registrant loop.

In S58, the matching unit 14 determines whether any candidate registrants remain among the multiple registrants. If no candidate registrants remain, the process proceeds to S59. If candidate registrants remain, the process proceeds to S60.

In S59, since there is no partial registered image that substantially matches the partial captured image, the matching unit 14 recognizes that the subject is a different person from any one of the multiple registered persons and exits the matching loop. After that, the recognition output unit 15 does not unlock the electromagnetic lock 30.

In S60, if i = 5, the matching unit 14 exits the matching loop and proceeds to S61, or increments i to i = i + 1 and moves on to the next matching loop.

In S61, the recognition output unit 15 recognizes that the target person is the same person as one of the multiple registered people because there is a registered person whose partial captured images substantially match all of the partial registration images. Therefore, the recognition output unit 15 unlocks the electromagnetic lock 30.

The above processing allows the subject to be recognized as a different person from the registrant as soon as it is determined that one partial captured image does not substantially match the partial registered image of any one of multiple registered persons, allowing the subject to be recognized as a different person at an early stage. However, to avoid mistakenly recognizing a registered person as a different person, it is preferable to set a loose similarity threshold for determining that a partial captured image and a partial registered image substantially match each other. Because the threshold is set loosely, subjects who are not registered persons may be determined to substantially match when matching by body part, but by using a large number of body parts in the matching loop, it is possible to ultimately output a subject who is not a registered person as a different person from the registrant.

In addition, the partial captured image is compared with the partial registered image, and if the comparison does not result in a substantial match, the registrant is removed from the list of candidates for multiple registrants. Therefore, as the comparison progresses, the number of candidates for multiple registrants is narrowed down. As a result, even if the target person is the same person as one of multiple registrants and the electromagnetic lock is to be opened, it can be opened more quickly than if a brute force comparison were used.

An actual example of Operation Example 4 is a device that is likely to match targets other than registered individuals, and is suitable for use in facial recognition with a device used by multiple registered individuals. For example, it could be used in an entrance gate where an unspecified number of people are expected to enter, allowing only registered individuals to enter. Since a substantial match in all body parts will determine that the individuals are the same, it could be used as an electromagnetic lock for a high-security entrance/exit gate.

§4. Functions and Effects The matching unit can determine whether a partial registration image and a partial captured image corresponding to the partial registration image substantially match based on the calculated similarity.

When there is only one registered person, if priority is given to the speed of recognition of the person, the subject is recognized as the same person as the registered person when a combination is obtained in which at least one partial registration image substantially matches a partial captured image. On the other hand, if priority is given to the speed of recognition of others, the subject is recognized as not the same person as the registered person when a combination is obtained in which at least one partial registration image substantially matches a partial captured image.

Furthermore, when there are multiple registered individuals, and priority is given to the speed of recognition of the individual, the target individual is recognized as the same person as the registered individual when a combination is obtained in which a partial registration image of at least one certain body part substantially matches a partial captured image. On the other hand, when priority is given to the speed of recognition of other individuals, the target individual is determined to not be the same person as the registered individual when a combination is obtained in which a partial registration image of at least one certain body part substantially does not match a partial captured image, and in subsequent processing, the registered individual is removed from the candidates for facial recognition.

As described above, image recognition of certain areas can influence the results of facial recognition. This makes it possible to terminate facial recognition earlier.

[Embodiment 2]
In the first embodiment, a method for increasing the speed of face recognition has been described. In contrast, in the second embodiment, a method for optimizing the priority of partial registration images in order to increase the speed of face recognition will be described.

Figure 8 is a block diagram showing the configuration of the main parts of a face recognition system 100a according to embodiment 2. Face recognition system 100a differs from face recognition system 100 in that face recognition device 10a includes a priority setting unit 16.

The priority setting unit 16 is a functional block that optimizes the priority of each partial registration image based on the history of similarities in past facial recognition results. The priority setting unit 16 first learns the similarity trends for each partial registration image based on the similarity history. As an example, the priority setting unit 16 calculates the average similarity based on the similarity history. Then, when prioritizing the recognition speed of the individual (operation examples 1 and 3), the priority setting unit 16 assigns priorities to each partial registration image in descending order of average similarity. As a result, partial registration images with higher priorities are more likely to have higher similarities, allowing facial recognition to be completed more quickly. Similarly, when prioritizing the recognition speed of others (operation examples 2 and 4), the priority setting unit 16 assigns priorities in descending order of average similarity. As a result, partial registration images with higher priorities are more likely to have lower similarities to others, allowing facial recognition to be completed more quickly.

In addition, the priority optimization process may be performed at any time while the face recognition device 10a is not performing face recognition processing, thereby enabling face recognition to always be performed with optimal priority.

Furthermore, the priority optimization in the priority setting unit 16 does not have to be performed automatically, but may be manually set in any order of priority by user operation. In this case, the priority setting unit 16 may display a list of partial registration images and images on a display device provided in the face recognition device 10a, and accept priority setting input for each partial registration image from the user via an input device provided in the face recognition device 10a. Furthermore, the priority setting unit 16 may acquire and set priority setting information received from the user on an external terminal device via communication means.

A specific example of assigning priorities is that, because the eyes are a part of the body that strongly expresses a person's characteristics, the priority of the eyes is set high, and because a user often wears a mask, the priority of the mouth is set low. It is also possible to set priorities according to one's own characteristics and preferences. For example, it is possible to set a low priority for the eyes because the user often wears sunglasses, or a low priority for the hair because the user often changes their hairstyle.

[Embodiment 3]
In the first embodiment, the description focuses on face recognition, but the object to be recognized is not limited to a face. In the third embodiment, object recognition in which the object to be recognized is an object will be described.

Figure 9 is a model diagram showing a captured image 113 and a partial captured image 114 according to embodiment 3. The captured image 113 is an image of an arbitrary object, and may be, for example, an image of a car as shown in Figure 9. The partial captured image 114 is, for example, an image of a portion of one of the four captured images 113, and is represented by partial captured images 114a to 114d. Each partial captured image represents a feature of the object.

For example, partial image 114a is an image of a side window, partial image 114b is an image of a windshield, partial image 114c is an image of a headlight, and partial image 114d is an image of a tire. The ranges of partial images 114 (partial images 114a to 114d are collectively referred to as partial images 114) may overlap with each other. The method for creating these partial images 114 and the method for assigning priority to partial images 114 are the same as the flowchart of face recognition operations shown in Figure 3.

Furthermore, the flowchart for the object recognition operation according to the third embodiment is the same as the flowchart for the face recognition operation shown in Figures 4 to 7. Using the same method as described above, the target object is photographed to obtain a captured image, a partial captured image corresponding to a partial registered image of the registered object is extracted from the captured image, and the partial registered image is compared with the partial captured image.

When priority is given to the speed at which the target object is recognized as a registered object based on the matching results, the target object is recognized as being the same as the registered object when the matching results for at least one part substantially match, as shown in the above-mentioned Operation Example 1. On the other hand, when priority is given to the speed at which the target object is recognized as not being the registered object, the target object is recognized as not being the same as the registered object when the matching results for at least one part do not substantially match, as shown in the above-mentioned Operation Example 2.

Next, we will explain the case where there are multiple registered objects. When priority is given to the speed at which the target object is recognized as one of the registered objects, as shown in the above-mentioned Operation Example 3, the target object is recognized as being the same as one of the multiple registered objects when the part matching for each priority substantially matches one of the multiple registered objects. When priority is given to the speed at which the target object is recognized as being none of the registered objects, as shown in the above-mentioned Operation Example 4, when the part matching process for each priority does not substantially match one of the multiple registered objects, the registered object is removed from the candidates, and when there are no more candidates, the target object is recognized as not being the same as any of the multiple registered objects.

[Software implementation example]
The functions of the face recognition device 10/10a and the object recognition device (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the face recognition device 10/10a).

In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in each of the above embodiments are realized by executing the program using this control device and storage device.

The above program may be stored non-transitory on one or more computer-readable storage media. These storage media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

Furthermore, some or all of the functions of each of the above control blocks can be realized by logic circuits. For example, integrated circuits incorporating logic circuits that function as each of the above control blocks are also included in the scope of the present invention. In addition, the functions of each of the above control blocks can also be realized by, for example, a quantum computer.

Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI may run on the control device, or on another device (for example, an edge computer or a cloud server).

[Additional Notes]
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

10, 10a Facial recognition device 12 Storage unit 13 Image extraction unit 14 Matching unit 15 Recognition output unit 16 Priority setting unit 20 Camera 30 Electromagnetic lock 100, 100a Facial recognition system 111, 113 Captured image 112, 112a to 112e, 114, 114a to 114d Partial captured image

Claims (10)

an image acquisition unit that acquires a captured image of a target object;
an image extraction unit that extracts images of a predetermined number of portions of the target object from the captured image to create a plurality of partial captured images;
a matching unit that matches a plurality of partial registered images, which are extracted as images of a plurality of predetermined portions of a registered object as a recognition target and are stored in a storage unit, with the partial captured image in an order according to the priority associated with each of the partial registered images;
a recognition output unit that outputs a recognition result of the target object when the matching in the matching unit satisfies a predetermined condition ,
the matching unit performs matching in the order of priority, and when it determines that the partial registration image and the partial captured image corresponding to the partial registration image substantially match, the recognition output unit outputs a recognition result of the target object;
the storage unit stores the partial registration images and priorities of the partial registration images for a plurality of the registered objects;
The object recognition device , wherein the matching unit performs matching against the partial registered images of all the registered objects at a certain priority, and then performs matching against the partial registered images of all the registered objects at a next priority . an image acquisition unit that acquires a captured image of a target object;
an image extraction unit that extracts images of a predetermined number of portions of the target object from the captured image to create a plurality of partial captured images;
a matching unit that matches a plurality of partial registered images, which are extracted as images of a plurality of predetermined portions of a registered object as a recognition target and are stored in a storage unit, with the partial captured image in an order according to the priority associated with each of the partial registered images;
a recognition output unit that outputs a recognition result of the target object when the matching in the matching unit satisfies a predetermined condition ,
the matching unit performs matching in the order of priority, and when it determines that the partial registration image and the partial captured image corresponding to the partial registration image do not substantially match, the recognition output unit outputs a recognition result of the target object;
the storage unit stores the partial registration images and priorities of the partial registration images for a plurality of the target objects;
an object recognition device, wherein the matching unit performs matching against the partial registered images of all the registered objects at a certain priority, and then performs matching against the partial registered images of all the registered objects at a next priority, and if it determines that the partial registered image and the partial captured image corresponding to the partial registered image do not substantially match, does not perform matching for the registered object at the next priority . The object recognition device according to claim 1 , wherein the matching unit performs matching by calculating a similarity between the partial registration image and the partial captured image and comparing the calculated similarity with a predetermined threshold value. the matching unit performs a process of storing the calculation result of the similarity as a similarity history;
The object recognition device according to claim 3 , further comprising a priority setting unit that sets priorities of the partial registration images based on the similarity history. The object recognition device according to claim 1 , further comprising a priority setting unit that sets priorities of the partial registration images based on a user input. The object recognition device according to claim 1 , wherein the target object is a human face. an image acquisition step of acquiring a captured image of a target object;
an image extraction step of extracting images of a predetermined plurality of portions of the target object from the captured image to create a plurality of partial captured images;
a matching step of matching the partial captured image with a plurality of partial registered images, each of which is an extracted image of a predetermined plurality of parts of a registered object as a recognition target and is stored in a storage unit, in an order according to the priority associated with each of the partial registered images;
a recognition output step of outputting a recognition result of the target object when the matching in the matching step satisfies a predetermined condition ,
In the matching step, matching is performed in the order of priority, and when it is determined that the partial registration image and the partial captured image corresponding to the partial registration image substantially match, a recognition result of the target object is output in the recognition output step;
the storage unit stores the partial registration images and priorities of the partial registration images for a plurality of the registered objects;
A control method for an object recognition device , wherein in the matching step, after matching is performed against the partial registered images of all the registered objects at a certain priority, matching is performed against the partial registered images of all the registered objects at a next priority . an image acquisition step of acquiring a captured image of a target object;
an image extraction step of extracting images of a predetermined plurality of portions of the target object from the captured image to create a plurality of partial captured images;
a matching step of matching the partial captured image with a plurality of partial registered images, each of which is an extracted image of a predetermined plurality of parts of a registered object as a recognition target and is stored in a storage unit, in an order according to the priority associated with each of the partial registered images;
a recognition output step of outputting a recognition result of the target object when the matching in the matching step satisfies a predetermined condition ,
In the matching step, matching is performed in an order according to the priority, and when it is determined that the partial registration image and the partial captured image corresponding to the partial registration image do not substantially match, the recognition output step outputs a recognition result of the target object;
the storage unit stores the partial registration images and priorities of the partial registration images for a plurality of the target objects;
a control method for an object recognition device, in which, in the matching step, after matching is performed against the partial registered images of all the registered objects at a certain priority, matching is performed against the partial registered images of all the registered objects at a next priority, and if it is determined that the partial registered image and the partial captured image corresponding to that partial registered image do not substantially match, matching is not performed for the registered object at the next priority. An object recognition program for causing a computer to function as the object recognition device described in claim 1, the object recognition program causing a computer to function as the image acquisition unit, the image extraction unit, the storage unit, the matching unit, and the recognition output unit. An object recognition program for causing a computer to function as the object recognition device described in claim 2 , the object recognition program causing a computer to function as the image acquisition unit, the image extraction unit, the memory unit, the matching unit and the recognition output unit.