JP7687038B2 - Wearing confirmation system, method and program - Google Patents

Description

This disclosure relates to a system, method, and program for verifying fit.

Conventionally, when entering areas where dangerous work is performed, such as construction sites or factories, workers are required to wear safety equipment such as helmets for their own safety. Technology has also been developed to check that workers are wearing helmets when entering such special areas (see Patent Document 1).

Patent Publication No. 2020-038456

However, the above-mentioned conventional technology has problems in that it does not allow entry to people without identification or other documents, and it is difficult to determine whether accessories such as helmets are worn correctly on a person's head.

The present disclosure aims to provide a system, method, and program for checking whether an article is being worn by a person with ease.

In order to solve the above problems, the present disclosure provides:
A photographing device for photographing visitors;
a person detection means for performing image analysis on the photographed image acquired by the photographing device and detecting a person in the photographed image;
an attachment detection means for detecting an attachment of the person by performing image analysis on the captured image;
a wearing determination means for determining a wearing state of an attachment based on detection results by the person detection means and the attachment detection means;
a storage means for storing the wearing state determined by the wearing determination means and the captured image as a confirmation result;
The present invention provides a wearing confirmation system having the following features.

In addition, the wearing confirmation system of the present disclosure includes:
The image capturing device may further include a display device for displaying the captured image, a detection reach of a person by the person detection means, and a detection reach of an article by the article detection means.

In addition, in the wearing confirmation system of the present disclosure,
The detection reach may be calculated by calculating the number of captured images in which a person or attached object is detected as a detection time based on the number of captured images subjected to image analysis in a specified time, and calculating the reach of the detection time relative to a predetermined detection reach time.

In addition, in the wearing confirmation system of the present disclosure,
The display device may further display a message according to a state of processing by the person detection means and the attachment detection means, and a processing rate of the captured image.

In addition, the wearing confirmation system of the present disclosure includes:
The device may further include a control device that unlocks an automatic door or a key box based on a result of the determination by the mounting determination means.

In addition, the wearing confirmation system of the present disclosure includes:
The person detection means may calculate a feature amount of a face or eye area of a person in a captured image, and detect the person based on the feature amount.

In addition, in the wearing confirmation system of the present disclosure,
The attachment detection means may detect the attachment in the photographed image by using a template image of the attachment and comparing it with a region of interest in the photographed image.

In addition, in the wearing confirmation system of the present disclosure,
The attachment detection means may have three modes: a normal mode using a single template image, a three-template mode using three template images of different sizes, and a random template mode using a scaled template image obtained by enlarging or reducing an original template image within a predetermined range.

In addition, in the wearing confirmation system of the present disclosure,
The template image may be an image of the attachment with a predetermined mark.

In addition, in this disclosure,
A method for checking the condition of an attachment by a computer connected to a photographing device that photographs visitors, the method comprising:
The computer
Performing image analysis on the captured image acquired by the photographing device to detect people in the captured image;
Performing image analysis on the captured image to detect clothing worn by the person;
a wearing determination means for determining a wearing state of an attachment based on the detection result of the person and the detection result of the attachment;
The wearing state judged by the wearing judging means and the photographed image are stored as a confirmation result.

In addition, in this disclosure,
A computer connected to a camera that photographs visitors,
a person detection means for performing image analysis on the photographed image acquired by the photographing device and detecting a person in the photographed image;
an attachment detection means for performing image analysis on the captured image and detecting attachments of the person;
a wearing determination means for determining a wearing state of an attachment based on detection results by the person detection means and the attachment detection means;
a storage means for storing the wearing state determined by the wearing determination means and the captured image as a confirmation result;
Provide a program to function as a

The present disclosure makes it possible to provide a system, method, and program for checking whether an article is being worn by a person.

FIG. 1 is a diagram illustrating a configuration of a wearing confirmation system according to an embodiment of the present disclosure. 1 is a schematic diagram showing an appearance of a wearing confirmation system according to an embodiment of the present disclosure. FIG. 1 is a schematic diagram of the mounting confirmation system as viewed from the side of the entrance. FIG. 2 is a functional block diagram showing details of the processing device 11. FIG. 2 is a diagram showing a display screen of a display device 13. 1 is a state transition diagram showing an overview of processing of a wearing confirmation system according to an embodiment of the present disclosure. 4A and 4B are diagrams showing the state of a screen displayed on a display device 13. FIG. 11 is a diagram for explaining details of human detection. FIG. 11 is a diagram for explaining details of attachment detection. FIG. 13 is a diagram for explaining an example of detecting a logo attached to the front of a helmet.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the drawings.
1. System configuration
Fig. 1 is a diagram showing a configuration of a wearing confirmation system according to an embodiment of the present disclosure. Fig. 2 is a schematic diagram showing an external appearance of the wearing confirmation system according to an embodiment of the present disclosure. Fig. 3 is a schematic diagram of the wearing confirmation system as viewed from the side direction of the entrance. In Figs. 1 to 3, 10 is a wearing confirmation system, 11 is a processing device, 12 is an image capturing device, 13 is a display device, 14 is a control device, 15 is an automatic door, 16 is a key box, and 17 is a tablet PC.

The processing device 11 is a device that performs various calculation processes in the wearing confirmation system 10, and is realized by incorporating a program for realizing the functions of the wearing confirmation system 10 into a computer having a CPU, main memory, non-volatile storage device, etc. The photographing device 12 is a photographing means that takes photographs and obtains the photographed images as image data, and is realized by a general-purpose photographing device such as a CCD camera. The display device 13 is a display means that displays based on the results processed by the processing device 11, and is realized by a general-purpose display device such as a liquid crystal display or an organic EL display. In this embodiment, the processing device 11, photographing device 12, and display device 13 are built into a single tablet PC 17.

The control device 14 is connected to the automatic door 15 and the key box 16, and transmits an unlocking signal to the automatic door 15 and the key box 16. The control device 14 is connected to the processing device 11, and transmits an unlocking signal to the automatic door 15 and the key box 16 based on a command from the processing device 11. The control device 14 is not shown in FIG. 3.

The automatic door 15 is a door that opens automatically. In this embodiment, the entrance is opened to allow entry by moving in a direction along the wall based on an unlocking signal from the control device 14. The key box 16 is a box into which a key is inserted. A physical key or a card key can be used as the key. If the door is not an automatic door, a locked door is unlocked using a key taken out from the key box 16. In this embodiment, the key box is unlocked based on an unlocking signal from the control device 14. This allows the user to take out the door key from the key box. The automatic door 15 and the key box 16 are examples of an entrance mechanism, and the entrance mechanism is not limited to the automatic door 15 and the key box 16 as long as it opens the entrance directly or indirectly based on the control of the control device 14.

Figure 4 is a functional block diagram showing details of the processing device 11. In this embodiment, the processing device 11 is realized by incorporating a dedicated program into a general-purpose computer. As shown in Figure 4, the processing device 11 has a person detection means 11a, an attachment detection means 11b, an attachment determination means 11c, and a confirmation result storage means 11d.

The person detection means 11a is a means for performing image analysis on the photographed image acquired by the photographing device 12 and detecting a person in the photographed image, and is realized by the CPU reading and executing a program stored in a non-volatile storage device. The attachment detection means 11b is a means for performing image analysis on the photographed image acquired by the photographing device 12 and detecting an attachment of a person, and is realized by the CPU reading and executing a program stored in a non-volatile storage device. The attachment determination means 11c is a means for determining the attachment state of an attachment based on the detection results by the person detection means 11a and the attachment detection means 11b, and is realized by the CPU reading and executing a program stored in a non-volatile storage device. The confirmation result storage means 11d is a means for storing the attachment state determined by the attachment determination means 11c and the photographed image as confirmation results, and is realized by a specified storage area of the non-volatile storage device.

Figure 5 is a diagram showing the display screen of the display device 13. In Figure 5, 13a is a captured image display area, 13b is a message display area, 13c is an achievement display area, 13d is a date and time display area, and 13e is a processing information display area. As shown in Figure 5, on the display screen of the display device 13, a captured image is displayed in the captured image display area 13a in the center of the screen. In practice, captured images captured at a predetermined rate (fps: frames (images) per second) are displayed by switching them sequentially. The captured images displayed may be at an analysis rate at which the images are analyzed by the processing device 11. In this embodiment, the captured images are displayed by switching them sequentially at the analysis rate. For example, when the shooting rate is 30 fps and the analysis rate is 20 fps, the captured images are displayed by switching them at a pace of 20 times per second.

A message for visitors is displayed in the message display area 13b at the top of the screen of the display device 13. This message is obtained from the processing device 11 and displayed according to the processing by the processing device 11. The achievement level display area 13c at the left side of the screen of the display device 13 displays the achievement level of the analysis by the processing device 11. The date and time display area 13d at the upper right of the screen of the display device 13 displays the current date and time. Other detailed information processed by the processing device 11 is displayed in the processing information display area 13e at the lower right of the screen of the display device 13. The processing information display area 13e can display information indicating units of date and time less than seconds, the processing rate of the captured image (fps: frames (images)/second), etc. The processing rate includes the shooting rate, which is the rate of shooting, and the analysis rate, which is the rate at which the captured image is analyzed. In the example of FIG. 5, the processing information display area 13e displays "45.40", which indicates units of date and time less than seconds, 30.0 fps, which indicates the shooting rate, and 20.0 fps, which indicates the analysis rate. As the processing rate, either the capture rate or the analysis rate may be displayed.

2. Processing Operation
Next, the processing operation of the wearing confirmation system according to this embodiment will be described together with the wearing confirmation method according to this embodiment. Fig. 6 is a state transition diagram showing an overview of the processing of the wearing confirmation system according to this embodiment. After being started, the wearing confirmation system according to this embodiment performs processing while transitioning through six states S1 to S6. Note that the wearable article is not particularly limited as long as it is something that is worn by a person, but here, a case where a helmet is used as an example of a wearable article will be described.

First, when the wearing confirmation system is started, processing begins. Then, the photographing device 12 takes pictures at all times, and captures the image data obtained by the photographing as a photographed image. The captured photographed image is passed to the processing device 11 for a predetermined process. The photographing device 12 takes pictures at a predetermined rate (fps: frames (images) per second), captures the captured images, and sends them to the processing device 11. The processing device 11 also passes the images acquired from the photographing device 12 to the display device 13. The display device 13 then displays the images received from the processing device 11 at any time. For example, when image analysis is performed at an analysis rate of 20 fps (frames per second), the photographed images are transmitted from the processing device 11 every 1/20 seconds and displayed on the screen of the display device 13. Therefore, the images are displayed like a moving image on the screen of the display device 13. The acquisition of photographed images by the photographing device 12 and the display of photographed images by the display device 13 are always performed while the wearing confirmation system is operating, regardless of the states of S1 to S6 shown in FIG. 6.

When processing is started in the wearing confirmation system, it first transitions to a standby state S1. In the standby state S1, the photographing device 12 takes photographs continuously, and captures the image data obtained by photographing as a photographed image. The processing device 11 then performs processing such as analysis on the captured photographed image. The analysis rate of the photographed image in the processing device 11 may be the same as the photographing rate by the photographing device 12, or it may be different. For example, if the photographing rate is 30 fps as described above, the analysis rate in the processing device 11 may be the same as the photographing rate, 30 fps, or may be a lower rate of 20 fps.

In standby state S1, the person detection means 11a in the processing device 11 performs person detection processing. Then, if the person detection means 11a detects a person of a predetermined size or larger at least once, the state transitions to person detection state S2. Figure 7 is a diagram showing the state of the screen displayed on the display device 13. In standby state S1, the display device 13 displays a screen as shown in Figure 7(a). Since standby state S1 is a state in which no person has been detected, a message urging the user to move is displayed in the message display area 13b. For example, as shown in Figure 7(a), a message such as "Please put on a helmet and approach the camera" is displayed.

If no person is detected from the captured image, the person detection means 11a repeats the person detection process for the captured image newly captured by the image capture device 12. Details of the person detection process will be described later. During the standby state S1, a message encouraging the user to come closer, such as "Please put on a helmet and come closer to the camera," is continuously displayed.

In the person detection state S2, the person detection means 11a continues the person detection process on the captured image. Specifically, the person detection means 11a calculates the person detection time PDT. The person detection time PDT is the time when a person is detected. Specifically, the person detection means 11a calculates the person detection time PDT by executing a process according to the following [Formula 1].

[Formula 1]
Person detection time PDT=number of person detections PDM×(1/analysis rate FPS)

In [Formula 1], the person detection count PDM is the number of times a person is detected, counted by the person detection means 11a. The analysis rate FPS is the number of captured images analyzed per second. As shown in [Formula 1], the person detection time PDT is calculated by multiplying the person detection count PDM by the reciprocal of the analysis rate FPS. Then, the person detection means 11a calculates the person detection reach PGL by executing processing according to the following [Formula 2].

[Formula 2]
Person detection reach level PGL = Person detection time PDT / Person detection reach time PGT

In [Formula 2], the person detection reach time PGT is a previously set time required for person detection. As shown in [Formula 2], the person detection reach level PGL is calculated by dividing the person detection time PDT calculated by executing processing according to [Formula 1] by the person detection reach time PGT. In other words, the person detection reach level is calculated by calculating the number of captured images in which a person is detected as the detection time based on the number of captured images analyzed in a specified time, and calculating the reach level of the person detection time relative to the previously set person detection reach time. In the example of [Formula 2], the person detection reach level PGL is calculated as a value between 0 and 1. The person detection reach time PGT can be set arbitrarily, but can be set to 1.5 seconds, for example.

The processing device 11 executes processing according to [Formula 2] and transmits the calculated human detection reach level PGL to the display device 13, and the display device 13 displays the human detection reach level PGL in the reach level display area 13c. Therefore, in the human detection state S2, the display device 13 displays a screen such as that shown in FIG. 7(b).

As shown in FIG. 7(b), the reach display area 13c displays the person detection reach PGL in the form of a bar extending vertically. In the reach display area 13c, the inside of the vertically extending bar is displayed as being filled from the bottom end to a height proportional to the person detection reach PGL. Therefore, immediately after the transition to the person detection state S2, the person detection reach PGL = 0, so there is no part filled inside the bar. Then, as the person detection reach PGL increases, the inside of the bar is filled from the bottom, and when the person detection reach PGL = 1, the entire inside of the bar is filled. The example in FIG. 7(b) shows a point in time when the person detection reach PGL = 0.

In addition, in person detection state S2, the current status, such as "Detecting person", is displayed in the message display area 13b. This display notifies the user of the current status. Then, a bar showing the person detection reach level PGL is displayed in the reach level display area 13c, allowing the visitor to check how far person detection has progressed. This allows the visitor to know how long they need to wait in front of the image capture device 12.

In the person detection state S2, if the processing device 11 is often unable to detect people from the captured image, the person detection count PDM does not increase easily, and therefore the person detection time PDT does not increase either. In such a case, it is not desirable to continue the detection process indefinitely. For this reason, the processing device 11 performs a process to return to the standby state S1 when a pre-set person detection limit time PRT has elapsed. That is, the person detection time PDT and person detection reach level PGL are once reset to 0, and the next time a person is detected, the processing device transitions to the person detection state S2 and performs person detection. The person detection limit time PRT can be set arbitrarily, but can be set to 10 seconds, for example.

In person detection state S2, if the person detection achievement level PGL reaches 1, it is assumed that the person has been correctly detected, and person detection is complete. When person detection is complete, the processing device 11 transitions to an attachment standby state S3. In the attachment standby state S3, the attachment detection means 11b performs attachment detection processing on the captured image. Then, if a helmet, which is an attachment, is detected even once, the state transitions to an attachment detection state S4.

In the accessory standby state S3, the display device 13 displays a screen as shown in FIG. 7(c). In the accessory standby state S3, a helmet, which is the object to be worn, has not been detected, so a message for the user is displayed in the message display area 13b. For example, as shown in FIG. 7(c), a message such as "Please wear a helmet" is displayed. Note that in the example of FIG. 7(c), a captured image in which the helmet is not worn is displayed in the captured image display area 13a, but in the accessory standby state S3, a similar message is displayed even if the captured image is one in which the helmet is worn.

If no attached object is detected from the captured image, the image capturing device 12 repeats the attached object detection process for the newly captured image. The attached object detection process will be described in detail later. During the helmet standby state S3, a message such as "Please put on your helmet" is continuously displayed to encourage the user to put on the helmet.

In the attached object detection state S4, the attached object detection means 11b continues the helmet detection process for the captured image. Specifically, the attached object detection means 11b calculates the attached object detection time HDT. The attached object detection time HDT is the time when the attached object, the helmet, is detected. Specifically, the attached object detection means 11b calculates the attached object detection time HDT by executing a process according to the following [Equation 3].

[Formula 3]
Wearing object detection time HDT = wearing object detection count HDM × (1/analysis rate FPS)

In [Formula 3], the number of times that the attached object is detected, HDM, is the value counted by the attached object detection means 11b for the number of times that a helmet is detected. As in [Formula 1], the analysis rate FPS is the number of captured images analyzed per second. As shown in [Formula 3], the attached object detection time HDT is calculated by multiplying the number of times that the attached object is detected, HDM, by the reciprocal of the analysis rate FPS. Then, the attached object detection means 11b calculates the attached object detection reach level HGL by carrying out processing according to the following [Formula 4].

[Formula 4]
Wearing object detection reach level HGL = wearing object detection time HDT / wearing object detection reach time HGT

In [Formula 4], the wearable object detection reach time HGT is a preset time required for wearable object detection. As shown in [Formula 4], the wearable object detection reach HGL is calculated by dividing the wearable object detection time HDT calculated by executing the process according to [Formula 3] by the wearable object detection reach time HGT. That is, the wearable object detection reach is calculated as the wearable object detection time based on the number of captured images analyzed in a specified time, and is calculated as the reach of the wearable object detection time relative to the wearable object detection reach time set in advance. In the example of [Formula 4], the wearable object detection reach HGL is calculated as a value between 0 and 1, similar to the person detection reach PGL in [Formula 2]. The wearable object detection reach time HGT can be set arbitrarily, but can be set to 1.5 seconds, for example.

The processing device 11 executes processing according to [Formula 4] and transmits the calculated wearable item detection reach HGL to the display device 13, and the display device 13 displays the wearable item detection reach HGL in the reach display area 13c. Therefore, in the wearable item detection state S4, the display device 13 displays a screen such as that shown in FIG. 7(e).

As shown in FIG. 7(e), the reach display area 13c displays the wearable item detection reach HGL in the form of a bar extending vertically. In the reach display area 13c, the inside of the vertically extending bar is displayed as being filled from its bottom end to a height proportional to the wearable item detection reach HGL. Therefore, immediately after the transition to the wearable item detection state S4, the wearable item detection reach HGL = 0, so there is no part filled inside the bar. Then, as the wearable item detection reach HGL increases, the inside of the bar is filled from the bottom, and when the wearable item detection reach HGL = 1, the entire inside of the bar is filled. The example in FIG. 7(e) shows a point in time when the wearable item detection reach HGL = 0.

In addition, in the attached object detection state S4, the current processing status, such as "Detecting helmet," is displayed in the message display area 13b. This display notifies the user of the current processing status. Then, by displaying a bar indicating the attached object detection progress level HGL in the progress level display area 13c, the user can check how far the attached object detection has progressed. This allows the user to know how long they need to wait in front of the image capture device 12.

In the wearable object detection state S4, if the wearable object detection means 11b is often unable to detect wearable objects from captured images, the wearable object detection count HDM does not increase, and therefore the wearable object detection time HDT does not increase either. In such a case, it is not desirable to continue the detection process indefinitely. For this reason, the wearable object detection means 11b performs a process to return to the standby state S1 when the wearable object detection limit time HRT set in advance has elapsed. That is, once the person detection time PDT, person detection reach level PGL, wearable object detection time HDT, and wearable object detection reach level HGL are all reset to 0, and when the next person is detected, the state transitions to the person detection state S2 and person detection is performed. The wearable object detection limit time HRT can be set arbitrarily, but can be set to 10 seconds, for example.

When the wearable item detection achievement level HGL reaches 1 in the wearable item detection state S4, it is determined that the helmet, which is the wearable item, has been correctly detected, and wearable item detection is complete. When wearable item detection is complete, the wear determination means 11c transitions to wearable item state S5, determining that the helmet, which is the wearable item, is correctly worn.

When the state transitions to the accessory wearing state S5, the display device 13 switches the display of the message display area 13b based on an instruction from the processing device 11. For example, as shown in FIG. 7(d), a message such as "[OK] Helmet wearing has been confirmed. The key box will be unlocked" is displayed. Then, the processing device 11 acquires a full-screen image captured from the screen of the display device 13 immediately after the transition to the accessory wearing state S5, as shown in FIG. 7(d). This full-screen image records the captured image in which person detection and accessory detection have been completed, and the date and time. The wearing determination means 11c saves the acquired full-screen image in the confirmation result storage means 11d. As a result, the face image and date and time of the person who is allowed to enter (normal entrant) are stored. In this embodiment, the full-screen image of the display screen of the display device 13 is stored in the confirmation result storage means 11d as the confirmation result, but as long as the good or bad of the wearing (admission or non-admission), the face image (the captured image showing the face), and the date and time are recorded, they may be stored in a format other than the full-screen image.

In the accessory attachment state S5, the processing device 11 then transmits an unlock signal to the control device 14. Specifically, if the determination result by the attachment determination means 11c is "attached correctly", the unlock signal is transmitted to the control device 14. Upon receiving the unlock signal from the processing device 11, the control device 14 further transmits an unlock signal to the automatic door 15 or the key box 16. Then, after transitioning to the accessory attachment state S5, if the preset accessory attachment limit time HST is exceeded, the state transitions to the standby state S1. The accessory attachment limit time HST can be set arbitrarily, but can be set to 15 seconds, for example.

In addition, in the accessory waiting state S3, if the accessory waiting time limit HWT is exceeded before the accessory detection is completed, the state transitions to the accessory not wearing state S6. The accessory waiting time limit HWT can be set arbitrarily, but can be set to 5 seconds, for example. When the state transitions to the accessory not wearing state S6, the display device 13 switches the display of the message display area 13b based on an instruction from the processing device 11. For example, as shown in FIG. 7(f), a message such as "[NG] Helmet wearing could not be confirmed" is displayed. Then, the processing device 11 acquires a full-screen image that captures the screen of the display device 13 immediately after the transition to the accessory not wearing state S6, as shown in FIG. 7(f). In this full-screen image, a captured image in which person detection was successful but accessory detection was not successful, and the date and time of the image are recorded. The wearing determination means 11c stores the acquired full-screen image in the confirmation result storage means 11d. As a result, the face image and date and time of the person who was not allowed to enter (violator, outsider) are stored.

After transitioning to the no-attachment state S6, if the pre-set no-attachment time limit NHT has elapsed, transitioning to the standby state S1 occurs. In other words, after transitioning to the no-attachment state S6, if the no-attachment time limit NHT has elapsed, transitioning to the standby state S1 occurs. The no-attachment time limit NHT can be set arbitrarily, but can be set to 15 seconds, for example. In the standby state S1, the person detection means 11a repeatedly performs person detection processing on the captured image.

3. Details of Person Detection Processing
As described above, in the standby state S1 and the person detection state S2, the person detection unit 11a performs a person detection process on the captured image. Here, the details of the person detection process by the person detection unit 11a will be described. As the person detection process, various techniques can be used as long as they detect a person from an image. In this embodiment, a feature amount in a face part or an eye part of a person in a captured image is calculated, and a person is detected based on the calculated feature amount. That is, a feature amount-based method is used. As a feature amount-based method, object detection using a Haar feature-based Cascade type classifier can be used. In this embodiment, the person detection unit 11a has two modes, a face detection mode for detecting a face and an eye detection mode for detecting eyes, and either one can be selected by setting. FIG. 8 is a diagram for explaining the details of person detection. FIG. 8(a) corresponds to the face detection mode, and FIG. 8(b) corresponds to the eye detection mode.

First, we will explain the face detection mode. The face detection mode is a mode for detecting a person using the overall facial features. As shown in FIG. 8(a), in the face detection mode, one target area Ff is set within the captured image. Then, a face is detected using the facial features contained in the target area Ff.

Next, we will explain the eye detection mode. The eye detection mode is a mode in which people are detected using only the features of the eyes. As shown in Figures 8(b) and (c), in eye detection mode, a total of two target areas Fe are set in the captured image, one for each eye. Then, detection is performed using the features of the eyes contained in the target areas Fe. In the case of eye detection mode, people can be detected not only when the entire face is exposed, as shown in the captured image in Figure 8(b), but also when a mask is worn, as shown in the captured image in Figure 8(c).

4. Details of helmet detection process
As described above, in the standby state S1 and the attached object detection state S4, the attached object detection means 11b performs an attached object detection process on the captured image. Here, the details of the attached object detection process will be described. As the attached object detection process, various techniques can be used as long as they detect an attached object such as a helmet from an image. In this embodiment, a template matching method is used. That is, in this embodiment, if the attached object is a helmet, an image of only the helmet is prepared as a template image, and the helmet is detected by comparing and matching with this template image.

As a template matching method, a method of comparing a template image with a region of interest in a captured image can be used. In this embodiment, three modes are provided: normal mode, 3 template mode, and random template mode, and any one of them can be selected by setting. Figure 9 is a diagram for explaining the details of attachment detection. Figure 9(a) corresponds to the normal mode, Figure 9(b) corresponds to the 3 template mode, and Figure 9(c) corresponds to the random template mode.

First, the normal mode will be described. In the normal mode, one template image, i.e., a single template image, is set in advance, and detection is performed by matching with the template image. As shown in FIG. 9(a), in the normal mode, one target area Fm is set within the captured image. Then, the image contained in the target area Fm is compared with the template image.

Next, the 3 template mode will be explained. In the 3 template mode, three template images of different sizes are set in advance, and if a match occurs with any of the three template images, it is considered to have been detected. In this case, three template images are used: actual size (magnification 1), less than magnification 1, and more than magnification 1. The template images less than magnification 1 and more than magnification 1 can be created by reducing or enlarging the actual size template image. In the 3 template mode, as shown in Figure 9 (a), a target area Fm is set within the captured image, and the image contained in the target area Fm is compared with the actual size template image.

Furthermore, in the 3 template mode, as shown in FIG. 9(b), a target area Fs is set within the captured image, and the image contained in the target area Fs is compared with a reduced template image. Also, as shown in FIG. 9(b), a target area Fl is set within the captured image, and the image contained in the target area Fl is compared with an enlarged template image. The reduction rate and enlargement rate can be set appropriately, but for example, a reduced template image is created with a magnification rate of 0.7 times, and an enlarged template image is created with a magnification rate of 1.2 times. This 3 template mode can tolerate a certain degree of distance between the imaging device 12 and the detection target.

The random template mode will be explained. In the random template mode, a value is randomly selected within a range of two set magnification ratios (less than 1, more than 1) for each detection, and a scaled template image (simply indicated as "template image" in FIG. 9(c)) is generated by scaling a template image at full size (original template image) using the selected value, and detection is considered to have occurred when the scaled template image matches. As shown in FIG. 9(c), for example, a scaled template image is generated within a range of magnification ratios from 0.7 to 1.2, and compared with the captured image. This random template mode can also tolerate a certain degree of distance between the imaging device 12 and the detection target.

In this embodiment, an example of using a helmet as an article of wear has been described, but even a specific logo attached to the front of the helmet can be detected. FIG. 10 is a diagram for explaining an example of detecting a logo attached to the front of a helmet. In this case, an image of a helmet with a logo attached to the front side is prepared as a template. Then, using this template, a comparison is made with the captured image, and if the logo matches in addition to the shape of the helmet, entry can be permitted. For example, a logo mark representing a specific company can be used as the logo. FIG. 10 shows an example in which the ABC logo is attached to the front side of the helmet. In this case, the article of wear detection means 11b detects an article of wear in the article of wear detection state S4, and if the logo matches, it transmits an unlocking signal to the control device 14, assuming that the person is wearing a legitimate helmet. In this way, by preparing an image of a helmet with a logo attached to the front side as a template image, only people wearing helmets that are permitted to be used are permitted to enter, and it is possible to prevent intrusion by outsiders.

<5. Inversion of the captured image>
The display device 13 may display the captured image in the captured image display area 13a without changing the orientation of the captured image, or may display the captured image in a left-right inverted state. A setting can be used to switch between a normal display in which the captured image is displayed as is and an inverted display in which the captured image is displayed in a left-right inverted state. By performing an inverted display in which the captured image is displayed in a left-right inverted state, the user can check the wearing state of the device with the same sensation as looking in a mirror.

Although the preferred embodiment of the present disclosure has been described above, the present disclosure is not limited to the above embodiment and various modifications are possible. For example, in the above embodiment, a computer is used as the processing device 11, and a dedicated program stored in a storage device is executed by a CPU to realize a person detection means, an attachment detection means, an attachment determination means, and a confirmation result storage means, but each of these means may be incorporated into hardware as an arithmetic circuit.

Reference Signs List 11: Processing device 11a: Person detection means 11b: Wearing object detection means 11c: Wearing determination means 11d: Confirmation result storage means 12: Photographing device 13: Display device 14: Control device 15: Automatic door 16: Key box 17: Tablet PC

Claims (8)

A photographing device for photographing visitors;
a person detection means for performing image analysis on the photographed image acquired by the photographing device and detecting a person in the photographed image;
an attachment detection means for detecting an attachment of the person by performing image analysis on the captured image;
a wearing determination means for determining a wearing state of an attachment based on detection results by the person detection means and the attachment detection means;
a storage means for storing the wearing state determined by the wearing determination means and the captured image as a confirmation result;
having
the person detection means calculates a feature amount of an eye portion of the person in the captured image, and detects the person based on the feature amount;
Displaying a detection reach of a person by the person detection means and a detection reach of an attachment by the attachment detection means;
The detection reachability is calculated based on the number of captured images in which a person or attached object is detected as a detection time based on the number of captured images analyzed in a specified time, and is calculated as the reachability of the detection time relative to a predetermined detection reachability time. 2. The wearing confirmation system according to claim 1, wherein the display device further displays a message according to a state of processing by the person detection means and the wearing object detection means, and a processing rate of the captured image. The mounting confirmation system according to claim 1 or 2, further comprising a control device that unlocks an automatic door or a key box based on the result of the determination by the mounting determination means. The wearing confirmation system according to any one of claims 1 to 3, wherein the wearing detection means detects the wearing object in the captured image by comparing a template image of the wearing object with a region of interest in the captured image. The wearing confirmation system according to claim 4, wherein the wearing detection means has three modes: a normal mode using a single template image, a three-template mode using three template images of different sizes, and a random template mode using a scaled template image obtained by enlarging or reducing the original template image within a predetermined range. The wearing confirmation system according to claim 4 or claim 5, wherein the template image is an image of the wearing object with a predetermined mark. A method for checking the condition of an attachment by a computer connected to a photographing device that photographs visitors, the method comprising:
The computer
performing image analysis on the captured image acquired by the photographing device, calculating a feature amount of an eye portion of a person in the captured image, and detecting a person in the captured image based on the feature amount;
Performing image analysis on the captured image to detect clothing worn by the person;
determining a wearing state of the attachment based on the detection result of the person and the detection result of the attachment;
The determined wearing state and the captured image are stored as confirmation results.
Displaying the detection reach of the person and the detection reach of the wearable item;
The detection reachability is calculated based on the number of captured images in which a person or attached object is detected as a detection time based on the number of captured images analyzed in a specified time, and is calculated as the reachability of the detection time relative to a predetermined detection reachability time. A computer connected to a camera that photographs visitors,
a person detection means for performing image analysis on a photographed image acquired by the photographing device, calculating a feature amount of an eye portion of a person in the photographed image, and detecting a person in the photographed image based on the feature amount;
an attachment detection means for performing image analysis on the captured image and detecting attachments of the person;
a wearing determination means for determining a wearing state of an attachment based on detection results by the person detection means and the attachment detection means;
a storage means for storing the wearing state determined by the wearing determination means and the captured image as a confirmation result;
calculating the number of photographed images in which a person is detected as a first detection time based on the number of photographed images that have been subjected to image analysis within a predetermined time period, and calculating a detection reachability of the person as a reachability of the first detection time relative to a detection reachability time that is set in advance;
a display means for calculating the number of photographed images in which the attachment is detected as a second detection time based on the number of photographed images analyzed in a predetermined time, and displaying a detection arrival degree of the attachment calculated as an arrival degree of the second detection time with respect to a detection arrival time set in advance;
A program to function as a