JP7639251B2 - False detection determination device, method, and program - Google Patents

Description

The present invention relates to a false positive determination device, method, and program.

Conventionally, technology is known that uses security and surveillance cameras to detect intruders who have entered offices, homes, cars, etc.

Patent document 1 discloses a technology for detecting people that compares the shape and size of the bounding box of a human body candidate detected from a fisheye image obtained by a fisheye camera with predetermined shape and size criteria.

JP 2020-107070 A

However, intruders in images captured by security and surveillance cameras may have parts of their body hidden (meaning that only parts of the body can be identified) or may be in a variety of postures (for example, not only standing, but also crouching or crawling). Therefore, when detecting intruders in security and surveillance camera images, using only the shape and size of the bounding box may result in the incorrect detection of objects other than people in the image.

Furthermore, when the technology in Patent Document 1 is combined with a night vision camera for nighttime surveillance, images of people taken at night resemble whiteout caused by insects or other objects approaching the camera lens, and whiteout caused by insects or other objects may be mistakenly detected as people.

The present invention aims to determine whether a system that detects intruders from images captured at night is erroneously detecting an object other than a human.

An erroneous detection determination device according to one embodiment of the present invention includes an image acquisition unit that acquires a plurality of images, an object detection unit that detects a human-like object in the images, a blown-out highlight determination unit that determines whether or not there is blown-out highlight in the area in the image in which the object is detected, a motion determination unit that, when blown-out highlight is present, determines whether or not the movement of the object is human movement, and, when it is determined that the movement of the object is not human movement, determines that the detection of the object is an erroneous detection.

The present invention makes it possible to determine whether or not an object other than a human has been erroneously detected.

FIG. 1 is a diagram for explaining an overview according to one embodiment of the present invention. 1 is a diagram showing an overall system configuration according to an embodiment of the present invention; 1 is a functional block diagram of an erroneous detection determination device according to an embodiment of the present invention; 11 is a flowchart of an erroneous detection determination process according to an embodiment of the present invention. 11 is a flowchart of an insect movement determination process according to an embodiment of the present invention. 11 is a flowchart of a human movement determination process according to an embodiment of the present invention. FIG. 13 is a diagram for explaining determination of insect movement according to one embodiment of the present invention. FIG. 1 is a diagram for explaining determination of human movement according to an embodiment of the present invention. 1 is a block diagram showing an example of a hardware configuration of an erroneous detection determination device according to an embodiment of the present invention;

The following describes an embodiment of the present invention based on the drawings.

This specification describes an embodiment in which an infrared camera equipped with infrared lighting (hereinafter simply referred to as an "infrared camera") is used at night to detect intruders, and whiteout caused by insects is erroneously detected as a person, but the present invention can also be applied to whiteout caused by small animals other than insects (any object other than a person will do).

Figure 1 is a diagram for explaining an overview of one embodiment of the present invention. In one embodiment of the present invention, when a human-like object (specifically, an object with a size and shape that is assumed to be a human) is detected in an image captured by an infrared camera installed in a surveillance area, it is possible to determine whether the object has not been erroneously detected (i.e., it is a human) or whether it has been erroneously detected (i.e., it is not a human) depending on whether there is blown-out highlights in the area in which the object was detected.

In step 1 (S1), an image captured by an infrared camera is acquired.

In step 2 (S2), assume that a human-like object (specifically, an object with a size and shape that is assumed to be a human) is detected in the image of S1.

In step 3 (S3), it is determined whether the detection score of S2 (specifically, the detection confidence level) is greater or less than a threshold. If it is greater than the threshold, the process proceeds to step 8, where it is determined that the object detected in S2 is a person (i.e., not a false detection). If it is less than the threshold, the process proceeds to step 4.

In step 4 (S4), it is determined whether or not there is blown-out highlights in the area in which the object was detected in the image in S2. If there is no blown-out highlights, the process proceeds to step 6. If there is blown-out highlights, the process proceeds to step 5.

In step 5 (S5), it is determined whether the object detected in S2 is moving like an insect (whether it is moving like an insect). If there is insect-like movement, the process proceeds to S9, and the detection of the human-like object in S2 is determined to be a false detection. If there is no insect-like movement, the process proceeds to step 6.

In step 6 (S6), it is determined whether the human-like object detected in S2 is moving. If there is movement, the process proceeds to S8, and the object detected in S2 is determined to be a human. If there is no movement, the process proceeds to S7, and the object detected in S2 is determined to be not a human.

Figure 2 is a diagram showing the overall system configuration according to one embodiment of the present invention. As shown in Figure 2, the false detection determination system 1 includes a false detection determination device 10 and an imaging device 20, which is an infrared camera. The false detection determination device 10 can acquire images captured by the imaging device 20 via any network or storage medium. Each of these will be described below.

When an object (specifically, an object with a size and shape that is assumed to be a person) is detected in an image captured by the imaging device 20, the false detection determination device 10 determines whether the object is not a false detection (i.e., it is a person) or whether it is a false detection (i.e., it is not a person). The false detection determination device 10 is a computer such as a server. The false detection determination device 10 will be described in detail later with reference to FIG. 3.

The imaging device 20 is an infrared camera such as a security or surveillance camera that captures images of people who intrude into an office, a residence, a car, etc. For example, when an intruder detection unit (not shown) such as an infrared sensor detects an object, the imaging device 20 records an image captured several seconds before and after the detection in an internal memory (not shown) and transmits the image to the false detection determination device 10. The infrared sensor or the like may be installed separately from the imaging device 20, and when the infrared sensor or the like detects an object, the imaging device 20 may be linked to capture an image of the target area. In addition, when the infrared sensor or the like transmits a detection signal to the false detection determination device 10, the false detection determination device 10 may request the imaging device 20 to transmit the captured image to itself according to the content of the detection signal.

Figure 3 is a functional block diagram of an incorrect detection determination device 10 according to one embodiment of the present invention. As shown in Figure 3, the incorrect detection determination device 10 can include an image acquisition unit 101, an object detection unit 102, a detection score determination unit 103, a highlight blowout determination unit 104, an insect movement determination unit 105 (having a similarity calculation unit 151 and a determination unit 152), a human movement determination unit 106 (having a similarity calculation unit 161 and a determination unit 162), and an incorrect detection determination unit 107. The insect movement determination unit 105 and the human movement determination unit 106 are collectively referred to as the movement determination unit. In addition, by executing a program, the misdetection determination device 10 can function as an image acquisition unit 101, an object detection unit 102, a detection score determination unit 103, a highlight blowout determination unit 104, an insect movement determination unit 105 (having a similarity calculation unit 151 and a determination unit 152), a human movement determination unit 106 (having a similarity calculation unit 161 and a determination unit 162), and a misdetection determination unit 107.

The image acquisition unit 101 acquires multiple images captured by the imaging device 20 (i.e., a video consisting of multiple frames (still images)). Alternatively, the image acquisition unit 101 may receive a detection signal from the aforementioned infrared sensor or the like, and request the imaging device 20 to transmit the captured images to itself according to the content of the detection signal, and acquire the multiple images transmitted by the imaging device 20 in response to the request.

The object detection unit 102 detects an object (specifically, a body part or the entire body that is assumed to be a person) in each of the multiple images (i.e., each frame) acquired by the image acquisition unit 101. The object detection unit 102 stores coordinate information of the area (e.g., a rectangle) in which the object is detected in the image, and the detection score (detection reliability) in a memory or the like (not shown) in the false detection determination device 10. Hereinafter, an image (frame) in which the object detection unit 102 detects an object is also referred to as an object detection image, and an area in which an object is detected in an object detection image is also referred to as an object detection area.

For example, the object detection unit 102 can detect objects in an image by machine learning such as deep learning. Specifically, the object detection unit 102 inputs an image into a trained model that has previously trained images of the entire and partial human body using a neural network or the like, detects any objects that are assumed to be human in the image, and outputs the object detection area in the image and the detection score (detection reliability). Here, "detection of people and objects using anchor boxes" is shown as an example. For example, in object detection methods such as "Single Shot Multibox Detector (SSD)", "M2Det", and "Faster R-CNN", anchor boxes with multiple aspect ratios are applied to each cell of a feature map obtained by inputting an image, and candidates for "human rectangles" and "human detection scores" are obtained. After that, NMS (Non-Maximum Suppression) is applied, and if the overlap of the "human rectangles" is equal to or greater than a certain level, only rectangles with high "human detection scores" are left and output. There are various methods for human and object detection, but any method is acceptable as long as it outputs a "human rectangle (object detection area)" and a "human detection score."

Note that objects may be detected in one or more images (frames).

The detection score determination unit 103 determines whether the score of the detection by the object detection unit 102 (detection reliability (indicating the accuracy of predictions/outputs by machine learning)) is smaller than a threshold value.

The blown-out highlight determination unit 104 determines whether or not there is blown-out highlight in an image in which an object has been detected. Specifically, the blown-out highlight determination unit 104 calculates the percentage of pixels whose brightness pixel value is equal to or greater than a predetermined value (e.g., 250) among all pixels in an area (e.g., a rectangle) that includes the object. The blown-out highlight determination unit 104 determines that there is blown-out highlight if the percentage of the number of pixels whose brightness pixel value is equal to or greater than a predetermined value (e.g., 250) to the total number of pixels (number of pixels in width x number of pixels in height) is equal to or greater than a predetermined numerical value (e.g., 20 percent).

The insect movement determination unit 105 determines whether or not there is movement of the object detected by the object detection unit 102 (whether or not it is movement of something other than a human, such as an insect). Specifically, the insect movement determination unit 105 determines whether or not there is movement of the object based on whether or not there is similarity between the object detection image (the frame at time t in which a human-like object was detected) and the images before and after it (the frame at time t-1 and the frame at time t+1). Below, the insect movement determination unit 105 will be explained separately as a similarity calculation unit 151 and a determination unit 152.

The similarity calculation unit 151 calculates the similarity between the object detection image (a frame at time t in which a human-like object was detected) and the images before and after it (a frame at time t-1 and a frame at time t+1). Specifically, the similarity calculation unit 151 compares an area (e.g., a rectangle) that contains an object in the object detection image with areas (e.g., rectangles) in the previous and next images that have the same coordinate position as the area in the object detection image.

When a human-like object is detected in multiple images (frames), the similarity calculation unit 151 calculates the similarity between each object detection image and the images before and after it (i.e., the frames before and after it included in the video).

<<Calculation of similarity>>
Here, the calculation of the similarity will be described in detail. For example, the similarity calculation unit 151 can use the method of ZNCC (Zero-mean Normalized Cross Correlation). For example, it is assumed that the similarity between a rectangle of M pixels x N pixels including an object and a rectangle of M pixels x N pixels in the previous and next images is calculated. The pixels (T(i, j) and I(i, j)) at the same coordinates of the two rectangles are used. The similarity calculation unit 151 calculates the similarity by:

を算出することができる。

can be calculated.

である。なお、

In addition,

である。類似度が１であると画像に変化がない、つまり画像内の物体の動きがないと判定することができ、類似度が－１に近づくほど画像が類似していない、つまり画像内の物体の動きが有ると判定することができる。

When the similarity is 1, it can be determined that there is no change in the image, that is, there is no movement of the object in the image, and as the similarity approaches -1, it can be determined that the images are less similar, that is, there is movement of the object in the image.

The similarity is not limited to the ZNCC method, but can be calculated using any template matching method such as NCC (Normalized Cross-Correlation), SSD (Sum of Squared Difference), or SAD (Sum of Absolute Difference).

For each object detection region in an object detection image in which the object detection unit 102 detects an object, the determination unit 152 calculates the similarity of the region at the same coordinate position as the object detection region in the previous and next images, and determines whether or not at least one of the similarities is equal to or less than a threshold. Specifically, the determination unit 152 determines that there is object movement when the similarity of at least one of the previous and next images for at least one object detection image is equal to or less than a threshold. The determination unit 152 determines that there is no object movement when the similarity of none of the previous and next images is equal to or less than a threshold in all object detection images. This utilizes the characteristics of a situation in which an insect is photographed in a position close to the camera. The shooting interval between each of the approximately 15 images photographed within a few seconds is about 0.2 seconds. If an object assumed to be a person in the image data is truly a person, the similarity will be high when comparing adjacent images before and after the object detection region, but in the case of a small living thing such as an insect photographed just in front of the camera lens, the movement will be captured significantly, resulting in a low image similarity. Using this, the image can be compared with the immediately preceding and following images, and if either one has a low similarity, it can be determined that the movement is due to something other than a human, such as an insect.

The human movement determination unit 106 determines whether or not there is movement (whether or not it is human movement) of the human-like object detected by the object detection unit 102. Note that, below, an embodiment is described in which the presence or absence of object movement is determined based on whether or not the object detection image (i.e., the frame in which the object is detected) is similar to other images (i.e., other frames included in the video), but this is not limited to this, and the movement of a human-like object may be determined by any method. Below, the human movement determination unit 106 will be described separately as a similarity calculation unit 161 and a determination unit 162.

The similarity calculation unit 161 calculates the similarity between the object detection image (a frame in which a human-like object is detected) and other images (i.e., other frames included in the video). Specifically, the similarity calculation unit 161 compares an area in the object detection image that contains an object (object detection area) with an area in the other image that has the same coordinate position as the object detection area in the object detection image. Note that the other frames may be all other frames included in the video, or may be some other frames included in the video.

When a human-like object is detected in multiple images (frames), the similarity calculation unit 161 calculates the similarity between each object detection image (each frame) and each of the other images (i.e., other frames included in the video).

<<Calculation of similarity>>
Here, the calculation of the similarity will be described in detail. For example, the similarity calculation unit 161 can use the method of ZNCC (Zero-mean Normalized Cross Correlation). For example, it is assumed that the similarity between a rectangle of M pixels x N pixels including an object and a rectangle of M pixels x N pixels in another image is calculated. The pixels (T(i, j) and I(i, j)) at the same coordinates of the two rectangles are used. The similarity calculation unit 161 calculates the similarity by:

を算出することができる。

can be calculated.

である。なお、

In addition,

である。類似度が１であると画像に変化がない、つまり画像内の物体の動きがないと判定することができ、類似度が－１に近づくほど画像が類似していない、つまり画像内の物体の動きが有ると判定することができる。

When the similarity is 1, it can be determined that there is no change in the image, that is, there is no movement of the object in the image, and as the similarity approaches -1, it can be determined that the images are less similar, that is, there is movement of the object in the image.

The similarity is not limited to the ZNCC method, but can be calculated using any template matching method such as NCC (Normalized Cross-Correlation), SSD (Sum of Squared Difference), or SAD (Sum of Absolute Difference).

The determination unit 162 determines whether or not there are a predetermined number or more of other images whose similarity is equal to or less than the threshold for each object detection region in the object detection image in which the object detection unit 102 detects an object. Specifically, the determination unit 162 determines that there is object movement when the number of other images whose similarity is equal to or less than the threshold is equal to or more than the predetermined number in at least one object detection region. The determination unit 162 determines that there is no object movement when the number of other images whose similarity is equal to or less than the threshold is less than the predetermined number in all object detection regions. Note that the predetermined number may be one or may be two or more. Furthermore, when there are multiple object detection regions in the same object detection image, these determination processes are performed for each object detection region, and if the number of other images whose similarity is equal to or less than the threshold for at least one object detection region is equal to or more than the predetermined number, it can be determined that there is object movement.

The false detection determination unit 107 determines that the object detected by the object detection unit 102 is a human because the object is moving, or determines that the object detected by the object detection unit 102 is not a human because the object is not moving.

Methods
Hereinafter, the erroneous detection determination process will be described with reference to FIG. 4, the insect movement determination process will be described with reference to FIG. 5, and the human-like object movement determination process will be described with reference to FIG.

Figure 4 is a flowchart of the false positive determination process according to one embodiment of the present invention.

In step 11 (S11), the image acquisition unit 101 acquires multiple images (i.e., a video consisting of multiple frames (still images)) captured by the imaging device 20.

In step 12 (S12), the object detection unit 102 detects an object that is assumed to be a person (human-like) in each of the multiple images (i.e., each frame) acquired in S11. Note that an object may be detected in one or more images (frames). If a human-like object is detected, the process proceeds to step 13, and if a human-like object is not detected, the process ends.

In step 13 (S13), the detection score determination unit 103 determines whether the detection score (detection reliability) of S12 is smaller than a threshold value. If it is determined that it is less than the threshold value, the process proceeds to step 14, and if it is determined that it is equal to or greater than the threshold value, it is determined that it is a person.

In step 14 (S14), the highlight blowout determination unit 104 determines whether or not there is highlight blowout in the area where the object was detected in S12. If there is highlight blowout, the process proceeds to step 15, and if there is no highlight blowout, the process proceeds to step 16.

In step 15 (S15), the insect movement determination unit 105 determines whether or not the object detected in S12 is moving (whether or not it is moving like an insect, etc.). If it is determined that there is no insect-like movement, the process proceeds to step 16, and if it is determined that there is insect-like movement, the process proceeds to step 17.

In step 16 (S16), the human movement determination unit 106 determines whether the human-like object detected in S12 is moving (whether the human-like object is moving). If it is determined that there is no movement, the process proceeds to step 17, and if it is determined that there is movement, the process proceeds to step 18.

In step 17 (S17), the erroneous detection determination unit 107 determines that the object detected in S12 is an erroneous detection (i.e., the object detected in S12 is other than a human).

In step 18 (S18), the erroneous detection determination unit 107 determines that the object detected in S12 is not an erroneous detection (i.e., the object detected in S12 is a person).

Figure 5 is a flowchart of the insect movement determination process (S15 in Figure 4) according to one embodiment of the present invention.

In step 21 (S21), the similarity calculation unit 151 selects one of the object detection regions in the object detection image in which the object detection unit 102 has detected an object.

In step 22 (S22), the similarity calculation unit 151 calculates the similarity between the object detection area in the object detection image selected in S21 (i.e., the frame in which the object was detected) and the area at the same coordinate position as the object detection area in the previous and next images (i.e., the previous and next frames included in the video).

In step 23 (S23), the similarity calculation unit 151 determines whether or not the calculation of similarities for the object detection regions in all images (object detection images) in which the object detection unit 102 has detected an object has been completed. If completed, the process proceeds to step 24, and if not completed, the process returns to step 21.

In step 24 (S24), for each object detection region in an image in which the object detection unit 102 detected an object (object detection image), the determination unit 152 calculates the similarity between regions in the previous and next images at the same coordinate position as the object detection region, and determines whether or not at least one of the similarities is equal to or less than a threshold value. If the similarity of at least one of the previous and next images for at least one object detection image is equal to or less than the threshold value, the process proceeds to step 25; if the similarity of none of the previous and next images for all object detection images is equal to or less than the threshold value, the process proceeds to step 26.

In step 25 (S25), the determination unit 152 determines that there is insect movement.

In step 26 (S26), the determination unit 152 determines that there is no insect movement.

Figure 6 is a flowchart of the human movement determination process (S16 in Figure 4) according to one embodiment of the present invention.

In step 31 (S31), the similarity calculation unit 161 selects one of the object detection regions in the object detection image in which the object detection unit 102 has detected an object.

In step 32 (S32), the similarity calculation unit 161 calculates the similarity between the object detection area in the object detection image selected in S31 (i.e., the frame in which the object is detected) and an area at the same coordinate position as the object detection area in another image (i.e., another frame included in the video).

In step 33 (S33), the similarity calculation unit 161 determines whether or not the calculation of similarities for the object detection regions in all images (object detection images) in which the object detection unit 102 has detected an object has been completed. If completed, the process proceeds to step 34, and if not completed, the process returns to step 31.

In step 34 (S34), the determination unit 162 determines whether or not there are a predetermined number or more of other images whose similarity is equal to or less than the threshold for each object detection region in which the object detection unit 102 has detected an object. If the number of other images whose similarity is equal to or less than the threshold in at least one object detection region is equal to or more than the predetermined number, the process proceeds to step 35, and if the number of other images whose similarity is equal to or less than the threshold in all object detection regions is less than the predetermined number, the process proceeds to step 36.

In step 35 (S35), the determination unit 162 determines that there is human-like object movement.

In step 36 (S36), the determination unit 162 determines that there is no movement of an object that resembles a human.

<Determining insect movement>
The determination of insect movement will now be described in detail.

Figure 7 is a diagram for explaining insect movement determination according to one embodiment of the present invention. Assume that a video captured by a security or surveillance camera (i.e., a video acquired by the image acquisition unit 101) is composed of 15 frames in time series from frame 1 to frame 15. Assume that an object is detected in the 10th frame (i.e., the object detection unit 102 detects an object in the 10th frame). Assume that the detection score (detection confidence) is 0.5, which is less than the threshold value.

First, the similarity calculation unit 151 of the insect movement determination unit 105 calculates the similarity between the frame in which the object is detected (the 10th frame in FIG. 7) and the previous and next frames in the video (the 9th and 11th frames in FIG. 7).

Next, the determination unit 152 of the insect movement determination unit 105 calculates the similarity for each object detection region in the object detection image for regions in the previous and next images at the same coordinate position as the object region, and determines whether at least one of the similarities is below a threshold. In the example of Figure 7, it is assumed that the similarity of the object detection region in the 11th frame is below a threshold. In this case, the insect movement determination unit 105 determines that there is insect movement, since the similarity of at least one of the previous and next images is below the threshold.

When an object is detected in multiple images (frames) (i.e., when an object is detected in frames other than the 10th frame), the similarity calculation unit 151 calculates the similarity between the object detection area in each object detection image (each frame) and the area at the same coordinate position as the object detection area in each previous and next images (i.e., the previous and next frames included in the video). Then, for each image in which an object is detected (object detection image), the determination unit 152 determines whether or not at least one of the previous and next images has a similarity below the threshold (i.e., a determination similar to the determination for the 10 frames described above is made). If at least one of the previous and next images has a similarity below the threshold in at least one object detection image, it is determined that there is movement, and if none of the previous and next images has a similarity below the threshold in all object detection images, it is determined that there is no movement.

<Determining human movement>
The determination of human movement will now be described in detail.

Figure 8 is a diagram for explaining human movement determination according to one embodiment of the present invention. Assume that a video captured by an imaging device 20 such as a security or surveillance camera and acquired by the image acquisition unit 101 is composed of 15 frames in time series from frame 1 to frame 15. Assume that the object detection unit 102 detects a human-like object within the 10th frame. Assume that the detection score (detection reliability) is 0.5, which is less than the threshold value.

First, the similarity calculation unit 161 of the human movement determination unit 106 calculates the similarity between the object detection region in which the object is detected in the image in which the object is detected (frame 10 in FIG. 8) and regions at the same coordinate position as the object detection region in which the aforementioned object is detected in other frames included in the video (frames 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, and 15 in FIG. 8).

Next, the determination unit 162 of the human movement determination unit 106 counts the number of similarities calculated by the similarity calculation unit 161 that are below the threshold, and determines whether this number is a predetermined number (e.g., 8) or more. In the example of Figure 8, the similarity threshold is set to 0.7, and there are 11 frames with similarities below the threshold (marked with a circle in Figure 8). In this case, the human movement determination unit 106 determines that there is movement in the human-like object, since the number of other images with similarities below the threshold is a predetermined number or more.

In addition, when an object is detected in multiple images (frames) (i.e., when an object is detected in frames other than the 10th frame), the similarity calculation unit 161 calculates the similarity between each object detection region of the image (object detection image) in which the object is detected and the region in the other image (i.e., the other frames included in the video) at the same coordinate position as the object detection region. Then, the determination unit 162 determines whether or not there is a predetermined number of regions at the same coordinate position as the object detection region of the other image in which the similarity is less than the threshold value for each object detection region in which the object is detected (i.e., a determination similar to the determination in the 10th frame described above is performed). If the number of regions at the same coordinate position as the object detection region of the other image in which the similarity is less than the threshold value is equal to or greater than the predetermined number in at least one object detection region, it is determined that there is movement, and if the number of similarities less than the threshold value in all object detection images is less than the predetermined number, it is determined that there is no movement. Similarly, when there are multiple object detection regions within the same image, the similarity between each object detection region and other images is calculated, and the number of other images (regions) whose similarity falls below a threshold is counted to determine movement.

Here, we will explain the determination based on the condition that there are multiple images with similarity below a threshold. When comparing the object detection area of an image in which a human-like object has been detected with other images, the similarity may be calculated to be low if there happens to be an insect or ambient light such as a car light reflected. In this case, if the method calculates and compares the similarity only with the area of the same position range in the previous and next images, the insect or ambient light may be erroneously detected as a person. By comparing not only the previous and next images but also multiple other images and setting the determination condition to include the number of images (areas) with a similarity below a threshold, it is possible to suppress the occurrence of erroneous detection due to noise such as insects or ambient light, and to accurately determine that it is a moving human body.

<Other embodiments>
Other embodiments will be described below.

<<Detecting the movements of insects and other non-human objects>>
In the above, an embodiment has been described in which the presence or absence of object movement (whether the movement is of an object other than a human, such as an insect) is determined based on whether the object detection image and the previous and following images are similar, but the present invention can also be applied to an embodiment in which the presence or absence of object movement is determined based on the characteristics of the object movement. For example, the insect movement determination unit 105 can determine that the movement is of an insect or the like moving at high speed when the change in the ratio of white pixels between the object detection area of the object detection image and the area at the same position as the object detection area in the previous and following images is equal to or greater than a threshold value. Also, for example, the insect movement determination unit 105 can determine that the movement is of an insect or the like moving at high speed when the change in the size of the area (e.g., a rectangle) containing the object between the object detection area of the object detection image and each object detection area in the previous and following images is equal to or greater than a threshold value. At this time, if there is no object detection area in the previous or following image, a rectangular area that specifies the object position in the previous or following image is specified using an object tracking technique to make the determination. Also, for example, when the change in the moving direction of an area (e.g., a rectangle) containing an object between the object detection area of the object detection image and each object detection area of the previous or next image is equal to or greater than a threshold value, the insect movement determination unit 105 can determine that the movement is that of a fast moving insect, etc. In this case, when there is no object detection area in the previous or next image, an object tracking technique is used to specify a rectangular area that identifies the object position in the previous or next image and make the determination.

<<Determining the movement of each body part>>
In one embodiment of the present invention, the false detection determination device 10 can detect parts of a person's body (e.g., not limited to the entire body, but parts of the head, arms, upper body, lower body, left or right half of the body, etc.) and determine the movement of the detected parts of the person's body.

Specifically, the object detection unit 102 detects human body parts by machine learning or the like. The detection score determination unit 103 determines whether the detection score (detection reliability) is smaller than a threshold value. The human movement determination unit 106 determines whether or not there is movement of a human body part. In this case, it may be possible to set a threshold value for the detection score (detection reliability) and a threshold value for determining human-like object movement for each human body part.

For example, when learning the shapes of heads and arms, a hat or helmet placed indoors may be erroneously detected as a "head," and long, thin objects such as chairs, desk frames, and pillars as "arms." In order to prevent such erroneous detections, if conditions are set to uniformly determine that a person (moving object) is present when the similarity is below a threshold in a larger number of images, there is a risk that a person will be missed. Therefore, for object detection areas detected as arms, legs, and other parts of the body that tend to move a lot in typical human movements, it is possible to accurately detect objects that are human by determining that the object is present as a person when the similarity is below a threshold in a larger number of images than for object detection areas detected as parts of the body that move less, such as the head.

<<Motion determination based on brightness>>
The above describes an embodiment in which whether or not the object detection areas of two images (frames) are similar is determined by template matching such as ZNCC. However, the present invention can also be applied to an embodiment in which whether or not two images (frames) are similar is determined based on an evaluation value based on the brightness of the object detection areas of the two images (frames) (e.g., the average brightness difference).

In addition, the multiple images obtained by comparing the luminance difference of each pixel when the object detection area is compared can be trained into an AI model that classifies moving images, such as 3DCNN (AI that inputs a series of multiple images (moving images) and outputs their contents), and output which movement pattern it corresponds to. The trained model may be a model trained based on the similarity between an image (e.g., the first frame) other than the image in which the object detection unit 102 detected the object (in the example of Figures 7 and 8, the tenth frame) and other images. It may also be a model trained based on the similarity between images of adjacent frames (e.g., the similarity between the images of the first and second frames, the similarity between the images of the second and third frames, etc.).

<<Excluding cyclical movements>>
In one embodiment of the present invention, the human movement determination unit 106 can further determine whether the object movement is periodic or not, and the erroneous detection determination unit 107 determines that the object detection is a erroneous detection if the object movement is periodic (for example, the detection of a light, flag, or curtain swaying in the wind, or a flashing car headlight, or other object other than a human).

Specifically, the human movement determination unit 106 can input the similarity between the object detection area in which the object detection unit 102 detected an object and an area in another image at the same coordinate position as the object detection area (in the example of FIG. 8, the similarity between the 10th frame and frames 1 to 9 and 11 to 15) into a trained model using a method such as a support vector machine or random forest, and output whether the movement is periodic or not. The trained model may be a model trained based on the similarity between an image (for example, the 1st frame) other than the image in which the object detection unit 102 detected an object (in the example of FIG. 8, the 10th frame) and another image. It may also be a model trained based on the similarity between images of adjacent frames (for example, the similarity between the images of the 1st and 2nd frames, the similarity between the images of the 2nd and 3rd frames, etc.).

<<Learning human-like movements>>
The trained model may have previously learned patterns of human-like movements (such as a person moving or standing still). The human movement determination unit 106 may output which movement pattern corresponds to which of multiple classes from multiple frames.

＜Effects＞
In this way, in one embodiment of the present invention, it is possible to determine whether an object detected in an image captured by a security or surveillance camera or the like is a false positive (i.e., a person) or a false positive (i.e., not a person). Furthermore, in one embodiment of the present invention, if there is blown out highlights and it is determined that the movement of an insect (i.e., a fast moving movement) is present, it is possible to reduce false positives of intruders by excluding the movement from the detection target.

<Hardware Configuration>
9 is a block diagram showing an example of a hardware configuration of an error detection determination device 10 according to an embodiment of the present invention. The error detection determination device 10 has a central processing unit (CPU) 1001, a read only memory (ROM) 1002, and a random access memory (RAM) 1003. The CPU 1001, the ROM 1002, and the RAM 1003 form a so-called computer.

The false detection determination device 10 may also have an auxiliary storage device 1004, a display device 1005, an operation device 1006, an I/F (Interface) device 1007, and a drive device 1008. Each piece of hardware in the false detection determination device 10 is connected to each other via a bus B.

The CPU 1001 is a computing device that executes various programs installed in the auxiliary storage device 1004.

The ROM 1002 is a non-volatile memory. The ROM 1002 functions as a primary storage device that stores various programs, data, etc. required for the CPU 1001 to execute various programs installed in the auxiliary storage device 1004. Specifically, the ROM 1002 functions as a primary storage device that stores boot programs such as the Basic Input/Output System (BIOS) and the Extensible Firmware Interface (EFI).

RAM 1003 is a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). RAM 1003 functions as a primary storage device that provides a working area in which various programs installed in the auxiliary storage device 1004 are expanded when they are executed by the CPU 1001.

The auxiliary storage device 1004 is an auxiliary storage device that stores various programs and information used when the various programs are executed.

The display device 1005 is a display device that displays the internal state of the false detection determination device 10, etc.

The operation device 1006 is an input device through which the administrator of the misdetection determination device 10 inputs various instructions to the misdetection determination device 10.

The I/F device 1007 is a communication device that connects to a network and communicates with other devices.

The drive unit 1008 is a device for setting the storage medium 1009. The storage medium 1009 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The storage medium 1009 may also include semiconductor memory that records information electrically, such as EPROM (Erasable Programmable Read Only Memory) and flash memory.

The various programs to be installed in the auxiliary storage device 1004 are installed, for example, by setting the distributed storage medium 1009 in the drive device 1008 and reading the various programs recorded on the storage medium 1009 by the drive device 1008. Alternatively, the various programs to be installed in the auxiliary storage device 1004 may be installed by downloading them from a network via the I/F device 1007.

Although the examples of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

Reference Signs List 1 False detection determination system 10 False detection determination device 20 Imaging device 101 Image acquisition unit 102 Object detection unit 103 Detection score determination unit 104 Whiteout determination unit 105 Insect movement determination unit 106 Human movement determination unit 107 False detection determination unit 151 Similarity calculation unit 152 Determination unit 161 Similarity calculation unit 162 Determination unit 1001 CPU
1002 ROM
1003 RAM
1004 Auxiliary storage device 1005 Display device 1006 Operation device 1007 I/F device 1008 Drive device 1009 Storage medium

Claims (6)

an image acquisition unit that acquires a plurality of images;
an object detection unit that detects a human-like object in the image;
a blown-out highlight determination unit that determines whether or not there is blown-out highlight in an object detection region, which is a region in which the object is detected , within an object detection image, which is an image in which the object is detected;
a motion determination unit that, when it is determined that there is whiteout , determines whether or not there is any movement of an object other than a human being among the detected objects, and, when it is determined that there is no movement of an object other than a human being among the detected objects, further determines whether or not there is any movement of a human being among the detected objects, and, when it is determined that there is no whiteout, determines whether or not there is any movement of the human being among the detected objects;
an erroneous detection determination unit that determines that the detection of the object is an erroneous detection when it is determined that the detected object does not include the human movement and when it is determined that the detected object includes a movement of an object other than the human ;
The movement determination unit determines whether or not there is movement of an object other than the person in the detected object based on a similarity between the object detection area of the object detection image and areas at the same coordinate position as the object detection area in each of an image immediately before and an image immediately after the object detection image.
13. An erroneous detection determination device comprising : The movement determination unit determines whether or not the detected object includes movement of the person based on a similarity between the object detection region of the object detection image and a region at the same coordinate position as the object detection region in an image other than the object detection image among the plurality of images.
2. The erroneous detection determination device according to claim 1 . 3. The erroneous detection determination device according to claim 1 , wherein the erroneous detection determination unit determines that the detection of the object is not an erroneous detection when a reliability of the detection of the object in the image is greater than a threshold value. the detected object is a part of a human body;
The false detection determination device according to claim 3 , further comprising: a threshold value for determining whether or not the detected object is indicative of the motion of the person , and a threshold value for determining the reliability of the detection of the object, for each part of the person's body. A method executed by an erroneous detection determination device, comprising:
acquiring a plurality of images;
detecting a human-like object within the image;
determining whether or not there is blown-out highlights in an object detection region, which is a region in which the object is detected, within an object detection image, which is an image in which the object is detected;
when it is determined that there is blown-out highlights, determining whether or not there is any movement of an object other than a human among the detected objects based on a similarity between the object detection area of the object detection image and areas at the same coordinate position as the object detection area in each of an image immediately before and an image immediately after the object detection image, and when it is determined that there is no movement of an object other than a human among the detected objects, further determining whether or not there is any movement of a human among the detected objects; when it is determined that there is no blown-out highlights, determining whether or not there is any movement of the human among the detected objects;
and if it is determined that the detected object does not have the movement of the person and if it is determined that the detected object has the movement of an object other than the person, determining that the detection of the object is a false detection . An image acquisition unit that acquires a plurality of images;
an object detection unit that detects a human-like object in the image;
a blown-out highlight determination unit that determines whether or not there is blown-out highlight in an object detection region, which is a region in which the object is detected, within an object detection image, which is an image in which the object is detected;
a motion determination unit which, when it is determined that there is blown- out highlights, determines whether or not there is any movement of an object other than a human being among the detected objects based on a similarity between the object detection area of the object detection image and areas at the same coordinate position as the object detection area in each of an image immediately before and an image immediately after the object detection image, and when it is determined that there is no movement of an object other than a human being among the detected objects, further determines whether or not there is any movement of a human being among the detected objects, and when it is determined that there is no blown-out highlights, determines whether or not there is any movement of the human being among the detected objects;
A program for functioning as an erroneous detection determination unit that determines that the detection of the object is an erroneous detection when it is determined that the detected object does not include the human movement and when it is determined that there is movement of an object other than the human.

Images