JP5333489B2 - Object detection device and object detection program - Google Patents

Description

  The present invention relates to an object detection apparatus and an object detection program for processing a frame image of a monitoring target area imaged by an imaging apparatus such as a video camera to detect an object being imaged.

  2. Description of the Related Art Conventionally, there is an apparatus that detects an object such as a person or an object located in a monitoring target area by imaging a monitoring target area with an imaging device such as a video camera and processing a frame image of the imaging device. This type of device generates a difference image between a captured image of a monitoring target area and a stored background image of the monitoring target area, and detects an object captured from this difference image (patent) Reference 1 etc.).

JP 2000-324477 A

However, the conventional apparatus disclosed in Patent Document 1 and the like needs to appropriately update the background image in accordance with environmental changes such as brightness around the monitoring target area in order to ensure object detection accuracy. . That is, the conventional device, depending on the environmental changes around the monitored area requires configuration to properly update the background image, the device body was expensive. Also, if the background image is not updated properly, the object detection accuracy will be reduced.

  An object of the present invention is to provide an object detection device that ensures object detection accuracy and reduces the cost of a main body.

  Another object is to provide an object detection program to be installed in the object detection apparatus.

  The object detection device of the present invention is configured as follows in order to solve the above problems and achieve the object.

A 1st time average image generation part produces | generates a 1st time average image from the frame image which the imaging device imaged the monitoring area each time the 3rd time passed during the predetermined 1st time. In addition, the second time average image generation unit generates a second time average image from the frame image obtained by imaging the monitoring area by the imaging device every second time during a predetermined second time. To do. The first time is longer than the second time. For example, the first time is about 2 to 4 times the second time. The third time is shorter than the second time. In addition, by setting both the first time and the second time to be an integral multiple of the third time, the load on the generation of the first time average image and the second time average image is suppressed. It is done.

In addition, the difference image generation unit generates a difference image between the first time average image generated every time the third time elapses and the second time average image. This difference image is a difference image between the first time average image and the second time average image in which the latest frame images used for generation thereof are the same. The frame images used for generating a second time-averaged image is that contained in the frame image used to generate the first time average image.

  For this reason, the frame image used to generate the first time average image having a large ratio of the object being captured in the frame image used to generate the second time average image, If the ratio of not being captured in the frame image that is not used to generate the second time average image is small, this object appears in the difference image.

Therefore, by generating a difference image between the first time average image repeatedly generated every third time shorter than the second time and the second time average image, the object is positioned in the monitoring target area. As the elapsed time since then increases, the ratio of the image captured in the frame image used to generate the second time average image increases, and this object appears in the difference image.

  In addition, the detection unit detects the presence or absence of the object being imaged from the difference image.

  In addition, a determination unit that determines the type of an object based on the shape or size of the object that appears in the difference image generated by the difference image generation unit may be provided.

  According to the present invention, the object detection accuracy can be ensured and the cost of the main body can be reduced.

It is a figure which shows the structure of the principal part of an object detection apparatus. It is a figure which shows a setting table. It is a flowchart which shows the outline | summary of operation | movement of an object detection apparatus. It is a figure explaining the process which an object detection apparatus produces | generates a 3rd average image. It is a figure explaining the process which an object detection apparatus produces | generates a difference image. It is a figure which shows the frame image of the imaging device which is imaging the monitoring object area. It is a figure which shows the frame image of the imaging device which is imaging the monitoring object area. It is a figure which shows the state in which the neglected object appears in a difference image with progress of time.

  Hereinafter, an object detection apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a diagram illustrating a configuration of a main part of the object detection device. The object detection device 1 includes a control unit 2, an image processing unit 3, a storage unit 4, an operation unit 5, and an output unit 6. The object detection device 1 detects an object such as a person or an object that is almost stationary in the monitoring target area. For example, in a monitoring target area, a person who feels bad and falls down or crawls, or an abandoned object is detected.

  The object detection device 1 does not detect a passerby moving in the monitoring target area at a certain speed or an object such as a bag possessed by the passerby.

  The control unit 2 controls the operation of each part of the main body.

  An imaging device 10 is connected to the image processing unit 3. The imaging device 10 is a video camera, and is installed in a direction for imaging a monitoring target area. For example, the imaging device 10 outputs a frame image obtained by imaging the monitoring target area to the image processing unit 3 every 100 ms. The image processing unit 3 has a configuration corresponding to the first average image generation unit, the second average image generation unit, the difference image generation unit, and the determination unit referred to in the present invention.

  The image processing unit 3 has an image memory (not shown) used as a working area. The image processing unit 3 generates a first time average image from the frame images captured by the imaging device 10 over a predetermined first time T1. In addition, the image processing unit 3 generates a second time average image from the frame image captured by the imaging device 10 over a predetermined second time T2. The image processing unit 3 generates a difference image between the first time average image and the second time average image. The first time T1 is preferably set to be at least twice as long as the second time T2 so that an object that is almost stationary in the monitoring target area is likely to appear in the difference image. Here, a case where the first time T1 is three times the second time T2 (T1 = 3 × T2) will be described as an example.

  In addition, the image processing unit 3 generates the difference image every predetermined third time T3. The third time T3 is shorter than the second time T2. The third time T3 may be set according to the speed of the person moving in the monitoring target area. The third time T3 may be set to a time in seconds of about 10 to 20 seconds so that the object can be detected quickly.

  The first time T1 and the second time T2 are preferably set to an integral multiple of the third time T3 in order to reduce the processing load on the image processing unit 3. For example, the third time T3 may be a time in seconds obtained by dividing 1 minute such as 20 seconds or 15 seconds by an integer, and the first time T1 and the second time T2 may be time in minutes. . Further, as the third time T3 is shortened, the object detection delay can be reduced, but the processing load of the image processing unit 3 increases.

  The storage unit 4 stores data related to the setting of the first time T1, the second time T2, and the third time T3 described above. Specifically, the ratio α (here α = 3) between the first time T1 and the second time T2, the setting table 4a shown in FIG. 2, and the third time T3 (here T3). = 15 seconds). The setting table 4a shown in FIG. 2 registers the time length for the first time T1 for each time zone, but registers the time length for the second time T2 for each time zone. There may be. It is only necessary that the first time T1 and the second time T2 can be determined based on the ratio α between the first time T1 and the second time T2 and the setting table 4a. The setting table 4a may be configured such that the first time T1 and the second time T2 are individually set for each time zone. In this case, the ratio α between the first time T1 and the second time T2 is unnecessary.

Moreover, the setting table 4a shown in FIG. 2 exemplifies the case where the station platform where the train arrives and departs is set as the monitoring target area, and there is no train arrival and departure and there is no user etc. The device 1 is stopped. In addition, the first time T1 is set to 15 minutes for the time zone from 7:00 to 10:00 and the time zone from 17:00 to 20:00 when the train departure and arrival interval is relatively short at rush hour. For the time zone, the first time T1 is set to 30 minutes. In addition, when the station platform is set as the monitoring target area, the second time T2 is longer than the train arrival / departure interval in order to suppress detection of passengers waiting for the arrival of the train as an object at the station platform. The operation unit 5 that is preferably set includes an input device such as a keyboard and a mouse, and the ratio α between the first time T1 and the second time T2 stored in the storage unit 4, the setting table 4a, and the third An operator's input operation for changing the setting at time T3 is accepted.

  The output unit 6 outputs an object detection result.

  Hereinafter, the operation of the object detection apparatus 1 will be described.

  Each time a frame image is input from the imaging device 10, the object detection device 1 generates an edge image of the frame image. Specifically, an edge image with a change in brightness of 256 gradations is generated using a known Laplacian filter. The image processing unit 3 generates an addition image obtained by adding an edge image generated from a frame image input until the next object detection timing from a frame image input after an object detection timing described later. The object detection timing is the above-described third time T3 interval. This process is executed separately from the process shown in FIG.

  FIG. 3 is a flowchart showing the operation of the object detection apparatus. The object detection device 1 determines whether or not the object detection timing has come (s1). As described above, this object detection timing is the timing when the third time T3 has elapsed from the previous object detection timing. That is, the object detection device 1 determines that the object detection timing has come every time the third time T3 elapses.

  When the object detection device 1 determines that the object detection timing is reached in s1, the image processing unit 3 generates a first time average image (s2) and a second time average image (s3).

  At this time, the image processing unit 3 obtains an added image obtained by adding the edge image generated from the frame image input until the current object detection timing from the frame image input after the previous object detection timing. . The image processing unit 3 generates a third time average image obtained by dividing the added image by the number of input frame images, and stores this in the image memory. As shown in FIG. 4, the image processing unit 3 stores a third time average image in the image memory at each successive object detection timing.

  When the first time T1 is n times the third time T3, the image processing unit 3 adds the latest n third time average images stored in the image memory, and divides this by n. By doing so, a first time average image is generated.

  In addition, when the second time T2 is m times the third time T3, the image processing unit 3 adds the m most recent third time average images stored in the image memory, and adds this to m The second time average image is generated by dividing by.

  Thus, since the image processing unit 3 is configured to store the third time average image in the image memory, it is possible to suppress the load on the generation of the first time average image and the second time average image. it can.

  The first time average image generated by the above-described processing is the latest frame image input from the imaging device 10 to the image processing unit 3 at the time when it is determined that the object detection timing is reached in s1, and is the first frame image. Yes. In addition, the first time average image uses a frame image input before the first time T1 before the first frame image as a rear end frame image. The first time average image is an average image of edge images generated from the frame image input between the first frame image and the rear frame image.

  The second time average image uses the latest frame image input from the imaging device 10 to the image processing unit 3 as the first frame image when it is determined that the object detection timing is reached in s1. In the second time average image, a frame image input before the second time T2 before the first frame image is used as a rear end frame image. The second time average image is an average image of edge images generated from the frame image input between the first frame image and the rear frame image.

  That is, the first time average image is an average image of the frame images input from the imaging device 10 to the image processing unit 3 over the first time T1. The second time average image is an average image of the frame images input from the imaging device 10 to the image processing unit 3 over the second time T2. Further, as is clear from the above description, the first frame image used for creating the first time average image and the second time average image is the same frame image. The frame images used for generating the second time average image are all included in the frame images used for generating the first frame image.

  The image processing unit 3 generates a difference image between the first time average image generated in s2 and the second time average image generated in s3 (s4).

  The image processing unit 3 extracts an imaged object from the difference image created in s4 (s5). In s5, the difference image is binarized, and a known contraction / expansion process is performed on the binarized image. The contraction process is a process of converting a black pixel having a white pixel in the vicinity into a white pixel. The expansion process is a process of converting a white pixel having a black pixel in the vicinity into a black pixel. By performing this contraction / expansion process, the object being imaged is broken and noise is removed. Then, the image processing unit 3 extracts an image area that exceeds a predetermined size as an object.

  The image processing unit 3 detects the position on the frame image for each object extracted in s5 (s6). If the object extracted in s5 is the first object extracted this time, an ID and a detection time are assigned to the object (s7, s8), and the detection is performed along with the position and size on the frame image detected in s6. Register in the table (s9). In s7, for each object extracted in s5, referring to the detection table, this is the first object extracted this time depending on whether or not the object associated with the position on the frame image detected in s6 has already been registered. Judging whether or not. As will be described later, this detection table has an area for storing the type and state for each object.

  It should be noted that the processing related to s8 and s9 is not performed for the object determined to have already been extracted in s7.

  If there is an object whose type (person, thing) or state has not been determined among the objects extracted in s5 (s10), the image processing unit 3 will determine the type (person) for each object that has not been determined. , Object) and state are discriminated (s11). In s11, a fallen person, a crouched person, a neglected bag, etc. are discriminated using the shape and size of the object, the angle formed by the floor normal and the principal axis of the object, and the like. The type and state determined in s11 are registered in the detection table (s12). In s12, the type and state determined in s11 are associated with the ID of the corresponding object registered in the detection table.

  Note that an object whose type or state cannot be determined in s11 is left unidentified in the detection table. In addition, the processing related to s11 and s12 is not performed for the object that has been determined that the type and state have already been determined in s10.

  The object detection apparatus 1 performs output related to detection of the object whose type and state are determined this time by the output unit 6 (s13), and returns to s1. For an object whose type and state have already been discriminated, no output related to detection of the object is performed. For this reason, the same object will not be issued twice, and the staff will not be confused.

  The object detection device 1 executes the process shown in FIG. 3, and as shown in FIG. 5, every time the third time T <b> 3 elapses (every object detection timing comes), 2 average images, and further, a difference image between the first average image and the second average image is generated.

  As shown in FIG. 5, the first average image has the latest frame image input to the image processing unit 3 from the imaging device 10 at the time when the object detection timing is reached, and the first frame image. The frame image input before the first time T1 is set as the rear end frame image. The first average image is an average image of frame images from the rear end frame image to the first frame image. The second average image has the latest frame image input to the image processing unit 3 from the imaging device 10 at the time of object detection timing as the first frame image, and the second time T2 from the first frame image. The frame image input before is set as the rear end frame image. The second average image is an average image of the frame images from the rear end frame image to the first frame image.

The first average image that generates the difference image and the second average image have the same top frame image. In addition, since the second time T2 is shorter than the first time T1, the frame image used for generating the second average image is included in the frame image used for generating the first average image.

  As can be seen from FIG. 5, the object appearing in the generated difference image has a ratio of the frame image in which the object is captured at substantially the same position on the frame image in the frame image used to generate the second average image. As it grows, its shape becomes clearer.

  On the other hand, an object moving in the monitoring area is a frame image in which the object is captured at substantially the same position on the frame image in the first average image and the frame image used to generate the second average image. Since the ratio is small, it is difficult to appear in the first average image and the second average image, and as a result, it is difficult to appear in the difference image.

  Furthermore, an object such as a bench installed in the monitoring area is located at substantially the same position on the frame image in almost all the frame images used for generating the first average image and the second average image. Since the image is captured, it appears clearly in the first average image and the second average image, so it is difficult to appear in the difference image.

  For example, it is assumed that the station platform shown in FIG. 6 is a frame image of the imaging device 10 that is imaging the monitoring target area. FIG. 6 shows the background of the monitoring target area, in which the object is not imaged. Here, in FIG. 7, when a package is left in an area surrounded by a broken line, immediately after the package is left, a frame in which the package is captured in the frame image used for generating the second time average image. Since the image ratio is small, it hardly appears in the difference image as shown in FIG.

  However, as the elapsed time after the baggage is left unattended, in the frame image used to generate the second time average image, the ratio of the frame image in which the baggage is captured increases. As shown in FIG. 8B and FIG. 8C, the difference image appears clearly.

  The same applies to a person who feels bad and falls down in the monitored area.

  As described above, the object detection apparatus 1 can detect an object located in the monitoring target area without registering a background image. Therefore, a configuration for updating the background image with respect to environmental changes such as brightness around the monitoring target area is unnecessary, and the cost of the apparatus main body can be reduced. In addition, since the first time average image and the second time average image change according to environmental changes such as the brightness around the monitored area, the influence due to the environmental changes such as the brightness around the monitored area. It is possible to detect an object with reduced noise.

  When the monitoring target area is a station platform, the second time T2 is set to be longer than the train arrival / departure interval in order to suppress detection of passengers waiting for the arrival of the train as an object at the station platform. Is preferred. By setting the second time T2 to be longer than the train arrival / departure interval, passengers waiting for the arrival of the train at the station platform can be prevented from appearing in the second average image. In addition, by setting the second time T2 to be longer than the train arrival and departure interval, the first time T1 is also longer than the train arrival and departure interval, so the train is waiting for arrival at the station platform. Passengers are also prevented from appearing in the first average image.

  That is, the first time T1 and the second time T2 may be set according to the train arrival / departure interval. This object detection apparatus 1 should just set the setting table 4a according to the train arrival / departure intervals.

  In addition, the image processing unit 3 may delete from the image memory the third time average image that is no longer used for generating the first time average image as time passes. Thereby, the image memory can be used efficiently.

  The object detection apparatus 1 may set the monitoring target area not only at the station platform but also at another location.

  The object detection device 1 may detect the object by processing the frame image captured by the imaging device 10 in real time, or the frame image captured by the imaging device 10 may be a recording medium such as a hard disk. The object may be detected by batch processing on the frame image recorded on the recording medium.

DESCRIPTION OF SYMBOLS 1 ... Object detection apparatus 2 ... Control part 3 ... Image processing part 4 ... Memory | storage part 4a ... Setting table 5 ... Operation part 6 ... Output part 10 ... Imaging device

Claims (8)

An object detection device that detects a captured object by processing a frame image of a monitoring target area captured by an imaging device,
A first time average image generating unit that generates a first time average image from a frame image captured by the imaging device during a predetermined first time;
A second time average image generating unit that generates a second time average image from a frame image captured by the imaging device during a predetermined second time shorter than the first time;
A difference image generation unit that generates a difference image between the first time average image and the second time average image;
A detection unit that detects the presence or absence of an imaged object from the difference image generated by the difference image generation unit ;
The first time average image generation unit generates the first time average image every time a predetermined third time shorter than the second time elapses,
The second time average image generation unit generates the second time average image every time the third time elapses,
The difference image generation unit is the first time average image and the second time average image generated every time the third time elapses, and the latest frame image used for the generation is the same. An object detection device that generates a difference image between a certain first time average image and the second time average image . The object device according to claim 1, wherein both the first time and the second time are times that are integral multiples of the third time. The object detection apparatus according to claim 1, wherein the first time is at least twice as long as the second time. The object detection apparatus according to claim 1, further comprising a determination unit that determines a type of the object detected by the detection unit. A storage unit for storing a ratio between the first time and the second time;
It said first time, or for one of the second time, the object detection apparatus according to any one of claims 1 to 4, which includes a setting acceptance unit that accepts a setting of the time length, the. The storage unit stores a setting table in which a time length is registered for one of the first time and the second time for each time zone,
The object detection apparatus according to claim 5 , wherein the setting reception unit receives a setting change for the setting table. Train in accordance with the arrival and departure intervals of the train in the station platform to depart, the first time, and defines the second time, the object detecting device according to any one of claims 1 to 6 The control unit of the object detection device that processes the frame image of the monitoring target area imaged by the imaging device and detects the imaged object,
A first time average image generating step of generating a first time average image from a frame image captured by the imaging device during a predetermined first time;
A second time average image generating step of generating a second time average image from a frame image captured by the imaging device during a predetermined second time shorter than the first time;
A difference image generation step for generating a difference image between the first time average image and the second time average image in which the latest frame images used for the generation are the same ;
A detection step of detecting the presence or absence of an imaged object from the difference image generated in the difference image generation step ,
The first time average image generation step is a step of generating the first time average image every time a predetermined third time shorter than the second time elapses,
The second time average image generation step is a step of generating the second time average image every time the third time elapses;
The difference image generation step includes the first time average image and the second time average image generated every time the third time elapses, and the latest frame image used for the generation is the same. An object detection program , which is a step of generating a difference image between a certain first time average image and the second time average image .