JP5222909B2 - Collapse detection system and collapse detection method - Google Patents

Description

  The present invention relates to a collapse detection system and a collapse detection method for detecting a collapse such as a cliff collapse, and more particularly, to a collapse detection system and a collapse detection for detecting a small collapse phenomenon such as a pebbles falling as a precursor of a large-scale collapse. Regarding the method.

  In general, in monitoring of a landslide, a large rock collapse is detected using a wire sensor. In the work at the risk point of earth and sand disaster such as restoration work at the disaster site, it is possible to reduce the risk by discovering small collapses (hereinafter referred to as “small collapses”) such as pebbles falling as a precursor to a large collapse. It is also important to avoid it. Usually, a supervisor is assigned to visually monitor a dangerous spot. However, it is difficult for a human to cover a wide range, and monitoring by a machine is desired as a means for supporting a human.

  There are not many precedent examples of video surveillance specializing in earth and sand disasters, but considering moving pebbles as moving objects, various moving object detection technologies are being studied in the field of monitoring humans and vehicles. However, in the surveillance video, the small collapse is very small compared to the monitored area, and the color is similar to the background slope, so the normal moving object detection method identifies the small collapse as noise etc. There was a problem that it was difficult. As a technique to solve this problem, focusing on the fact that noise becomes an isolated point in the space-time where the frames of the monitoring video are arranged in time series, the collapse forms a continuous trajectory, There is a technique using a spatio-temporal Gabor filter characterized by improving the discrimination performance from noise (Non-patent Document 1).

  The spatiotemporal Gabor filter obtains local spatiotemporal frequency information of an input image by a convolution operation using a three-dimensional Gabor function in which a sine wave is localized using a Gaussian function as a window function. Usually, a plurality of types of spatiotemporal Gabor filters are prepared and a filter set is configured. By calculating the motion energy with all the spatiotemporal Gabor filters prepared, it is possible to obtain a response to the motion that causes various trajectories. Specifically, given that the spatio-temporal Gabor filter has a strong response when the corresponding parameter matches the moving direction / velocity of the object, Gabor for the speed, moving direction, etc. that is assumed to be small collapse. When the filter response (kinetic energy) exceeds a predetermined threshold, it is determined that a small collapse has occurred.

Ken Okamoto et al. "Detection of landslide disaster precursors using spatio-temporal Gabor filter" IEICE technical report. PRMU, Pattern recognition and media understanding Vol. 108, No. 327, pp.139-144, 2008.

  By the way, in video surveillance, in addition to identifying a detection target and noise included in the video, it is also necessary to identify a change (disturbance phenomenon) other than a collapse phenomenon such as flying insects. Small collapse is a phenomenon that occurs from the top to the bottom of the slope, and its direction can be assumed in advance, so a spatio-temporal Gabor filter that responds strongly to the vertical movement of the slope is used. If the response characteristic with respect to the moving direction is used, it is possible to distinguish a certain level of small collapse and disturbance.

  However, the spatio-temporal Gabor filter response is affected not only by the movement direction (spatio-temporal characteristics) of the detection target but also by spatial characteristics such as the shape of the detection target and the contrast (brightness difference) with the background. . The small collapse can be regarded as a forward motion with a low contrast, but for example, when a bird with a high contrast with the background performs an orthogonal motion in the direction of the spatiotemporal Gabor filter, it is theoretically difficult to distinguish between the two. That is, the method based on the response characteristics of the spatiotemporal Gabor filter in the moving direction and speed cannot always distinguish between the disturbance phenomenon and the small collapse.

  An object of the present invention is to provide a collapse detection system and a collapse detection method that can accurately detect a collapse that is a detection target as a disturbance phenomenon, in view of the conventional technology described above.

  In order to achieve the above object, the invention according to the first aspect is a collapse detection system that detects a collapse, and strongly responds to two different moving directions from a shooting unit that captures a monitoring video to be detected. Using two spatio-temporal Gabor filters, the orthogonal projection component in the direction of the two-dimensional velocity of each pixel in the image captured by the imaging unit is set as the speed at which the movement energy is maximized for each of the two moving directions. A calculation unit that calculates a two-dimensional velocity by assuming that the maximum value of motion energy for each of the two moving directions is equal to or greater than a predetermined threshold, and the two-dimensional velocity is within a predetermined range, The gist of the present invention is that it includes a determination unit that determines that a collapse has occurred at the location.

  The gist of the invention according to the second aspect is that, in the invention according to the first aspect, the two moving direction components detected by the spatiotemporal Gabor filter are orthogonal to each other.

  The invention according to a third aspect is the invention according to the first or second aspect, comprising a plurality of sets of moving directions detected by the spatio-temporal Gabor filter, wherein the calculating unit has the movement energy in each moving direction. The gist is that an average value of the two-dimensional velocities calculated from the set of the spatio-temporal Gabor filters exceeding a predetermined threshold is used as the two-dimensional velocities of the pixel.

  The invention according to a fourth aspect is the invention according to any one of the first to third aspects, wherein the inter-frame difference value obtained from the video imaged by the imaging unit is superimposed on the original image and collapsed. The gist is that an enhancement processing unit for emphasizing a region is provided.

  The invention according to a fifth aspect is the invention according to any one of the first to fourth aspects, wherein the determination unit performs a morphological process for excluding spatially isolated pixels, and the morphologically processed image thereof The gist is to determine whether or not a collapse has occurred.

  In order to achieve the above object, the invention according to the sixth aspect is a collapse detection method for detecting collapse, and is strong against two different moving directions, and a shooting step for capturing a monitoring image to be detected. Using the two spatio-temporal Gabor filters that respond, the speed at which the movement energy is maximized in each of the two moving directions is determined so that the two-dimensional speed of each pixel in the image taken in the photographing step is positive in that direction. A calculation step of calculating a two-dimensional velocity by regarding it as a projection component, and when the maximum value of motion energy for each of the two movement directions is equal to or greater than a predetermined threshold, and the two-dimensional velocity is within a predetermined range, The gist of the invention is that it includes a determination step for determining that a collapse has occurred at the location of the pixel.

  The gist of the seventh aspect of the invention is that, in the sixth aspect of the invention, the two moving direction components detected by the spatiotemporal Gabor filter are orthogonal to each other.

  The invention according to an eighth aspect is the invention according to the sixth or seventh aspect, comprising a plurality of sets of movement directions detected by the spatio-temporal Gabor filter, wherein the movement energy in each movement direction is determined in the calculating step. The gist is that an average value of the two-dimensional velocities calculated from the set of the spatio-temporal Gabor filters exceeding a predetermined threshold is used as the two-dimensional velocities of the pixel.

  The invention according to the ninth aspect is the invention according to any one of the sixth to eighth aspects, wherein the inter-frame difference value obtained from the video shot in the shooting step is further superimposed on the original image and collapsed. The gist is that an emphasis processing step for emphasizing a region is provided.

  The invention according to a tenth aspect is the invention according to any one of the sixth to ninth aspects, wherein a morphological process for excluding spatially isolated pixels is performed in the determination step, and the morphologically processed image The gist is to determine whether or not a collapse has occurred.

  According to the present invention, it is possible to provide a collapse detection system and a collapse detection method that can accurately detect a collapse that is a detection target by identifying it as a disturbance phenomenon.

It is a lineblock diagram of a collapse detection system in a 1st embodiment of the present invention. It is a flowchart which shows operation | movement of the collapse detection system in the 1st Embodiment of this invention. It is explanatory drawing of the velocity vector (v) in the 1st Embodiment of this invention. It is a flowchart which shows an example of the determination process in the 1st Embodiment of this invention. It is explanatory drawing of the identification by the velocity vector (v) in the 1st Embodiment of this invention. It is explanatory drawing of the filter pair in the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the collapse detection system in the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
First, the outline of the present invention will be described. That is, in the present invention, a set of two spatio-temporal Gabor filters that respond strongly to two different moving directions is used, and the speed at which the movement energy is maximized for each of the two moving directions is determined as a two-dimensional speed on the screen. The two-dimensional velocity is calculated by assuming that it is an orthogonal projection component in the corresponding direction. Thus, a two-dimensional vector amount (hereinafter referred to as “speed vector” or “two-dimensional speed”) that is the speed of each pixel on the monitoring screen is obtained. Then, it is determined whether or not the response (motion energy) to the spatiotemporal Gabor filter is greater than or equal to a predetermined threshold value and the velocity vector is within a range that is assumed to be small collapse. In this way, it is possible to identify whether the moving object on the screen detected by the set of spatio-temporal Gabor filters is a small collapse or other disturbance phenomenon.

  FIG. 1 is a configuration diagram of a collapse detection system 10 according to an embodiment of the present invention. The collapse detection system 10 is a system that detects a collapse such as a cliff collapse by video. Functionally, the imaging unit 11, the three-dimensional data creation unit 12, the calculation unit 13, the determination unit 14, An output unit 15 and a storage unit 15 are provided. The imaging unit 11 captures a monitoring image to be detected. The three-dimensional data creation unit 12 creates two-dimensional image × time three-dimensional data using the video imaged by the imaging unit 11. The calculation unit 13 uses two spatio-temporal Gabor filters that respond strongly to two different movement directions, and the three-dimensional data generation unit 12 generates a speed that maximizes the movement energy for each of the two movement directions. The two-dimensional velocity is calculated by regarding the two-dimensional velocity of each pixel in the three-dimensional data (video captured by the imaging unit 11) as an orthogonal projection component in the direction. The determination unit 14 determines that a collapse has occurred at the location of the pixel when the maximum value of the motion energy for each of the two moving directions is equal to or greater than a predetermined threshold and the two-dimensional velocity is within a predetermined range. To do. The output unit 15 outputs the determination result of the determination unit 14 to a liquid crystal display or the like. The storage unit 16 stores various parameters (such as motion energy and velocity vector) used for the determination process of the determination unit 14.

  FIG. 2 is a flowchart showing the operation of the collapse detection system 10 according to the embodiment of the present invention. Hereinafter, the configuration of the collapse detection system 10 will be described with reference to FIG.

  First, the monitoring video is converted into two-dimensional image × time three-dimensional data, and a spatio-temporal Gabor filter process is performed as a spatio-temporal feature extraction process of the three-dimensional data (steps S 1 → S 2 → S 3, S5). The velocity components in two different directions are calculated by the spatio-temporal Gabor filter (steps S4 and S6), and these are added together to capture the movement of each pixel on the screen as a two-dimensional velocity vector (step S7). Is used to carry out a small collapse determination process (step S8).

The spatio-temporal Gabor filter process is a feature extraction process that captures the motion characteristics of the input image. Specifically, the input image (I) and a linear filter (g _φ ) are subjected to a convolution operation (represented by *) to react The movement energy (E) is obtained from the strength (r _φ ).

The linear filter (g _φ ) is expressed by the following formula.

Where γ is the size of the spatial scale, σ is the standard deviation of the Gaussian function applied to the spatial plane, v is the velocity representing the center position of the Gaussian function applied to the spatial plane, λ is the spatial period (1 / λ is the spatial frequency) ), Φ is the phase correction, μ _t is the average value of the Gaussian function applied in the time direction, τ is the standard deviation of the Gaussian function applied in the time direction, and θ represents the spatial direction of the filter. θ corresponds to the direction of collapse, and v corresponds to the speed of collapse. When moving in the opposite direction, the velocity v has a negative value. The linear filter is composed of parameters of direction (θ), velocity (v), and period (λ). For example, if you want to detect a collapse that moves about 3 pixels per frame downwards by about 4 x 4 pixels, you can set θ = 0 ° (downward), v = 3 pixels / frame, and λ = 8 pixels. Is appropriate. Of course, other values can be used as parameters, and the detection characteristics can be changed by changing the parameters.

Obtaining velocity components in two different directions means that the velocity (v ₁ , v ₂ ) maximizes the motion energy (E ₁ , E ₂ ) in each direction with respect to the two directions (θ ₁ , θ ₂ ). Is to seek. FIG. 3 shows the relationship between the velocity (v ₁ , v ₂ ) obtained in two directions and the two-dimensional velocity vector (v). As shown in this figure, the velocity (v ₁ , v ₂ ) is a value obtained by orthogonally projecting the velocity vector (v) with respect to each direction (θ ₁ , θ ₂ ). When unit vectors in two directions are set as e ₁ and e ₂ , the two-dimensional velocity vector is obtained as follows.

v = ((v ₁ -v ₂ cos (Δθ)) e ₁ + (v ₂ -v ₁ cos (Δθ)) e ₂ ) / (1- cos ² (Δθ))
However, Δθ = θ ₁ -θ ₂
Note that the two-dimensional velocity vector can be calculated if velocity components in any two different moving directions are obtained. If the two moving directions are orthogonal to each other, the velocity vector (v) can be simplified to the following equation.

v = v ₁ e ₁ + v ₂ e ₂
FIG. 4 is a flowchart illustrating an example of the determination process of the determination unit 14. First, it is determined whether or not there is a moving object depending on whether or not the motion energy (E ₁ , E ₂ ) exceeds a predetermined threshold (steps S11 → S12). That is, if the motion energy (E ₁ , E ₂ ) exceeds a predetermined threshold, it is determined that there is a moving object, while the motion energy (E ₁ , E ₂ ) does not exceed a predetermined threshold. Determines that there is no moving object (step S13). Next, it is determined whether or not the velocity vector (v) is within a range assumed in advance, and whether or not a small collapse is detected (step S14). That is, if the velocity vector (v) is within the range assumed in advance, it is determined that there is a small collapse (step S16). On the other hand, if the velocity vector (v) is not within the range assumed in advance, the insect It is determined that the disturbance is a flight phenomenon (step S15).

FIG. 5 is a diagram for explaining identification by the velocity vector (v). As shown in this figure, if the monitoring target slope 21 and the monitoring condition (imaging condition) are determined, the velocity vector assumed to be collapsed therein is limited to a certain range 22. Therefore, when the velocity vector (v) exists in the assumed range 22 at the place recognized as the moving object by the motion energy (E ₁ , E ₂ ), the determination unit 14 is small collapsed. Judge that there is. On the other hand, when the velocity vector (v ′) exists outside the assumed range 22, it is determined that the moving object is a disturbance phenomenon such as flying insects.

  As described above, according to the collapse detection system 10 in the first embodiment of the present invention, the collapse that is the detection target can be identified as a disturbance phenomenon and accurately detected. In other words, landslides and landslides can be automatically detected by detecting them from disturbances such as flying insects from the precursor stage. Prevention). Although false alarms are a problem in these automatic detection systems, the present invention solves the problem of discrimination from disturbance phenomenon and contributes to improvement of practicality. As a function of the disaster prevention system, early detection in sediment disaster monitoring becomes possible by combining it with the function of automatically sending alarms such as alarms and e-mail delivery to the monitoring center and disaster prevention manager from the judgment result of collapse occurrence.

Note that the determination unit 14 may perform a morphological process that excludes pixels that are spatially isolated, and may determine whether a collapse has occurred using the morphologically processed image. In spatio-temporal Gabor filter processing (kinetic energy (E ₁ , E ₂ )), noise cannot necessarily be completely removed. However, as disclosed in Non-Patent Document 1, spatio-temporal Gabor filter processing is not It has excellent discrimination performance, and it is possible to further improve accuracy by combining morphological processing, which is a general isolated point removal technique.

Further, here, the monitoring video is used as it is, but the present invention is not limited to this. That is, an enhancement processing unit that emphasizes the collapsed region by superimposing the inter-frame difference value obtained from the video captured by the imaging unit 11 on the original image may be provided. More specifically, by taking the absolute value of the time difference of each pixel value and adding it to the original image as in the following equation, it is possible to emphasize a slight change due to collapse.

Here, _I't is the pixel value of the frame at time t after enhancement, the pixel value of frame I _t is the time t, the pixel value of the frame I _t-1 at time t-1, alpha is the degree of emphasis The coefficient shown, n is an index. The change due to noise is also emphasized, so it is not suitable for surveillance images with excessive noise, but the spatio-temporal Gabor filter has excellent discrimination performance from noise, so it is effective for increasing the sensitivity of images with low contrast. is there.

(Second Embodiment)
In the second embodiment, when calculating the velocity vector (v), a set of parameters (θ ₁ , θ ₂ ) of the spatio-temporal Gabor filter is implemented in parallel in a plurality of directions, and the average value of the results is obtained. replace. That is, the calculation unit 13 includes a plurality of sets of moving directions detected by the spatiotemporal Gabor filter, and the calculation unit 13 calculates the average of the two-dimensional velocities calculated from the set of spatiotemporal Gabor filters in which the movement energy in each moving direction exceeds a predetermined threshold. Let the value be the two-dimensional velocity of the pixel.

FIG. 6 shows a case where four sets of filters (hereinafter referred to as “filter pairs”) in which orthogonal parameters (θ ₁ , θ ₂ ) are shifted every 22.5 ° are used, and FIG. The operation of the collapse detection system 10 is shown. Of course, the number of filter pairs is not limited to four. In the present embodiment, basically, an averaging process (S31) for integrating velocity vectors obtained by a plurality of filter pairs is added to the first embodiment. However, this averaging process is different from a simple averaging process in that the identification is performed based on the motion energy threshold value and the averaging process is performed using only effective velocity vectors (the process for performing the motion energy threshold process). The motion energy threshold value is stored in the storage unit 16 in advance. The determination part 14 determines that there is no moving object in the said location, when there is no filter pair whose motion energy exceeds a threshold value in the process of averaging (step S32).

The response to the spatiotemporal Gabor filter has a direction dependency depending on the shape of the collapsed object, disturbance phenomenon, or other spatial characteristics. By averaging using a plurality of filter pairs, the influence of these dependencies can be reduced. Further, in calculating the velocity (v ₁ , v ₂ ) in each direction by the spatio-temporal Gabor filter, it is practically obtained from calculation of discrete representative values, and it is difficult to avoid rounding errors and the like. It is also possible to reduce the influence of rounding errors and the like by the conversion processing. On the other hand, providing a filter more than necessary is disadvantageous from the viewpoint of calculation cost and the like. However, by increasing the number of filter pairs in accordance with the ability of the computer, it is possible to calculate a speed vector with higher accuracy, and to improve the accuracy of identifying small collapses and disturbance phenomena.

  As described above, according to the collapse detection system 10 according to the second exemplary embodiment of the present invention, the averaging process for integrating the velocity vectors obtained by the plurality of filter pairs is performed. Can be identified as a disturbance phenomenon.

  It should be noted that the present invention is also effective for general movement detection of microscopic objects having the same color as the background environment similar to the landslide disaster precursor. For example, it can be applied to monitoring of living things in a low-contrast environment where illumination is not available.

DESCRIPTION OF SYMBOLS 10 ... Collapse detection system 11 ... Imaging | photography part 12 ... Three-dimensional data preparation part 13 ... Calculation part 14 ... Determination part 15 ... Output part 16 ... Memory | storage part

Claims (10)

A collapse detection system for detecting a collapse,
A shooting section for shooting the surveillance video to be detected;
Using two spatio-temporal Gabor filters that respond strongly to two different moving directions, the speed at which the movement energy is maximized for each of the two moving directions is determined for each pixel in the image taken by the photographing unit. A calculation unit that calculates a two-dimensional velocity by regarding the orthogonal projection component of the two-dimensional velocity in the direction;
A determination unit that determines that a collapse has occurred at a location of the pixel when the maximum value of the motion energy for each of the two moving directions is equal to or greater than a predetermined threshold and the two-dimensional velocity is within a predetermined range. When,
A collapse detection system characterized by comprising:   The collapse detection system according to claim 1, wherein the two moving direction components detected by the spatiotemporal Gabor filter are orthogonal to each other. A plurality of sets of moving directions detected by the spatiotemporal Gabor filter,
The calculation unit uses the average value of the two-dimensional velocities calculated from the set of spatio-temporal Gabor filters in which movement energy in each moving direction exceeds a predetermined threshold value as the two-dimensional velocity of the pixel. The collapse detection system according to claim 1 or 2.   4. The image processing apparatus according to claim 1, further comprising an enhancement processing unit that emphasizes a collapsed region by superimposing an inter-frame difference value obtained from a video photographed by the photographing unit on an original image. The collapse detection system according to claim 1.   The said determination part implement | achieves the morphological process which excludes the pixel isolated spatially, and determines whether collapse occurred using the morphological process image. 5. The collapse detection system according to any one of 4 above. A collapse detection method for detecting collapse,
A shooting step for shooting the surveillance video to be detected;
Using two spatio-temporal Gabor filters that respond strongly to two different movement directions, the speed at which the movement energy is maximized for each of the two movement directions is determined for each pixel in the image taken in the photographing step. A calculation step for calculating a two-dimensional velocity by regarding it as an orthogonal projection component in the direction of the two-dimensional velocity,
A determination step of determining that a collapse has occurred at the location of the pixel when the maximum value of the motion energy for each of the two moving directions is equal to or greater than a predetermined threshold and the two-dimensional velocity is within a predetermined range. When,
A collapse detection method characterized by comprising:   The fall detection method according to claim 6, wherein the two moving direction components detected by the spatiotemporal Gabor filter are orthogonal to each other. A plurality of sets of moving directions detected by the spatiotemporal Gabor filter,
In the calculating step, an average value of the two-dimensional velocities calculated from the set of the spatiotemporal Gabor filters in which the movement energy in each moving direction exceeds a predetermined threshold is used as the two-dimensional velocity of the pixel. The collapse detection method according to claim 6 or 7.   9. The method according to claim 6, further comprising an emphasis processing step of emphasizing a collapsed region by superimposing an inter-frame difference value obtained from the video imaged in the imaging step on an original image. The collapse detection method according to claim 1.   7. The determination step according to claim 6, wherein a morphological process for excluding spatially isolated pixels is performed, and it is determined whether or not a collapse has occurred using the morphologically processed image. The collapse detection method according to claim 9.

Images