JP5968752B2 - Image processing method, image processing apparatus, and image processing program for detecting flying object - Google Patents

Description

  The present invention relates to an image processing method, an image processing apparatus, and an image processing program for detecting a flying object. More specifically, the present invention relates to a technique for detecting the entry of a low-brightness moving body flying into a monitoring target area and a technique for measuring the number of times a low-brightness moving body entering the monitoring target area enters.

  If there are many target areas to be monitored for the presence or absence of moving objects, or if the monitoring time is extended over a long period of time, the imaging equipment or processing equipment may be It is often desirable to be able to determine the presence or absence of a moving object mechanically and automatically.

  As a conventional mechanism for detecting the ingress of a moving body into a monitored area, for example, an intruder monitoring device, a video monitoring device installed at a position where the intruder monitoring area can be monitored, these intruder monitoring devices, and A communication line connecting the video surveillance device, the video surveillance device includes a surveillance camera and a video surveillance board for imaging the intruder monitoring area, and the video surveillance board is an object imaged by the surveillance camera. An alarm signal transmitter that sends an alarm signal to the intruder monitoring device when the image image and the pre-stored object assumed image are compared and matched, and an object image image data request from the intruder monitoring device And an object video image data generating means for transmitting an object video image data signal to the intruder monitoring device by receiving the signal. The alarm device that issues an alarm by receiving an alarm signal transmitted from the visual device side, and the object video image captured on the video monitoring device side and the target object image are almost the same on the intruder monitoring device side A monitor screen provided so that an object video image data request signal can be transmitted to the intruder monitoring device side when the intruder monitoring device side determines that the object video image data signal is In some cases, the object video image data is displayed on the monitor screen by receiving (Patent Document 1).

JP 2006-119762

  However, in the mechanism for detecting the entry of the moving body of Patent Document 1, it is necessary to store the image of the moving body in advance, and when the direction in which the moving body enters the monitoring target area is not specified, There is a problem that it is necessary to prepare a large number of images from all directions in advance, which is costly and takes a lot of labor and time. In particular, recording a flying object such as a bird and preparing a large number of images in advance requires a lot of labor and time. Therefore, the mechanism of Patent Document 1 cannot be said to be highly versatile for detecting flying objects such as birds.

  Therefore, the present invention detects the entry of a moving object such as a flying object into a monitored area without taking time and effort such as preparing an image of the moving object in advance and without requiring a long time for processing. It is another object of the present invention to provide an image processing method, an image processing apparatus, and an image processing program that can measure the number of times a moving object such as a flying object enters the monitoring target area.

  Here, in the present invention, the movement trajectory of a moving object such as a flying object having a lower luminance than the background of the shooting range of the imaging apparatus is set as a detection target. In the present invention, it is possible to detect any moving object whose luminance is lower than that of many parts of the background of the shooting range, and a part of the background of the shooting range has the same luminance as the moving object to be detected. There may be an object or an object having a lower luminance than a moving object to be detected.

  In order to achieve such an object, an image processing method for detecting a flying object according to the present invention is data of a captured image obtained by photographing a monitoring target area, and is continuous in time series over a predetermined processing unit time. A high-luminance image in which the maximum value of the luminance value for each pixel coordinate (x, y) during the processing unit time is the pixel value H (x, y) and the minimum value of the luminance value A low luminance image having a value L (x, y) is created, and Max [H (x, y) −, which is a difference value of pixel values for each pixel coordinate (x, y) between the high luminance image and the low luminance image. A difference image having L (x, y), 0] as a pixel value is created, and the difference image is binary converted to create a moving trajectory image of a moving object flying.

  Further, the image processing apparatus for detecting a flying object according to the present invention is data of a captured image obtained by photographing a monitoring target area and is continuous in time series over a predetermined processing unit time. A high-luminance image in which the maximum luminance value for each pixel coordinate (x, y) during the processing unit time is the pixel value H (x, y) and the minimum luminance value A means for creating a low-brightness image having a value L (x, y) and Max [H (x, y), which is a difference value between pixel values (x, y) between the high-brightness image and the low-brightness image. ) -L (x, y), 0] having a means for creating a difference image and a means for creating a movement trajectory image of a moving object by binary conversion of the difference image. Yes.

  In addition, the image processing program for detecting a flying object according to the present invention is data of a captured image obtained by capturing a monitored area, and is captured image data that is continuous in time series over a predetermined processing unit time. Means for reading from the storage device, a high luminance image having the maximum luminance value for each pixel coordinate (x, y) during the processing unit time as the pixel value H (x, y), and the minimum luminance value as the pixel value Means for creating a low luminance image with L (x, y), Max [H (x, y) −, which is a difference value of pixel values for each pixel coordinate (x, y) between the high luminance image and the low luminance image The computer is made to function as a means for creating a difference image having a pixel value of L (x, y), 0] and a means for creating a moving trajectory image of a moving object by binary conversion of the difference image. .

  Therefore, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, it is possible to compare the high luminance image and the low luminance image in the time series continuous image group in a predetermined time segment. Thus, the movement trajectory image is generated by the above-described method, so that the processing content is simple, and therefore, extraction of the moving object from the captured image is performed with high accuracy while suppressing the calculation load of the device.

  According to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, it is also possible to detect the moving object from the image captured by one imaging apparatus and the number of times the moving object enters the monitoring target area. Measurement can be performed, and it is not necessary to use a special imaging device or a plurality of imaging devices.

An image processing method for detecting the flying object of the present invention also pixel values of time identifier t of the original captured image with pixel values selected as the luminance value of each of the low-intensity image pixel coordinates (x, y) create a value T (x, y) time image recorded by converting the pixel coordinates (x, y) of the time each image pixel value T (x, y) time identifier t of captured images that appear as The pixel coordinates (x, y) satisfying the value of the time identifier t of the captured image as the pixel value T (x, y) in the time image satisfying a predetermined condition based on the appearance frequency for each value of the value of the moving object Add the movement locus of the moving object flying to the movement trajectory image as a moving track so that to create a movement locus complementary image. The image processing apparatus for detecting a flying object according to the present invention also uses the value of the time identifier t of the original captured image having the pixel value selected as the luminance value for each pixel coordinate (x, y) of the low luminance image as the pixel. Means for generating a time image converted and recorded to a value T (x, y), and a time identifier t of a captured image appearing as a pixel value T (x, y) for each pixel coordinate (x, y) of the time image The pixel coordinates (x, y) satisfying the value of the time identifier t of the captured image as the pixel value T (x, y) in the time image satisfying a predetermined condition based on the appearance frequency for each value of the value of the moving object movement trajectory image by adding the moving locus of the moving object flying have to have a means for creating a movement locus complementary image as a moving track. The image processing program for detecting a flying object according to the present invention also uses the value of the time identifier t of the original captured image having the pixel value selected as the luminance value for each pixel coordinate (x, y) of the low luminance image. Means for generating a time image converted and recorded to a value T (x, y), a time identifier t of a captured image appearing as a pixel value T (x, y) for each pixel coordinate (x, y) of the time image Pixel coordinates (x, y) satisfying the value of the time identifier t of the captured image as the pixel value T (x, y) in the time image satisfying a predetermined condition based on the appearance frequency for each value move the moving object movement trajectory image by adding the moving locus of the moving object flying in the means for creating a movement locus complementary image so that to function computer as a trajectory. Therefore, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, the movement trajectory image is complemented using the time image. A moving trajectory of a moving body that cannot be captured only by comparison is detected.

  Further, the image processing method for detecting a flying object according to the present invention is a method of flying to a monitored area during a processing unit time according to the appearance state of a moving track of a moving object flying in the moving track image or the moving track complement image. It is also possible to determine whether or not a moving body is entering. Further, the image processing apparatus for detecting a flying object according to the present invention can fly to the monitoring target area during the processing unit time according to the appearance state of the moving locus of the moving object flying in the movement locus image or the movement locus complement image. It is also possible to further include means for determining whether or not a moving body is entering. In addition, the image processing program for detecting a flying object according to the present invention is a method for flying to a monitoring target area during a processing unit time according to the appearance state of a moving track of a moving object flying in a moving track image or a moving track complement image. It is also possible to cause the computer to function as a means for determining whether or not a moving body has entered. In these cases, the above-described effect is exhibited in the measurement of the number of times the moving body that has flew into the monitored area.

  In the image processing method, the image processing apparatus, and the image processing program for detecting a flying object according to the present invention, a kite is selected as a flying object to be detected (in other words, a kite is targeted). good. In these cases, the kite can be a moving object with lower brightness than many parts of the background of the shooting range, so that the above-described effect is exhibited in the kite detection and measurement of the number of times the kite has entered the monitored area. Is done.

  According to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, the processing content is simple, and thus the extraction of the moving object from the captured image is highly accurate while suppressing the calculation load of the device. Therefore, it is possible to improve versatility as an image processing technique by shortening the processing time and improving the quick reportability.

  Further, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object according to the present invention, it is not necessary to use a special imaging apparatus or a plurality of imaging apparatuses. Costs for construction can be reduced, and versatility as an image processing technique can be improved.

  In addition, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, it is possible to detect a moving locus of a moving body that cannot be captured only by comparing a high luminance image and a low luminance image. Because this is not an extremely complicated process, it is possible to extract a moving object from a captured image with higher accuracy while avoiding an increase in processing time and impairing promptness. It becomes possible to improve the reliability.

  In addition, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, the above-described effects can be exhibited in the measurement of the number of times the moving body that has jumped into the monitored area. Therefore, it is possible to improve versatility and reliability as an image processing technique for measuring the number of times a moving body enters the monitoring target area.

  Further, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, in the detection of the wrinkle and the measurement of the number of times the wrinkle enters the monitoring target area as described above, Such effects can be exhibited.

It is a flowchart explaining an example of embodiment of the image processing method which detects the flying object of this invention. FIG. 3 is a functional block diagram of an image processing apparatus that detects a flying object realized by the program when the image processing method for detecting a flying object according to the embodiment is performed using an image processing program for detecting a flying object. 6 is a diagram illustrating an example of a captured image in Embodiment 1. FIG. It is a figure which shows an example of a high-intensity image among the images classified by luminance in Example 1. It is a figure which shows an example of the low-intensity image among the images classified by luminance in Example 1. 6 is a diagram illustrating an example of a time image in Embodiment 1. FIG. 6 is a diagram illustrating an example of a difference image in Embodiment 1. FIG. 6 is a diagram illustrating an example of a flight trajectory image in Embodiment 1. FIG. It is a figure which shows the measurement result of the approach frequency | count of a ridge in Example 1. FIG.

  Hereinafter, the configuration of the present invention will be described in detail based on an example of an embodiment shown in the drawings.

  1 and 2 show an example of an embodiment of an image processing method, an image processing apparatus, and an image processing program for detecting a flying object according to the present invention.

As shown in FIG. 1, the image processing method for detecting a flying object according to the present embodiment takes an image of a monitoring target area using an imaging device (S1), and is data of a captured image obtained by the imaging. The maximum value of the luminance value for each pixel coordinate (x, y) during the processing unit time is used as the pixel value H (x, y) using the data of the captured images that are continuous in time series over the predetermined processing unit time. And a low-brightness image L having a minimum luminance value as a pixel value L (x, y) (S2-1), and pixel coordinates (x, y) of the low-brightness image L A time image T recorded by converting the value of the time identifier t of the original captured image having the pixel value selected as the luminance value for each pixel to the pixel value T (x, y) is created (S2-2). Max [H (x, y) −L (x, y), which is a difference value of pixel values for each pixel coordinate (x, y) between the luminance image H and the low luminance image L. , 0] as a pixel value S (x, y) is created (S3), and the difference image S is binary-transformed to make a moving object, specifically, the background of the imaging range of the imaging device. A moving trajectory basic image S _b of a moving object flying with low luminance is created (S4), and a captured image that appears as a pixel value T (x, y) for each pixel coordinate (x, y) of the time image T is generated. The value of the time identifier t of the captured image as the pixel value T (x, y) in the time image T satisfies a predetermined condition based on the appearance frequency Hist [T (x, y)] for each value of the time identifier t. pixel coordinates (x, y) is the movement locus basic image by adding the moving locus of the moving object in S _b creates a movement track complement image S _c as a moving locus of the moving object to the flying (S5), the mobile removing noise locus complementary image S _c (S6), and, of the moving body in the moving locus complementary image after noise removal shifting To determine the presence or absence of entry of the moving body flying to the monitored area during the processing unit time in accordance with the revealing situations trajectory are in (S7) as.

In addition, the image processing apparatus for detecting a flying object according to the present embodiment is data of a captured image obtained by photographing a monitoring target area using the imaging apparatus, and is time-series over a predetermined processing unit time. And a means (11a) for reading data of continuous captured images from a data server (16) as a storage device, and a maximum luminance value for each pixel coordinate (x, y) during a processing unit time as a pixel value H ( a high-brightness image H having x, y) and a low-brightness image L having a minimum luminance value as a pixel value L (x, y) are created and the luminance for each pixel coordinate (x, y) of the low-brightness image L Means (11b) for creating a time image T recorded by converting the value of the time identifier t of the original captured image having the pixel value selected as the value into the pixel value T (x, y); Max [H (x, which is a difference value of pixel values for each pixel coordinate (x, y) between the low-brightness image L and the low-luminance image L. , Y) −L (x, y), 0] as a pixel value S (x, y), a means (11c) for creating a difference image S, and binary image conversion of the difference image S to convert a moving object, specifically luminance is lower flying than the background of the imaging range of the imaging apparatus to means for creating a movement track basic image S _b of the moving body and (11d), the pixel coordinates (x, y) of the time the image T pixel values for each T Imaging with a predetermined condition based on the appearance frequency Hist [T (x, y)] for each value of the time identifier t of the captured image appearing as (x, y) as the pixel value T (x, y) in the time image T time identifier t pixel coordinates value meets the image (x, y) is the movement locus basic image S _b to the mobile movement locus complementary image movement trajectory by adding as a moving locus of the moving object to the flying and means for creating a S _c (11e), means for removing noise of the movement trajectory complementary image S _c and (11f), the movement locus complementing image after noise removal Means (11g) for determining whether or not a flying mobile body has entered the monitoring target area during the processing unit time in accordance with the appearance of the moving locus of the mobile body in the image.

Furthermore, the image processing program for detecting a flying object according to the present embodiment is data of a captured image obtained by capturing an image of the monitoring target area using the image capturing apparatus, and is time-series over a predetermined processing unit time. Means (11a) for reading continuous captured image data from the storage device, and high luminance with the maximum value of the luminance value for each pixel coordinate (x, y) during the processing unit time as the pixel value H (x, y). Means (11b) for creating a low-brightness image L in which the minimum value of the image H and the luminance value is the pixel value L (x, y), and is selected as the luminance value for each pixel coordinate (x, y) of the low-brightness image L A means (11b) for creating a time image T obtained by converting the value of the time identifier t of the original captured image having the pixel value into the pixel value T (x, y), and the high luminance image H and the low luminance image. Max [H (x, y) −L (x) which is the difference value of the pixel value for each pixel coordinate (x, y) with L means (11c) for creating a difference image S with y), 0] as a pixel value S (x, y), and binary conversion of the difference image S to obtain a moving object, specifically, an imaging range of the imaging apparatus. and means (11d) for creating a movement track basic image S _b of the mobile object luminance is less flying than the background, the pixel coordinates (x, y) of the time the image T each pixel value T (x, y) appears as The value of the time identifier t of the captured image is set as a pixel value T (x, y) in the time image T under a predetermined condition based on the appearance frequency Hist [T (x, y)] for each value of the time identifier t of the captured image. It meets and pixel coordinates (x, y) is means for creating a movement track basic image S _b moving locus complementary image S _c movement trajectory by adding the moving object as a moving locus of the moving object to the flying (11e ) and, means for removing noise of the movement trajectory complementary image S _c and (11f), moves in the movement trajectory complementary image after denoising The computer is caused to function as means (11g) for determining whether or not the moving object flying into the monitored area during the processing unit time is present in accordance with the appearance state of the moving locus of the moving object.

  In executing the image processing method for detecting the flying object, first, the monitoring target area is photographed using the imaging device (S1).

  In the present invention, an image pickup apparatus (specifically, for example, a CCD-type digital video camera or a CMOS-type digital video camera, which includes an image pickup element and can continuously grasp and output at least luminance values for each pixel in time series. ), The imaging device is fixed at an appropriate location, and the surveillance area is photographed. In the present invention, since the luminance value for each pixel is used, it is sufficient that at least the luminance value for each pixel can be grasped (in other words, calculated). The captured image itself may be a grayscale image or a color image. Either way.

  In view of the processing contents of the present invention, the imaging speed (frame rate) of the imaging device is preferably 30 [fps] or more, but the assumed moving speed of the moving object to be detected is not so high, for example. In some cases, it may be less than 30 [fps].

  Here, the present invention performs a process of extracting a moving object recorded as an object with low brightness in the captured image. For this reason, in the present invention, a large number of planar objects having lower brightness than a moving object assumed as a detection target, for example, planar black objects such as black walls and dark shadows, are not included in the imaging range. In other words, when shooting in the monitoring target area, the fixed position and shooting direction of the imaging device are adjusted so that a higher brightness than the moving object assumed as a detection target does not occupy a large part of the shooting range as a background. The Moreover, the data of the image imaged in the time slot | zone where the brightness is ensured with sunlight are made into the object of a process. Furthermore, in particular, in order to detect a flying object, the fixed position and the shooting direction of the imaging apparatus are adjusted so that a range that can be assumed as the flight path of the flying object or can be a flight path is captured.

  Next, a high-brightness image and a low-brightness image are created using the captured image data obtained by the process of S1 (S2-1).

  Here, the processing after S2 in the image processing method for detecting a flying object of the present invention can be executed by the image processing apparatus for detecting the flying object of the present invention.

  And the process after S2 in the image processing method for detecting a flying object of the present invention and the image processing apparatus for executing these processes are realized by executing the image processing program for detecting the flying object of the present invention on a computer. Can be done. In the present specification, a case where an image processing apparatus that executes processing after S2 is realized by executing an image processing program for detecting a flying object on a computer, and processing after S2 in the image processing method is executed. explain.

  FIG. 2 shows an overall configuration of a computer 10 (which is also the image processing apparatus 10 in this embodiment) for executing the image processing program 17. The computer 10 (image processing apparatus 10) includes a control unit 11, a storage unit 12, an input unit 13, a display unit 14, and a memory 15, and is connected to each other by a signal line such as a bus. Further, a data server 16 as a storage device is connected to the computer 10 via a signal line such as a bus, and signals such as data and control commands are transmitted and received (that is, input / output) through the signal line. .

  The control unit 11 performs calculation related to the control of the entire computer 10 and the detection of the moving body from the captured image and the measurement of the number of times the moving body enters the monitoring target area by the image processing program 17 stored in the storage unit 12. For example, a CPU (Central Processing Unit).

  The storage unit 12 is a device that can store at least data and programs, and is, for example, a hard disk.

  The memory 15 serves as a memory space that is a work area when the control unit 11 executes various controls and operations, and is a RAM (Random Access Memory), for example.

  The input unit 13 is an interface for giving at least an operator's command to the control unit 11, and is, for example, a keyboard.

  The display unit 14 performs drawing / display of characters, graphics, and the like under the control of the control unit 11 and is, for example, a display.

  In the present embodiment, time-series continuous captured image data captured and acquired in the above-described processing of S1 and including at least a luminance value for each pixel is stored in the data server 16 as the image database 18 ( Saved).

Then, the control unit 11 of the computer 10 (which is also the image processing apparatus 10 in the present embodiment) executes the image processing program 17 so that the image data captured in the processing of S1 is predetermined. A data reading unit 11a that performs processing for reading data of captured images that are continuous in time series over the processing unit time from the data server 16 serving as a storage device, and luminance for each pixel coordinate (x, y) during the processing unit time A process for creating a high-intensity image H in which the maximum value is the pixel value H (x, y) and a low-intensity image L in which the minimum value is the pixel value L (x, y); A time image T is generated by converting the value of the time identifier t of the original captured image having the pixel value selected as the luminance value for each pixel coordinate (x, y) into the pixel value T (x, y) and recording it. Brightness-based image creation unit 11b, Max [H (x, y) −L (x, y), 0], which is the difference value of the pixel value for each pixel coordinate (x, y) between the high luminance image H and the low luminance image L, is set as the pixel value. A difference image creation unit 11c that performs a process of creating a difference image S as S (x, y), and the binary image of the difference image S to convert the difference image S into a moving object, more specifically, the brightness of the background of the imaging range of the imaging device. low flying movement trajectory image generating unit 11d which performs processing for creating a movement track basic image S _b of the movable body that, at the time the image T pixel coordinates (x, y) for each pixel value T (x, y) captured appear as The value of the time identifier t of the captured image as the pixel value T (x, y) in the time image T satisfies a predetermined condition based on the appearance frequency Hist [T (x, y)] for each value of the time identifier t of the image. and that the pixel coordinates (x, y) is moved by adding a movement locus of the moving object in the moving locus basic image S _b as a moving locus of the moving object to the flying A complementary image creation unit 11e that performs a process of creating a motion trajectory complement image S _c , a noise removal unit 11f that performs a process of removing noise from the travel trajectory supplement image S _c , and movement of the moving object in the motion trajectory complement image after noise removal An entry number measuring unit 11g is configured to perform a process of determining whether or not a moving object flying into the monitoring target area during the processing unit time corresponds to the appearance state of the trajectory.

  As specific processing by executing the image processing program 17, first, the data reading unit 11 a configured in the control unit 11 of the computer 10 (image processing apparatus 10) reads captured image data.

  Specifically, the data reading unit 11a captures data of captured images that are captured and acquired in the processing of S1 and recorded in the image database 18 stored in the data server 16 in time series (hereinafter, continuous images). Data) is read from the data server 16. Then, the data reading unit 11a stores the read continuous image data in the memory 15. The data reading unit 11a assigns a continuous number from 1 to each image in time-series order as a time identifier t for each image in the time-series continuous image, and assigns each image constituting the continuous image data to this time identifier t. Is stored in the memory 15 in association with the value of.

  Subsequently, the luminance-specific image creation unit 11b of the control unit 11 creates a high-luminance image and a low-luminance image as luminance-specific images.

ここに、Ｈ(ｘ，ｙ)：高輝度画像の画素座標(ｘ，ｙ)の画素値(輝度値)，
Ｌ(ｘ，ｙ)：低輝度画像の画素座標(ｘ，ｙ)の画素値(輝度値)，
ｔ：連続画像における画像毎の時刻識別子，
ｔ₁：処理対象とする連続画像中の画像時刻識別子の開始値，
ｔ₂：処理対象とする連続画像中の画像時刻識別子の終了値，
ｆ(ｘ，ｙ，ｔ)：時刻識別子ｔの画像の画素座標(ｘ，ｙ)での輝度値
をそれぞれ表す。
また、Ｍax[ｆ(ｘ，ｙ，ｔ)]は与えられたｔの範囲内におけるｆ(ｘ，ｙ，ｔ)の最大値を抽出する関数を表し、Ｍin[ｆ(ｘ，ｙ，ｔ)]は与えられたｔの範囲内におけるｆ(ｘ，ｙ，ｔ)の最小値を抽出する関数を表す。 Specifically, the brightness-specific image creation unit 11b reads the continuous image data stored in the memory 15 by the data reading unit 11a from the memory 15, and uses the continuous image data for a predetermined processing unit time. A high-intensity image H in which the brightest (that is, the largest) luminance value is recorded for each pixel is created according to Formula 1-1, and a low-intensity image L in which the darkest (that is, the smallest) luminance value is recorded for each pixel. Created according to Formula 1-2.

Here, H (x, y): pixel value (luminance value) of pixel coordinates (x, y) of the high luminance image,
L (x, y): pixel value (luminance value) of pixel coordinates (x, y) of the low luminance image,
t: time identifier for each image in the continuous image,
t ₁ : start value of image time identifier in continuous images to be processed,
t ₂ : end value of the image time identifier in the continuous image to be processed,
f (x, y, t): luminance value at pixel coordinates (x, y) of the image of the time identifier t
Respectively.
Max [f (x, y, t)] represents a function for extracting the maximum value of f (x, y, t) within a given range of t, and Min [f (x, y, t) ] Represents a function for extracting the minimum value of f (x, y, t) within a given range of t.

  The processing unit time is a time division for dividing a continuous image group to which the processing from S2 (that is, S2-1 and 2-2, the same applies hereinafter) to S6 is applied as a series. The length of the processing unit time is not limited to a specific length, and specifically, for example, it may be set in a range of about several tens of seconds to several minutes.

  Since the image having the value calculated by Equation 1-1 as the pixel value for each pixel is an image created by combining the maximum luminance values for each pixel coordinate (x, y) during the processing unit time, Even if a moving object with low luminance enters the shooting range during the processing unit time, the image of the moving object is not reflected.

  On the other hand, an image having the value calculated by Formula 1-2 as the pixel value for each pixel is an image created by combining and combining the minimum luminance values for each pixel coordinate (x, y) during the processing unit time. Therefore, when a moving body with low luminance enters the shooting range during the processing unit time, the image of the moving body is reflected. Specifically, the moving state of the moving object having a low luminance is an image that reflects the movement locus.

  Whether the captured image obtained by the processing of S1 is a grayscale image or a color image, the data of the captured image is converted into a luminance value by an appropriate method, and the high luminance image H and the low A luminance image L is created. Note that the method of calculating the luminance value from the image data (pixel value for each pixel) is a well-known technique, and therefore details thereof are omitted here.

Further, the start value t ₁ and the end value t ₂ of the image time identifier in the continuous image to be processed are processing units that are time divisions for dividing a continuous image group to which the processing from S2 to S6 is applied as a series. If the continuous image data recorded in the image database 18 are separated from each other by the unit of processing time to form separate data files, the values related to time are read for each file in the processing of S1. The processing in S2-S6 may be performed for each file, or if the continuous image data recorded in the image database 18 is not divided by processing unit time, the processing unit time specified in advance is used. What is necessary is just to read the data in the processing of S1 and perform the processing of S2-S6 with the corresponding number of images as a delimiter. It should be noted that if the imaging speed (frame rate) of the imaging device used in the imaging of the monitoring target area in the processing of S1 is R _F [fps] and the processing unit time (time division) is m seconds, the processing target is continuous. The number of images n = R _F × m, and the time identifiers t ₁ and t of the images so that the number of images from the start value t ₁ to the end value t ₂ of the image time identifier is [R _F × m]. While data is read in the processing of S1 while designating the value of ₂ , the processing from S2 to S6 is applied as a series to [R _F × m] continuous image groups.

  Then, the luminance-specific image creation unit 11b stores the value of H (x, y) for each pixel coordinate (x, y) in the memory 15 as the high luminance image H, and at the same time L for each pixel coordinate (x, y). The value of (x, y) is stored in the memory 15 as the low luminance image L.

  In addition, a time image is created in parallel with the process of S2-1 (S2-2).

ここに、Ｔ(ｘ，ｙ)：時刻画像の画素座標(ｘ，ｙ)の画素値を表す。 Specifically, the brightness-specific image creation unit 11b is selected as the brightness value of the pixel coordinates (x, y) of the low-brightness image L according to Formula 2 simultaneously with the creation of the low-brightness image L according to Formula 1-2. A time image T is generated by converting the value of the time identifier t of the original image having a pixel value (that is, having the pixel value of the minimum luminance value) into a pixel value and recording it.

Here, T (x, y) represents the pixel value of the pixel coordinate (x, y) of the time image.

  Note that the pixel value for each pixel of the time image T is actually the value of the time identifier t for each image in the continuous image, but this time identifier t is sequentially given to each image in the continuous image group in time series. It is a value related to time and is a value corresponding to the actual time when the image was captured, and is composed of the value of the time identifier t for each pixel coordinate (x, y). The data group is called a time image T.

  Then, the luminance-specific image creation unit 11b stores the value of T (x, y) for each pixel coordinate (x, y) in the memory 15 as the time image T.

  Next, a difference image is created using the high brightness image H and the low brightness image L created by the process of S2-1 (S3). However, 0 is set when the difference is negative.

ここに、Ｓ(ｘ，ｙ)：差分画像の画素座標(ｘ，ｙ)の画素値，
Ｈ(ｘ，ｙ)：高輝度画像の画素座標(ｘ，ｙ)の画素値(輝度値)，
Ｌ(ｘ，ｙ)：低輝度画像の画素座標(ｘ，ｙ)の画素値(輝度値)
をそれぞれ表す。
また、Ｍax[Ｈ(ｘ，ｙ)−Ｌ(ｘ，ｙ)，０]は「Ｈ(ｘ，ｙ)−Ｌ(ｘ，ｙ)」と「０」とのうちの大きい方を抽出する関数を表す。 Specifically, the difference image creation unit 11c determines the value of H (x, y) for each pixel coordinate (x, y) and the low luminance as the high luminance image H stored in the memory 15 in the process of S2-1. The value of L (x, y) for each pixel coordinate (x, y) as the image L is read from the memory 15, and the luminance value and the low luminance in the high luminance image H for each pixel coordinate (x, y) according to Equation 3 The difference with the luminance value in the image L is calculated. However, 0 is set when the difference is negative.

Here, S (x, y): the pixel value of the pixel coordinate (x, y) of the difference image,
H (x, y): pixel value (luminance value) of pixel coordinates (x, y) of the high luminance image,
L (x, y): Pixel value (luminance value) of pixel coordinates (x, y) of the low luminance image
Respectively.
Max [H (x, y) −L (x, y), 0] is a function for extracting the larger one of “H (x, y) −L (x, y)” and “0”. Represents.

  An image having a value calculated by Equation 3 as a pixel value for each pixel is a pixel in the high-intensity image H when the moving object does not enter the shooting range and the luminance value does not change greatly over the processing unit time. Since there is no large difference between the value H (x, y) and the pixel value L (x, y) in the low luminance image L, the value of S (x, y), which is the difference between these pixel values, becomes small. .

  On the other hand, when the luminance value temporarily decreases due to the moving object having low luminance entering the imaging range and being imaged during the processing unit time, the pixel value H in the high luminance image H in the imaging part (pixel). Since there is a large difference between (x, y) and the pixel value L (x, y) in the low luminance image L, the value of S (x, y), which is the difference between these pixel values, is large. Become. That is, the difference value S (x, y) at the pixel coordinates (x, y) related to the movement trajectory of the low-luminance moving body is larger than the difference value S at the other pixel coordinates.

  Then, the difference image creation unit 11c stores the value of S (x, y) for each pixel coordinate (x, y) in the memory 15 as the difference image S.

  Next, a basic image for the movement trajectory of the moving object is created using the difference image S created by the process of S3 (S4).

ここに、Ｓ_b(ｘ，ｙ)：移動軌跡基本画像の画素座標(ｘ，ｙ)の画素値，
Ｓ(ｘ，ｙ)：差分画像の画素座標(ｘ，ｙ)の画素値，
α：二値変換の閾値
をそれぞれ表す。 Specifically, the movement trajectory image creation unit 11d reads from the memory 15 the value of S (x, y) for each pixel coordinate (x, y) as the difference image S stored in the memory 15 in the process of S3. , The value of S (x, y) is converted into binary for each pixel coordinate (x, y) according to Equation 4.

Here, S _b (x, y): the pixel value of the pixel coordinate (x, y) of the movement trajectory basic image,
S (x, y): pixel value of pixel coordinates (x, y) of the difference image,
α: Binary conversion threshold
Respectively.

  Here, the threshold value α for binary conversion is a value (specifically, a luminance value) serving as a reference for extracting only a low-luminance moving object from the difference image S. The value of the threshold value α of the binary conversion is not limited to a specific value, and the tendency of the entire image in the continuous image data captured and acquired in the processing of S1 and the characteristics of the moving object to be detected (specifically Is appropriately set based on the brightness of the image of the captured moving body assumed based on the degree of blackness of the moving body. For example, detection of a moving object to be detected based on an average value of S (x, y) values for each pixel coordinate (x, y) in the difference image S obtained by the processing up to S3, distribution characteristics, and the like. It is conceivable that an appropriate value is adjusted and set appropriately.

  Based on the difference image S, the pixel value of the pixel having a small value of S (x, y) without a large change in the luminance value because there is no reflection of the moving body over the processing unit time. On the other hand, the pixel value of a pixel having a large S (x, y) value due to the reflection of the moving object and a large value of S (x, y) (that is, a pixel relating to the moving locus of the moving object). Becomes 1.

  As a result of the above processing, when a moving body enters the shooting range, an image in which the locus of movement of the moving body appears is created. As is clear from the above processing contents, in the image obtained as a result of the processing of S4, the pixel value is obtained in the image from which the moving object is extracted and the moving locus of the moving object appears. In an image in which 1 appears and a moving object is not extracted and a moving locus of the moving object does not appear, all pixel values are 0 (zero).

Then, the movement trajectory image creation unit 11d stores the value of S _b (x, y) for each pixel coordinate (x, y) in the memory 15 as the movement trajectory basic image S _b .

  Here, in the conversion process to binary using the threshold value α in the process of S4, the moving object and the background having a low luminance in a portion (pixel) having a low-luminance (that is, black) surface background in the captured image. The brightness difference between the moving object and the moving locus of the moving object may not be detected well.

  For this reason, if necessary, an image supplementing the moving locus of the moving body is created using the data of the captured images before and after the captured image that greatly contributed to the detection of the moving locus by the processing up to S4 (S5). May be.

ここに、Ｓ_c(ｘ，ｙ)：移動軌跡補完画像の画素座標(ｘ，ｙ)の画素値，
Ｓ_b(ｘ，ｙ)：移動軌跡基本画像の画素座標(ｘ，ｙ)の画素値，
Ｔ(ｘ，ｙ)：時刻画像の画素座標(ｘ，ｙ)の画素値，
ｔ_max：Ｈist[Ｔ(ｘ，ｙ)]の値が最大である画像毎の時刻識別子ｔの値，
δ₁，δ₂：補完範囲の閾値
をそれぞれ表す。
また、Ｈist[Ｔ(ｘ，ｙ)]は、時刻画像Ｔにおける画素座標(ｘ，ｙ)毎のＴ(ｘ，ｙ)の値（即ち、画像毎の時刻識別子ｔの値）の出現頻度を表す関数である。 Specifically, the complementary image creation unit 11e stores the value of T (x, y) for each pixel coordinate (x, y) as the time image T stored in the memory 15 in the process of S2-2 from the memory 15. The value of S _b (x, y) is updated for each pixel coordinate (x, y) according to Equation 5.

Here, S _c (x, y): the pixel value of the pixel coordinate (x, y) of the movement locus complement image,
S _b (x, y): pixel value of pixel coordinates (x, y) of the movement trajectory basic image,
T (x, y): pixel value of pixel coordinates (x, y) of the time image,
t _max : the value of the time identifier t for each image having the maximum value of Hist [T (x, y)],
δ ₁ , δ ₂ : complementary range threshold
Respectively.
Hist [T (x, y)] represents the appearance frequency of the value of T (x, y) for each pixel coordinate (x, y) in the time image T (that is, the value of the time identifier t for each image). It is a function to represent.

Here, the thresholds δ ₁ and δ ₂ of the complementary range are pixels corresponding to the movement trajectory of the moving body that has not been extracted as the movement trajectory basic image (strictly speaking, there is a probability that it corresponds to the movement trajectory of the moving body. It is a value (specifically, the number of captured images) that serves as a guideline for determining whether or not to add a high pixel) as a movement trajectory of the moving body. The values of the complementary range threshold values δ ₁ and δ ₂ are not limited to specific values, and the distribution characteristics of the appearance frequency Hist [t] for each value of the time identifier t for each image and the moving object to be detected. It is set as appropriate based on the moving speed of. For example, in consideration of the moving speed of the moving object to be detected, it is assumed that the time required to pass across the imaging range is about 3 seconds, and the appearance frequency Hist for each value of the time identifier t for each image. When the spread of the distribution of [t] is analyzed, when most of the appearance frequency Hist [t] is in the range of 1.5 seconds before and after the image identification value _tmax where the appearance frequency Hist [t] is maximum, It is conceivable to set the values of the complementary range thresholds δ ₁ and δ ₂ based on the number of captured images corresponding to a real time of 1.5 seconds. Note that the relationship between the threshold values δ ₁ and δ ₂ of the complementary range may be δ ₁ = δ ₂ or δ ₁ ≠ δ ₂ .

By the processing using Equation 5, the pixel value (luminance value) at the pixel coordinate (x, y) actually corresponding to the trajectory of the moving object in the captured image and the pixel coordinate (x, y) of the low luminance image L Although the pixel value L (x, y) is selected, the luminance value of the background at the pixel coordinate (x, y) is low, so the pixel value H (x , Y) and the pixel having the binary conversion value S _b (x, y) = 0 as a result of the process of S4 because the difference value S (x, y) is small is a pixel of the moving locus of the moving object. Expected to be replenished. Note that, as shown in Equation 5, the binary conversion value S _b (x, y) determined in the process of S4 remains as it is for the pixels that are not related to the replenishment as the pixels of the moving locus of the moving object.

Then, the complementary image creation unit 11e stores the value of S _c (x, y) for each pixel coordinate (x, y) in the memory 15 as the movement locus complementary image S _c .

Furthermore, the removal of noise with respect to the moving locus complementary image S _c created by the processing of S5 (S6) as may be.

Specifically, the noise removing unit 11f stores the value of S _c (x, y) for each pixel coordinate (x, y) as the movement trajectory complement image S _c stored in the memory 15 in the process of S5. Remove noise from isolated points. A method for removing noise such as an isolated point is not limited to a specific method, and various methods for removing noise are well known, and the details are omitted here. Specifically, for example, there is a method of setting the pixel value of the pixel of pixel value = 1 surrounded by pixel value = 0 to 0.

  Through the above processing, one movement trajectory image in the processing unit time to be processed is created. Then, by repeating the processes from S2 to S6 according to the time segment divided as the processing unit time, one movement trajectory image is created for each time segment divided as the processing unit time.

Then, the noise removing unit 11f uses the pixel value for each pixel coordinate (x, y) after removing the noise based on the value of S _c (x, y) for each pixel coordinate (x, y) as a movement locus. The image (after the movement trajectory interpolation process and the noise removal process) is stored in the memory 15 in association with the time identifier of the processing unit time (time segment) that is the processing target. Furthermore, if necessary, the noise removing unit 11f may store pixel value data of the moving track image (after the moving track complementing process and the noise removing process), for example, a moving track image data file stored in the storage unit 12 or the data server 16. Are recorded and stored so as to be accumulated for each processing unit time (time division).

  Subsequently, in the present embodiment, the number of times the moving body enters the monitoring target area is further measured using the movement trajectory image created by the processing up to S6 (S7).

  In the present invention, the moving object to the shooting range (in other words, the monitoring target area) according to the appearance state of the pixel whose pixel value is 1 in the movement trajectory image for each processing unit time (time segment) created by the processing up to S6. Determine whether there is an entry.

  Specifically, if the number of pixels with pixel value 1 in the movement trajectory image of a certain processing unit time is less than the entry determination threshold value γ, the moving object has not entered the monitoring target area during the processing unit time. On the other hand, if the number of pixels with the pixel value 1 is equal to or greater than the entry determination threshold γ, it is determined that the moving object has entered the monitoring target area during the processing unit time.

  Therefore, for example, the number of pixels having a pixel value of 1 is compared with the entry determination threshold value γ for each of 60 moving trajectory images obtained by repeating the processing from S2 to S6 60 times with a processing unit time of 1 minute. By determining whether or not a moving body has entered the monitoring target area, the number of times the moving body has entered the monitoring target area in one hour can be measured.

  Here, the entry determination threshold value γ is a value (specifically, the number of pixels) that serves as a guideline for determining whether or not the moving track of the moving object appears in the moving track image. The entry determination threshold value γ is not limited to a specific value, and takes into account the resolution of the imaging means used in the processing of S1 (that is, the number of pixels of the captured image) and forms an image of the moving body in the captured image. It is set as appropriate based on the assumption of the number of pixels. For example, in the captured image obtained by capturing the monitoring target area, 1.5 (minimum number of pixels) (the assumed number of pixels) constituting the image of the moving body when the moving body existing in the monitoring target area is captured. It is conceivable to set to double or double.

  In addition, when measuring the number of times the moving body has entered the monitoring target area, which is the process of S7 described above, the time required for the moving body to cross the shooting range shooting the monitoring target area. It is preferable to set the processing unit time in consideration of the appearance frequency of the moving object.

  That is, if it is assumed that the time required for the moving body to pass across the imaging range is large, for example, about 3 seconds, and if the processing unit time is 2 seconds, the passing of one moving body is a processing unit of two times. The possibility of straddling time increases, and even though it is a single entry, a movement locus appears in multiple movement locus images, and as a result, the moving object's entry count is counted as multiple times. The measurement accuracy decreases.

  On the other hand, when it is assumed that the time required for the moving body to pass across the imaging range is about 3 seconds, for example, and the appearance frequency of the moving body is assumed to be about once every 3 minutes, for example, the processing unit. If the time is 10 minutes, the possibility that a plurality of moving bodies will pass within one processing unit time is increased, and the movement locus appears in only one movement locus image even though it is a plurality of times of entry. As a result, the number of times the moving body enters is counted as one, and the measurement accuracy is lowered.

  In the present embodiment, as a specific process, the approach number measuring unit 11g is recorded and saved in the movement trajectory image data file stored in the memory 15 or stored in the storage unit 12 or the data server 16 in the process of S6. Of the movement trajectory image data for each processing unit time, the processing unit time data corresponding to the measurement target time zone of the number of times the moving body has entered the monitoring target area is read from the memory 15, the storage unit 12, or the data server 16.

  Subsequently, the entry number measurement unit 11g initializes the number N of movements of the moving body (that is, N = 0), and the number of pixels with the pixel value 1 for each movement trajectory image for each processing unit time becomes the entry determination threshold γ. It is determined whether or not this is the case, and if it is equal to or greater than the entry determination threshold γ, 1 is added to the number N of times the mobile body has entered. This process is repeated for each of the movement trajectory images for each processing unit time read as corresponding to the measurement target time zone for the number of times of entering the moving body.

  When there are a plurality of measurement target time zones, the above-described processing is repeated for each measurement target time zone.

  Then, the entry frequency measurement unit 11g displays the value of the moving object entry frequency N as the measurement result by the process of S7, in association with the measurement target time zone, if necessary, on the display unit 14, or stores, for example, The data is stored in the unit 12 or the data server 16 as a measurement result data file.

  And the control part 11 complete | finishes the process of the detection of the mobile body from a captured image, and the measurement of the frequency | count of the mobile body approaching into the monitoring object area (END).

  According to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention having the above-described configuration, a high-intensity image H and a low-intensity image in a time-series continuous image group in a predetermined time segment. Since the movement trajectory image is created by comparing with L, the processing content is simple as a processing content, and therefore the moving object can be extracted from the captured image with high accuracy while suppressing the calculation load of the device. Therefore, it is possible to shorten the processing time, improve the quick reportability, and improve the versatility as an image processing technique. Furthermore, since the movement trajectory image can be complemented using the time image T, in this case, the movement trajectory of the moving body that cannot be captured only by the comparison between the high luminance image H and the low luminance image L is obtained. Since it can be detected, this is not an extremely complicated process, and it is possible to extract a moving body from a captured image with higher accuracy while avoiding an increase in processing time and impairing quickness, It becomes possible to improve the reliability as an image processing technique.

  Further, according to the image processing method, the image processing apparatus, and the image processing program for detecting a flying object of the present invention, a difference between frames or a frame interval between captured images that are continuous in time series as seen in conventional image processing is provided. Since the time difference is not used, objects reflected in the background due to changes in the solar radiation caused by clouds or slight changes in the background in the monitored area (shooting range), etc., caused by frame differences or time differences, etc. Can be avoided, and it is possible to improve the versatility as an image processing technique by increasing the accuracy of extracting a moving body from a captured image. In addition, according to the present invention, the moving direction and speed of a pixel that is regarded as particles by comparing a plurality of images as in the movement trajectory detection process by the particle tracking method as seen in conventional image processing. Since it is not a calculation, the luminance value of the pixel that looks like a particle changes randomly due to the noise of solar radiation and background changes that occurs in the particle tracking method, so it can be extracted stably unless the shooting conditions are limited. The problem of being unable to do so can also be avoided, and it becomes possible to improve the versatility of the image processing technique by increasing the accuracy of extraction of the moving object from the captured image. Furthermore, in the present invention, it is possible to detect a moving body and measure the number of times the moving body enters the monitoring target area from an image captured by a single imaging apparatus, and to install a special imaging apparatus or a plurality of imaging apparatuses. Since it is not necessary to use it, it becomes possible to reduce the cost for construction of equipment necessary for measurement, and it is possible to improve versatility as an image processing technique.

  Although the above-described embodiment is an example of a preferred embodiment of the present invention, the embodiment of the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention. For example, in the above-described embodiment, the storage unit in which captured image data is stored is the data server 16, but the storage unit 12 or another storage device may be used. Alternatively, the captured image data may be directly input from the imaging device to the computer 10 (image processing device 10) without being temporarily stored in the storage device, and processing may be sequentially performed from the input data.

Further, although as in the embodiments described above to create a time image T (S2-2) by utilizing the time image T to create a movement locus complementary image S _c (S5), these images T, S _c Is not an essential process in the present invention. That is, when there is no planar object with low luminance in the imaging range of the monitoring target area, the luminance difference between the low luminance moving body and the background is large over the entire captured image, and the high luminance image H Since the moving locus of the moving object can be detected satisfactorily only by comparison with the low-luminance image L, it is not necessary to create the moving locus complement image using the time image T.

ここに、Ｓ_c(ｘ，ｙ)：移動軌跡補完画像の画素座標(ｘ，ｙ)の画素値，
Ｓ_b(ｘ，ｙ)：移動軌跡基本画像の画素座標(ｘ，ｙ)の画素値，
β：補完採否の閾値
をそれぞれ表す。
また、Ｈist[Ｔ(ｘ，ｙ)]は、時刻画像Ｔにおける画素座標(ｘ，ｙ)毎のＴ(ｘ，ｙ)の値（即ち、画像毎の時刻識別子ｔの値）の出現頻度を表す関数である。 In the above-described embodiment, the movement trajectory complement image S _c is created according to Equation 5 in the processing of S5, but S _b (x, y) for each pixel coordinate (x, y) according to Equation 6 below. The movement trajectory complement image may be created by updating the value.

Here, S _c (x, y): the pixel value of the pixel coordinate (x, y) of the movement locus complement image,
S _b (x, y): pixel value of pixel coordinates (x, y) of the movement trajectory basic image,
β: Threshold value for complementary adoption
Respectively.
Hist [T (x, y)] represents the appearance frequency of the value of T (x, y) for each pixel coordinate (x, y) in the time image T (that is, the value of the time identifier t for each image). It is a function to represent.

  Here, the threshold value β for complementation acceptance / rejection is a pixel corresponding to a movement trajectory of a moving object that has not been extracted as the movement trajectory basic image (strictly speaking, a pixel having a high probability corresponding to the movement trajectory of the moving object). This is a value (specifically, the number of pixels in the time image T) that serves as a guideline for determining whether or not to add complementarily as a movement locus of the moving object. The value of the threshold value β for complementation acceptance / rejection is not limited to a specific value, and is set as appropriate based on the tendency of the appearance frequency Hist [t] for each value of the time identifier t for each image. For example, an appropriate value for complementing the trajectory of the moving object is appropriately selected based on the average value, minimum value / maximum value, intermediate value, or distribution characteristics of the appearance frequency Hist [t] for each value of the time identifier t for each image. Adjustment and setting are possible.

In the above-described embodiment, noise is removed from the movement trajectory complement image S _c (the movement trajectory basic image S _b when the processing of S5 is not performed as described above) (S6). However, removal of noise from the movement trajectory image is not an essential process in the present invention. In other words, when the presence of noise in the movement trajectory image is not a problem, it is not necessary to remove noise. When measuring the number of times the moving object enters the monitoring target area (S7), the number of times the moving object has entered without the influence of noise is adjusted by appropriately adjusting the value of the entry determination threshold γ. In this case, it is not necessary to remove noise particularly from the moving trajectory image.

  In the above-described embodiment, the number of times the moving body enters the monitoring target area is measured (S7). However, the measurement of the number of times the moving body has entered is not an essential process in the present invention. That is, when the purpose is to detect a situation in which a moving body has entered the monitoring target area, the number of times the moving body has entered may not be measured.

  An embodiment in which the image processing method for detecting a flying object according to the present invention is applied to actual detection of a moving object and measurement of the number of times the moving object enters the monitoring target area will be described with reference to FIGS. In this embodiment, a case will be described in which an image processing apparatus is realized by executing an image processing program for detecting a flying object on a computer, and processing in the image processing method is executed.

  In this embodiment, assuming that a black object that passes through the imaging range in a short time is a kite, the kite that flies and enters the monitored area is the detection target, and the kite flight trajectory is detected as the movement track. did.

  Since spider webs can cause power outages, power companies monitor the nesting status of crows in unmanned substations and sales offices by patrol. Since there are many monitoring points such as unmanned substations at locations away from the sales office, there is a limit to the time that patrolmen can be assigned to one monitoring point. Therefore, it is desired to continuously grasp the arrival situation of crows using images from a surveillance camera and use it as an aid for patrol. Specifically, it is desired to provide information to the patrol staff by measuring the appearance status of the spider prior to the creation of the nest, the frequency of carrying branches, etc., and using the place where the flying frequency is high as an alert location. However, at present, there is no mechanism for detecting the flying of a kite from video captured by a surveillance camera and outputting the detection result in a form that is useful for patrol. Due to the above circumstances, it is meaningful to mechanically / automatically detect the moving object and measure the number of times the moving object enters the monitoring target area with the monitoring target area as a substation and the detection target as a trap. large.

  In this embodiment, a substation, which is a monitoring target area, is photographed using a CCD digital color video camera (frame rate: 30 [fps]) having a resolution of 350,000 pixels (horizontal 730 × vertical 480 pixels) as an imaging device. (S1).

  Here, it is assumed that the kite flies into the monitored area and enters the area to be monitored. Since it affects the detection of the body, we took an image from the position where the camera was fixed at an elevation angle, and the range of air that was assumed to be the flying altitude of the kite. The shooting period was from 6 am to 6 pm. An example of the captured image is shown in FIG.

  In this embodiment, the continuous image data obtained by the above photographing is stored as an image database in a hard disk as a storage device of a computer.

  Next, using the captured image data obtained by the process of S1, a high-luminance image H is created according to Formula 1-1 and a low-luminance image L is created according to Formula 1-2 (S2-1). A time image T was created according to Equation 2 (S2-2).

  Here, in this embodiment, the processing unit time is 1 minute. Therefore, the processing from S2 to S6 was applied to a series of continuous images of n = 30 [fps] × 60 [seconds] = 1800 continuous images per processing unit time. The value of the time identifier t for each image in the continuous image is 1 to 1800.

  An example of the high luminance image H is shown in FIG. 4A and an example of the low luminance image L is shown in FIG. 4B. An example of the time image T is shown in FIG.

  From FIG. 4B, which is an example of the low-intensity image L, it is confirmed that the flight trajectory of the kite that proceeds to the left from the position near the lower end of the right end of the image and then rises is selected as a part of the low-intensity image L. The

  An example of the time image T shown in FIG. 5 is obtained by converting the value of the time identifier t (specifically 1 to 1800) for each image into a luminance value (specifically 0 to 255) and displaying it. It is. In FIG. 5, the brightness of the flying locus part (pixel) of the kite is high because the value of the time identifier t of the image in which the kite flying is captured because the kite flew at the end of the processing unit time. Therefore, the luminance value is increased and the luminance is increased.

  Next, a difference image S is created according to Equation 3 using the high-intensity image H and the low-intensity image L created by the process of S2-1 (S3).

  An example of the difference image S is shown in FIG.

Next, create a basic image S _b for trajectories crow using a differential image S created by the process of S3 (S4).

In the present embodiment, the luminance value for each pixel of the difference image S is set to 0 to 255 gradations, and the threshold value α for binary conversion is set to 50, and the luminance value for each pixel of the difference image S is binary converted according to Equation 4. We have created a basic image S _b of the flight trajectory Te.

Further, in this embodiment, creating the image S _c which complements trajectories crow using data before and after the captured image of especially large contributions captured image to detect the trajectories by the processing up to S4 (S5).

In this embodiment, the flight trajectory complement image _Sc is created according to Equation 6. It should be noted that the threshold value β for complementary adoption is set to 50.

Further, in this embodiment, it was removed of noise on trajectories complementary image S _c created by the processing of S5 (S6).

An example of trajectories basic image complementation of trajectories with respect to S _b (S5) and the flight path image after the noise removal processing (S6) shown in FIG.

  As a result of inspection of the picked-up image by human visual observation, it was confirmed that all the white portions in FIG. In addition, it was confirmed that the discontinuous white part on the lower side of FIG. 7 is also a flight trajectory of the kite that flew between the facilities in a low sky. From this result, it was confirmed that the moving object can be detected with very good accuracy by the image processing method for detecting a flying object of the present invention.

  In the present embodiment, the number of times that the kite entered the substation as the monitoring target area was measured based on the flight trajectory image created by the processing up to S6 (S7).

  In the present embodiment, the above processing is repeated for each processing unit time by setting the processing unit time (time division) as one minute for a continuous image group captured from 6 am to 6 pm, which is the photographing time zone. , And when the number of pixels with pixel value 1 is greater than or equal to the entry determination threshold γ, 1 is added to the number of times the kite enters N and the number of times the kite enters the substation is measured. did. Note that the entry determination threshold γ = 100.

  The results of the measurement shown in FIG. 8 were obtained by measuring the number N of the soot entry every hour from 6 am to 6 pm. The result shown in FIG. 8 is in good agreement with the number of times manually measured by a person. From this result, the number of times the moving object enters the monitoring target area is detected by the image processing method for detecting a flying object of the present invention. It was confirmed that can be performed with very good accuracy.

  In this embodiment, since the monitoring target area is a substation, electric wires etc. are included in the imaging range, and the shaking of the electric wires etc. is imaged, and the solar radiation change from 6 am to 6 pm Although the change in the brightness of the equipment wall due to the weather change is imaged, it has been confirmed that the detection process is not affected by such a background change according to the process of the present invention. That is, according to the present invention, it has been confirmed that the extraction process of the detection target can be satisfactorily performed without being affected by the background change.

  In this embodiment, a kite flying in the monitoring target area is used as a detection target, and a kite flight trajectory is detected as a movement track. However, what is assumed as a detection target in the present invention is not limited to kites. The object to be detected is a moving object recorded as a low-luminance object in the captured image, more specifically, a moving object recorded as an object having a lower luminance than the background occupying a large part of the shooting range. It can be.

10 Image processing device 17 Image processing program

Claims (9)

The pixel coordinates (x, x, x) during the processing unit time using the data of the captured image obtained by the imaging of the monitoring target area and the data of the captured image continuous in time series over a predetermined processing unit time. y) Creates a high-brightness image having a maximum luminance value for each pixel value H (x, y) and a low-brightness image having a minimum luminance value as a pixel value L (x, y). A difference image having a pixel value of Max [H (x, y) −L (x, y), 0], which is a difference value of the pixel value for each pixel coordinate (x, y) between the image and the low luminance image. The difference image is binary-converted to generate a moving trajectory image of the flying moving object, and the pixel value selected as the luminance value for each pixel coordinate (x, y) of the low luminance image value the pixel value of the time identifier t of the original captured image T (x, y) create a time image recorded by converting into, before the time the image A predetermined condition based on the appearance frequency for each value of the time identifier t of the captured image that appears as a pixel value T (x, y) for each pixel coordinate (x, y) is defined as a pixel value T (x, y in the time image). ), The pixel coordinate (x, y) that satisfies the value of the time identifier t of the captured image is the movement trajectory of the flying mobile object, and the movement trajectory of the mobile object flying in the movement trajectory image. image processing method for detecting the flying coming object you characterized by creating a moving locus complementary image by adding. It is determined whether or not the flying mobile object has entered the monitoring target area during the processing unit time according to the appearance status of the moving trajectory of the flying mobile object in the movement locus image or the movement locus complement image. The image processing method for detecting a flying object according to claim 1 . Image processing method for detecting the flying object according to claim 1 or 2, wherein the moving body to the flying is crow. Means for reading captured image data obtained by photographing the monitoring target area, which is continuous in time series over a predetermined processing unit time, from a storage device; and between the processing unit times Creates a high-intensity image with the maximum luminance value for each pixel coordinate (x, y) as the pixel value H (x, y) and a low-intensity image with the minimum luminance value as the pixel value L (x, y) Max [H (x, y) −L (x, y), 0], which is a difference value of pixel values for each pixel coordinate (x, y) between the high luminance image and the low luminance image A means for creating a difference image having a pixel value as a pixel value, a means for creating a moving trajectory image of a moving object that is binary-converted from the difference image, and the pixel coordinates (x, y) of the low-luminance image Convert the value of the time identifier t of the original captured image having the pixel value selected as the luminance value to the pixel value T (x, y) It means for creating a time images recorded Te, the pixel coordinates (x, y) of the time the image pixel value T (x, y) for each of the value by the frequency of appearance of the time identifiers t of the captured image to appear as The pixel coordinates (x, y) satisfying a predetermined condition based on the value of the time identifier t of the captured image as the pixel value T (x, y) in the time image is a movement trajectory of the flying moving object. the image processing device for detecting flying come object you; and a means for creating a movement trajectory complement image movement trajectory added to the mobile that the flying to the movement track image as being. It is determined whether or not the flying mobile object has entered the monitoring target area during the processing unit time according to the appearance status of the moving trajectory of the flying mobile object in the movement locus image or the movement locus complement image. 5. The image processing apparatus for detecting a flying object according to claim 4 , further comprising means for performing the operation. 6. The image processing apparatus for detecting a flying object according to claim 4 or 5, wherein the flying mobile body is a kite. Means for reading captured image data obtained by photographing the monitoring target area, which is continuous in time series over a predetermined processing unit time, from a storage device; pixels in the processing unit time A high luminance image having a maximum luminance value for each coordinate (x, y) as a pixel value H (x, y) and a low luminance image having a minimum luminance value as a pixel value L (x, y) are created. Means, Max [H (x, y) −L (x, y), 0], which is a difference value of the pixel value for each pixel coordinate (x, y) between the high-intensity image and the low-intensity image. A means for creating a difference image as a value, a means for creating a moving trajectory image of a moving object that binary-converts the difference image, and a luminance value for each pixel coordinate (x, y) of the low-luminance image The value of the time identifier t of the original captured image having the selected pixel value is converted into the pixel value T (x, y) and recorded. A predetermined time based on the appearance frequency of each time identifier t of the captured image that appears as a pixel value T (x, y) for each pixel coordinate (x, y) of the time image. The pixel coordinates (x, y) satisfying the condition of the time identifier t of the captured image as the pixel value T (x, y) in the time image are the movement trajectory of the flying mobile body. an image processing program for detecting the flight came object you characterized by causing add a movement locus of the moving object to the flying to the moving path image as a means of creating movement locus complementary image to function computer. It is determined whether or not the flying mobile object has entered the monitoring target area during the processing unit time according to the appearance status of the moving trajectory of the flying mobile object in the movement locus image or the movement locus complement image. The image processing program for detecting a flying object according to claim 7, wherein the computer is also functioned as a means for performing the operation. The image processing program for detecting a flying object according to claim 7 or 8, wherein the flying mobile body is a kite.

Images