JP3649277B2 - Speed measurement system and speed measurement method by image recognition - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a speed measurement system and a speed measurement method based on image recognition.
[0002]
[Prior art]
Various methods are used to determine the moving speed. There are a method in which a signal is transmitted to the mobile body itself and processing is performed according to a signal interval between two points, and a method in which the signal is obtained from the movement time between a plurality of measurement points.
[0003]
[Problems to be solved by the invention]
In the above method, a dedicated device must be installed for speed detection, and the device cannot be used for other purposes.
[0004]
An object of the present invention is to provide a speed measurement system and a speed measurement method based on image recognition that calculate a moving speed from a moving distance of a moving body that is captured in continuous images using video continuously captured from one point. There is.
[0005]
[Means for Solving the Problems]
Humans can judge the speed of a moving object by watching an image. There are many factors in the determination, and depending on the object, two or more still images are compared, and the determination is made based on the moving distance. The present invention provides a configuration for recognizing and notifying the moving speed of a moving object by cutting out a camera image continuously shooting the moving object, treating it as a still image, and comparing the moving distance in the continuous still image. Is.
[0006]
The initial background data acquisition means reads the detection target area from the background image data in the storage device. (This means is intended to improve detection accuracy at the same time as initialization, and can also be handled by using initial background data based on the first image acquired by the next video acquisition means.)
The video acquisition means acquires and stores one frame of the video shot by the video shooting device as still image data via the video input device. Based on the stored still image data of one frame and the initial background data stored in the initial background data acquisition means, the background estimation means estimates the background data using a Kalman filter on each pixel and cuts out only the foreground data. Do.
[0007]
The labeling means concatenately classifies and labels bits having the same gray value in the foreground data, and the feature quantity extraction means recognizes the speed detection object that matches the feature quantity condition recorded in the individual environment data of the storage device. To do.
[0008]
The background update means uses the video acquired by the video acquisition means as the next background image.
[0009]
When the feature amount extraction unit recognizes the detection target, it records the detection by the detection recording unit, and further measures and records the position of the center of gravity of the object.
[0010]
Again, the procedure from the image acquisition means to the background update means is repeated.
[0011]
In the next video, if an object that is judged to be the same object from the characteristics and features of the speed detection target is shot, the speed calculation means compares the center of gravity and divides the moving distance by the shooting time difference. To calculate the moving speed of the object.
[0012]
Therefore, speed measurement is possible regardless of the type of detection object by the configuration of the present invention.
[0013]
Further, the configuration of the present invention enables speed measurement in combination with existing surveillance cameras and monitors.
[0014]
Further, the configuration of the present invention enables coexistence with other image recognition functions.
[0015]
In addition, by realizing image recognition with software, customization according to the environment is facilitated, and thresholds can be set with individual environment data, so that it is easy to correct functions accompanying changes in the environment.
[0016]
In addition, by realizing image recognition with software, various detection result notification means can be realized, and it is easy to cope with and add various support functions.
[0017]
In addition, by performing background estimation using the Kalman filter, noise can be eliminated and the amount of calculation can be reduced.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment of the invention)
Referring to FIG. 1, the present embodiment includes a video photographing device 1 such as a camera, a video input device 2 such as an image input board and a network interface board, a data processing device 3 such as an electronic computer, and a storage device 4. .
[0019]
The video shooting device 1 always sends a shooting result to the video input device 2.
[0020]
The data processing device 3 includes an initial background data acquisition unit 31, a video acquisition unit 32, a background estimation unit 33, a labeling unit 34, a feature amount extraction unit 35, a background update unit 36, a detection recording unit 37, and a speed calculation unit 38. The background data is estimated from the background image data and the acquired still image data using a Kalman filter, and the foreground portion is cut out. The extracted image portion is further narrowed down to the detection target area, and the detection target is recognized. These processes are repeated, and when the same object to be detected is photographed in continuous still images, the positions of the centers of gravity are compared, and the moving speed of the object is calculated by dividing the moving distance by the photographing time difference.
[0021]
The initial background data acquisition means 31 acquires the background image data 41 stored in the storage device 4 and in which the detection target is not photographed, and sets it as an initial value for background estimation.
[0022]
The video acquisition means 32 issues and obtains a command for acquiring the latest one frame of still image data to the video input device 2.
[0023]
The background estimation means 33 adapts the background image data 41 acquired by the initial background data acquisition means 31 using the Kalman filter for all the pixels of the still image image data acquired by the video acquisition means 32, estimates the background, Cut out only the foreground part other than the background.
[0024]
The labeling means 34 gives the same label to the gray value pixel clusters connected to each other for the foreground portion, outlines it, recognizes it as a figure, and performs labeling after discrimination.
[0025]
The feature amount extraction unit 35 is configured to provide elemental conditions (area, roundness, ellipse major axis / minor axis ratio, characteristic of the shape of the velocity detection object previously defined in the individual environment data 42 for each labeled region. An object that satisfies an extraction condition combining length, width, etc.) is recognized as a detection target.
[0026]
The background update unit 36 updates the still image data acquired by the video acquisition unit 32 as background image data for the next process.
[0027]
As a result of the processing of the feature quantity extraction means 35, when a figure that is a detection target is recognized, the detection recording means 37 obtains the position of the center of gravity.
[0028]
When the processing from the image acquisition means 32 to the detection recording means 38 is repeated again, and when an object that is determined to be the same object from the characteristics and feature quantities of the object of speed measurement is photographed on a continuous still image, the speed calculation means 38 is obtained. Then, the center of gravity position is compared, and the moving speed of the object is calculated by dividing the moving distance by the photographing time difference.
[0029]
Next, the operation of the present embodiment will be described in detail with reference to FIGS.
[0030]
The video imaged by the video imaging device 1 is constantly transmitted to the video input device 3 in real time.
[0031]
In the data processing device 3, the initial background data acquisition means 31 reads the background image data 41 and the individual environment data 42 stored in the storage device 4 during the initial operation (step A1). The read background image data is stored in “Background” as a data set of an initial video state for the gray value for each bit.
[0032]
The video acquisition means 32 issues a command to acquire one frame data of video to the video input device 2 and stores it in “Image [n] (n = 1)” (step A2). When the camera video is a black and white 8-bit video shot with NTSC, this data is a video with a gray value distribution of 640 × 480 and gray values ranging from 0 to 255.
[0033]
The background estimation means 33 uses the gray value for each bit as a transition video state data set for the still image data “Image [1]” acquired by the video acquisition means 32, and is set by the initial background data acquisition means 31. The background is estimated and calculated using the Kalman filter with the data set “Background” of the initial video state for the gray value for each bit. In general, the predicted value can be calculated by an ordinary differential equation of the r-th order of each predicted element, but the Kalman filter is an algorithm used for prediction and smoothing based on observation, and is obtained by an ordinary differential equation of the r-th order. The estimated value and the expected value obtained are replaced by r first-order ordinary differential equations. The part that was not estimated as the background is conceptually considered that the background is the lower layer, and objects such as humans and vehicles are photographed on the layer above it, and that part other than the background = foreground "Foreground" (Step A3).
[0034]
For the foreground “Foreground” that has been cut out, the labeling means 34 detects and joins bits having the same gray value, thereby judging and classifying the objects present in the image, labeling each, and arranging them. As "labeled [n]". By this means, those connected with the same gray value are regarded as connected components, the figures surrounded by the connected components are recognized as the same object, and an ID number is assigned to each object (step A4).
[0035]
The feature quantity extraction means 35 is an area, roundness, length, width, ellipse major axis and minor axis which are characteristic elements as the shape of the speed detection object with respect to the labeled image “labeled [1]”. When the ratio is set as a threshold value and these conditions are combined and there is a satisfactory figure, it is determined that the speed detection object has been recognized (step A5) and stored in “selected [1]”.
[0036]
The background update unit 36 stores the still image data “Image [1]” acquired by the video acquisition unit 32 in “Background” to be background data for the next process.
[0037]
As a result of the feature amount extraction means 35, when a figure to be detected is recognized, the detection recording means 37 obtains the center-of-gravity position coordinates.
[0038]
When the image acquisition means 32 to the detection recording means 37 are repeated again, and an object that is determined to be the same object from the characteristics and feature quantities of the object of speed measurement is captured in successive frames “Image [2]”, the speed The calculating means 38 compares the center of gravity position and calculates the moving speed of the object by dividing the moving distance obtained from the coordinate position by the photographing time difference.
[0039]
(Second Embodiment of the Invention)
Next, a second embodiment of the present invention will be described in detail with reference to FIG.
[0040]
Referring to FIG. 3, in this embodiment, the data processing device 3 does not have the initial background data acquisition unit 31, the background estimation unit 34, and the background update unit 36, and instead of the initial background data acquisition unit 31, the background image data acquisition unit 39. However, instead of the background estimation means 33, an image difference detection means 310 is provided.
[0041]
Frame data showing only the background in advance is prepared in advance, and is stored in “Basic Image” as background image data 42.
[0042]
In the first embodiment of the invention, the background image data does not necessarily need to be only the background, and the video acquired by the video acquisition means 32 can be used as an alternative means. Features cannot be applied, and only background data is required. The image acquisition means 32 acquires one frame acquired by the image capturing device 1 as image data via the image input device 2 and stores it in “Image [n] (n = 1)”.
[0043]
The image difference detection means 39 compares the still image data (“Image [1]”) for one frame acquired by the video acquisition means 32 with the background image data “Basic Image” acquired by the background image data acquisition means 310. , Store the difference part in "Difference [1]". For example, if the background is an image of only a road, and the vehicle is shown in the acquired 1-frame image, the road that is the same part of the background and 1 frame is recognized as the background, and where the road should originally exist The foreign object (vehicle) shown in the image is captured as something other than the background (other than the road), and only the vehicle is cut out as the foreground.
[0044]
The labeling means 34 assigns the same label to gray blocks of gray value pixels connected to each other from “Difference [1]”, classifies the figures, and stores them in “Labeled [1]”. When two cars, a red vehicle and a white vehicle, are shown, the gray pixel block near red color becomes a single figure, and the gray pixel block near white color becomes a single figure. The first and second management is performed.
[0045]
The feature quantity extraction unit 35 is a detection target predetermined in the individual environment data 42 for each figure included in “Labeled [1]”, such as the area, the major axis / minor axis ratio of the ellipse, and the roundness. It is determined whether or not the detection target exists in the video depending on whether or not the characteristic condition of the object is met. For example, if the vehicle to be photographed fits within a certain area within the size of the image, and if it includes the same area, it is a vehicle depending on the aspect ratio of each area, etc. It is to recognize whether it is a thing.
[0046]
The processing for starting the detection recording means 38 and the speed calculation means 39 based on the recognized result is the same as in the first embodiment of the present invention.
[0047]
【The invention's effect】
As described above, the present invention has the following effects.
[0048]
The first effect is that speed measurement is possible regardless of the type of detection object by the configuration of the present invention.
[0049]
The second effect is that the speed can be measured in combination with an existing surveillance camera and monitor by the configuration of the present invention.
[0050]
The third effect is that the configuration of the present invention can coexist with other image recognition functions.
[0051]
The fourth effect is that by realizing image recognition with software, customization according to the environment is easy, and thresholds can be set with individual environment data, so it is easy to modify functions accompanying changes in the environment. It is a point.
[0052]
The fifth effect is that various recognition result notifying means can be realized by realizing image recognition by software, and it is easy to cope with and add various support functions.
[0053]
The sixth effect is that noise can be eliminated and the amount of calculation can be reduced by performing background estimation using the Kalman filter.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a first exemplary embodiment of the present invention.
FIG. 2 is a flowchart showing the operation of the first exemplary embodiment of the present invention.
FIG. 3 is a diagram showing a configuration of a second exemplary embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Video imaging device 2 Video input device 3 Data processing device 4 Storage device 31 Initial background data acquisition means 32 Video acquisition means 33 Background estimation means 34 Labeling means 35 Feature amount extraction means 36 Background update means 37 Detection recording means 38 Speed calculation means 39 Background image data acquisition means 41 Background image data 42 Individual environment data 310 Image difference detection means

Claims (5)

Is composed of a Film image input device, a data processing device, a storage device,
Said storage device stores a characteristic individual environment data for each image movies and background image data,
The data processing device uses a Kalman filter to estimate background data from the background image data and the still image data acquired from the video input device , cuts out the foreground portion, and uses the cut-out image portion as the individual environment data. Based on the above, it further narrows down the detection target area, recognizes the detection target, updates the acquired still image data as background image data for the next processing, repeats the processing, and is the same as the continuous still image A speed measurement system based on image recognition that calculates the moving speed of an object by comparing the position of the center of gravity and dividing the moving distance by the difference in shooting time. The data processing device includes initial background data acquisition means, video acquisition means, background estimation means, labeling means, feature amount extraction means, background update means, detection recording means, and speed calculation means. ,
The initial background data acquisition means acquires background image data that is stored in the storage device and the detection object is not photographed, and sets the initial value of background estimation,
The video acquisition means issues and obtains an instruction to acquire the latest one frame of still image data for the video input device,
The background estimation means adapts the background image data acquired by the initial background data acquisition means using the Kalman filter for all the pixels of the still image image data acquired by the video acquisition means, estimates the background, Cut out only the foreground part other than
The labeling means attaches the same label to the gray value pixel clusters connected to each other for the foreground part, takes an outline, recognizes it as a figure, performs labeling after discrimination,
The feature amount extraction means, for each labeled region, area, roundness, ellipse major axis minor axis ratio, which are characteristic element conditions as the shape of the speed detection target object previously defined in the individual environment data, Recognize an object that satisfies the extraction conditions combining length and width as a detection target,
The background update means updates the still image data acquired by the video acquisition means as background image data for the next processing,
When the feature amount extraction unit recognizes a figure that is a detection target, the detection recording unit obtains the center of gravity position,
When the processing from the video acquisition unit to the detection recording unit is repeated again, and when an object that is determined to be the same object from the characteristics and feature quantities of the target object for speed measurement is photographed on a continuous still image, the speed calculation unit The speed measurement system by image recognition according to claim 1, wherein the center of gravity position is compared, and the moving speed of the object is calculated by dividing the moving distance by the photographing time difference. The storage peripherals, and storing the characteristic individual environment data for each image movies and background image data,
By using a Kalman filter from the background image data and the still image data acquired from the video input device , the data processing device estimates the background data, cuts out the foreground portion, and extracts the cut-out image portion based on the individual environment data. Furthermore, it narrows down to the detection target area, recognizes the detection target, updates the acquired still image data to the background image data of the next process, repeats these processes, and detects the same in consecutive still images A method of measuring a speed by image recognition, comprising: calculating a moving speed of an object by comparing the position of the center of gravity when the object is shot and dividing a moving distance by a shooting time difference. The storage peripherals, and storing the characteristic individual environment data for each image movies and background image data,
The step of acquiring the background image data stored in the storage device by the initial background data acquisition means and the detection target is not photographed and setting the initial value of the background estimation;
Issuing and obtaining a command for obtaining the latest one frame of still image data from the video input device to the video input device; and
The background estimation means applies the background image data acquired by the initial background data acquisition means to the background image data acquired by the initial background data acquisition means for all pixels using the Kalman filter for the still image image data acquired by the video acquisition means, and other than the background Cutting out only the foreground part of
The labeling means, the foreground portion of the gray value pixel block connected to each other, the same label is attached and contoured, recognized as a figure, distinguished, and labeling;
For each area labeled by the feature quantity extraction means, the area, roundness, ellipse major axis minor axis ratio, length, which are characteristic element conditions as the shape of the speed detection object defined in the individual environment data in advance Well, a step of recognizing an object that satisfies the extraction condition combined with the width as a detection target;
Updating the background image data acquired by the video acquisition means as background image data for the next processing by the background update means;
When the feature quantity extraction means recognizes a figure that is a detection object, a step of obtaining the center of gravity position by the detection recording means;
When the processing from the video acquisition unit to the detection recording unit is repeated again, and when an object that is determined to be the same object from the characteristics and feature quantities of the target of speed detection is photographed on a continuous still image, the speed calculation unit And a step of calculating the moving speed of the object by comparing the position of the center of gravity and dividing the moving distance by the difference in photographing time. A program for causing a computer or a microprocessor to function as a speed measurement system by image recognition according to claim 1 or 2.

Images