JP4032938B2 - Distance index detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a distance index detection device for a contour of a target.
[0002]
[Prior art]
[Patent Document 1]
JP 2000-113201 A (page 4-7, FIG. 1)
2. Description of the Related Art In recent years, a system has been developed that detects a vehicle in the vicinity of a traveling vehicle from an image taken by a camera mounted on the vehicle, presents the positional relationship to the driver, and gives an alarm based on the inter-vehicle distance.
As a method for detecting a vehicle from a photographed image, a method is used in which the rear part of a preceding vehicle is photographed with a video camera to generate image data, and a horizontal edge line segment and a vertical edge line segment in the image are extracted based on the image data. It has been known.
[0003]
In this method, among the extracted vertical edge line segments, pairs of vertical edge line segments that satisfy the criteria that the horizontal distance and the vertical length are related to the vehicle width and the vehicle height, respectively, are extracted.
Next, for each pair of vertical edge line segments, the horizontal edge line segments are selected in the horizontal direction between the vertical edge line segment pairs, and the vertical edges are selected based on the number, position, and length. An evaluation value related to the possibility that the line segment pair is a line segment representing both side ends of the vehicle is calculated. The evaluation value is compared with a reference value to determine whether or not the pair of vertical edge line segments is a line segment representing both side ends of the vehicle.
In this way, the end of the vehicle can be identified to detect the presence or absence of the vehicle (see Patent Document 1).
[0004]
[Problems to be solved by the invention]
In the above-mentioned known technology, since there is a structure having similar characteristics in the background because the edge of the vehicle image is used, or a similar characteristic to the edge of the rear contour of the vehicle due to mutual overlap between the background structure and the detection target vehicle. May occur, the background may be erroneously detected as the entire vehicle or a part thereof.
[0005]
In order to solve the above problems, the present invention provides a distance index that can prevent erroneous detection in detecting the edge of a detection target contour even when there is a structure having characteristics similar to the edge of the detection target contour in the background. An object is to provide a detection device.
[0006]
[Means for Solving the Problems]
For this reason, the present invention provides an image pickup means having an optical system, an optical axis driving means for moving the optical axis of the optical system in a direction perpendicular to the optical axis, and an edge extraction means for extracting an edge of a picked-up image of the image pickup means. And an image accumulation processing means connected to the edge extraction means, and adding and accumulating a plurality of edge images extracted at a constant time interval while the optical axis moves in one direction to generate an accumulated edge image, An image storage means for storing one edge image from the edge extraction means as a representative edge image; connected to the image storage processing means and the image storage means; and the density of each pixel of the representative edge image and the stored edge image A calculation means for calculating the ratio, and the concentration ratio is used as a distance index.
[0007]
【The invention's effect】
According to the present invention, since the density ratio of each pixel of the representative edge image and the accumulated edge image is calculated and used as an index according to the distance, the same density ratio is used when extracting the detection target edge from the edge image of the captured image. Since the edge portions having the same distance, that is, at the same distance, are extracted, it is possible to easily detect the edge of the target pair.
As a result, even when there is a structure having characteristics similar to the edge of the contour of the detection target in the background, erroneous detection can be prevented in edge detection of the contour of the detection target.
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below.
FIG. 1 is a block diagram of a vehicle detection apparatus to which the distance index detection apparatus of the present invention is applied.
The vehicle detection device includes an image processing device 1 and a camera unit 2.
The camera unit 2 includes a lens 21, an image sensor 22, a shutter 24, and a driving device 23 that drives the lens 21 in the horizontal direction.
As shown in FIG. 2, the subject image 26 of the subject 20 is formed on the imaging surface 25 of the image sensor 22 by the lens 21. The lens 21 is driven in the horizontal direction as indicated by an arrow a at a right angle to the optical axis by the driving device 23. The drive device 23 is composed of, for example, an electromagnetic actuator.
The image sensor 22 is a device that converts an optical image such as a CCD into an electric signal. A shutter 24 is provided in front of the imaging surface 25, and exposure of the image sensor 22 is controlled by a signal from the control unit 16. The shutter 24 is a liquid crystal shutter, for example.
[0009]
The image processing apparatus 1 performs image processing on the image signal from the camera unit 2 and detects the edge of the contour of the target. As shown in FIG. 1, first, the image signal of the image sensor 22 is input to the edge detection unit 11. The edge detection unit 11 creates an edge image of the image captured by the image sensor 22 and outputs it to the image accumulation processing unit 12 and the image storage unit 13. Hereinafter, the edge image stored in the image storage unit 13 is referred to as a representative edge image.
The image accumulation processing unit 12 adds and accumulates a certain number of edge images and generates an accumulated edge image.
[0010]
The accumulated edge image and the representative edge image are input from the image accumulation processing unit 12 and the image storage unit 13 to the calculation unit 14, respectively.
The calculation unit 14 calculates the density ratio of each pixel of the representative edge image and the accumulated edge image and outputs it to the detection processing unit 15. In addition, the image storage unit 13 outputs the representative edge image to the detection processing unit 15.
The detection processing unit 15 performs pattern processing on the representative edge image and extracts edges corresponding to the same distance using density ratio data.
[0011]
The control unit 16 is connected to and controlled by the driving device 23, the shutter 24, the image sensor 22, the edge detection unit 11, the image accumulation processing unit 12, the image storage unit 13, the calculation unit 14, and the detection processing unit 15.
The control unit 16 is a drive signal for driving the lens 21 via the drive unit 23, an exposure control signal SC1 for controlling the timing at which the image sensor 22 acquires a captured image and the edge detection unit 11 generates an edge image, and image storage. An image accumulation control signal SC2 that causes the processing unit 12 to perform addition accumulation processing, an image storage control signal SC3 that causes the image storage unit 13 to store a representative edge image, and an arithmetic control that controls the arithmetic timing of the arithmetic unit 14 and the detection processing unit 15. The signal SC4 is output.
The image processing apparatus 1 is configured as a microcomputer as a whole.
[0012]
The operation of this embodiment will be described below.
The control unit 16 inputs a sine wave to the driving device 23. The driving device 23 operates according to the input control signal, and reciprocates the lens 21 in the horizontal direction perpendicular to the optical axis direction.
As shown in FIG. 2, the exposure of an image of the subject 20 is controlled by a shutter 24 through a lens 21. As the optical axis of the lens 21 is moved horizontally, the subject is shifted in parallel to the horizontal direction and the subject image 26, The image is formed as 26 '.
[0013]
At this time, when the distance between the subject 20 and the lens 21 is L1, the distance between the lens 21 and the imaging surface 25 of the image sensor 22 is L2, and the horizontal movement amount of the optical axis of the lens 21 is d1, the subject on the imaging surface 25 The moving amount d2 of the image 26 can be expressed as follows.
d2 = (1 + L2 / L1) × d1 (1)
As shown in Expression (1), the movement amount d2 of the subject image 26 on the imaging surface 25 is larger as the distance L1 between the subject 20 and the lens 21 is smaller.
[0014]
The image sensor 22 performs exposure control with the shutter 24 in synchronization with the exposure control signal sent from the control unit 16, and sends the captured image to the edge detection unit 11 for each exposure.
When the captured image is input, the edge detection unit 11 detects an edge in the vertical direction of the image and generates an edge image. This edge detection is performed using, for example, a Sobel filtering technique. The density (intensity) of the edge image generated here takes a positive or negative value depending on the density relationship with surrounding pixels of the pixel of interest.
A normal captured image output from the image sensor 22 is schematically shown in FIG. 5, and an edge image processed by the edge detection unit 11 is schematically shown in FIG. In FIG. 6, compared with FIG. 5, the vertical part is emphasized and the horizontal part is weakened in the contour of the captured image.
[0015]
The edge image is output to the image accumulation processing unit 12 and the image storage unit 13.
The image accumulation processing unit 12 adds and accumulates a plurality of edge images created from captured images while the subject image moves in one direction from one of the left and right maximum movement positions to the other maximum movement position as the lens 21 moves. To process.
[0016]
FIGS. 3 and 4 show a method for adding and storing edge images in the image storage processing unit 12.
FIG. 3 shows a more detailed configuration of the image storage processing unit 12, and the image storage processing unit 12 includes an image memory 41, an adder 42, a selector 43, a sign comparator 44, and a selector controller 45.
The signal S1 is the pixel density of the edge image output from the edge detection unit 11, the signal S2 is the pixel density of the accumulated edge image accumulated and stored in the image memory 41, and the signal S3 is the adder 42 with the above-described signals S1 and S2. The result of addition operation is shown.
[0017]
FIG. 4 shows a control signal for processing a captured image by the image processing apparatus 1 corresponding to the driving of the lens in the horizontal direction, and the vertical axis indicates the signal level “High” (“H”), “Low” (“L”). The horizontal axis is shown as time t.
(1) to (5) shown in FIG. 4 indicate control timing.
The lens 21 is displaced by a sine wave with a period of T1 seconds with respect to the time axis from the center position to the left and right (±) by the drive signal.
Correspondingly, for the period during which the subject image 26 moves in the left and right directions on the imaging surface 25 between one maximum value and the other maximum value, that is, in one direction leftward or rightward. Several times, here, five times, the exposure control signal SC1 rises at the timing of {circle around (1)} to {circle around (5)} in the period T2 seconds, and the shutter 24 opens for exposure during the “H” level, and the image sensor 22 is exposed. Obtains five captured images.
[0018]
First, at timing (1), the exposure control signal SC1 becomes “H” level, and the image sensor 22 outputs the image signal to the edge detector 11.
The edge detection unit 11 processes the input captured image and generates an edge image.
[0019]
Further, during the period from (1) to (2), the image accumulation control signal SC2 is at the “L” level.
The “L” level image accumulation control signal SC2 is input to the selector controller 45, and during the period when the signal SC2 is “L” level, the selector controller 45 receives the pixel density signal S1 and the pixel density input to the selector 43. The signal S1 is selected from the signal S2 and the pixel density signal S3, and is controlled to be output to the image memory 41.
That is, the edge image of the captured image acquired at the timing (1) is stored in the image memory 41 as an initial image of the addition accumulation process of the image accumulation processing unit 12.
[0020]
In FIG. 4, the image accumulation control signal SC2 changes to the “H” level at the timing (2) and is maintained at the “H” level until the next repeated timing (1).
At timing {circle around (2)}, the exposure control signal SC <b> 1 becomes “H” level, the second captured image is acquired, and the edge detection unit 11 generates the second edge image.
When the “H” level of the image accumulation control signal SC2 is input to the selector controller 45, the selector is selected according to a signal indicating whether or not the signs match, which is input from the sign comparator 44 to the selector controller 45. Among the signals S1, S2, and S3 of the pixel density input to 43, either the signal S2 or S3 is selected and output to the image memory 41.
That is, the density signals S1 and S2 of the pixels at the same position input to the sign comparator 44 are compared, and in the case of the same sign, the pixel density signal S3 obtained by adding the signals S1 and S2 by the adder 42 is selected. In the case of different signs, the signal S2 is selected and stored in the image memory 41.
This addition accumulation process is performed for all pixels.
[0021]
FIG. 7 shows an example in which the pixel density is plotted in the horizontal direction in the vicinity of the edge in the vertical direction of the edge image after the rod-shaped subject at a predetermined distance from the lens 21 is image-processed by the edge detection unit 11. The vertical axis represents the pixel density (edge strength) of the edge image, and the horizontal axis X represents the horizontal position of the pixel.
In the above-described edge image addition and accumulation process, the signal S3 of the adder 42 is written and accumulated in the image memory 41 only when the pixels compared by the sign comparator 44 have the same sign. As shown in Fig. 2, the pixel density of the edge with the opposite sign is generated near the contour of the subject.Therefore, it is highly possible that the edges overlap due to the movement of the image, and adding the pixel density with the opposite sign cancels the edge pixel density. This is to prevent becoming.
[0022]
The processing in the image storage processing unit 12 during the period in which the image storage control signal SC2 is at “H” level is the timing of (3), (4), and (5) when the exposure control signal SC1 is at “H” level. And so on.
In other words, the edge image addition and accumulation processing is performed an even number of times at the period T2 while the lens is moving in the same direction in the half period of the period T1, as can be seen from the lens driving of FIG.
[0023]
When the lateral density distribution of each pixel of the edge image is as shown in FIG. 7, if this image is added and accumulated while moving to the right in the horizontal direction, the density of the accumulated edge image as shown by the thick broken line in FIG. Distribution. Here, the peak value Va is taken at the midpoint of the moving range when the image is moved in the X direction.
When the edge density is different in the addition and accumulation process, the pixel density signal S2 accumulated in the image memory 41 is selected, that is, it is not added, so the pixel density to be added on the plus side in FIG. A dotted line is shown on the right side from the position where the pixel density exists for reference.
[0024]
When the subject 20 having the same edge strength contour as shown in FIG. 7 is closer to the lens 21 than in the case of FIG. 7, the subject image movement amount relative to the horizontal movement amount of the same lens Therefore, as shown in FIG. 9, the accumulated edge image has a wide edge portion. As a result, the peak value Va ′ is lower than Va in FIG. Thus, the accumulation edge image of the image accumulation processing unit 12 has a larger peak value Va as the movement amount of the subject image is smaller, that is, as the distance between the subject 20 and the lens 21 is longer.
[0025]
The control unit 16 sets the image storage control signal SC3 to be sent to the image storage unit 13 to the “H” level during the period from (3) to (4) in FIG. 4, and the timing of (3) output from the edge detection unit 11. An edge image at the center of the odd-numbered images acquired during movement in one direction of the subject image, that is, the edge image by exposure, is stored in the image storage unit 13 as a representative edge image.
[0026]
After the image storage processing unit 12 adds and stores the edge image at the timing (5) and stores it in the image memory 41, the control unit 16 performs a period until the timing (1) when the next cycle exposure starts. The calculation control signal SC4 sent to the calculation unit 14 and the detection processing unit 15 is set to the “H” level.
The calculation unit 14 receives the “H” level of the calculation control signal SC 4, reads the final accumulated edge image from the image memory 41, and reads the representative edge image from the image storage unit 13. The density of the representative edge image / the density of the final accumulated edge image) is calculated and output to the detection processing unit 15.
[0027]
However, when the density value of the pixel of the accumulated edge image is zero, the calculation is not performed and the ratio is set to zero. When the pixel density of the representative edge image in the image storage unit 13 is zero, the calculation result is zero. Therefore, the pixel density ratio of the portion where no edge exists is zero.
As described in FIGS. 8 and 9, the farther the subject is from the lens 21, the higher the pixel density peak value of the final accumulated edge image at the position corresponding to the pixel density peak value of the edge part of the representative edge image. The density ratio calculated in 14 is a smaller value for the edge position of a subject at a long distance.
Thus, the density ratio of each pixel of the edge calculated by the calculation unit 14 becomes a distance index. This density ratio for each pixel is hereinafter referred to as a distance index data map.
[0028]
A distance index data map is output from the calculation unit 14 and a representative edge image is output from the image storage unit 13 to the detection processing unit 15.
The detection processing unit 15 temporarily stores these and performs target detection processing.
As a first step, pattern processing is performed on the representative edge image to extract a vertical edge line, and a pair of vertical edge candidates in the outline of the vehicle is used.
In the second step, for the pair of vertical edge candidates, the density ratio of the distance index data map corresponding to the pixel position of the edge portion is read, and the density ratio indicated by the pixel positions of both vertical edge portions is almost equal. When the values are the same, the pair of vertical edges is determined to be the vertical edge of the rear contour of the other vehicle that is equidistant from the host vehicle.
[0029]
In the present embodiment, the lens 21, the image sensor 22, and the shutter 24 controlled by the control unit 16 are the imaging unit of the present invention, the driving device 23 is the optical axis driving unit, and the edge detection unit 11 is the edge extracting unit. The image accumulation processing unit 12 constitutes an image accumulation processing unit, the image storage unit 13 constitutes an image storage unit, the calculation unit 14 constitutes a calculation unit, and in particular, the lens 21 constitutes an optical system.
[0030]
As described above, according to the present embodiment, the optical axis of the lens for forming the subject image on the imaging surface is moved in the horizontal direction, and a plurality of edge images are acquired and added and accumulated during image movement. By creating a stored edge image and taking the density ratio of each pixel with the representative edge image, using the fact that the density ratio of the pixels in the edge portion corresponds to the distance of the subject, a pair of vertical edges corresponding to the same distance Can be easily detected.
As a result, the pattern processing alone can prevent erroneous detection of a background structure or a similar edge pattern generated by overlapping the background and the target as a pair of vertical edges at the same distance.
That is, since the vertical edge of the rear contour pair of the vehicle ahead can be easily extracted from the image captured by the video camera, it can be used for vehicle detection.
[0031]
Since the representative edge image is selected at the center position of the left and right maximum amplitudes of the captured image and compared with the final accumulated edge image, the peak value of the pixel density at the edge portion of the representative edge image and the final accumulated edge image are compared. The peak part of the pixel density at the edge part is compared, and the density ratio is an effective distance index.
Further, in the edge image addition and accumulation processing in the image accumulation processing unit 12, when the positive side pixel density and the negative side pixel density appear adjacent to each other at the edge part of the edge image, the image density of the same sign is added. When different signs are used, no addition is performed, so that edge cancellation can be prevented and edge density originally desired to be extracted and emphasized can be emphasized.
[0032]
The edge detection unit 11 emphasizes and extracts vertical edge portions, and acquires and accumulates a plurality of edge images while the subject image moves in the horizontal direction. Thus, the edge portion is expanded in the horizontal direction, and the enhancement of the density of the edge portion can be effectively obtained in the addition and accumulation process.
[0033]
In the above embodiment, the edge detection unit 11 extracts edges so as to emphasize vertical edges, and the lens 21 is driven in the horizontal direction. However, the edge detection unit 11 emphasizes horizontal edges. Edge extraction may be performed, and the lens may be driven in the vertical direction.
[0034]
Further, in the embodiment, the subject image on the imaging surface is moved by moving the optical axis by driving the lens. However, for example, a plurality of microlenses are arranged side by side, and a liquid crystal shutter is placed on the front or back of the microlens. And a configuration in which light can be transmitted or not transmitted for each microlens, and a configuration in which the optical axis movement is controlled by switching the transmission position of the liquid crystal shutter according to an exposure control signal from the control unit 16. Good.
With such a configuration, the optical axis can be moved with a highly reliable structure without a dynamic mechanism.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an embodiment of the present invention.
FIG. 2 is a diagram illustrating optical axis movement of a camera unit.
FIG. 3 is a diagram illustrating an addition accumulation processing method of the image processing apparatus.
FIG. 4 is a diagram illustrating image processing timing control of the image processing apparatus.
FIG. 5 is a diagram illustrating a captured image acquired by a camera unit.
FIG. 6 is a diagram illustrating an edge image generated by an edge detection unit.
FIG. 7 is a diagram illustrating a change in pixel density in the horizontal direction in the vicinity of an edge portion in an edge image.
FIG. 8 is a diagram showing a change in pixel density in the horizontal direction in the vicinity of the edge portion of the accumulated edge image when the subject is far away.
FIG. 9 is a diagram illustrating a change in pixel density in the horizontal direction in the vicinity of the edge portion of the accumulated edge image when the subject is close.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Camera part 11 Edge detection part 12 Image accumulation process part 13 Image storage part 14 Calculation part 15 Detection process part 16 Control part 20 Subject 21 Lens 22 Image sensor 23 Driving device 24 Shutter 25 Imaging surface 26, 26 'Subject Image 41 Image memory 42 Adder 43 Selector 44 Sign comparator 45 Selector controller

Claims (4)

Imaging means having an optical system;
Optical axis driving means for moving the optical axis of the optical system in a direction perpendicular to the optical axis;
Edge extraction means for extracting an edge of a captured image of the imaging means;
An image accumulation processing means connected to the edge extraction means, and adding and accumulating a plurality of edge images extracted at a constant time interval while the optical axis moves in one direction to generate an accumulated edge image;
Image storage means for storing one edge image from the edge extraction means as a representative edge image;
A calculation unit connected to the image storage processing unit and the image storage unit, calculating a density ratio of each pixel of the representative edge image and the storage edge image, and outputting the density ratio as a distance index; A distance index detection device. The optical axis driving means performs movement of the optical axis by periodic vibration of the optical system, and the image accumulation processing means performs addition accumulation processing of the edge image between the maximum amplitudes of the periodic vibration,
The distance index detection apparatus according to claim 1, wherein the image storage unit stores an edge image of a captured image at a center position of the amplitude of the periodic vibration.   2. The image accumulation processing means performs addition accumulation only when the density of a pixel of an edge image to be newly added has the same sign as the density of the pixel to be added, and does not add if the density is different. Or the distance parameter | index detection apparatus of 2.   4. The optical axis driving unit moves the optical axis in a direction perpendicular to the direction of the edge extracted by the edge extraction unit by enhancing the edge from the captured image. The distance index detection device according to 1.

Images