JP2551324B2 - Method and device for recognizing environment of moving vehicle - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting an object moving with respect to the environment, such as a preceding vehicle or an oncoming vehicle, from an image captured by a camera mounted on a vehicle, and particularly, the calculation amount BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving vehicle environment recognition method and apparatus for reducing the size of a device.

[0002]

2. Description of the Related Art As a method for detecting a moving object using an image picked up by a moving camera, there is a "moving image analysis method" (Japanese Patent Application No. 3-308623). There, the motion vector that occurs on the image when the camera moves in a stationary environment is calculated, and from the motion vector, the reciprocal of the movement parameter of the camera and the distance from the camera to the object in the image (reciprocal distance). Is solved by the method of least squares, and the place where the squared error becomes large is regarded as the moving object.

[0003]

In the prior art,
Since the moving object is detected after the camera motion parameter and the inverse distance are calculated from the motion vector on the image, and the motion vector must be calculated accurately for that purpose, the calculation cost is high and it is large as an in-vehicle device. It has the drawback of becoming too much.

It is an object of the present invention to provide a method and apparatus for recognizing the environment of a moving vehicle, which is capable of detecting a moving body with a smaller vehicle-mounted device which has a low calculation cost.

[0005]

A method for recognizing a moving vehicle environment according to the present invention sets a window in an image captured by a camera mounted on the vehicle, and calculates a motion vector calculated at each point in the window. In the environment recognition method of a moving vehicle that detects a moving body, when a window is set in the image and it is assumed that the moving body does not exist in the set window, the distribution of motion vectors calculated at each point in the window Is calculated, the motion vector at each point in the set window is calculated, and the object is detected using the motion vector outside the distribution of the motion vectors.

The environment recognizing device for a moving vehicle according to the present invention sets a window in an image taken by a camera mounted on the vehicle and detects an object from a motion vector calculated at each point in the window. In the environment recognition device of
Window setting means for setting a window in an image and calculating a distribution of motion vectors calculated at each point in the window when a moving object does not exist in the set window, and a window setting means A motion vector calculation means for calculating a motion vector at each point, and an object detection means for detecting an object using a motion vector out of the distribution obtained by the window setting means among the motion vectors. It is characterized by

[0007]

In the present invention, the method of calculating the distribution of the motion vector calculated at each point in the window when the moving body is not present in the set window, and the motion calculated at each point in the window Regarding a method of detecting an object using a motion vector out of the distribution on the motion vector plane among the vectors, refer to FIGS. 2, 3, 4, 5, 6, 7, and 8. Describe.

FIG. 2 shows an image pickup system using a camera mounted on a vehicle.
Express with. The xyz system is a coordinate system fixed to the vehicle, and the xy plane coincides with the road surface. The lens center of the camera is fixed to the point P on the z axis, and XYZ is the camera coordinate system. The optical axis of the camera coincides with the X-axis, which points downwards from the horizon by an angle θ. The image plane is located at a distance f from the point P on the X axis so as to be parallel to the YZ plane, and the ξη coordinate system is set on the image plane. Let ξ be the point at which the point R on the road surface (XY plane) is projected on the image surface.
_{Let R} and η _R be

[0009]

[Equation 1]

[0010] However, (X _R , Y _R , Z _R ) is the coordinates of the point R in the camera coordinate system,

[0011]

[Equation 2]

Is given by

Now, by the motion of the vehicle, xyz system and XY
It is assumed that the Z system moves as shown in FIG. However, FIG. 3 is a view of the movement from above. That is, the camera moves from the x axis in the direction of φ _{p by} the distance d _p, and the projection of the optical axis of the camera on the xy plane coincides with the x ′ axis. If the coordinates of the point R on the image plane after movement are ξ ′ _R and η ′ _R ,

[0014]

(Equation 3)

[0015] Therefore, the motion vector generated when the point R moves on the image due to the motion of the camera is

[0016]

[Equation 4]

It is

The angle φ _p can be calculated from the angle of the front wheels of the vehicle. The motion of the vehicle is approximated by the bicycle model shown in FIG. The cutting angle of the front wheels is φ ₀ , and the center of rotation at that time is the point Q. It is sufficient if φ ₀ is the average of the angles of the left and right wheels. The distance between the rear wheel and the front wheel is l _0, and the distance between the rear wheel and the camera is l _p . The radius of gyration r ₀ is

[0019]

(Equation 5)

The angle φ _p indicating the moving direction of the camera is

[0021]

(Equation 6)

Is given by Also, video rate 1/3
The movement amount d _p of the camera during 0 seconds is determined by the vehicle speed sk.
When m / h,

[0023]

(Equation 7)

Is given by

Therefore, when the window a is set in the shaded area in FIG. 5, the coordinates of points in the window (ξ
_i , η _i ) has a one-to-one correspondence with the coordinates (x _i , y _i , 0) on the road surface by the equations (1) and (2). Therefore, the window a on the image corresponds to the area b on the xy plane (road surface), and the motion vector generated by the points in the area b is given by the equation (1).
~ (6), the set becomes the region c on the motion vector plane.

Therefore, when the motion vector is detected at each point in the window a and the motion vector is plotted on the motion vector plane, the motion vector outside the area c is not the motion vector generated by the movement of the road surface. Therefore, the moving body can be detected.

As a method for calculating the motion vector, Naoya Ota, “Detection of Movement Vector with Reliable Coordinates”, Computer Vision '90, Symposium Papers, 199.
0-8, pp. The method described in 21-30 can be used. Further, the direction reliability r of the calculated motion vector is

[0028]

(Equation 8)

Is given by However, E (ξ, η) is the pixel value at the position (ξ, η) of the input image,

[0030]

[Outside 1]

Is a partial differential with respect to ξ and η of the pixel value, respectively.

The angle θ ₁ appearing here is an angle from the horizontal line as shown in FIG. 6, but normally 0 or 18
0 degree is selected. However, when there is a road white line in the window, the line segment s is selected so as to be in the same direction as the white line.

Next, the "moving voting method" will be described. Figure 7
As shown in the above, among the motion vectors calculated in the window, a motion vector that exists outside the region c in the motion vector plane, or a motion vector that exists outside and has a direction reliability greater than a constant value r _0. Consider an image in which 1 is assigned to the point where is calculated and 0 is assigned to the other points, and this is used as the initial detection image. Although this initial detection image may be used as the detection result, a method for performing better detection by performing temporal interpolation is the “moving voting method”, and the moving body is detected according to the rules shown in FIG. 1. Obtain the initial detection image. 2. Calculate on the voting image. The pixel value h of the voting image at the position corresponding to the pixel value 1 of the initial detection image is replaced with the value h + d obtained by adding the constant d. However, when h + d exceeds a specific numerical value h _max , h
_Replace with _max . -The pixel value of the voting image at the position corresponding to 0 in the initial detection image is divided by the constant e. 3. The voting image is duplicated with a certain threshold value T (pixel value 1 is the moving body region) to obtain a detection image, which is output. 4. The moving amount of the moving body area is calculated, and the voting image is moved by the moving amount. Return to 5.1.

For the calculation of the movement amount, the average of the motion vectors in the moving body area is used.

[0035]

Embodiments of the present invention will now be described with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of the present invention. Referring to FIG. 1, an embodiment of the present invention comprises an image input device 1 such as a camera, an environment recognition device 2 for a moving vehicle according to the present invention, and an output device 3 such as a display device.

The environment recognizing device 2 for a moving vehicle sets a window in the image and calculates the distribution of the motion vector detected when the moving body is not present from the moving information of the vehicle, and the window setting means 21. A motion vector calculating means 22 for calculating a motion vector at each point in the window, a direction reliability calculating means 23 for calculating the direction reliability of the calculated motion vector, and a direction reliability among the motion vectors outside the distribution. The object area detection means 24 detects a moving body by using a motion vector whose degree is larger than a certain numerical value.

Next, the operation of this embodiment will be described with reference to FIGS. 3, 6, 8 and 9.

The image acquired by the image input device 1 is supplied to the window setting means 21.

The window setting means 21 sets windows a1 and a2 in the image as shown in FIG. At this time, the distribution of the motion vector detected when the moving body does not exist is calculated according to the equations (1) to (6). At that time, using the movement information of the vehicle, equation (7),
By (8), the movement parameters φ _p and d _p of the camera in FIG. 3 are calculated and used. Since there is an error in the obtained movement parameter, the calculated distribution is made slightly larger and the areas c1 and c2 are set on the motion vector plane.

The motion vector calculation means 22 calculates a motion vector at each point in the window set by the window setting means 21.

The direction reliability calculation means 23 calculates the direction reliability r at each point in the window according to the equations (9) to (11). The angle θ ₁ is 0. However, when there is a road white line in the window, the line segment s in FIG.
So that it matches the direction of the white line.

In the object area detecting means 24, the motion vector of each point in the window a1 calculated by the motion vector calculating means 22 is not included in the area c1 on the motion vector plane, and further in the direction reliability calculating means 23. An initial detection image 1 in which a point where a motion vector whose calculated direction reliability r is larger than a fixed numerical value r ₀ exists is 1, and other points are 0.
Create At the same time, among the motion vectors detected at each point in the window a2, the point where there is a motion vector whose direction reliability r is larger than the numerical value r ₀ is set to 1, and the other points are set to 0, and the initial detection image 2 is set. create. Since the calculated motion vector also contains an error, when selecting the motion vector, among the motion vectors obtained in a1,

[0044]

[Equation 9]

And among the motion vectors found in a2

[0046]

[Equation 10]

It is sufficient to choose one that is r ₀
Is adopted as 2.0. For each initial detection image, a detection image is obtained by the “moving voting method” shown in FIG. 8 and supplied to the output device 3. Regarding parameters, d = 10, h _max = 200, T = 1.0, e =
3.0 is a typical value.

[0048]

As described above, in the method and apparatus for recognizing the environment of a moving vehicle according to the present invention, by calculating the distribution of the motion vector detected at each point in the window when there is no moving body, It is possible to detect a moving body from a motion vector that does not belong to that distribution. Since the cost of the calculation amount is smaller than that of the conventional method, the moving body can be detected by a small vehicle-mounted device.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a model of an image pickup system.

FIG. 3 is an explanatory diagram showing movement of a camera.

FIG. 4 is an explanatory diagram showing a relationship between a moving direction of a camera and a moving direction of a vehicle.

FIG. 5 is an explanatory diagram showing that a motion vector calculated in a window has a certain distribution.

FIG. 6 is an explanatory diagram showing angles of directional reliability.

FIG. 7 is an explanatory diagram showing an initial detection image.

FIG. 8 is an explanatory diagram showing a “moving voting method”.

FIG. 9 is an explanatory diagram showing the distribution of windows and motion vectors in the example.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 image input device 2 environment recognition device for moving vehicle 3 output device 21 window setting means 22 motion vector calculation means 23 direction reliability calculation means 24 object area detection means

Claims (6)

(57) [Claims] 1. An environment recognition method for a moving vehicle, wherein a window is set in an image captured by a camera mounted on a vehicle, and a moving body is detected from a motion vector calculated at each point in the window. Set the window to and set the area of the motion vector calculated at each point in the window on the motion vector plane when there is no moving object in the set window.
A method for recognizing an environment of a moving vehicle, comprising: setting, calculating a motion vector at each point in the set window, and detecting an object using a motion vector that does not belong to the region among the motion vectors. 2. The environment recognition method for an automobile according to claim 1, wherein the traveling information of the vehicle is used when the area is set . 3. The object is detected by using a motion vector whose direction reliability in a certain direction is larger than a specific value among motion vectors not belonging to the area.
Alternatively, the method for recognizing the environment of a moving vehicle according to item 2. 4. An environment recognition device for a moving vehicle, wherein a window is set in an image captured by a camera mounted on a vehicle, and an object is detected from a motion vector calculated at each point in the window. Set the window and set the area of the motion vector calculated at each point in the window on the motion vector plane when the moving object does not exist in the set window.
A window setting means for setting, a motion vector calculating means for calculating a motion vector at each point in the set window, and a motion vector among the motion vectors that does not belong to the region obtained by the window setting means are used. 2. An environment recognition device for a moving vehicle, comprising: 5. The environment recognizing device for a moving vehicle according to claim 4, wherein the window setting means uses traveling information of the vehicle. 6. The direction reliability calculation means for calculating reliability of a motion vector calculated by the motion vector calculation means in a specific direction by the object detection means, and the motion vector calculated by the motion vector calculation means. Out of the win
Object area detection means for detecting an object using a motion vector that does not belong to the area obtained by the dough setting means, and whose direction reliability calculated by the direction reliability calculation means is larger than a specific value. The environment recognition device for a mobile vehicle according to claim 4 or 5.