JP4175087B2 - Vehicle external recognition device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for fusing a device that detects a relative position with an object by scanning using an electromagnetic wave such as a laser and a device that performs image processing from an imaging result such as a camera.
[0002]
[Prior art]
There has been proposed an object detection apparatus that divides a laser irradiation range into a plurality of detection areas and determines the presence of an object from the reception intensity of each detection area (see Patent Document 1). In this apparatus, since the added value of the received intensity in each detection area is compared with the threshold value, the object can be reliably detected regardless of the relative distance to the object.
[0003]
[Patent Document 1]
JP 2000-28718 A (pages 4 to 6)
[0004]
[Problems to be solved by the invention]
However, in the apparatus as described in Patent Document 1 described above, when the reflecting object that reflects the electromagnetic wave from the radar cannot completely reflect the electromagnetic wave as a whole (because a part of the object is out of the detection range or the like, If the reflective area for the true size is different), the correct relative lateral position may not be detected. This will be described with reference to FIGS. At time t0, the entire stopped vehicle is within the radar viewing angle. At this time, the reflective area is detected, and the center positions of the left end and the right end of the detection reflector group are recognized as the lateral position of the vehicle. Since part of the stopped vehicle goes out of the radar viewing angle at time t1, the stopped vehicle is positioned at the position indicated by the solid line in FIG. 8, and the lateral position is recognized by the reflector on the right end side only. Thereafter, the entire vehicle goes out of the viewing angle at time t2. FIG. 9 is a time chart showing the relationship between the lateral position of the stopped vehicle recognized by the object detection device and the width of the object (vehicle width). As shown in the time chart of the horizontal position in FIG. 9, when the vehicle is recognized by only one reflector, the vehicle width changes greatly. As a result, a step is generated between the actual lateral position and the relative lateral position of the radar output.
[0005]
By the way, a fusion system that redundantly detects an object A detected by a radar from a monocular camera image (a system that captures the same object A as a radar with a monocular image in preparation for a temporary loss of radar due to pitching fluctuations, etc. ) Realizes a reduction in processing time and an improvement in reliability as compared to external recognition using a single image. As an example, a typical processing sequence by the fusion system is shown in FIG. When an object is found in process 1, an attention area is determined from the imaging result based on the position result detected by the radar in process 2. Then, in process 3, a region Ω having a predetermined feature is extracted from a monocular image, and calibration is performed to determine how far the region Ω in the monocular image has. Thereafter, in the process 4, the object A is tracked by a process of tracking using only the image (pattern matching, edge detection, etc.), and the position of the object A is calculated based on the tracking result and the calibration result.
[0006]
In general, object detection by a camera is less reliable than radar object detection. Therefore, in order to increase certainty, if {the radar detection position is compared with the detection position by image processing and the difference is small, when the difference continues for a predetermined period τ or more = this is set as processing ε}, the current traveling The camera image processing in the situation performs processing 5 for determining whether or not the reliability is high.
[0007]
When tracking an object with an image, if the tracking period becomes long, the object may not be detected and tracked correctly from the image. This is caused by a background or the like entering the target object during tracking. For this reason, it is common to determine whether the radar and the image captured by the image are the same object or not. This avoids the problem of not knowing which one to believe when both the radar and the image move separately. In particular, except for the sudden appearance of an interrupting vehicle or the like, it takes a long time to track the radar detection object with a monocular camera, so the reliability of the detection result in the image is regularly determined by the above processing ε. .
[0008]
If it is determined in the process 6 that the reliability is low, the initial image processing state to be executed again from the process 1 (the characteristic area Ω of the camera image is again determined based on the radar detection position, and the calibration between the image and the radar is performed. Return to the process to be performed.
On the contrary, when it is determined that the reliability is high, tracking of the object position is continued only by the image processing, and braking and warning having different timing and size are performed as compared with the case where the reliability is low.
[0009]
In such a fusion system, when a radar position change as shown in FIGS. 7 to 9 occurs, it is determined that the difference between the radar detection position and the image detection position is large, and the following problem occurs.
-Less than a predetermined period τ (before reliability is increased): The period until reliability is further increased.
-After a predetermined period τ (after reliability is increased): Return to the calibre state (initial state with low reliability).
[0010]
In view of the above-described problems, the present invention can continue image processing even when there is an object to be detected near the left and right ends of the radar viewing angle and the radar erroneously detects the lateral position of the detected object. An object of the present invention is to provide a vehicle external environment recognition device.
[0011]
[Means for Solving the Problems]
  In order to achieve the above object, in the present invention,By scanning the electromagnetic wave with respect to the predetermined range,Detect the object and determine the relative position between the vehicle and the detected object.detectionScanning type distance measuring means, imaging means for imaging the situation around the vehicle,Of the imaging meansImaging resultsPerforming image processing of either object tracking processing by an image or processing of performing edge detection processing with an area corresponding to the position of the object detected by the scanning distance measuring means in the image as a region of interest, Detects the relative position of the vehicle and objects around the vehicleImage processing meansThe aboveBased on the output of the scanning rangefinderAnd grasping the distance measurement state that is the degree that the object detected by the scanning distance measuring means is located at the end of the predetermined range.Ranging status grasping means,If the distance measuring status grasping means finds that the object detected by the scanning distance measuring means is not located at the end of the predetermined range, the own vehicle detected by the scanning distance measuring means The relative position between the vehicle and the object is compared with the relative position between the vehicle and the object detected by the image processing unit, and the relative position between the vehicle and the object detected by the scanning distance measuring unit When the difference between the vehicle and the object detected by the image processing means is smaller than a predetermined value, the processing of the image processing means is changed to the object tracking process, and the vehicle detected by the scanning distance measuring means When the difference between the relative position between the object and the object and the vehicle detected by the image processing means is equal to or greater than a predetermined value, the processing of the image processing means In the case of changing to the edge detection processing and the distance measurement status grasping means grasping that the object detected by the scanning distance measurement means is highly located at the end of the predetermined range, the scanning type The relative position between the vehicle and the object detected by the distance measuring unit and the relative position between the vehicle and the object detected by the image processing unit are not checked, and the processing of the image processing unit is not changed.Image processing change meansHaving establishedThus, the above problem has been solved.
[0012]
【The invention's effect】
In the present invention, the detection status of each object detected by scanning is grasped. Then, the processing content of the image is temporarily changed according to the necessity of changing the image processing from the result of grasping the detection status. In other words, even if a step is generated at the radar detection position and a difference between the original object position and the detection position by the image processing occurs, the object can be detected accurately without causing any trouble in object tracking by the image.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, although the embodiment of the outside world recognition device for vehicles in the present invention is described based on an example, the present invention is not limited to an example.
[0014]
(First embodiment)
FIG. 1 is a diagram showing a first embodiment of the present invention. First, the configuration will be described. A scanning type laser radar 1 is provided at the head of the vehicle. The laser radar 1 is connected to a radar processing device 2 that extracts obstacle candidates from the scanned result, and the radar processing device 2 has a two-dimensional origin with the vehicle as the origin for one or more obstacle candidates. (Vehicle distance direction and vehicle width direction) Calculation of coordinate values and calculation of the width (size) of obstacle candidates are performed.
[0015]
In addition, a progressive scan type 3CCD camera 3 is installed to quickly grasp the situation ahead of the vehicle. The imaging result of the 3CCD camera 3 is connected to the image processing device 4. The image processing device 4 stores the image data near the coordinates of the obstacle candidate captured by the radar processing device 2, and detects the object by image processing when the radar detection object is lost due to the pitching variation of the host vehicle. Is implemented.
[0016]
The output of the radar processing device 2 and the output of the image processing device 4 are connected to the external recognition device 5. A vehicle speed detection device 6 that detects the dependent left and right wheel speeds and a steering angle detection device 7 that detects the front wheel steering angle are connected to the external environment recognition device 5 in order to estimate the state quantity of the host vehicle.
[0017]
From the hardware configuration as described above, calculation processing such as distance measurement status grasping means of the present invention and subsequent processing (relative speed calculation processing and obstacle judgment processing) changing means, etc. are performed, and thereby, high-level external recognition for vehicles. The system is implemented.
[0018]
The outside recognition device 5 accurately determines whether each object detected by the radar processing device is an obstacle for the host vehicle, and the determination result is output to the automatic brake control device 8. A negative pressure brake booster 9 that achieves an arbitrary braking force is connected to the front and rear wheels, and a braking force command voltage from the automatic brake control 8 of the host vehicle is applied to the solenoid valve of the negative pressure brake booster 9. Done.
[0019]
Each of the radar processing device 2 and the automatic brake control device 8 includes a microcomputer and its peripheral components, various actuator drive circuits, and the like, and transmits and receives information to and from each other via a communication circuit.
[0020]
FIG. 2 is a block diagram illustrating a control configuration of the vehicle external environment recognition apparatus according to the first embodiment. First, the configuration will be explained. 101 in the figure is a scanning distance measuring means for outputting a relative position with respect to an object, 102 is an imaging means for imaging the situation around the own vehicle, and 103 is based on a distance measurement result of the scanning distance measuring means. Image processing means for performing pattern matching processing on the imaging result of the imaging means 102, 104 a distance measurement status grasping means for grasping the distance measurement status of the object detected by the scanning distance measuring means 101, and 105 a distance measurement status grasping means 104 The image processing change means changes the image processing of the image processing means 103 using an output as a trigger.
[0021]
  FIG. 3 is a flowchart showing object detection control in the first embodiment. This example isClaims 1 and 2From the scanning laser radar scan results (position change and width change) to grasp the situation in front of the vehicle, the distance measurement status of the detected object is grasped, and the contents of the processing related to image processing are temporarily This is a case where image processing with high reliability is performed by making a change to the above. This control is performed every 100 [ms].
[0022]
In step 201, the position vector (horizontal direction: Px_z) of each object detected by the scanning laser radar 1 is detected.₀, Longitudinal direction Py_z₀) And the width of the object (size: W_z₀). Furthermore, the camera imaging result is also captured. The subscript z₀Is the current value, z₁Is the past value of 1 sampling (100ms), z_nMeans n sampling past values respectively.
[0023]
In step 202, the following expression (1) is calculated from the vertical position and horizontal position of the detected object read in step 201.
α = func1 [abs {atan (Px_z0 / Py_z0)} − K1 Th_R] (1)
Here, func1 (A) is a function having the characteristics shown in FIG. 4 and has a range of func1∈ [0, 1]. atan (A) is a function that outputs the arc tangent value of A (the first letter of the argument argument), and abs (A) is a function that outputs the absolute value of A. K1 is a positive number less than 1, and Th_L and Th_R represent the angles of the left and right ends in the viewing angle of the scanning laser radar. For example, when the scanning laser radar with a viewing angle of 12 deg is used and the right side in the direction of travel of the vehicle is positive, Th_L = −6 [deg], Th_R = +6 [deg] with the front of the vehicle as the center (zero deg) .
[0024]
In step 203, the following equation (2) is calculated from the width of the detected object read in step 201.
β = func2 {W_z0−Th_W} (2)
Here, Th_W is a threshold value determined from the width of an object handled as an object to be alarmed or controlled. func2 (A) is a function having the characteristics shown in FIG. 5 and takes a range of func2ε [0, 1].
[0025]
In step 204, the process proceeds to step 208 if the initial image processing end flag: IP_State, which will be described later, satisfies the following expression (3) (when the initial process is completed), and to step 205 otherwise.
IP_State> 0 (3)
[0026]
In step 205, the image processing area is expanded in the direction outside the radar viewing angle, and perspective transformation is performed to determine a region of interest for image processing.
disp_obj_YA = (y_o + (focusV * CAM_h2 / Py_z₀ ))
disp_obj_YB = (y_o+ (focusV * CAM_h / Py_z₀)) (4)
disp_obj_XL = (x_o+ (focusH / Py_z₀* Px_z₀))-(focusH * wide / Py_z₀)
disp_obj_XR = (x_o+ (focusH / Py_z₀* Px_z₀)) + (focusH * wide / Py_z₀)
Here, disp_obj _ ** is the coordinate value of the edge of the rectangular area where image processing is performed, disp_obj_YA is the upper side of the rectangle, disp_obj_YB is the lower side of the rectangle, disp_obj_XL is the left side of the rectangle, and disp_obj_XR is the image coordinate on the right side of the rectangle Represents. Y_oIs the ordinate [pix] of the vanishing point, x_oIs the abscissa [pix] of the vanishing point (y_o, x_oIs a parameter determined by the camera mounting position and orientation), focusV is the vertical focal length [pix] of the camera converted to pixels, and the focal length [pix] of the horizontal direction of the camera converted to focusH pixels, and the light receiving surface is square In the case of a lattice, focusV = focusH (focusV and focusH are parameters determined by the camera angle of view and the resolution of the light receiving element). CAM_h is the camera mounting height [m], CAM_h2 is the value obtained by subtracting the object height to be considered as an obstacle candidate: obj_H [m] from CAM_h, and wide is the horizontal position of the laser radar. This is a value [pix] obtained by pixel-converting half the width [m] determined by the sum of the detection accuracy (standard deviation [m]) and the maximum value of the object to be considered as an obstacle.
[0027]
In step 206, initial feature detection processing is performed on the region of interest limited by the above equation (4) in step 205 in the following manner.
(1) A differential image of a region of interest is obtained by a sobel filter or the like. Here, the sobel filter is an operation for obtaining a luminance change between pixels adjacent to a certain pixel.
{Circle around (2)} Binarization is performed with a threshold value set by using the average value and the variance value regarding the luminance of the obtained differential image. Here, binarization is a process of discretizing two colors, black when the luminance of each pixel (= the luminance of the differential image here) is weaker than a threshold value, and white when the luminance is higher than the threshold value. is there.
(3) A binarized image is projected. Specifically, the number of white pixels having a strong luminance change is counted to be added in the horizontal and vertical directions of the screen, respectively. Then, since the luminance change between the background and the vehicle is strong near the left and right ends and the upper and lower ends of the radar detection vehicle, white pixels are counted more than other places.
(4) From the projection results, the area corresponding to the four corners of the radar-detected vehicle in the binary image has a larger projection amount than the other areas, so the edge of the vehicle is detected. In the present embodiment, a pair of vertical edge pairs corresponding to the left and right ends of the vehicle is detected, and the in-screen coordinates are obtained. Here, the positions (horizontal coordinate values) of the obtained edge pairs on the image are set as SideEdge_R and SideEdge_L, respectively.
(5) Width on radar detection vehicle image: EdgeWidth is obtained from the edge pair interval [pix] detected by the projection process (EdgeWidth = SideEdge_R−SideEdge_L).
{Circle around (6)} In the projection result, it is checked whether there is a horizontal edge between SideEdge_R and SideEdge_L.
(7) When a strict feature (H-shaped edge consisting of a combination of (4) vertical edge pair and (6) horizontal edge) can be detected from the attention area of the imaging result, the initial feature detection process is completed. Set IP_State = 1, otherwise, the initial feature detection process fails = SideEdge_R = 0, SideEdge_L = 0, EdgeWidth = 0,
Set IP_State = 0.
[0028]
In step 207, the position of the radar detection object is calculated as the position [m] from the image by the following equations (5) to (7), and the process proceeds to step 213.
CamTate = Py_z₀                                            ... (5)
CamYoko
= ((((SideEdge_R + SideEdge_L) / 2) -x_o) * Py_z₀) / focusH (6)
RealWidth = EdgeWidth * Py_z₀ / focusH (7)
Here, when SideEdge_R = 0 and SideEdge_L = 0, CamTate and CamYoko are set as output values meaning lost in the image. For example, CamTate = CamYoko = 256 [m]. Further, the counter: LR_IP_Cnt indicating the comparison time between the radar and the image is cleared to zero.
[0029]
In step 208, feature tracking processing is performed on the region of interest in the following manner.
(1) A differential image of a region of interest is obtained by a sobel filter or the like. Specifically, the differential image may be obtained from the entire imaging result, or the differential image may be obtained by focusing only on a region slightly wider than the region including SideEdge_R and SideEdge_L in the previous sampling.
{Circle around (2)} Binarization is performed with a threshold value set by using the average value and the variance value regarding the luminance of the obtained differential image.
{Circle around (3)} Two small areas only in the vicinity of SideEdge_R and SideEdge_L obtained in the previous sampling are obtained from the binarized image.
(4) A projection process is performed for each of the two small areas.
(5) From the two projection processing results, the most frequently counted (long) edge is obtained one by one from each small area. The two vertical edges obtained here are SideEdge_R and SideEdge_L in the current sampling.
(6) Width on image of radar detected vehicle: EdgeWidth is obtained from the interval [pix] between edge pairs.
(7) If a simplified feature (vertical edge pair of (5)) can be detected from the region of interest of the imaging result, set IP_State = 1, which means the success of the feature tracking process, otherwise Then, SideEdge_R = 0, SideEdge_L = 0, EdgeWidth = 0, and IP_State = 0 are set so as to mean that the feature tracking process fails = lost image.
[0030]
In step 209, the position of the radar detection object is calculated as the position from the image by the following equations (8) to (10).
CamTate = (focusV * RealWidth) / (EdgeWidth) (8)
CamYoko
= ((((SideEdge_R + SideEdge_L) / 2) -xo) * CamTate) / focusH (9)
RealWidth = EdgeWidth * CamTate / focusH (10)
Here, when SideEdge_R = 0 and SideEdge_L = 0, CamTate and CamYoko are set as output values meaning lost in the image. For example, CamTate = CamYoko = 256 [m].
[0031]
  In step 210, if α and β obtained in step 202 and step 203 satisfy the following expression (11), the process proceeds to step 212, and if not, the process proceeds to step 211.
α + β> Th_Crit1 (11)
Here, Th_Crit1 is a positive threshold value regarding the frequency of performing temporary processing change in an image, and takes a numerical value less than 2. In other words, when the initial image processing is completed, and then the detected object is highly present at the left and right ends of the radar viewing angle, the image detection position is emphasized without matching with the radar detection position. Even when object tracking is being performed, tracking can be continued without being returned to the initial image processing state due to erroneous determination (Claim 1Corresponding).
[0032]
  In step 211, the radar detection position read in step 201 is compared with the detection position in the image calculated in step 207 or 209 from the following equation (12), and the counter: LR_IP_Cnt representing the collation time is changed.
if (abs (CamTate−Py_z0) + abs (CamYoko−Px_z0)< Th_D) LR_IP_Cnt
= LR_IP_Cnt + 1
else IP_State = 0, LR_IP_Cnt = 0 (12)
Here, if (expression) statement1 else statement2 is a function that executes statement1 when expression is satisfied and executes statement2 when expression is not satisfied. Th_D is a positive threshold value [m] that means an allowable range of positional deviation between the radar and the image.
[0033]
In step 212, the reliability of the image processing is grasped. If the following expression (13) is satisfied, the process proceeds to step 213, and if not, the process proceeds to step 214.
LR_IP_Cnt <Th_Cnt (13)
Here, Th_Cnt is a positive integer threshold value determined by the operation sampling period, and if it is equal to or less than this threshold value, the image reliability is low.
[0034]
In step 213, when outputting the integrated position of the image output to the subsequent stage and the radar, since the reliability of the image is low, the radar detection position is regarded as important, and the following equation is set, and the process proceeds to step 217.
integratedPx = Px_z₀                                  (14)
integratedPy = Py_z₀                                  ... (15)
[0035]
In step 214, the integrated position of the image output to the subsequent stage and the radar is obtained.
Here, the horizontal position obtained by the integration is integratedPx, and the vertical position is integratedPy.
[0036]
In step 215, it is determined whether or not to place importance on the detection result in the image based on the radar detection status according to the following equation (16). If the expression (16) is satisfied, the process proceeds to step 216, and if not, the process proceeds to step 217.
α + β> Th_Crit2 (16)
Here, Th_Crit2 is a positive threshold value for determining that the accuracy of the radar detection position is deteriorated, and takes a numerical value less than 2. That is, when the detected object is present at the left and right ends of the radar viewing angle, the accuracy of the radar detection position may be poor.
[0037]
  In step 216, the integrated position is corrected by the following equation so that the image processing position is emphasized at the integrated position of the image output to the subsequent stage and the radar.
integratedPx = CamTate (17)
integratedPy = CamYoko (18)
That is, when integrating the position of the image and the radar, if there is a detected object at the left and right ends of the radar viewing angle, a step may occur in the horizontal position as shown in FIG. Misdetection may occur. Therefore, by emphasizing the object detection position from the image processing, even if the difference between the detection position and the image detection position is large, or when object tracking by image is being performed, the initial image processing can be performed by incorrect judgment. Tracking can be continued without returning to the state. In the first embodiment, only the position of the image processing is used. However, the detection position obtained by multiplying the detection position by the radar by the first certainty factor, the detection position by the image processing by the second certainty factor, and the respective certainty factors. It is good also as composition which unifies a position. At this time, even if the integrated position is obtained by changing to increase the second certainty factorgood.
[0038]
In step 217, the position obtained by integrating the radar detection position and the detection position from the image is output to the subsequent stage, the past value is updated, and the process ends.
[0039]
Next, the image processing control will be described for each situation with reference to FIGS.
(When there is a stopped vehicle ahead and it is at the left or right edge of the radar viewing angle)
When there is a stopped vehicle ahead (time t0 to t1), the process proceeds from step 201 → step 202 → step 203 → step 204 → step 205 → step 206 → step 207 → step 212 to improve the reliability of the image detection position. Check. When the reliability of the image detection position is high, the process proceeds from step 214 to step 215 to step 217, where the radar detection position and the image detection position are integrated to output the integrated position, and the past value is updated. When the reliability of image detection is low, the process proceeds to step 213, and the radar detection position is output as the integrated position. Once the initial image processing is completed, the process proceeds from step 204 to step 208 to step 209 to step 210 to step 211, and the image reliability is verified by collating with the radar detection position while continuing the image tracking. The position of the detected object is output while checking.
[0040]
(There is a stopped vehicle ahead, and the vehicle goes further out of the radar viewing angle from the left and right ends of the radar viewing angle.)
When there is a stopped vehicle ahead and the vehicle has moved further from the left and right ends of the radar viewing angle to the side farther from the own vehicle (time t1 to t2), step 201 → step 202 → step 203 → step 204 → step 208 Step 209 → Step 210 → Step 210 In other words, the width of the sensing object is small when the sensing object is present at the left and right ends of the radar viewing angle, but when it moves gradually away from the vehicle and goes out of the radar viewing angle, it recognizes the reflector on the right side only. The width of the detected object is rapidly reduced (see FIGS. 7 to 9). In this case, although the error in the radar detection position is large, the deviation from the image detection position becomes large, so that it is determined that the reliability of the image processing is low, and the initial state is returned to the low reliability again. And the problem that the period until the reliability becomes high is delayed. Therefore, when the detection vehicle is present at the left and right ends of the radar viewing angle, it is not recognized that the reliability of the image processing is low by not comparing the radar detection position and the image detection position. Does not interfere with tracking.
[0041]
As described above, in the first embodiment, during long-term tracking in image processing, there is a step or error in the radar detection position in the process of grasping the difference from the radar as a judgment of the quality of the tracking state. In such a case, the image failure determination is not performed in the position matching between the radar and the image. Further, when integrating the positions of the radar and the image, the image is emphasized. In this way, by temporarily changing the image processing, when the radar detection accuracy is poor, it is not mistaken that there is a mistake on the image side, so that the tracking performance by the image is improved, The accuracy of the obstacle determination process at the later stage can be improved.
[0042]
  (Reference example)
nextReference exampleWill be described. Since the basic hardware configuration is the same as that of the first embodiment, only different points will be described.. FirstIn the first embodiment, the image detection position is emphasized without performing the image processing failure judgment near the laser radar viewing angle.Reference exampleThen, a more accurate fusion process is performed by changing the radar lateral position step so as to temporarily correct the lateral position used in the image processing.
[0043]
  Figure 4Reference example5 is a flowchart showing relative speed calculation control and obstacle determination control in FIG. This embodiment is also performed every 100 ms, and step 301 is the same as step 201 in the first embodiment, and is therefore omitted.
[0044]
  Step 302 is the same as step 202, but proceeds to 305 if the following equation (19) is satisfied, otherwise proceeds to 303.
K1 Th_L <atan (Px_z₀ / Py_z₀ <K1 Th_R (19)
Here, if Equation 19 is not satisfied, it can be seen that the detected object is in a delicate distance measuring state existing near the left and right ends of the radar viewing angle.The
[0045]
  In step 303, if the lateral position of the detected object read in step 201 and its past value satisfy the following equation (20), the process proceeds to step 304. If not, the process proceeds to step 306.
abs (Px_z₀ − Px_z₁)> Th_Px (20)
Here, Th_Px is a positive threshold for the change in the lateral position, and is usually a value smaller than the size of the object that is an obstacle. That is, if Expression 20 is satisfied, it means a sudden change in the detection position, and it can be seen that one of the vehicle reflectors has changed due to going outside the radar viewing angle.The
[0046]
In step 304, the correction amount of the object lateral position used later is calculated.
cmpPx = Px_z₀ − Px_z₁ (21)
[0047]
In step 305, the correction amount of the object lateral position used later is reset.
cmpPx = 0 (22)
[0048]
Step 306 is the same as step 204 in the first embodiment, and will not be described.
[0049]
  In step 307, the same as step 205 in the first embodiment, but the region of interest is calculated from equation (23) in consideration of the correction of equations (21) and (22).
disp_obj_YA = (y_o + (focusV * CAM_h2 / Py_z₀ ))
disp_obj_YB = (y_o+ (focusV * CAM_h / Py_z₀)) ... (23)
disp_obj_XL = (x_o+ (focusH / Py_z₀* (Px_z₀-CmpPx)))-(focusH * wide / Py_z₀)
disp_obj_XR = (x_o+ (focusH / Py_z₀* (Px_z₀-CmpPx))) + (focusH * wide / Py_z₀)
As described above, the object tracking performance by image processing is improved by correcting the detection position of the radar and determining the attention area of the image based on the corrected value.The
[0050]
Steps 308 to 311 are the same as steps 206 to 209 in the first embodiment, and are therefore omitted.
[0051]
  Step 312 is the same as step 211 in the first embodiment, but the position of the radar and the image are compared as in the following equation (24) in consideration of the correction of equations (21) and (22).
if (abs (CamTate−Py_z₀) + abs (CamYoko− (Px_z₀-CmpPx))< Th_D) LR_IP_Cnt = LR_IP_Cnt + 1
else IP_State = 0, LR_IP_Cnt = 0 (24)
As described above, by comparing the corrected radar detection position with the image detection position, the radar detection position and the image detection position do not greatly deviate despite high reliability of image processing. The horizontal position of the radar detection position is held as the value before the sudden change, only the vertical position of the radar detection position is updated, and this radar detection position and the image detection position may be collated.Yes.
[0052]
Since step 313 is the same as step 212 in the first embodiment, it is omitted.
[0053]
Step 314 is the same as step 213 in the first embodiment, but corrects the lateral position at the integrated position as in the following expression (25) in consideration of the correction of expressions (21) and (22). (The vertical position is omitted because it is the same as Expression 15.) Then, the process proceeds to Step 316.
integratedPx = Px_z₀-CmpPx (25)
[0054]
Step 315 is the same as step 214 in the first embodiment, and is omitted.
[0055]
Since step 316 is the same as step 217 in the first embodiment, it is omitted.
[0056]
Next, the image processing control will be described for each situation with reference to FIGS.
(When there is a stopped vehicle ahead and it is at the left or right edge of the radar viewing angle)
When there is a stopped vehicle ahead (time t0 to t1), the process proceeds from step 301 to step 302, step 303, step 306, step 307, step 308, and step 309 to confirm the reliability of the image detection position. When the reliability of the image detection position is high, the process proceeds from step 315 to step 316, the radar detection position and the image detection position are integrated to output the integrated position, and the past value is updated. When the reliability of image detection is low, the process proceeds to step 314, and the radar detection position is output as the integrated position. Once the initial image processing is completed, the process proceeds from step 306 to step 310 to step 311 to step 312 and the image detection is confirmed by collating with the radar detection position while continuing image tracking. Outputs the position of the sensing object. At this stage, since both the left and right reflectors of the detected vehicle are detected, the lateral position is not particularly corrected even at the left and right ends of the radar viewing angle.
[0057]
(There is a stopped vehicle ahead, and the vehicle goes further out of the radar viewing angle from the left and right ends of the radar viewing angle.)
When there is a stopped vehicle ahead and the vehicle has moved further from the left and right ends of the radar viewing angle to the side farther from the own vehicle (time t1 to t2), step 301 → step 302 → step 303 → step 304 → step 306 Go to Step 310 → Step 311 → Step 312. That is, even if the detected object exists at the left and right ends of the radar viewing angle, the width of the detected object is large when the left and right reflectors are detected. However, when it gradually moves away from the host vehicle and goes out of the radar viewing angle, the width of the detected object decreases rapidly because only one (for example, the right side) reflector is recognized, and the center position of the detected object is further reduced. It changes in the direction which approaches the own vehicle side rapidly. (See FIGS. 7-9). In this case, although the error in the radar detection position is large, the deviation from the image detection position is large, so that it is determined that the reliability of the image processing is low, and the initial state is returned to the low reliability again. If the detection vehicle exists at the left and right ends of the radar viewing angle and the radar detection position changes suddenly, the lateral position change amount is corrected to correct the difference between the radar detection position and the image detection position. Even if the collation is performed, it is not recognized that the reliability of the image processing is low, and the object tracking by the image is not hindered.
[0058]
  As explained above,Reference exampleTherefore, during long-term tracking in image processing, if there is a step or error in the radar detection position in the process of grasping the difference from the radar as a judgment of the quality of the tracking state, the radar position is corrected. Do. As described above, by temporarily changing the image processing, it is possible to appropriately cope with the deterioration of detection accuracy at the left and right ends of the viewing angle of the radar by the image processing, so that the tracking performance by the image is improved. In addition, the accuracy of the obstacle determination process in the subsequent stage can be improved.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a basic configuration in a first embodiment.
FIG. 2 is a block diagram showing a control configuration of an external environment recognition device in the first embodiment.
FIG. 3 is a flowchart showing image processing control in the first embodiment.
FIG. 4 is a map showing characteristics of a function fanc1 in the first embodiment.
FIG. 5 is a map showing characteristics of a function fanc2 in the first embodiment.
FIG. 6 is a block diagram showing a control configuration of an external environment recognition device in the second embodiment.
FIG. 7 is a schematic diagram showing a distance measurement situation when a stopped vehicle is detected.
FIG. 8 is a diagram illustrating a relationship between a reflector and a vehicle center when a stopped vehicle is detected.
FIG. 9 is a diagram illustrating a relationship between a lateral position and a vehicle width when a stopped vehicle is detected.
FIG. 10 is a diagram illustrating an image processing sequence.
[Explanation of symbols]
1 Laser radar
2 Radar processing equipment
3 CCD camera
4 Image processing device
5 Outside world recognition device
6 Vehicle speed detection device
7 Steering angle detector
8 Automatic brake control device
9 Negative pressure brake booster

Claims (2)

Scanning type distance measuring means for detecting an object within the predetermined range by scanning electromagnetic waves with respect to a predetermined range around the own vehicle, and detecting a relative position between the own vehicle and the detected object;
Imaging means for imaging the situation around the vehicle;
Either an object tracking process using an image that is an imaging result of the imaging unit , or a process that performs an edge detection process using an area corresponding to the position of the object detected by the scanning distance measuring unit in the image as an attention area Image processing means for performing image processing and detecting a relative position between the vehicle and an object around the vehicle ;
Based on the output of the scanning type distance measuring means, a distance measuring state acquisition section that the object detected by the scanning type distance measuring means to grasp the ranging status is the degree situated at the end of the predetermined range,
If the distance measuring status grasping means finds that the object detected by the scanning distance measuring means is not located at the end of the predetermined range, the vehicle detected by the scanning distance measuring means The relative position between the vehicle and the object is compared with the relative position between the vehicle and the object detected by the image processing unit, and the relative position between the vehicle and the object detected by the scanning distance measuring unit When the difference between the vehicle and the object detected by the image processing means is smaller than a predetermined value, the processing of the image processing means is changed to the object tracking process,
When the relative position between the vehicle and the object detected by the scanning distance measuring unit and the difference between the vehicle and the object detected by the image processing unit are equal to or greater than a predetermined value, the processing of the image processing unit is performed. Change to the edge detection process,
When the distance measuring state grasping means grasps that the object detected by the scanning distance measuring means is highly positioned at the end of the predetermined range, the own vehicle detected by the scanning distance measuring means An image processing changing unit that does not perform a comparison between the relative position between the object and the object and the relative position between the vehicle and the object detected by the image processing unit and does not change the processing of the image processing unit;
The vehicle environment recognizing apparatus, wherein a is provided. The vehicle external environment recognition device according to claim 1,
The distance measurement status grasping means detects an object near the left and right ends of the viewing angle in the scanning distance measuring means, and when the width of the detected object is small, the object detected by the scanning distance measurement means is within the predetermined range. An outside recognition device for vehicles characterized by grasping that the degree to which it is located in the end of is high .

Images