JP7307699B2 - object detector - Google Patents

Description

The present invention relates to an object detection device for a moving object such as a vehicle.

2. Description of the Related Art Conventionally, there is known a device that mounts a camera on a vehicle and detects an object existing around the vehicle based on a photographed image input from the camera (for example, Patent Document 1). This device identifies an image area of an object from an image input from a camera, and calculates a travel control amount of the vehicle based on the size of this image area.

Japanese Unexamined Patent Application Publication No. 2013-33312

The travel control amount includes the target vehicle speed of the vehicle, and in calculating the target vehicle speed, the distance between the vehicle and the object is calculated. That is, the distance between the moving object and the object is calculated based on the size of the object on the image. Here, the size of an object on an image is expressed using the number of pixels, which is the number of pixels corresponding to the object. In this case, the size of the object in the image has an error that depends on the size of one pixel at the maximum, and the distance between the moving object and the object that is calculated based on this size also has an error. There is concern about problems that may occur.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object thereof is to provide an object detection apparatus capable of accurately calculating a movement control amount of a moving body based on an image.

Means for solving the above problems is applied to a moving object equipped with a first camera and a second camera for capturing a first image and a second image including parallax, and An object detection device for detecting an object existing around the moving object based on the parallax obtained from the position of the object on the first image and the position of the object on the second image a compound eye distance calculator that calculates the distance between the moving object and the object as a compound eye distance; and a size of the object on the one image using one of the first image and the second image a monocular distance calculation unit that calculates the distance between the moving body and the object as a monocular distance based on, and a distance that determines that at least one of the compound eye distance and the monocular distance is greater than a predetermined distance threshold a determination unit, and a movement control amount for controlling a movement mode of the moving object based on the monocular distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is greater than the distance threshold. and a control amount calculation unit that calculates the movement control amount based on the compound eye distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is smaller than the distance threshold; , provided.

Using one of the first image and the second image captured by the first camera and the second camera, the distance between the moving object and the object is calculated as the monocular distance based on the size of the object on the one image. In this case, the monocular distance has an error depending on the length corresponding to one pixel at the maximum, and the magnitude of the error is roughly proportional to the distance between the moving object and the object. Further, when the distance between the moving object and the object is calculated as the compound eye distance based on the parallax between the position of the object on the first image and the position of the object on the second image, the compound eye distance is and the parallax in the second image by the principle of triangulation. In this case, the compound eye distance and parallax are in an inversely proportional relationship, and the longer the compound eye distance, the smaller the parallax. Also, the smaller the parallax, the lower the calculation accuracy of the compound eye distance. Therefore, the error included in the compound eye distance rapidly increases as the parallax decreases. Therefore, if the object is far away from the moving object, the monocular distance error is smaller than the compound eye distance error, and if the object is near the moving object, the monocular distance error is equal to the compound eye distance is larger than the error of

Therefore, in the above configuration, it is determined that at least one of the compound eye distance and the monocular distance is greater than the distance threshold, and if it is determined to be greater, the movement control amount of the moving object is calculated based on the monocular distance and determined to be small. In this case, the movement control amount of the moving object is calculated based on the compound eye distance. By calculating the movement control amount of the moving object based on the distance with the smallest error out of the compound eye distance and the monocular distance, the movement control amount of the moving object can be calculated with high accuracy.

FIG. 1 is an overall configuration diagram of an object detection system; The figure explaining the principle of triangulation. The figure explaining a width expansion rate. 4 is a flowchart showing the procedure of switching processing according to the first embodiment; The graph which shows transition of compound eye distance and monocular distance. 6 is a graph showing the relationship between compound eye distance and compound eye error and the relationship between monocular distance and monocular error according to the first embodiment; A graph showing changes in relative velocity. 10 is a flowchart showing the procedure of switching processing according to the second embodiment; 11 is a flowchart showing the procedure of switching processing according to the third embodiment; 10 is a graph showing the relationship between the compound eye distance and the compound eye error and the relationship between the monocular distance and the monocular error according to the third embodiment; 4 is a graph showing the relationship between vehicle width and distance threshold; 14 is a flowchart showing the procedure of switching processing according to the fourth embodiment; 14 is a graph showing the relationship between the compound eye distance and the compound eye error and the relationship between the monocular distance and the monocular error according to the fourth embodiment; 4 is a graph showing the relationship between reference length and distance threshold;

(First embodiment)
Embodiments embodying the present invention will be described below with reference to the drawings. This embodiment embodies an object detection system mounted on a vehicle. The system is an example of a vehicle system mounted on a vehicle, for example, and detects objects (eg, other vehicles) existing around the vehicle.

First, a schematic configuration of a vehicle object detection system according to the present embodiment will be described with reference to FIG. A vehicle 50 as a moving object includes a stereo camera 10, an ECU 20 as an object detection device, and a controlled device 30 as a control target of the ECU 20. FIG.

The stereo camera 10 is installed in the front part of the vehicle 50 with the optical axis facing the front of the vehicle 50 so that the front of the vehicle 50 can be photographed. A stereo camera 10 includes a right camera 11 and a left camera 12 . The right camera 11 and the left camera 12 are arranged at the same height in the gravitational direction and are arranged at positions apart in the left-right (horizontal) direction. In this embodiment, the horizontal direction corresponds to the "predetermined direction", and the right camera 11 and the left camera 12 respectively correspond to the "first camera and second camera".

The right camera 11 and the left camera 12 photograph the same other vehicle positioned in front of the vehicle 50 from different directions. The right camera 11 and the left camera 12 take pictures simultaneously at a predetermined cycle, and the right image taken by the right camera 11 and the left image taken by the left camera 12 are output to the ECU 20, respectively. The right camera 11 and the left camera 12 are each composed of a CCD image sensor or a CMOS image sensor, for example. In addition, in this embodiment, the right image and the left image respectively correspond to the "first image and the second image".

The ECU 20 is mainly composed of a computer including a CPU, various memories, and the like. The ECU 20 detects other vehicles positioned in front of the vehicle based on the right and left images respectively output by the right camera 11 and the left camera 12 of the stereo camera 10, and calculates the distance to the other vehicles.

A method using well-known stereo matching processing is applied to calculate the distance to other vehicles. Briefly, the ECU 20 obtains a right image and a left image photographed at the same time, and based on the parallax DP (displacement amount on the image) of the position of the other vehicle commonly present in these images, the principle of triangulation is used to calculate the distance to the other vehicle. Hereinafter, the distance to the other vehicle calculated based on the parallax DP will be referred to as compound eye distance LP.

Further, the distance to other vehicles can also be calculated by the following method. Specifically, the ECU 20 acquires an image width WG, which is the size in the left-right direction of the other vehicle on one of the right images of the right image and the left image, and obtains the image width WG and the width of the other vehicle in the left-right direction. The distance to the other vehicle is calculated based on the ratio to the vehicle width WR, which is the dimension. Hereinafter, the distance to the other vehicle calculated based on the image width WG of the other vehicle on the image is referred to as a monocular distance LS. The image used for calculating the monocular distance LS may be the left image.

The ECU 20 also calculates the relative speed VR of the other vehicle with respect to the vehicle 50 based on the calculated compound eye distance LP or monocular distance LS, and performs follow-up control processing such as ACC (Adaptive Cruise Control) for the vehicle. Specifically, the accelerator device 31 and the brake device 32, which are the devices to be controlled 30, are operated at predetermined operation timings. The accelerator device 31 applies driving force to the vehicle 50 according to a driver's accelerator operation or a control command from the ECU 20 . The brake device 32 also includes a brake mechanism that changes the braking force of the vehicle 50 and a brake ECU that controls the operation of this brake mechanism. The brake ECU is communicably connected to the ECU 20 and controls the brake mechanism according to control commands from the ECU 20 .

By the way, the image width WG used for calculating the monocular distance LS is expressed using the number of pixels, which is the number of pixels corresponding to other vehicles. In this case, the monocular distance LS has an error depending on the length corresponding to one pixel at maximum, and the magnitude of the error is generally proportional to the distance between the vehicle 50 and the other vehicle. Specifically, the monocular distance LS is represented by the following (Equation 1) using the focal length LF of the right camera 11 and the left camera 12, the vehicle width WR, and the image width WG.

LS=WR×LF/WG (Formula 1)
Here, when the type of an object on the right image or the left image is determined by known pattern matching, the vehicle width WR of the other vehicle is set to a constant value determined according to the type.

Further, in the calculation of the compound eye distance LP, the compound eye distance LP is calculated from the parallax DP between the right image and the left image according to the principle of triangulation. The principle of triangulation will be described with reference to FIG. The upper side of FIG. 2 shows the right camera 11, the left camera 12, and the other vehicle viewed from above in the gravity direction, and the lower side of FIG. Right and left images are shown respectively. As shown in FIG. 2, the compound eye distance LP includes a reference length LC that is the distance in the horizontal direction between the right camera 11 and the left camera 12, the focal length LF of the right camera 11 and the left camera 12, the right image and the left image. Using the parallax DP in the image, it is represented by the following (Equation 2).

LP=LC×LF/DP (Formula 2)
According to (Equation 2), the compound eye distance LP and the parallax DP are in an inversely proportional relationship, and the longer the compound eye distance LP, the smaller the parallax DP. Also, the smaller the parallax DP, the lower the calculation accuracy of the compound eye distance LP. Therefore, the error included in the compound eye distance LP rapidly increases as the parallax DP decreases. Therefore, if the other vehicle is far away from the vehicle 50, the error in the monocular distance LS is smaller than the error in the compound eye distance LP, and if the other vehicle is close to the vehicle 50, the monocular distance LS is larger than the error of the compound eye distance LP.

Therefore, in the present embodiment, it is determined that the monocular distance LS is greater than the distance threshold Lth. The relative velocity VR is calculated based on the distance LP. In other words, switching processing for switching the distance used for calculating the relative velocity VR is performed depending on whether or not the monocular distance LS is greater than the distance threshold Lth. In the present embodiment, the relative velocity VR corresponds to "movement control amount for controlling the movement mode of the moving body".

A specific configuration of the ECU 20 in this embodiment will be described with reference to FIG. As shown in FIG. 1, the ECU 20 includes an image acquisition unit 21, an area identification unit 22, a monocular distance calculation unit 23, a compound eye distance calculation unit 24, a width expansion ratio calculation unit 25, a distance determination unit 26, and a relative speed calculation unit 27. Function.

The image acquisition unit 21 acquires a right image and a left image captured by the right camera 11 and the left camera 12, respectively. The right image and the left image are images captured at the same time, and the image acquisition unit 21 acquires these paired images at a predetermined cycle. When there is another vehicle common to the right image and the left image acquired by the image acquisition unit 21, the area specifying unit 22 uses one of the right image and the left image, and determines whether the other vehicle is located on the one image. An image area XA, which is an existing area, is specified. The size (the number of pixels) of the image area XA specified by the area specifying unit 22 in the horizontal direction is the image width WG.

The monocular distance calculation unit 23 calculates the monocular distance LS using the image width WG identified by the region identification unit 22, and the compound eye distance calculation unit 24 uses the right image and the left image obtained by the image obtaining unit 21. to calculate the compound eye distance LP. The width enlargement ratio calculation unit 25 calculates a width enlargement ratio RW, which is the ratio between the image width WG specified by the area specifying unit 22 and the reference width WK. Here, the reference width WK is, for example, an image width on the right image or the left image acquired at a predetermined reference timing. A memory in the ECU 20 stores a reference width WK for the right image and a reference width WK for the left image. A width enlargement ratio RW is calculated using the image reference width WK. Further, when the image area XA of the left image is specified by the area specifying unit 22, the width enlargement ratio RW is calculated using the reference width WK for the left image.

The width enlargement ratio RW will be described with reference to FIG. For example, when the other vehicle is faster than the vehicle 50 and the relative speed VR of the other vehicle with respect to the vehicle 50 is positive during the period TA from the reference timing to the present, the specified image is displayed as shown in FIG. The image width WG of the area XA becomes smaller than the reference width WK shown in FIG. 2(B). In this case, the width enlargement ratio RW is greater than "1". Further, when the other vehicle is slower than the vehicle 50 and the relative speed VR of the other vehicle with respect to the vehicle 50 is negative during the period TA from the reference timing to the present time, as shown in FIG. The width WG becomes larger than the reference width WK. In this case, the width enlargement ratio RW is smaller than "1".

The distance determination unit 26 determines that the monocular distance LS calculated by the monocular distance calculation unit 23 is greater than the distance threshold Lth. Here, the distance threshold Lth is a distance at which the error of the compound eye distance LP and the error of the monocular distance LS are substantially equal, and is a value determined by the vehicle width WR and the reference length LC of the stereo camera 10 . The error in the compound eye distance LP increases as the distance between the vehicle 50 and the other vehicle increases. On the other hand, the error of the monocular distance LS is substantially constant regardless of the distance between the vehicle 50 and the other vehicle. Therefore, when the distance determination unit 26 determines that the monocular distance LS is greater than the distance threshold Lth, the error in the compound eye distance LP is greater than the error in the monocular distance LS. Also, when the distance determination unit 26 determines that the monocular distance LS is smaller than the distance threshold Lth, the error in the compound eye distance LP is smaller than the error in the monocular distance LS.

The relative velocity calculator 27 calculates the relative velocity VR based on the compound eye distance LP or the monocular distance LS. The relative speed VR is expressed by the following (Equation 3) using a reference distance LK which is the distance between the vehicle 50 and another vehicle at the reference timing, a period TA from the reference timing to the present, and a width expansion rate RW. expressed.

VR=LK×(1−RW)/(TA×RW) (Formula 3)
Then, the relative velocity calculator 27 switches the reference distance LK used for calculating the relative velocity VR between the compound eye distance LP and the monocular distance LS.

Next, the procedure of switching processing in this embodiment will be described with reference to the flowchart of FIG. This process is repeatedly performed by the ECU 20 at predetermined intervals.

First, in step S10, the right image captured by the right camera 11 is acquired. In step S12, the left image captured by the left camera 12 is obtained. The right image and left image acquired in steps S10 and S12 are images shot simultaneously by the right camera 11 and the left camera 12 .

In step S14, it is determined whether or not the same other vehicle exists in the right image and left image acquired in steps S10 and S12. If a negative determination is made in step S14, the correction process ends. On the other hand, if an affirmative determination is made in step S14, the image area XA is specified in step S16.

In step S16, one of the right image and the left image acquired in steps S10 and S12 is selected to be used for specifying the image area XA. Specifically, it is determined whether the other vehicle exists on the right camera 11 side or the left camera 12 side with respect to the center between the right camera 11 and the left camera 12 in the horizontal direction. . That is, it is determined whether the camera exists on the right camera 11 side or the left camera 12 side of the perpendicular bisector of the line connecting the right camera 11 and the left camera 12 . Which side of the right camera 11 or the left camera 12 the other vehicle is on can be determined, for example, from the positions of the other vehicle on the right image and the left image in the horizontal direction.

In step S16, when it is determined that another vehicle exists on the right camera 11 side, the image area XA of the right image is specified. Further, when it is determined that another vehicle exists on the left camera 12 side, the image area XA of the left image is specified.

In step S18, the monocular distance LS is calculated based on the image width WG of the image area XA identified in step S16. Therefore, in step S16, the image used for calculating the monocular distance LS is switched between the right image and the left image based on the determination result as to which side of the right camera 11 or left camera 12 the other vehicle is present. In subsequent step S20, it is determined whether or not the monocular distance LS calculated in step S18 is greater than the distance threshold Lth. In this embodiment, the process of step S18 corresponds to the "monocular distance calculator", and the process of step S20 corresponds to the "distance determination unit".

If it is determined that the monocular distance LS is greater than the distance threshold Lth, an affirmative determination is made in step S20. In this case, the width enlargement ratio RW is calculated in step S22. In subsequent step S24, the relative velocity VR is calculated based on the monocular distance LS calculated in step S18 and the width enlargement ratio RW calculated in step S22. That is, when it is determined that the monocular distance LS is greater than the distance threshold Lth, the relative velocity VR is calculated based on the monocular distance LS out of the compound eye distance LP and the monocular distance LS.

In step S22, the image width WG specified in the previous switching process is used as the reference width WK. That is, in this embodiment, the reference timing is the timing at which the previous switching process was performed. According to (Formula 3), in step S24, the relative velocity VR is calculated based on the monocular distance LS calculated in the previous switching process. When the switching process is repeatedly performed at predetermined intervals, the monocular distance LS calculated in the previous switching process and the monocular distance LS calculated in the current switching process are substantially the same. Therefore, in step S24, the relative velocity VR is calculated based on the monocular distance LS calculated in the current switching process. The same applies to steps S28 and S30.

If it is determined that the monocular distance LS is smaller than the distance threshold Lth, a negative determination is made in step S20. In this case, in step S26, the compound eye distance LP is calculated using the right and left images obtained in steps S10 and S12. In subsequent step S22, the width enlargement ratio RW is calculated. It should be noted that in the present embodiment, the process of step S26 corresponds to the "compound eye distance calculator".

In step S30, the relative velocity VR is calculated based on the compound eye distance LP calculated in step S26 and the width enlargement ratio RW calculated in step S28. That is, when it is determined that the monocular distance LS is smaller than the distance threshold Lth, the relative velocity VR is calculated based on the compound eye distance LP out of the compound eye distance LP and the monocular distance LS. In this embodiment, the processing of steps S24 and S30 corresponds to the "control amount calculator".

5 to 7 show an example of switching processing. FIG. 5 shows changes in the compound eye distance LP and the monocular distance LS when the vehicle 50 moves away from another vehicle at a constant speed. As shown in FIG. 5, when another vehicle is present near the vehicle 50, the compound eye error ΔLP, which is the difference between the actual distance LT (broken line) to the other vehicle and the compound eye distance LP (solid line), is small. The error becomes smaller as the vehicle 50 moves away from the other vehicle. In the stereo camera 10, depending on the specifications of the stereo camera 10, there are cases where the compound eye distance LP deviates to the larger side from the actual distance LT, and cases where the compound eye distance LP deviates to the smaller side, and FIG. It is Below, the compound eye error ΔLP of the compound eye distance LP that deviates to the larger side with respect to the actual distance LT is referred to as the compound eye upper error ΔLPU, and the compound eye error ΔLP of the compound eye distance LP that deviates to the smaller side from the actual distance LT is referred to as the compound eye lower error. It is called ΔLPD.

Also, the monocular distance LS (chain line) is calculated based on the image width WG of the other vehicle on the right image or the left image. The image width WG is expressed using the number of pixels, which is the number of pixels corresponding to other vehicles, and increases or decreases in units of pixels. Specifically, when the monocular distance LS increases, it decreases by one pixel regardless of the distance between the vehicle 50 and the other vehicle. increases by one pixel regardless of the distance between Therefore, the monocular distance LS varies as the number of pixels increases and decreases, and the maximum value and the minimum value are repeatedly generated. In the following, the difference between the variable monocular distance LS and the actual distance LT will be referred to as the monocular upper error ΔLSU, and the difference between the minimum values and the actual distance LT will be referred to as the monocular lower error. The sum of the upper monocular error ΔLSU and the lower monocular error ΔLSD is called the monocular error ΔLS.

FIG. 6 shows the relationship between compound eye distance LP and compound eye error ΔLP and the relationship between monocular distance LS and monocular error ΔLS corresponding to FIG. As shown in FIG. 6, when another vehicle exists near the vehicle 50, the compound eye error .DELTA.LP is smaller than the monocular error .DELTA.LS. On the other hand, when another vehicle exists far from the vehicle 50, the compound eye error ΔLP is larger than the monocular error ΔLS.

Therefore, in the present embodiment, the distance at which the compound eye error ΔLP and the monocular error ΔLS are equal is set as the distance threshold Lth. Specifically, in the stereo camera 10 in which the compound eye distance LP deviates to the larger side than the actual distance LT, the distance at which the compound eye upper error ΔLPU and the monocular upper error ΔLSU are equal is set as the distance threshold Lth. Further, in the stereo camera 10 in which the compound eye distance LP shifts to the smaller side with respect to the actual distance LT, the distance at which the compound eye lower error ΔLPD and the monocular lower error ΔLSD are equal is set as the distance threshold Lth. Then, when it is determined that the monocular distance LS is greater than the distance threshold Lth, the relative velocity VR is calculated based on the monocular distance LS. Also, when it is determined that the monocular distance LS is smaller than the distance threshold Lth, the relative velocity VR is calculated based on the compound eye distance LP. That is, the distance used for calculating the relative velocity VR is switched between the compound eye distance LP and the monocular distance LS depending on the distance between the vehicle 50 and the other vehicle.

FIG. 7 shows changes in the relative speed VR when the vehicle 50 moves away from another vehicle at a constant speed. FIG. 7 shows the amount of speed deviation between the actual relative speed VR and the target relative speed Vtg when a constant target relative speed Vtg is set. As shown by the dashed line in FIG. 7, when the compound eye distance LP is used as the reference distance LK regardless of the distance between the vehicle 50 and the other vehicle, the compound eye error ΔLP increases due to the increase in the compound eye distance LP. , the amount of speed deviation increases. In this embodiment, the distance used for calculating the relative velocity VR is switched between the compound eye distance LP and the monocular distance LS depending on the distance between the vehicle 50 and the other vehicle. Therefore, as indicated by the solid line in FIG. 7, an increase in the amount of speed deviation can be suppressed.

According to this embodiment described above, the following effects are obtained.

Using one of the right image and the left image captured by the right camera 11 and the left camera 12, the distance between the vehicle 50 and the other vehicle is calculated based on the image width WG, which is the size of the other vehicle on the one image. When the distance is calculated as the monocular distance LS, the monocular distance LS has an error that depends on the length corresponding to one pixel at the maximum, and the magnitude of the error generally depends on the distance between the vehicle 50 and the other vehicle. be proportional. Further, when the distance between the vehicle 50 and the other vehicle is calculated as the compound eye distance LP based on the parallax DP obtained from the position of the other vehicle on the right image and the position of the other vehicle on the left image, the compound eye distance LP is calculated by the principle of triangulation from the parallax DP in the right and left images. In this case, the compound eye distance LP and the parallax DP are in an inversely proportional relationship, and the longer the compound eye distance LP, the smaller the parallax DP. Also, the smaller the parallax DP, the lower the calculation accuracy of the compound eye distance LP. Therefore, the error included in the compound eye distance LP rapidly increases as the parallax DP decreases. Therefore, if the other vehicle is far away from the vehicle 50, the error in the monocular distance LS is smaller than the error in the compound eye distance LP, and if the other vehicle is close to the vehicle 50, the monocular distance LS is larger than the error of the compound eye distance LP.

Therefore, in the present embodiment, it is determined that the monocular distance LS is greater than the distance threshold Lth. The relative velocity VR is calculated based on the distance LP. By calculating the relative velocity VR based on the distance with the smallest error among the compound eye distance LP and the monocular distance LS, the relative velocity VR can be calculated with high accuracy.

When the other vehicle exists on the right camera 11 side with respect to the center between the right camera 11 and the left camera 12 in the horizontal direction, the deviation width from the optical axis of the right camera 11 to the other vehicle is the left camera It is smaller than the deviation width from the optical axis of No. 12 to the other vehicle. Therefore, by calculating the monocular distance LS using the right image captured by the right camera 11, the monocular distance LS can be calculated with high accuracy, and errors in the compound eye distance LP can be suppressed. Further, when another vehicle exists on the left camera 12 side with respect to the center between the right camera 11 and the left camera 12 in the horizontal direction, the deviation width from the optical axis of the left camera 12 to the other vehicle is the right camera It is smaller than the deviation width of other vehicles from the optical axis of 11. Therefore, by calculating the monocular distance LS using the right image captured by the left camera 12, the monocular distance LS can be calculated with high accuracy, and errors in the compound eye distance LP can be suppressed.

In this embodiment, when switching the distance used for calculating the relative velocity VR, the monocular distance LS is compared with the distance threshold value Lth among the compound eye distance LP and the monocular distance LS. Although the monocular distance LS includes variations due to changes in the number of pixels, the error is proportional to the distance between the vehicle 50 and the other vehicle, and does not increase rapidly. Therefore, by comparing the monocular distance LS with the distance threshold Lth, it is possible to properly determine whether or not the distance between the vehicle 50 and the other vehicle is greater than the distance threshold Lth.

(Second embodiment)
The second embodiment will be described below with reference to FIG. 8, focusing on differences from the first embodiment. This embodiment differs from the first embodiment in that the distance threshold Lth to be compared with the monocular distance LS is switched depending on whether the distance between the vehicle 50 and the other vehicle is increasing or decreasing. .

FIG. 8 shows a flowchart of the switching process of this embodiment. In addition, in FIG. 8, for convenience, the same step numbers are assigned to the same processes as those shown in FIG. 4, and the description thereof is omitted.

In the switching process of the present embodiment, once the monocular distance LS is calculated in step S18, it is determined in step S40 whether or not the monocular distance LS has increased. The monocular distance LS calculated in step S<b>18 is stored in the memory within the ECU 20 . In step S40, it is determined whether or not the monocular distance LS is increasing based on the tendency of change in the monocular distance LS calculated in a plurality of switching processes performed up to a predetermined period from now.

If it is determined that the monocular distance LS is increasing, that is, if the distance between the vehicle 50 and the other vehicle is increasing, an affirmative determination is made in step S40. In this case, in step S42, the distance threshold value Lth is set to the increase threshold value LU previously stored in the memory within the ECU 20, and the process proceeds to step S20. In this case, in step S20, it is determined whether or not the monocular distance LS calculated in step S18 is greater than the increase threshold LU.

On the other hand, if it is determined that the monocular distance LS is decreasing, that is, if the distance between the vehicle 50 and the other vehicle is decreasing, a negative determination is made in step S40. In this case, in step S42, the distance threshold value Lth is set to the decrease threshold value LD previously stored in the memory within the ECU 20, and the process proceeds to step S20. In this case, in step S20, it is determined whether or not the monocular distance LS calculated in step S18 is greater than the reduction threshold LD.

In this embodiment, the increase threshold LU is set to a value larger than the decrease threshold LD. If the distance threshold value Lth is set to the same value regardless of whether the distance between the vehicle 50 and the other vehicle is increasing or decreasing, the monocular distance LS may be different from the distance threshold value Lth due to chattering, for example. , the switching of the distance used for calculating the relative velocity VR is repeated. As a result, the relative velocity VR fluctuates due to the difference between the compound eye distance LP and the monocular distance LS, which affects the calculation of the relative velocity VR.

In this regard, in the present embodiment, the increase threshold value LU used when the distance between the vehicle 50 and the other vehicle is increasing is used when the distance between the vehicle 50 and the other vehicle is decreasing. is set to a value greater than the decrease threshold LD. That is, hysteresis is provided when the distance used to calculate the relative velocity VR is switched. As a result, frequent switching of the distance used for calculating the relative velocity VR can be suppressed, and the relative velocity VR can be calculated appropriately.

(Third embodiment)
The third embodiment will be described below with reference to FIGS. 9 to 11, focusing on differences from the first embodiment. This embodiment differs from the first embodiment in that information indicating the vehicle width WR of the other vehicle is obtained separately from the image area XA. Specifically, it differs from the first embodiment in that the vehicle type classified according to the size of the vehicle is specified as the information indicating the vehicle width WR of the other vehicle. Vehicle types are, for example, passenger cars (compact and medium-sized vehicles) and trucks (large-sized vehicles). This embodiment differs from the first embodiment in that the vehicle width WR is acquired based on the specified vehicle type. Therefore, the acquired vehicle width WR differs from a constant value determined for the vehicle (hereinafter simply a constant value).

FIG. 9 shows a flowchart of the switching process of this embodiment. In FIG. 9, the same steps as those shown in FIG. 4 are denoted by the same step numbers for convenience, and description thereof will be omitted.

In the switching process of the present embodiment, once the image area XA is specified in step S16, the vehicle type of the other vehicle existing in the image area XA is specified in step S50. Specifically, the vehicle type of the other vehicle is specified from the extracted feature points, such as the width in the left-right direction of the tail lamps present at the rear of the other vehicle. A memory in the ECU 20 stores correspondence information in which vehicle types and vehicle widths WR are associated with each other, and the vehicle width WR corresponding to the identified vehicle type is obtained. In this embodiment, the process of step S50 corresponds to the "dimension acquisition unit", and the vehicle type corresponds to "dimension information".

At step S52, a distance threshold Lth is set based on the vehicle width WR obtained at step S50. Specifically, the larger the vehicle width WR acquired in step S50, the smaller the distance threshold Lth is set. Therefore, for example, the distance threshold Lth for trucks is set to a smaller value than the distance threshold Lth for passenger cars.

FIG. 10 shows the relationship between compound eye distance LP and compound eye error ΔLP and the relationship between monocular distance LS and monocular error ΔLS in this embodiment. In FIG. 10, the other vehicle is a truck, and the vehicle width WR of the other vehicle is larger than the constant value set as the average value of the vehicle width of passenger cars and the vehicle width of trucks. As the vehicle width WR increases, the effect of the image width WG represented by the number of pixels becomes relatively small, so the monocular error ΔLS becomes small. Therefore, as shown in FIG. 10, the monocular error ΔLSA (solid line) corresponding to the vehicle width WR is smaller than the monocular error ΔLSB (broken line) corresponding to a constant value, and the compound eye error ΔLP and the monocular error ΔLS are equal. distance becomes shorter.

Therefore, in the present embodiment, the distance threshold Lth is set so that the larger the vehicle width WR, the smaller the distance threshold Lth. Specifically, as shown in FIG. 11, the distance threshold LthA corresponding to the monocular error ΔLSA is set to a value smaller than the distance threshold LthB corresponding to the monocular error ΔLSB. As a result, the magnitude relationship between the compound eye error ΔLP and the monocular error ΔLS can be appropriately determined using the distance threshold Lth set according to the monocular error ΔLS, and the relative velocity VR can be calculated based on the distance with the small error. It can be calculated with high accuracy.

(Fourth embodiment)
The fourth embodiment will be described below with reference to FIGS. 12 to 14, focusing on differences from the first embodiment. This embodiment differs from the first embodiment in that the reference length LC between the right camera 11 and the left camera 12 is changeable. Specifically, the stereo camera 10 has three or more cameras arranged in a row in the horizontal direction, and two cameras are selected from the three or more cameras as the right camera 11 and the left camera 12. do. Therefore, if the selected cameras are different, the reference length LC between the right camera 11 and the left camera 12 is also different. This embodiment differs from the first embodiment in that information about the selected camera is acquired.

FIG. 12 shows a flowchart of the switching process of this embodiment. In addition, in FIG. 12, for convenience, the same step numbers are assigned to the same processes as those shown in FIG. 4, and the description thereof is omitted.

In the switching process of this embodiment, first, in step S60, a selection result indicating which camera has been selected is obtained. The memory in the ECU 20 stores distance information indicating the distance between the cameras in the horizontal direction, and acquires the reference length LC according to the acquired selection result. In the present embodiment, the process of step S60 corresponds to the "reference length obtaining section", and the selection result corresponds to "reference length information".

At step S62, a distance threshold Lth is set based on the reference length LC obtained at step S60. Specifically, the longer the reference length LC acquired in step S60, the larger the distance threshold Lth is set.

FIG. 13 shows the relationship between the compound eye distance LP and the compound eye error ΔLP and the relationship between the monocular distance LS and the monocular error ΔLS in this embodiment. In FIG. 13, the reference length LC of the compound eye error ΔLPC indicated by the solid line is longer than the reference length LC of the compound eye error ΔLPB indicated by the dashed line. As the reference length LC increases, the influence of the decrease in the detection accuracy of the parallax DP becomes relatively small, so the compound eye error ΔLP becomes smaller. Therefore, as shown in FIG. 13, the compound eye error ΔLPC indicated by the solid line is smaller than the compound eye error ΔLPB indicated by the dashed line, and the distance at which the compound eye error ΔLP equals the monocular error ΔLS increases.

Therefore, in the present embodiment, the distance threshold Lth is set such that the longer the reference length LC, the larger the distance threshold Lth. Specifically, as shown in FIG. 13, the distance threshold LthC corresponding to the compound eye error ΔLPC is set to a value larger than the distance threshold LthB corresponding to the compound eye error ΔLPB. As a result, the magnitude relationship between the compound eye error ΔLP and the monocular error ΔLS can be appropriately determined using the distance threshold Lth set according to the compound eye error ΔLP, and the relative velocity VR can be calculated based on the distance with the small error. It can be calculated with high accuracy.

(Other embodiments)
The present invention is not limited to the description of the above embodiments, and may be implemented as follows.

- The moving object is not limited to the vehicle 50, and may be, for example, a drone.

The distance used for comparison with the distance threshold Lth is not limited to the monocular distance LS, and may be, for example, the compound eye distance LP. That is, it may be determined that at least one of the compound eye distance LP and the monocular distance LS is greater than the distance threshold Lth.

Alternatively, the compound eye distance LP and the monocular distance LS may be switched. For example, when the distance between the vehicle 50 and the other vehicle is increasing, the current distance between the vehicle 50 and the other vehicle is often shorter than the distance threshold Lth, so the error is relatively small. The compound eye distance LP and the distance threshold Lth may be compared. Further, when the distance between the vehicle 50 and the other vehicle is decreasing, the current distance between the vehicle 50 and the other vehicle is often longer than the distance threshold Lth, so the error is relatively small. The monocular distance LS and the distance threshold Lth may be compared.

The movement control amount for controlling the movement mode of the moving body is not limited to the relative velocity VR, but may be a control amount included in a control command to the accelerator device 31, a control amount of the brake mechanism, acceleration or deceleration of the vehicle 50, or the like. good too.

- In 2nd Embodiment, the method of determining the increase or decrease of the distance between the vehicle 50 and another vehicle is not restricted to the method using the monocular distance LS. For example, compound eye distance LP may be used. Furthermore, based on information such as the speed of the vehicle 50 and the amount of accelerator operation, an increase or decrease in the distance between the vehicle 50 and the other vehicle may be determined.

- Although the example which specifies the vehicle type which is dimension information from the right image or the left image was shown in 3rd Embodiment, it is not restricted to this. For example, the vehicle type of the other vehicle may be obtained through inter-vehicle communication with the other vehicle.

- In the fourth embodiment, an example in which three or more cameras are arranged in a row in the left-right direction has been described as a configuration in which the reference length can be changed, but the configuration is not limited to this. For example, the stereo camera 10 may have two cameras, at least one of which may be configured to be slidable in the horizontal direction. In this case, the slide amount of the camera may be acquired as the reference length information.

- The controller and method described in the present disclosure can be performed by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program; may be implemented. Alternatively, the controller and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring the processor with one or more dedicated hardware logic circuits. Alternatively, the control apparatus and techniques described in this disclosure can be implemented by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. It may also be implemented by one or more dedicated computers configured. The computer program may also be stored as computer-executable instructions on a computer-readable non-transitional tangible recording medium.

11... Right camera, 12... Left camera, 20... ECU, 50... Vehicle.

Claims (7)

Applied to a moving body (50) equipped with a first camera (11) and a second camera (12) for capturing a first image and a second image including parallax, based on the first image and the second image an object detection device (20) for detecting an object existing around the moving object,
Compound eye distance calculation for calculating a distance between the moving body and the object as a compound eye distance based on parallax obtained from the position of the object on the first image and the position of the object on the second image. Department and
A monocular distance calculation unit that uses one of the first image and the second image and calculates a distance between the moving body and the object as a monocular distance based on the size of the object on the one image. and,
a distance determination unit that determines that at least one of the compound eye distance and the monocular distance is greater than a predetermined distance threshold;
calculating a movement control amount for controlling a movement mode of the moving body based on the monocular distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is greater than the distance threshold; a control amount calculation unit that calculates the movement control amount based on the compound eye distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is smaller than the distance threshold,
The first camera and the second camera are arranged side by side in a predetermined direction,
a dimension acquisition unit that acquires dimension information indicating a dimension of the object in the predetermined direction;
The object detection device, wherein the distance determination unit sets the distance threshold smaller as the size of the object indicated by the size information is larger. The distance determination unit determines whether the distance between the moving body and the object increases or decreases,
an increase threshold that is the distance threshold used when the distance between the moving body and the object is increasing;
a decrease threshold that is the distance threshold used when the distance between the moving body and the object is decreasing;
The object detection device according to claim 1 , wherein the increase threshold is set to a value larger than the decrease threshold. Applied to a moving body (50) equipped with a first camera (11) and a second camera (12) for capturing a first image and a second image including parallax, based on the first image and the second image an object detection device (20) for detecting an object existing around the moving object,
Compound eye distance calculation for calculating a distance between the moving body and the object as a compound eye distance based on parallax obtained from the position of the object on the first image and the position of the object on the second image. Department and
A monocular distance calculation unit that uses one of the first image and the second image and calculates a distance between the moving body and the object as a monocular distance based on the size of the object on the one image. and,
a distance determination unit that determines that at least one of the compound eye distance and the monocular distance is greater than a predetermined distance threshold;
calculating a movement control amount for controlling a movement mode of the moving body based on the monocular distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is greater than the distance threshold; a control amount calculation unit that calculates the movement control amount based on the compound eye distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is smaller than the distance threshold,
The distance determination unit determines whether the distance between the moving body and the object increases or decreases,
an increase threshold that is the distance threshold used when the distance between the moving body and the object is increasing;
a decrease threshold that is the distance threshold used when the distance between the moving body and the object is decreasing;
The object detection device, wherein the increase threshold is set to a value larger than the decrease threshold. A reference length, which is the distance between the first camera and the second camera, can be changed,
a reference length acquisition unit that acquires reference length information indicating the reference length;
The object detection device according to any one of claims 1 to 3 , wherein the distance determination unit sets the distance threshold larger as the reference length indicated by the reference length information is longer. Applied to a moving body (50) equipped with a first camera (11) and a second camera (12) for capturing a first image and a second image including parallax, based on the first image and the second image an object detection device (20) for detecting an object existing around the moving object,
Compound eye distance calculation for calculating a distance between the moving body and the object as a compound eye distance based on parallax obtained from the position of the object on the first image and the position of the object on the second image. Department and
A monocular distance calculation unit that uses one of the first image and the second image and calculates a distance between the moving body and the object as a monocular distance based on the size of the object on the one image. and,
a distance determination unit that determines that at least one of the compound eye distance and the monocular distance is greater than a predetermined distance threshold;
calculating a movement control amount for controlling a movement mode of the moving body based on the monocular distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is greater than the distance threshold; a control amount calculation unit that calculates the movement control amount based on the compound eye distance out of the compound eye distance and the monocular distance when the distance determination unit determines that the distance is smaller than the distance threshold,
A reference length, which is the distance between the first camera and the second camera, can be changed,
a reference length acquisition unit that acquires reference length information indicating the reference length;
The object detection device, wherein the distance determination unit sets the distance threshold larger as the reference length indicated by the reference length information is longer. The first camera and the second camera are arranged side by side in a predetermined direction,
The monocular distance calculator,
determining whether the object exists on the first camera side or the second camera side with respect to the center between the first camera and the second camera in the predetermined direction; ,
when determining that the object exists on the first camera side, calculating the monocular distance using the first image;
The object detection device according to any one of claims 1 to 5 , wherein the monocular distance is calculated using the second image when it is determined that the object exists on the second camera side. The object detection device according to any one of claims 1 to 6 , wherein the movement control amount is a relative speed of the moving body with respect to the object.

Images