JP7491024B2 - Object detection device and program - Google Patents

Description

The present invention relates to an object detection device and a program for detecting an object in a moving body such as a vehicle.

Conventionally, there is known a device that mounts a stereo camera on a vehicle and detects objects around the vehicle based on a pair of captured images (for example, see Patent Document 1). This device calculates the distance between the vehicle and the object based on the parallax between a first image and a second image captured by the two cameras that make up the stereo camera.

Japanese Patent No. 4814669

In the above distance calculation, the distance between the vehicle and the object is calculated from the parallax in the first image and the second image using the principle of triangulation. In this case, the distance and the parallax are inversely proportional to each other, and the longer the distance, the smaller the parallax. Furthermore, the smaller the parallax, the lower the accuracy of the distance calculation, and there is a concern that the error contained in this distance will rapidly increase as the error becomes smaller.

The present invention has been made in consideration of the above-mentioned circumstances, and its purpose is to provide an object detection device and a program that can accurately calculate the distance between a moving body and an object based on the disparity between a first image and a second image.

The means for solving the above problem is an object detection device that is applied to a moving body equipped with a first camera and a second camera that capture a first image and a second image including parallax, and detects an object present around the moving body based on the first image and the second image, and includes a compound eye distance calculation unit that calculates the distance between the moving body and the object as a compound eye distance based on the parallax obtained from the position of the object on the first image and the position of the object on the second image, a monocular distance calculation unit that uses one of the first image and the second image to calculate the distance between the moving body and the object as a monocular distance based on the size of the object on the one image, and a distance correction unit that obtains a difference value between the compound eye distance and the monocular distance multiple times and corrects the compound eye distance based on the intermediate value of the obtained multiple difference values.

When the distance between a moving body and an object is calculated as the compound eye distance based on the parallax obtained from the position of the object on the first image captured by the first camera and the position of the object on the second image captured by the second camera, if the object is located far away from the moving body, the error in the compound eye distance increases rapidly and the calculation accuracy decreases.

In this regard, in the above configuration, one of the first image and the second image is used, and the distance between the moving body and the object is calculated as the monocular distance based on the size of the object in that one image. Here, the size of the object in the image is expressed using the number of pixels, which is the number of pixels corresponding to the object. In this case, an error occurs in the monocular distance that depends on the length of up to one pixel, and the magnitude of the error is roughly proportional to the distance between the moving body and the object. Therefore, for example, when the object is located far away from the moving body, the error in the monocular distance is smaller than the error in the compound eye distance. Therefore, the error in the compound eye distance can be suppressed by correcting the compound eye distance based on the difference value between the compound eye distance and the monocular distance.

In this case, in the above configuration, the difference value between the compound eye distance and the monocular distance is acquired multiple times, and the compound eye distance is corrected based on the intermediate value of the acquired multiple difference values. In the compound eye distance, the direction of deviation from the actual distance between the moving body and the object is constant, whereas in the monocular distance, the direction of deviation from the actual distance varies due to fluctuations associated with an increase or decrease in the number of pixels. Therefore, by correcting the compound eye distance based on the intermediate value of multiple difference values, the influence of fluctuations in the direction of deviation of the monocular distance can be suppressed, and the distance between the moving body and the object can be calculated with high accuracy based on the parallax in the first image and the second image.

1 is an overall configuration diagram of an object detection system according to a first embodiment. FIG. 1 is a diagram explaining the principle of triangulation. 5 is a flowchart showing a procedure of a correction process according to the first embodiment. Graph showing the change in compound eye distance. 13 is a graph showing the change in monocular distance. 11 is a graph showing a change in distance difference. 6 is a graph showing a transition of an average parallax shift amount according to the first embodiment. FIG. 11 is an overall configuration diagram of an object detection system according to a second embodiment. 10 is a flowchart showing a procedure of a correction process according to a second embodiment. 13 is a graph showing changes in compound eye distance and monocular distance. 11 is a graph showing a change in average parallax shift amount in a comparative example. 13 is a graph showing a transition of an average parallax shift amount according to the second embodiment.

First Embodiment
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment is an object detection system mounted on a vehicle. The system is an example of a vehicle system mounted on a vehicle, and detects objects (e.g., other vehicles, road structures, etc.) present around the vehicle.

First, the schematic configuration of the vehicle object detection system according to this embodiment will be described with reference to FIG. 1. A vehicle 50 as a moving body includes a stereo camera 10, an ECU 20 as an object detection device, and a controlled device 30 as a control target of the ECU 20.

The stereo camera 10 is installed at the front of the vehicle 50 with its optical axis facing forward of the vehicle 50 so that it can capture images of the area in front of the vehicle 50. The stereo camera 10 is equipped with a right camera 11 and a left camera 12. The right camera 11 and the left camera 12 are disposed at the same height in the direction of gravity and are disposed at positions separated in the left-right (horizontal) direction. In this embodiment, the left-right direction corresponds to the "predetermined direction", and the right camera 11 and the left camera 12 correspond to the "first camera and the second camera", respectively.

The right camera 11 and the left camera 12 capture images of the same object located in front of the vehicle 50 from different directions. The right camera 11 and the left camera 12 capture images simultaneously at a predetermined cycle, and the right image captured by the right camera 11 and the left image captured by the left camera 12 are each output to the ECU 20. The right camera 11 and the left camera 12 are each configured, for example, with a CCD image sensor or a CMOS image sensor. In this embodiment, the right image and the left image correspond to the "first image and the second image", respectively.

The ECU 20 is mainly composed of a computer equipped with a CPU, various memories, etc. The ECU 20 detects an object located in front of the vehicle based on the right image and the left image output by the right camera 11 and the left camera 12 of the stereo camera 10, respectively, and calculates the distance to the object.

A method using well-known stereo matching processing is applied to calculate the distance to an object. Simply put, the ECU 20 acquires right and left images captured simultaneously, and calculates the distance to the object using the principles of triangulation based on the parallax DP (amount of shift on the image) of the position of an object that is commonly present in these images. Hereinafter, the distance to the object calculated based on the parallax DP is referred to as the compound eye distance LP.

The ECU 20 also performs contact avoidance processing for the object based on the calculated compound eye distance LP. Specifically, it activates the controlled devices 30, the warning device 31 and the brake device 32, at a predetermined activation timing. The warning device 31 warns the driver of the presence of an object in front of the vehicle in response to a control command from the ECU 20. The warning device 31 is composed of, for example, a speaker installed in the vehicle cabin and a display unit that displays images. The brake device 32 also includes a brake mechanism that changes the braking force of the vehicle 50, and a brake ECU that controls the operation of the brake mechanism. The brake ECU is connected to the ECU 20 so as to be able to communicate with it, and controls the brake mechanism in response to a control command from the ECU 20.

The compound eye distance LP is calculated from the parallax DP in the right and left images by the principle of triangulation. The principle of triangulation will be explained with reference to FIG. 2. The upper side of FIG. 2 shows the right camera 11, the left camera 12, and the object as viewed from above in the direction of gravity, and the lower side of FIG. 2 shows the right and left images taken by the right and left cameras 11 and 12, respectively. As shown in FIG. 2, the compound eye distance LP is expressed as follows (Equation 1) using a reference length LC, which is the distance in the left-right direction between the right camera 11 and the left camera 12, the focal length LF of the right camera 11 and the left camera 12, and the parallax DP in the right and left images.

LP = LC × LF / DP (Equation 1)
According to formula 1, the compound eye distance LP and the parallax DP are inversely proportional to each other, and the longer the compound eye distance LP, the smaller the parallax DP. Also, the smaller the parallax DP, the lower the calculation accuracy of the compound eye distance LP, and the error included in the compound eye distance LP increases rapidly as the parallax DP decreases.

In this embodiment, one of the right and left images is used, and the distance between the vehicle 50 and the object is calculated as the monocular distance LS based on the size of the object on the one image. In other words, the compound eye distance LP is calculated using both the right and left images, and the monocular distance LS is calculated using one of the right and left images. Then, the parallax shift amount ΔZ between the compound eye distance LP calculated using the same right and left images and the monocular distance LS is calculated, and a correction process is performed to correct the compound eye distance LP based on the average parallax shift amount ZC, which is the average value of the parallax shift amounts ΔZ calculated multiple times. The parallax shift amount ΔZ is obtained by converting the distance difference ΔL between the compound eye distance LP and the monocular distance LS into the shift amount of parallax DP, and the calculation method of the monocular distance LS and the parallax shift amount ΔZ will be described later.

The specific configuration of the ECU 20 in this embodiment will be described with reference to FIG. 1. As shown in FIG. 1, the ECU 20 functions as an acquisition unit 21, a distance calculation unit 22, a distance determination unit 23, a correction value calculation unit 25, a distance correction unit 26, an alarm control unit 27, and a stop control unit 28.

The acquisition unit 21 acquires the right and left images captured by the right and left cameras 11 and 12, respectively. The right and left images are captured simultaneously, and the acquisition unit 21 acquires these paired images at a predetermined cycle. The distance calculation unit 22 calculates the compound eye distance LP and the monocular distance LS using the right and left images acquired by the acquisition unit 21. The distance calculation unit 22 calculates the compound eye distance LP and the monocular distance LS each time a right and left image is acquired.

The distance determination unit 23 determines that the compound eye distance LP calculated by the distance calculation unit 22 is greater than the distance threshold Lth. Here, the distance threshold Lth is the distance at which the error in the compound eye distance LP and the error in the monocular distance LS are approximately equal, and is a value determined by the size of the object and the reference length LC of the stereo camera 10. If the distance determination unit 23 determines that the compound eye distance LP is greater than the distance threshold Lth, the error in the compound eye distance LP will be greater than the error in the monocular distance LS. Also, if the distance determination unit 23 determines that the compound eye distance LP is smaller than the distance threshold Lth, the error in the compound eye distance LP will be smaller than the error in the monocular distance LS.

The correction value calculation unit 25 calculates the parallax shift amount ΔZ between the compound eye distance LP and the monocular distance LS, and calculates a correction value for correcting the compound eye distance LP based on the average parallax shift amount ZC of the calculated multiple parallax shift amounts ΔZ. In detail, the correction value calculation unit 25 calculates the parallax shift amount ΔZ between the compound eye distance LP and the monocular distance LS calculated using the same right and left images, and calculates the average parallax shift amount ZC of the multiple parallax shift amounts ΔZ calculated for multiple of these paired images as the correction value. Note that in this embodiment, the parallax shift amount ΔZ corresponds to the "difference value", and the average parallax shift amount ZC corresponds to the "intermediate value".

The distance correction unit 26 corrects the compound eye distance LP using the average parallax shift amount ZC calculated by the correction value calculation unit 25. Hereinafter, the corrected compound eye distance LP is referred to as the corrected distance LPR.

When it is determined that the corrected distance LPR calculated by the distance correction unit 26 has reached a predetermined warning distance, the warning control unit 27 transmits a control command to the warning device 31 to notify the driver that an object is approaching. When it is determined that the corrected distance LPR has reached a predetermined stopping distance, the stop control unit 28 transmits a control command to the brake device 32 to avoid contact between the vehicle 50 and the object.

Next, the procedure of the correction process in this embodiment will be described with reference to the flowchart in FIG. 3. This process is repeatedly performed by the ECU 20 at a predetermined interval under conditions in which the monocular distance LS is stable, such as when the vehicle 50 and the object are both traveling at a constant speed in a straight line, or when the object is present in front of the vehicle 50.

First, in step S10, a right image captured by the right camera 11 is acquired. In step S12, a left image captured by the left camera 12 is acquired. The right image and left image acquired in steps S10 and S12 are images captured simultaneously by the right camera 11 and the left camera 12.

In step S14, it is determined whether or not the same object is present in the right image and the left image acquired in step S10 and step S12. If a negative determination is made in step S14, the correction process is terminated. On the other hand, if a positive determination is made in step S14, the monocular distance LS is calculated in step S16. Note that in this embodiment, the process of step S16 corresponds to the "monocular distance calculation unit."

In step S16, one of the right and left images acquired in steps S10 and S12 is selected to be used to calculate the monocular distance LS. Specifically, it is determined whether the object is on the right camera 11 side or the left camera 12 side with respect to the center between the right camera 11 and the left camera 12 in the left-right direction. In other words, it is determined whether the object is on the right camera 11 side or the left camera 12 side of the perpendicular bisector of the line segment connecting the right camera 11 and the left camera 12. Whether the object is on the right camera 11 side or the left camera 12 side can be determined, for example, from the position of the object on the right image and the left image in the left-right direction.

In step S16, if it is determined that the object is present on the right camera 11 side, the monocular distance LS is calculated using the right image. If it is determined that the object is present on the left camera 12 side, the monocular distance LS is calculated using the left image.

In step S18, the compound eye distance LP is calculated using the right and left images acquired in steps S10 and S12. In the following step S20, it is determined whether the compound eye distance LP calculated in step S18 is greater than the distance threshold Lth. Note that in this embodiment, the process of step S18 corresponds to the "compound eye distance calculation unit", and the process of step S20 corresponds to the "distance determination unit".

If it is determined that the compound eye distance LP is greater than the distance threshold Lth, a positive determination is made in step S20. In this case, in step S22, the average parallax shift amount ZC is calculated, and the process proceeds to step S26. In calculating the average parallax shift amount ZC, first, a parallax conversion amount DS that converts the monocular distance LS into parallax is calculated. The parallax conversion amount DS is expressed as follows (Equation 2) using the reference length LC, focal length LF, and monocular distance LS.

DS=LC×LF/LS (Equation 2)
Next, the parallax shift amount ΔZ is calculated. The parallax shift amount ΔZ is expressed as the following (Equation 3) using the parallax DP of the compound eye distance LP and the parallax conversion amount DS of the monocular distance LS.

ΔZ=DP-DS (Equation 3)
The calculated parallax shift amount ΔZ is stored in the memory in the ECU 20. Then, an average parallax shift amount ZC is calculated using the plurality of parallax shift amounts ΔZ stored in the memory. The average parallax shift amount ZC is a cumulative average of the plurality of parallax shift amounts ΔZ stored in the memory, and the average parallax shift amount ZC is updated each time a new parallax shift amount ΔZ is stored in the memory. The calculated average parallax shift amount ZC is stored in the memory in the ECU 20.

On the other hand, if it is determined that the compound eye distance LP is smaller than the distance threshold Lth, a negative determination is made in step S20. In this case, in step S24, the previous average parallax shift amount ZC is read from the memory in the ECU 20 without calculating the average parallax shift amount ZC, and the process proceeds to step S26. Here, the previous average parallax shift amount ZC means the average parallax shift amount ZC stored in the memory in the ECU 20 in the correction process in which the process in step S22 was performed. In this embodiment, in step S24, the average parallax shift amount ZC last stored in the memory in the ECU 20 is read.

In step S26, the compound eye distance LP is corrected based on the average parallax shift amount ZC calculated in step S22 or read out in step S24, and the correction process ends. In other words, in step S26, the compound eye distance LP is corrected based on the average parallax shift amount ZC regardless of the determination result of step S20. Note that in this embodiment, the process of step S26 corresponds to the "distance correction unit."

To calculate the correction distance LPR, first, the correction parallax DPR is calculated by subtracting the average parallax shift amount ZC from the parallax DP at the compound eye distance LP. The correction parallax DPR is expressed as follows (Equation 4) using the parallax DP and the average parallax shift amount ZC.

DPR=DP-ZC (Equation 4)
Next, the correction distance LPR is calculated. The correction distance LPR is expressed as the following (Equation 5) using the reference length LC, the focal length LF, and the correction parallax DPR.

LPR=LC×LF/DPR (Equation 5)
Next, an example of the correction process is shown in Fig. 4 to Fig. 7. Fig. 4 shows the transition of the compound eye distance LP when the vehicle 50 approaches an object at a constant speed. As shown in Fig. 4, when the compound eye distance LP is larger than the distance threshold Lth and the object is far from the vehicle 50, the error of the compound eye distance LP, which is the difference between the actual distance LT (dashed line) to the object and the compound eye distance LP (solid line), is large, and the error becomes smaller as the vehicle 50 approaches the object. Note that in the stereo camera 10, there are cases where the deviation is larger than the actual distance LT and cases where the deviation is smaller than the actual distance LT depending on the specifications of the stereo camera 10, and Fig. 4 shows the transition of both of these. Below, an example in which the compound eye distance LP deviates to the smaller side than the actual distance LT will be described.

Figure 5 shows the progression of the monocular distance LS corresponding to Figure 4. The monocular distance LS is calculated based on the ratio of the actual dimension WR of the object to the image dimension WG, which is the size of the object on the right image or the left image. Specifically, the monocular distance LS is expressed as follows (Equation 6) using the focal length LF, the actual dimension WR, and the image dimension WG.

LS=WR×LF/WG (Equation 6)
Here, the actual dimension WR of the object is obtained by determining the object on the right or left image by well-known pattern matching.

The image dimension WG is expressed using the number of pixels, which is the number of pixels that correspond to the object. In this case, an error occurs in the monocular distance LS that depends on the length of up to one pixel, and the magnitude of the error is roughly proportional to the distance between the vehicle 50 and the object. Therefore, when an object is located far away from the vehicle 50, the error in the monocular distance LS is smaller than the error in the compound eye distance LP.

Therefore, in this embodiment, when an object is present far away from the vehicle 50, the compound eye distance LP is corrected based on the monocular distance LS. Specifically, as shown in FIG. 6, before time t1 at which the compound eye distance LP becomes equal to the distance threshold Lth, the parallax shift amount ΔZ corresponding to the distance difference ΔL between the compound eye distance LP and the monocular distance LS is calculated. Then, the compound eye distance LP is corrected based on this parallax shift amount ΔZ.

In addition, the direction of deviation of the monocular distance LS from the actual distance LT varies due to fluctuations associated with an increase or decrease in the number of pixels. Therefore, if the compound eye distance LP is corrected based on the monocular distance LS, the correction distance LPR after correction will also vary due to the influence of the fluctuation in the deviation direction of the monocular distance LS that remains in the parallax deviation amount ΔZ.

Therefore, in this embodiment, an average parallax shift amount ZC, which is the average value of multiple parallax shift amounts ΔZ, is calculated, and the compound eye distance LP is corrected based on this average parallax shift amount ZC. By calculating the average parallax shift amount ZC, the influence of fluctuations in the shift direction of the monocular distance LS that remain in the parallax shift amount ΔZ is suppressed. As a result, as shown in FIG. 7, the longer the elapsed time from the processing start time t0, the more the fluctuations in the correction distance LPR are suppressed.

On the other hand, as shown in FIG. 5, when the compound eye distance LP is smaller than the distance threshold Lth and an object is present near the vehicle 50, the error in the monocular distance LS may be larger than the error in the compound eye distance LP. In this case, the parallax shift amount ΔZ between the compound eye distance LP and the monocular distance LS and the average parallax shift amount ZC cannot be calculated properly. Therefore, in this embodiment, when an object is present near the vehicle 50, that is, after time t1, the average parallax shift amount ZC is not calculated (no arrow is shown in FIG. 6).

In addition, if the error in the monocular distance LS becomes larger than the error in the compound eye distance LP, there is no need to correct the compound eye distance LP based on the monocular distance LS. Therefore, it is also possible to stop the correction of the compound eye distance LP when an object is present close to the vehicle 50. However, if the correction of the compound eye distance LP is stopped when an object is present close to the vehicle 50, when the compound eye distance LP changes beyond the distance threshold Lth, the compound eye distance LP changes in a step-like manner between the correction distance LPR shown by the solid line in Figure 7 and the compound eye distance LP shown by the two-dot chain line in Figure 7. It is considered that this change will prevent the ECU 20 from properly performing contact avoidance processing for the object.

Therefore, in this embodiment, regardless of whether the compound eye distance LP is greater than the distance threshold Lth, the compound eye distance LP is corrected based on the average parallax shift amount ZC (indicated by an arrow in Figure 7). Specifically, if the compound eye distance LP is greater than the distance threshold Lth, the compound eye distance LP is corrected based on the average parallax shift amount ZC calculated in the correction process in which the compound eye distance LP was calculated. Also, if the compound eye distance LP is smaller than the distance threshold Lth, the compound eye distance LP is corrected based on the previous average parallax shift amount ZC calculated in the correction process prior to the correction process in which the compound eye distance LP was calculated. This makes it possible to suppress changes in the compound eye distance LP (corrected distance LPR) when the compound eye distance LP changes beyond the distance threshold Lth.

The above-described embodiment provides the following advantages:

-When the distance between the vehicle 50 and an object is calculated as the compound eye distance LP based on the parallax DP calculated from the position of the object on the right image captured by the right camera 11 and the position of the object on the left image captured by the left camera 12, if the object is located far from the vehicle 50, the error in the compound eye distance LP increases rapidly and the calculation accuracy decreases. In addition, when the distance between the vehicle 50 and an object is calculated as the monocular distance LS based on the image dimension WG on the right image or the left image, an error occurs in the monocular distance LS that depends on the length of up to one pixel, and the magnitude of the error is roughly proportional to the distance between the vehicle 50 and another vehicle. Therefore, for example, when an object is located far from the vehicle 50, the error in the monocular distance LS is smaller than the error in the compound eye distance LP. Therefore, the error in the compound eye distance LP can be suppressed by correcting the compound eye distance LP based on the parallax shift amount ΔZ between the compound eye distance LP and the monocular distance LS.

In this case, in this embodiment, the compound eye distance LP is corrected based on the average parallax shift amount ZC, which is the average value of the parallax shift amount ΔZ calculated over multiple times. In the compound eye distance LP, the shift direction with respect to the actual distance LT between the vehicle 50 and the object is constant, whereas in the monocular distance LS, the shift direction with respect to the actual distance LT fluctuates due to fluctuations associated with an increase or decrease in the number of pixels. Therefore, by correcting the compound eye distance LP based on the average parallax shift amount ZC, the influence of fluctuations in the shift direction of the monocular distance LS remaining in the parallax shift amount ΔZ can be suppressed, and the distance between the vehicle 50 and the object can be calculated with high accuracy based on the parallax DP in the right image and the left image.

- When the compound eye distance LP is greater than the distance threshold Lth and the object is located far from the vehicle 50, the error in the compound eye distance LP is greater than the error in the monocular distance LS, so the parallax shift amount ΔZ is calculated to calculate the average parallax shift amount ZC, and the compound eye distance LP is corrected. On the other hand, when the compound eye distance LP is smaller than the distance threshold Lth and the object is located close to the vehicle 50, the error in the compound eye distance LP may be smaller than the error in the monocular distance LS, and the average parallax shift amount ZC cannot be calculated properly. In this regard, in this embodiment, when it is determined that the compound eye distance LP is smaller than the distance threshold Lth, the average parallax shift amount ZC is not calculated. This makes it possible to suppress a decrease in the calculation accuracy of the compound eye distance LP due to a decrease in the calculation accuracy of the average parallax shift amount ZC.

- When an object is present near the vehicle 50, the error in the compound eye distance LP may be smaller than the error in the monocular distance LS, and in this case, there is no need to correct the compound eye distance LP based on the monocular distance LS. Therefore, when an object is present near the vehicle 50, it is possible to stop the correction of the compound eye distance LP. However, if the correction of the compound eye distance LP is stopped in this case, when the compound eye distance LP changes beyond the distance threshold Lth, the compound eye distance LP (corrected distance LPR) will change by the average parallax shift amount ZC, and this step-like change is considered to affect the implementation of the contact avoidance process. In this regard, in this embodiment, regardless of whether the compound eye distance LP is greater than the distance threshold Lth, the compound eye distance LP is corrected by the average parallax shift amount ZC. Therefore, it is possible to suppress the change in the compound eye distance LP when the compound eye distance LP changes beyond the distance threshold Lth.

- When an object is present on the right camera 11 side with respect to the center between the right camera 11 and the left camera 12 in the left-right direction, the deviation width from the optical axis of the right camera 11 to the object is smaller than the deviation width from the optical axis of the left camera 12 to the object. Therefore, by calculating the monocular distance LS using the right image captured by the right camera 11, the monocular distance LS can be calculated with high accuracy, and errors in the compound eye distance LP can be suppressed. Also, when an object is present on the left camera 12 side with respect to the center between the right camera 11 and the left camera 12 in the left-right direction, the deviation width from the optical axis of the left camera 12 to the object is smaller than the deviation width of the object from the optical axis of the right camera 11. Therefore, by calculating the monocular distance LS using the right image captured by the left camera 12, the monocular distance LS can be calculated with high accuracy, and errors in the compound eye distance LP can be suppressed.

Second Embodiment
The second embodiment will be described below with reference to Figures 8 to 12, focusing on the differences from the first embodiment. As shown in Figure 8, this embodiment differs from the first embodiment in that the ECU 20 functions as a change amount determination unit 24. In Figure 8, the same components as those shown in Figure 1 are denoted by the same reference numerals and will not be described for convenience.

The change amount determination unit 24 determines that the change amount VC in the distance between the vehicle 50 and the object during a specified period is greater than the change amount threshold Vth. In this embodiment, the change amount VC means the maximum value of the difference between the compound eye distance LP calculated in multiple correction processes performed from the present to the specified period ago. The change amount threshold Vth is the amount of change in the compound eye distance LP at which a specified fluctuation occurs in the monocular distance LS. Here, the specified fluctuation is the fluctuation corresponding to between adjacent maximum and minimum values among the maximum and minimum values that occur in the monocular distance LS due to an increase or decrease in the number of pixels.

Figure 9 shows a flowchart of the correction process of this embodiment. Note that in Figure 9, the same processes as those shown in Figure 3 above are given the same step numbers for convenience, and descriptions are omitted.

In the correction process of this embodiment, if a positive determination is made in step S20, the amount of change VC is calculated in step S30. In the following step S32, it is determined whether the amount of change VC calculated in step S30 is greater than the amount of change threshold Vth. In this embodiment, the process of step S30 corresponds to the "amount of change determination unit."

If it is determined that the change amount VC of the compound eye distance LP during the specified period is greater than the change amount threshold Vth, a positive determination is made in step S32. In this case, the process proceeds to step S22, where the average parallax shift amount ZC is calculated using the parallax shift amount ΔZ acquired during the specified period.

On the other hand, if it is determined that the change amount VC of the compound eye distance LP during the specified period is smaller than the change amount threshold Vth, a negative determination is made in step S32. In this case, the process proceeds to step S24, and the previous average parallax shift amount ZC is read from the memory in the ECU 20 without calculating the average parallax shift amount ZC.

Next, an example of the correction process of this embodiment is shown in Figures 10 to 12. Figure 10 shows the changes in the compound eye distance LP and the monocular distance LS when the vehicle 50 approaches an object. In Figure 10, the object is a moving body (e.g., another vehicle), and in a first period TA from processing start time t0 to time t2, the vehicle 50 approaches the object at a constant speed. In the subsequent second period TB from time t2 to time t3, the distance between the vehicle 50 and the object is maintained constant. In the subsequent third period TC from time t3, the changes in the compound eye distance LP and the monocular distance LS are shown when the vehicle 50 approaches the object at a constant speed again.

When the distance between the vehicle 50 and the object is maintained constant during the second period TB, the compound eye distance LP and the monocular distance LS calculated in the correction process are also maintained constant. If the compound eye distance LP maintained constant is greater than the distance threshold Lth and the object is located far away from the vehicle 50, it is also possible that the parallax shift amount ΔZ is calculated during the second period TB.

In FIG. 11 showing a comparative example, the parallax shift amount ΔZ is calculated in the second period TB. In the second period TB, the compound eye distance LP and the monocular distance LS are maintained constant, so the distance difference ΔL between the compound eye distance LP and the monocular distance LS is maintained constant, and the parallax shift amount ΔZ corresponding to this distance difference ΔL is also maintained constant. For example, as shown in FIG. 11, when the distance difference ΔL maintained constant is small, the calculated parallax shift amount ΔZ is also a small value. As a result, the average parallax shift amount ZC gradually decreases, and when the compound eye distance LP is corrected based on an excessively small average parallax shift amount ZC, the corrected distance LPR moves away from the actual distance LT.

In addition, in the second period TB, fluctuations in the shift direction of the monocular distance LS are suppressed, and the monocular distance LS is maintained in a state where it is shifted in a specific direction (the smaller side in FIG. 11) relative to the actual distance LT. Therefore, the parallax shift amount ΔZ calculated in a specified period and the average parallax shift amount ZC calculated based on this parallax shift amount ΔZ are still affected by the fluctuations in the shift direction of the monocular distance LS, and the compound eye distance LP cannot be accurately corrected based on the average parallax shift amount ZC.

Therefore, in this embodiment, it is determined that the change amount VC of the compound eye distance LP in a predetermined period is greater than the change amount threshold Vth, and if it is determined that it is greater, the average parallax shift amount ZC is calculated, and if it is determined that it is smaller, the average parallax shift amount ZC is not calculated. Specifically, the vehicle 50 approaches the object at a constant speed, and in the first period TA and the third period TC, in which the change amount VC is greater than the change amount threshold Vth, the average parallax shift amount ZC is calculated. On the other hand, in the second period TB, in which the distance between the vehicle 50 and the object is kept constant and the change amount VC is smaller than the change amount threshold Vth, the average parallax shift amount ZC is not calculated, and the compound eye distance LP is corrected based on the previous average parallax shift amount ZC calculated in the correction process before the correction process in which the change amount VC was calculated. In other words, when the change in the compound eye distance LP and the monocular distance LS is small, the average parallax shift amount ZC is not calculated. As a result, as shown in FIG. 12, it is possible to prevent a decrease in the calculation accuracy of the compound eye distance LP due to a decrease in the calculation accuracy of the average parallax shift amount ZC.

The above-described embodiment provides the following advantages:

- When the change amount VC of the compound eye distance LP in a specified period is smaller than the change amount threshold Vth, for example when the distance between the vehicle 50 and an object is maintained constant, fluctuations in the shift direction of the monocular distance LS are suppressed. As a result, the monocular distance LS is maintained in a state where it is shifted in a specific direction relative to the actual distance LT, and the parallax shift amount ΔZ acquired in the specified period and the average parallax shift amount ZC calculated based on this parallax shift amount ΔZ are still affected by the fluctuations in the shift direction of the monocular distance LS. Therefore, the compound eye distance LP cannot be accurately corrected based on this average parallax shift amount ZC.

In this regard, in the present embodiment, if it is determined that the change amount VC of the compound eye distance LP during a predetermined period is smaller than the change amount threshold Vth, the average parallax shift amount ZC is not calculated during that predetermined period. This makes it possible to prevent a decrease in the calculation accuracy of the compound eye distance LP due to a decrease in the calculation accuracy of the average parallax shift amount ZC.

Other Embodiments
The present invention is not limited to the above-described embodiment, and may be implemented as follows.

- The moving object is not limited to a vehicle 50 and may be, for example, a drone.

- An example has been shown in which the parallax shift amount ΔZ is used as the difference value between the compound eye distance LP and the monocular distance LS, but this is not limited to this, and the distance difference ΔL between the compound eye distance LP and the monocular distance LS may also be used as the difference value.

- The intermediate value of the difference values is not limited to the average value of the difference values, but may be, for example, the median value of the difference values.

The distance used for comparison with the distance threshold Lth is not limited to the compound eye distance LP, but may be, for example, the monocular distance LS. In other words, it is sufficient to determine that at least one of the compound eye distance LP and the monocular distance LS is greater than the distance threshold Lth.

The distance used to calculate the change amount VC is not limited to the compound eye distance LP, but may be, for example, the monocular distance LS. Furthermore, the change amount VC in the distance between the vehicle 50 and the object may be calculated based on information such as the speed of the vehicle 50 and the accelerator operation amount.

- In the above embodiment, an example was given in which when it is determined that the compound eye distance LP is smaller than the distance threshold Lth, the previous average parallax shift amount ZC is read from the memory in the ECU 20, but this is not limited to this.

For example, if the change amount VC of the compound eye distance LP is determined to be greater than the change amount threshold Vth and the calculated average parallax shift amount ZC is monotonically decreasing at a constant slope, the compound eye distance LP may be calculated using the average parallax shift amount ZC that is monotonically decreasing at that slope from the average parallax shift amount ZC last stored in the memory in the ECU 20.

- For example, by using the average parallax shift amount ZC last stored in the memory in the ECU 20 and the compound eye distance LP calculated in the correction process in which the average parallax shift amount ZC was calculated, a straight line or curve may be set so that the average parallax shift amount ZC becomes zero when the compound eye distance LP becomes zero, and the average parallax shift amount ZC may be reduced based on the set straight line or curve.

The control device and method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, the control device and method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control device and method described in the present disclosure may be realized by one or more dedicated computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transient tangible recording medium as instructions executed by the computer.

11...right camera, 12...left camera, 20...ECU, 50...vehicle.

Claims (7)

An object detection device (20) is applied to a moving body (50) equipped with a first camera (11) and a second camera (12) for capturing a first image and a second image including parallax, and detects an object present around the moving body based on the first image and the second image,
a compound eye distance calculation unit that calculates a distance between the moving body and the object as a compound eye distance based on a parallax obtained from a position of the object on the first image and a position of the object on the second image;
a monocular distance calculation unit that uses one of the first image and the second image and calculates a distance between the moving body and the object as a monocular distance based on a size of the object in the one image;
a distance determination unit that determines whether at least one of the compound eye distance and the monocular distance is greater than a predetermined distance threshold;
a correction value calculation unit that, on condition that at least one of the compound eye distance and the monocular distance is determined to be greater than the distance threshold, acquires a difference value between the compound eye distance and the monocular distance multiple times and calculates a correction value for correcting the compound eye distance based on an intermediate value of the acquired multiple difference values;
and a distance correction unit that corrects the compound eye distance using the correction value . The object detection device according to claim 1 , wherein the distance correction unit corrects the compound eye distance using the correction value regardless of a determination result of the distance determination unit. a change amount determination unit that determines whether a change amount of the distance between the moving body and the object in a predetermined period of time is greater than a predetermined change amount threshold,
The object detection device of claim 1 or 2, wherein the correction value calculation unit calculates the correction value using the difference value acquired during the specified period, on condition that it is determined that the amount of change in the compound eye distance during the specified period is greater than the change amount threshold. The first camera and the second camera are arranged side by side in a predetermined direction,
The monocular distance calculation unit
determining whether the object is present on the first camera side or the second camera side with respect to a center between the first camera and the second camera in the predetermined direction;
When it is determined that the object is present on the first camera side, the monocular distance is calculated using the first image;
The object detection device according to claim 1 , wherein when it is determined that the object is present on the second camera side, the monocular distance is calculated using the second image. An object detection device (20) is applied to a moving body (50) equipped with a first camera (11) and a second camera (12) that are arranged side by side in a predetermined direction and capture a first image and a second image including parallax, and detects an object present around the moving body based on the first image and the second image,
a compound eye distance calculation unit that calculates a distance between the moving body and the object as a compound eye distance based on a parallax obtained from a position of the object on the first image and a position of the object on the second image;
a monocular distance calculation unit that uses one of the first image and the second image and calculates a distance between the moving body and the object as a monocular distance based on a size of the object in the one image;
a distance correction unit that acquires a difference value between the compound eye distance and the monocular distance a plurality of times and corrects the compound eye distance based on an intermediate value of the acquired plurality of difference values ,
The monocular distance calculation unit
determining whether the object is present on the first camera side or the second camera side with respect to a center between the first camera and the second camera in the predetermined direction;
When it is determined that the object is present on the first camera side, the first image is used to calculate the monocular distance;
An object detection device that calculates the monocular distance using the second image when it is determined that the object is present on the second camera side . A program that is applied to a moving body (50) equipped with a first camera (11) and a second camera (12) that capture a first image and a second image including parallax, and detects an object present around the moving body based on the first image and the second image,
On the computer,
a compound eye distance calculation process for calculating a distance between the moving body and the object as a compound eye distance based on a parallax obtained from a position of the object on the first image and a position of the object on the second image;
a monocular distance calculation process for calculating a distance between the moving body and the object as a monocular distance based on a size of the object in one of the first image and the second image, using the one of the first image and the second image;
a distance determination process for determining whether at least one of the compound eye distance and the monocular distance is greater than a predetermined distance threshold;
a correction value calculation process for obtaining a difference value between the compound eye distance and the monocular distance multiple times, on condition that at least one of the compound eye distance and the monocular distance is determined to be greater than the distance threshold, and calculating a correction value for correcting the compound eye distance based on an intermediate value of the obtained difference values;
A distance correction process for correcting the compound eye distance using the correction value. A program applied to a moving body (50) equipped with a first camera (11) and a second camera (12) that are arranged side by side in a predetermined direction and capture a first image and a second image including parallax, the program detecting an object present around the moving body based on the first image and the second image,
On the computer,
a compound eye distance calculation process for calculating a distance between the moving body and the object as a compound eye distance based on a parallax obtained from a position of the object on the first image and a position of the object on the second image;
a monocular distance calculation process for calculating a distance between the moving body and the object as a monocular distance based on a size of the object in one of the first image and the second image, using the one of the first image and the second image;
a distance correction process for acquiring a difference value between the compound eye distance and the monocular distance multiple times and correcting the compound eye distance based on an intermediate value of the acquired multiple difference values;
The monocular distance calculation process includes:
determining whether the object is present on the first camera side or the second camera side with respect to a center between the first camera and the second camera in the predetermined direction;
When it is determined that the object is present on the first camera side, the first image is used to calculate the monocular distance;
A program that calculates the monocular distance using the second image when it is determined that the object is present on the second camera side.

Images