JP7765503B2 - Imaging device, parallax displacement correction method, and parallax displacement correction program - Google Patents

Description

This disclosure relates to an imaging device, a parallax displacement correction method, and a parallax displacement correction program.

Stereo cameras are known as devices for recognizing objects in three dimensions. Stereo cameras use the differences in the images captured by multiple cameras positioned at different locations to detect parallax between the multiple cameras based on trigonometry, and then use this parallax to detect the distance and position of an object from the camera. Stereo cameras are installed in vehicles, for example, and used as cameras that make up driver assistance systems.

However, if the optical axis of a stereo camera shifts due to deterioration over time, accurate parallax calculation becomes impossible. This makes it impossible to accurately measure the distance from the camera to an object, increasing the likelihood that the driving assistance system will not function properly. For this reason, various technologies have been proposed to calibrate the optical axis (in other words, to correct parallax shift) in order to accurately calculate parallax (see, for example, Patent Document 1). Patent Document 1 discloses a technology that corrects parallax shift by calculating the parallax between the position of the road surface near the vehicle (nearby road surface) and the position of the road surface far from the vehicle (far road surface position).

JP 2017-44573 A

However, the device disclosed in Patent Document 1 pre-sets the road surface position used to calculate the parallax shift, which limits the environment in which the parallax shift can be corrected. Furthermore, because the road surface texture is monotonous, the reliability of the parallax calculation may be low. This is particularly true on unclear roads lacking white lines or various signs. Calculating the amount of parallax shift using a disparity with low reliability may result in a decrease in the accuracy of the parallax shift correction, and ultimately in a decrease in the accuracy of distance measurement.

This disclosure proposes an imaging device, a parallax displacement correction method, and a parallax displacement correction program that can detect parallax displacement with high accuracy.

An imaging device according to one aspect of the present disclosure is characterized by comprising: a first imaging unit that acquires a first captured image; a second imaging unit that is positioned a certain distance from the first imaging unit and acquires a second captured image; a parallax calculation unit that calculates the parallax for each pixel of the first captured image and the second captured image; a region selection unit that selects, from the first captured image or the second captured image, a region where the parallax value has a reliability equal to or greater than a threshold; a parallax shift amount calculation unit that compares the parallax value in the region selected by the region selection unit with an ideal value of parallax that should be acquired in that region to determine the parallax shift amount for each region; and a parallax shift correction unit that corrects the parallax shift in the first imaging unit and the second imaging unit using a correction amount determined by statistically processing the parallax shift amount for each region.

The present invention provides an imaging device, a parallax displacement correction method, and a parallax displacement correction program that can detect parallax displacement with high accuracy.

1 is a schematic diagram illustrating the overall configuration of a stereo camera 100 as an imaging device according to a first embodiment. FIG. 2 is a block diagram illustrating an example of the detailed configuration of the image processing device 2 of FIG. 1A. 10 is an explanatory diagram illustrating a method for calculating a parallax D in the parallax calculation unit 13. FIG. 10 is an explanatory diagram illustrating a method for calculating a parallax D in the parallax calculation unit 13. FIG. 10 is an explanatory diagram illustrating a region selection method in a region selection unit 14. FIG. 10 is an explanatory diagram illustrating a method for calculating a parallax D in the parallax calculation unit 13. FIG. 10 is an explanatory diagram illustrating a method for calculating a parallax D in the parallax calculation unit 13. FIG. 10 is an explanatory diagram illustrating a method for calculating a parallax D in the parallax calculation unit 13 and a method for calculating a parallax shift amount ΔD in the parallax shift amount calculation unit 15. FIG. 10 is an explanatory diagram illustrating a method for calculating a parallax D in the parallax calculation unit 13 and a method for calculating a parallax shift amount ΔD in the parallax shift amount calculation unit 15. FIG. 10 is a flowchart illustrating a specific example of a procedure for calculating a parallax displacement correction amount ΔC in the stereo camera 100 according to the first embodiment. 10A and 10B are explanatory diagrams illustrating the operation of an area selection unit 14 in the stereo camera 100 according to the second embodiment. 10 is a flowchart illustrating a specific example of a procedure for calculating a parallax displacement correction amount ΔC in the stereo camera 100 according to the second embodiment. 10A and 10B are explanatory diagrams illustrating the operation of an area selection unit 14 in the stereo camera 100 according to the second embodiment. FIG. 11 is an explanatory diagram illustrating the operation of the third embodiment. FIG. 11 is an explanatory diagram illustrating the operation of the third embodiment. FIG. 11 is an explanatory diagram illustrating the operation of the third embodiment.

The present embodiment will be described below with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be indicated by the same numerals. Note that the accompanying drawings show embodiments and implementation examples in accordance with the principles of the present disclosure, but these are intended to aid in understanding the present disclosure and should not be used to interpret the present disclosure in any way as limiting. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

In this embodiment, the description has been provided in sufficient detail to enable those skilled in the art to implement the present disclosure, but it should be understood that other implementations and forms are possible, and that changes to the configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[First embodiment]
1A, the overall configuration of a stereo camera 100 as an imaging device according to a first embodiment will be described. The stereo camera 100 is an in-vehicle stereo camera device mounted on a vehicle such as an automobile, and includes a stereo camera main body 1, an image processing device 2, and an interface 5. The stereo camera main body 1, the image processing device 2, and the interface 5 are connected via a communication line such as a bus BU.

The stereo camera main body 1 is configured by multiple cameras, for example, two cameras, a first imaging unit 11 (left camera) and a second imaging unit 12 (right camera), arranged apart by a predetermined baseline length B. The first imaging unit 11 and the second imaging unit 12 each capture images of the area in front of the vehicle from positions that differ by the baseline length B to obtain the images. The image processing device 2 may include, for example, a CPU 3 as a computing device and a memory 4 as a storage device. The memory 4 may be RAM, ROM, a hard disk drive (HDD), or a combination thereof. The stereo camera 100 may be connected to an external device 200 via an interface 5.

The external device 200 may include various sensors that detect the vehicle's accelerator opening (i.e., throttle opening), brake operation amount (brake pedal operation amount), steering angle, vehicle speed, acceleration, temperature, humidity, etc., as well as an ECU (Electronic Control Unit) that controls various vehicle operations.

Memory 4 is a recording medium that stores programs (such as a parallax displacement correction program) used for various processes in image processing device 2, as well as various types of information. CPU 3 performs predetermined calculations on signals received via interface (I/F) 5 in accordance with the control program stored in memory 4, detecting three-dimensional objects and road surfaces, and calculating the position, distance, direction, etc. of objects. The results of calculations by CPU 3 are output to external device 200 via interface 5 and are used to determine and control various vehicle operations such as acceleration, braking, and steering.

An example of the detailed configuration of the image processing device 2 will be described with reference to the block diagram in Figure 1B. The image processing device 2 is, as an example, composed of a parallax calculation unit 13, an area selection unit 14, a parallax shift amount calculation unit 15, and a parallax shift correction unit 16, which are realized by a program. The image processing device 2 calculates the distance Z from the stereo camera 100 to an object using the parallax D calculated by the parallax calculation unit 13 and corrected by the parallax shift correction unit 16, and supplies the calculation result to the external device 200.

The parallax calculation unit 13 calculates the parallax D based on the image captured by the first imaging unit 11 and the image captured by the second imaging unit 12. Specifically, the parallax calculation unit 13 sets, for example, the image captured by the first imaging unit 11 as a reference image and extracts feature points with variations in shading in the reference image. Next, the parallax calculation unit 13 uses the other image captured by the second imaging unit 12 as a reference image and searches for the positions of reference points in the reference image that reflect the same subject as the feature points extracted in the reference image. Template matching such as SAD (Sum of Absolute Difference) can be used for the search. The difference between the position of the extracted feature points in the reference image and their positions in the reference image is calculated as the parallax D. The calculated parallax D is calculated for each region containing the extracted feature points and temporarily stored in memory 4 in association with the position of the region. That is, the parallax calculation unit 13 is configured to extract multiple feature points from a pair of reference and reference images and calculate the parallax D for each of multiple regions containing the extracted feature points.

The region selection unit 14 selects a region from which a highly reliable parallax D has been obtained from among multiple regions in the captured image from which parallax D data has been obtained. By selecting a region from which a highly reliable parallax D has been obtained and calculating the parallax shift amount ΔD using the parallax D in the selected region, it becomes possible to perform parallax shift correction with high precision. The method for determining a highly reliable parallax D will be described later. The number of regions that can be selected is not important.

The parallax shift calculation unit 15 calculates the parallax shift ΔD for each region using the values of parallax D for the multiple regions selected by the region selection unit 14. The parallax shift ΔD is calculated for each region as the difference (D - Di) between the parallax D calculated for a given region by the parallax calculation unit 13 and the ideal value Di of parallax that should be obtained for that region. Here, the "ideal value Di of parallax" is determined by the heights of the first and second imaging units 11 and 12 from the object and the angle of view of the camera that captures the region. The specific procedure for calculating the ideal value Di of parallax will be described later.

The parallax shift correction unit 16 determines a parallax shift correction value ΔC for the captured image from the multiple parallax shift amounts ΔD calculated for each of the selected multiple regions.

Next, with reference to Figures 2A to 2C, we will explain how the parallax calculation unit 13 calculates the parallax D and how the area selection unit 14 selects areas with highly reliable parallax D. As shown in Figure 2A, for example, an image of a road including a white line WL is captured by the first imaging unit 11 and the second imaging unit 12, resulting in a first captured image IM1 and a second captured image IM2. Here, the first captured image IM1 captured by the first imaging unit 11 is used as the base image, and the second captured image IM2 captured by the second imaging unit 12 is used as the reference image. The parallax calculation unit 13 extracts a certain feature point 21 in the first captured image IM1 and extracts a reference point 21' that has the same feature as the feature point 21 in the second captured image IM2. The parallax calculation unit 13 then performs SAD template matching between the feature point 21 and the reference point 21', and calculates the similarity SM, as shown in the graph of Figure 2B, for example. The horizontal axis of the graph in FIG. 2B indicates the horizontal coordinate of the second captured image IM2, and the vertical axis indicates the SAD value (similarity SM).

When similarity SM is calculated for feature point 21 in the first captured image IM1 (base image) at reference point 21' in the second captured image IM2 (reference image) where an object identical to the object captured at feature point 21 is captured, similarity SM will be a small value. On the other hand, when similarity SM is calculated for feature point 21 in the first captured image IM1 (base image) at a position in the second captured image IM2 (reference image) where an object different from the object captured at feature point 21 is captured (a position other than the reference point), similarity SM will be a high value. Here, the horizontal coordinate value of the position (first peak P1) where similarity SM is minimum is the disparity D of feature point 21.

When the protrusion amount of the first peak P1 (the amount of decrease in the similarity SM at the first peak P1) is k, the larger the value of the protrusion amount k, the greater the feature amount (e.g., brightness difference) of the feature point 21 relative to adjacent regions. The greater the brightness difference relative to adjacent regions, the greater the calculated protrusion amount k, and the higher the reliability of the disparity D calculated in that region. Therefore, the average value Av of the similarity SM is calculated for the region excluding the vicinity of the first peak P1 (the region surrounded by the dashed line in the figure), and this is used as the reference value. The difference between this average value Av (reference value) and the reliability SM of the minimum value of the first peak P1 is calculated as the protrusion amount k. The protrusion amount k is calculated for multiple regions, and the region selection unit 14 selects multiple regions with high k values. For example, as shown in Figure 2C, the protrusion amount k is calculated for each of multiple regions (each assigned a unique region ID), and the regions are sorted in descending order of protrusion amount k. A predetermined number of regions are selected in descending order of protrusion amount k by the region selection unit 14. As will be described later, a threshold value THk may be set for the protrusion amount k, and if the number of regions where the protrusion amount k exceeds the threshold value THk does not reach the minimum value Nreg, the operation of parallax displacement correction may be stopped.

In Figures 2A and 2B, the protrusion amount k is calculated from the difference between the average value Av of the similarity SM (excluding values near the first peak P1) and the lower limit value of the first peak P1. However, as shown in Figures 3A and 3B, it is also possible to identify a second peak P2 other than the first peak P1 (for example, a peak with the next largest protrusion amount k after the first peak P1), and calculate the difference between the lower limit value of this first peak P1 and the lower limit value of the second peak as the protrusion amount k.

Next, using Figure 4A, we will describe the method of calculating the parallax D in the parallax calculation unit 13 and the method of calculating the parallax shift amount ΔD in the parallax shift amount calculation unit 15. If the focal length of the lens L of the first imaging unit 11 and the second imaging unit 12 is f (mm), the pixel pitch of the imaging elements (e.g., CMOS sensors) of the first imaging unit 11 and the second imaging unit 12 is δ (mm/pixel), the position where the road surface at a horizontal distance Z (mm) from the lens L is projected onto the imaging surface is J (pixel), and the height of the lens L from the road surface is h (mm), then the relationship shown in the following equation (1) holds.

[Equation 1]
Z=(f×h)/(J×δ)…(1)

Furthermore, the parallax D (pixels) at a distance Z (mm) satisfies the following equation (2), where B (mm) is the distance (baseline length) between the first imaging unit 11 and the second imaging unit 12.

[Equation 2]
D=(B×f)/(Z×δ)…(2)

Substituting equation (1) into equation (2) gives the following equation (3).

[Equation 3]
D=B×J/h…(3)

In this way, the parallax D (pixels) of the road surface can be calculated from the baseline length B (mm) and height h of the stereo camera 100, and the coordinate J (pixels) of the road surface on the imaging surface. Because the baseline length B and height h (mm) are values that can be determined when the first imaging unit 11 and the second imaging unit 12 are mounted on a vehicle, the parallax D (mm) can be uniquely determined by the coordinate J (pixels) of the road surface.

The ideal value Di of parallax D of the road surface can be calculated based on the value of the baseline length B and the value of the height h when the stereo camera 100 is mounted on the vehicle. Here, instead of calculating the ideal value Di of parallax at the road surface, it is also possible to calculate the ideal value Di of parallax at an object Ob (e.g., a curb, guardrail, etc.) whose height from the road surface is known, as shown in Figure 4B. However, because the road surface is a feature that is always present in a typical driving environment, it is preferable in principle to calculate the ideal value Di of parallax at the road surface. While the ideal value Di of parallax is calculated at the road surface as a general rule, it is also possible to calculate the ideal value Di of parallax at objects other than the road surface in special cases.

The parallax shift amount calculation unit 15 calculates the difference between the parallax D actually calculated in the parallax calculation unit 13 according to the current baseline length B and height h, and the ideal parallax value Di calculated according to the baseline length B and height h at the time of design, as parallax shift amount ΔD = D - Di. The parallax shift amount calculation unit 15 calculates the parallax shift amount ΔD for each of the multiple road surface areas selected by the area selection unit 14.

Next, referring to the flowchart in Figure 5, a specific example of the procedure for calculating the parallax shift correction amount ΔC in the stereo camera 100 of the first embodiment will be described.

First, the road surface that is the basis for calculating the parallax displacement correction amount ΔC is identified (step S101). Then, in the manner described above, the parallax calculation unit 13 calculates the parallax D on that road surface for each region that includes the feature point 21 (step S102), and calculates the value of the protrusion amount k for each region (step S103).

Once the feature value k has been calculated for each region, the region selection unit 14 selects regions where the protrusion amount k is greater than the threshold value THk (step S104). If the number of regions Nreg selected in step S104 is greater than or equal to the threshold value THreg, the process proceeds to step S106. However, if it is less than the threshold value THreg, the calculation of the parallax displacement correction amount ΔC and the parallax displacement correction process are stopped (step S105). This is because if a certain number of regions where the protrusion amount k is greater than or equal to the threshold value THk cannot be obtained, it is not possible to calculate the parallax displacement amount with high accuracy.

In step S106, the parallax shift calculation unit 15 calculates the ideal parallax value Di for each of the selected regions in the manner described above, and calculates the difference between this ideal value Di and the actual parallax D as the parallax shift ΔD = D - Di (step S107).The parallax shift correction unit 16 then statistically processes the parallax shift ΔD obtained in this manner for the multiple regions to calculate the parallax shift ΔDfix for the entire image, and calculates the parallax shift correction amount ΔC according to this parallax shift ΔDfix (step S108).

As described above, according to the stereo camera 100 of the first embodiment, an area where a high protrusion amount k is obtained is selected, and the parallax shift amount ΔD is calculated based on the selected area, thereby providing an imaging device that can detect parallax shift with high accuracy.

Second Embodiment
Next, a stereo camera 100 according to a second embodiment will be described with reference to Figures 6A to 6C. The overall configuration of the stereo camera 100 according to the second embodiment is the same as that of the first embodiment (Figures 1A and 1B), so a duplicated description will be omitted.

In the first embodiment described above, the region selection unit 14 calculates the prominence k of the similarity SM (see Figure 2B) and selects multiple regions to use in calculating the parallax shift amount ΔD according to the magnitude of the prominence k. In contrast, in the second embodiment, instead of obtaining a graph of the similarity SM, the region selection unit 14 determines the magnitude of the luminance difference between the region of interest 41 and a corresponding region, for example, an adjacent region 42 adjacent to the region of interest 41, as shown in Figure 6A, and selects a region with a highly reliable parallax based on the magnitude of the luminance difference.

With reference to Figure 6B, a specific example of the procedure for calculating the parallax displacement correction amount ΔC in the stereo camera 100 of the second embodiment will be described. First, the road surface that serves as the basis for calculating the parallax displacement correction amount ΔC is identified (step S201). Then, the area selection unit 14 calculates the luminance difference between a focus area 41 on the identified road surface and an adjacent area 42 adjacent to the focus area 41 (step S202). Once the luminance difference has been calculated for each area, the area selection unit 14 selects areas where the luminance difference is greater than the threshold value THin (step S203).

Next, it is determined whether the number of regions Nreg selected in step S203 is equal to or greater than the threshold value THreg' (step S205). If the determination is Yes, the process proceeds to step S206, but if the determination is No (less than the threshold value THreg), the calculation of the parallax displacement correction amount ΔC and the parallax displacement correction process are stopped.

In step S206, the ideal parallax value Di for each of the selected regions is calculated in the manner described above, and the difference between this ideal value Di and the actual parallax D is calculated as the parallax shift amount ΔD = D - Di in the parallax shift amount calculation unit 15 (step S207).The parallax shift amount ΔD obtained in this way for the multiple regions is then statistically processed to calculate the parallax shift amount ΔDfix for the entire image, and the parallax shift correction amount ΔC is calculated according to this parallax shift amount ΔDfix (step S208).

As shown in Figure 6C, it is also possible to calculate the brightness difference between the region of interest 41 and a nearby region 42' that is not adjacent to it but is located a short distance away. The nearby region 42 is a region located at no distance from the region of interest 41, while the nearby region 42' is a region that is, for example, one region away from the region of interest 41. If the edge of the region of interest 41 (for example, the edge between the road surface and the white line) becomes blurred due to the resolution of the camera, it may not be possible to correctly calculate the brightness difference between the region of interest 41 and the nearby region 42. However, by calculating the brightness difference using the nearby region 42', which is located a short distance away, in this way, it is possible to calculate the brightness difference more accurately.

As described above, according to the stereo camera 100 of the second embodiment, the area to be used for calculating the parallax shift amount is selected according to the magnitude of the brightness difference, and the parallax shift amount ΔD is calculated based on the selected area, so that, as with the first embodiment, an imaging device can be provided that can detect parallax shift with high accuracy.

[Third embodiment]
Next, a stereo camera 100 according to a third embodiment will be described with reference to FIGS. 7 and 8. The overall configuration of the stereo camera 100 according to the third embodiment is the same as that according to the first embodiment (FIGS. 1A and 1B), so a duplicated description will be omitted. The third embodiment differs from the previous embodiments in the method of calculating the parallax displacement amount in the parallax displacement correction unit 16. Specifically, after region selection by the region selection unit 14 as in the previous embodiments, a representative value (e.g., median Median(J)) of the parallax displacement amount ΔD of a plurality of regions for each ordinate J (1, 2, ... m, ...) is calculated in the parallax displacement correction unit 16, as shown in FIG. 7.

Next, the average value of the calculated median values Median(1) to (m) is calculated as the average value ΔDav of the parallax shift amount ΔD. This parallax shift amount ΔDav is used as input to calculate the parallax shift correction amount ΔC in the parallax shift correction unit 16. However, if the standard deviation SD of the median value Median(J) used to calculate the average value ΔDav is greater than a threshold value, it is preferable not to perform parallax shift correction on the captured image. This is because if the standard deviation SD is greater than a threshold value, the reliability of the area containing that coordinate is determined to be low.

In the example shown in Figure 8, one median value Median(j) is calculated for one coordinate J. However, one median value Median(k to k+x) may also be calculated for multiple coordinates k to k+x. It is particularly preferable to calculate one median value for multiple vertical coordinates for road surfaces far from the vehicle (road surfaces shown at the top of the screen). This is because road surfaces far from the vehicle are captured by the first and second image capture units 11 and 12 with a compressed amount of information per pixel compared to nearby road surfaces (road surfaces shown at the bottom of the screen). This leads to a decrease in the accuracy of parallax displacement correction. As described above, for distant road surfaces, calculating a median value of 1 for multiple coordinates can prevent a decrease in accuracy.

Also, as shown in Figure 9, the median of the parallax shift amount ΔD for all selected areas can be calculated rather than for each coordinate, and this median can be determined as the parallax shift amount ΔD for the captured image, and used as the parallax shift correction amount ΔC.

Although various embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those that include all of the configurations described. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

100...Stereo camera, 200...External device, 1...Stereo camera main body, 2...Image processing device, 3...CPU, 4...Memory, 5...Interface, BU...Bus, 11...First imaging unit, 12...Second imaging unit, 13...Parallax calculation unit, 14...Area selection unit, 15...Parallax shift amount calculation unit, 16...Parallax shift correction unit.

Claims (13)

a first imaging unit that acquires a first captured image;
a second imaging unit disposed at a position spaced a predetermined distance from the first imaging unit and configured to acquire a second captured image;
a parallax calculation unit that calculates parallax in each region of the first captured image and the second captured image;
an area selection unit that selects an area from the first captured image or the second captured image, the area having a reliability of the disparity value equal to or greater than a threshold;
a parallax shift amount calculation unit that compares the parallax value in the region selected by the region selection unit with an ideal value of parallax to be obtained in the region, and calculates the parallax shift amount for each region;
a parallax displacement correction unit that corrects parallax displacement in the first imaging unit and the second imaging unit using a correction amount that is determined in accordance with the parallax displacement amount of each of the regions ,
When the number of regions selected by the region selection unit is less than a predetermined number, the parallax displacement correction unit stops the parallax displacement correction.
An imaging device characterized by: the parallax calculation unit calculates parallax by performing template matching using the first captured image as a base image and the second captured image as a reference image;
The imaging device according to claim 1 , wherein the region selection unit selects the region based on a prominence that is a difference between a minimum value of a peak in a graph of the similarity obtained in the template matching and a reference value. The imaging device of claim 2, wherein the reference value is the average value of the similarity in an area excluding the vicinity of the peak. The imaging device of claim 2, wherein the reference value is the minimum value of a peak other than the peak. The imaging device of claim 1, wherein the parallax displacement correction unit calculates a representative value of the parallax displacement amount calculated for each region and calculates the parallax displacement correction amount based on the representative value. The imaging device of claim 1, wherein the region selection unit selects the region based on the magnitude of the difference in brightness between the region of interest in the first captured image or the second captured image and a region corresponding to the region of interest. acquiring a first captured image captured by a first imaging unit and a second captured image captured by a second imaging unit disposed at a predetermined distance from the first imaging unit;
calculating a parallax between each region of the first captured image and the second captured image;
selecting an area from the first captured image or the second captured image, the area having a reliability of the disparity value equal to or greater than a threshold value;
a step of comparing the parallax value in the selected region with an ideal value of parallax to be obtained in the region, and determining the parallax displacement amount for each region;
correcting the parallax displacement in the first imaging unit and the second imaging unit by a correction amount determined in accordance with the parallax displacement amount of each of the regions,
When the number of regions selected in the region selection step is less than a predetermined number, the parallax displacement correction is stopped.
Parallax correction method. the step of calculating the parallax calculates the parallax by performing template matching using the first captured image as a base image and the second captured image as a reference image,
8. The parallax displacement correction method according to claim 7 , wherein the step of selecting the region selects the region based on a prominence, which is a difference between a minimum value of a peak in a graph of the similarity obtained in the template matching and a reference value. 9. The parallax displacement correcting method according to claim 8 , wherein the reference value is an average value of the similarity in a region excluding a region near the peak. The parallax displacement correcting method according to claim 8 , wherein the reference value is a minimum value of a peak other than the peak. 8. The parallax displacement correcting method according to claim 7 , wherein the correction of the parallax displacement comprises calculating a representative value of the parallax displacement amounts calculated in each region, and calculating a parallax displacement correction amount in accordance with the representative value. The parallax displacement correction method according to claim 7 , wherein the step of selecting the region selects the region based on a magnitude of a luminance difference between a region of interest in the first captured image or the second captured image and a region corresponding to the region of interest. acquiring a first captured image captured by a first imaging unit and a second captured image captured by a second imaging unit disposed at a predetermined distance from the first imaging unit;
calculating a parallax between each region of the first captured image and the second captured image;
selecting an area from the first captured image or the second captured image, the area having a reliability of the disparity value equal to or greater than a threshold value;
a step of comparing the parallax value in the selected region with an ideal value of parallax to be obtained in the region, and determining the parallax displacement amount for each region;
correcting the parallax displacement in the first imaging unit and the second imaging unit by a correction amount determined in accordance with the parallax displacement amount of each of the regions ,
When the number of regions selected in the region selection step is less than a predetermined number, the parallax displacement correction is stopped.
Parallax correction program.