JP4670528B2 - Imaging device deviation detection method, imaging device deviation correction method, and imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus displacement detection method for detecting an attachment displacement of an in-vehicle camera to a vehicle body, and an imaging apparatus displacement correction method for correcting an imaging apparatus displacement based on a detection result of the imaging apparatus displacement detection method. In particular, the present invention relates to an imaging apparatus deviation detection method, an imaging apparatus deviation correction method, and an imaging apparatus that are suitable when deviation occurs in the roll direction.

2. Description of the Related Art Conventionally, an apparatus that corrects a deviation of a camera optical axis of an in-vehicle camera based on a vanishing point of a captured image of the in-vehicle camera has been proposed.
JP 2002-259995 A

In the conventional apparatus, the deviation in the yaw direction and the pitch direction of the in-vehicle camera is corrected based on the vanishing point. However, the conventional apparatus cannot detect a deviation in the roll direction of the in-vehicle camera and cannot correct the deviation. For example, if the in-vehicle camera has a deviation in the roll direction, it becomes difficult to accurately identify the position of the vehicle in the traveling lane from the captured image.
The present invention has been made in view of the above problems, and is based on an imaging device that detects a shift in the roll direction of an in-vehicle camera, a shift detection method for the imaging device, and a detection result of the shift detection method for the imaging device. An object of the present invention is to provide a deviation correction method for an imaging apparatus that corrects a deviation in the roll direction of the imaging apparatus.

The shift detection method of the imaging apparatus according to claim 1 is a shift detection method of the imaging apparatus that detects a shift in a roll direction of an in-vehicle imaging apparatus that is mounted so as to image the front of the host vehicle.
In this imaging apparatus shift detection method, an intersection formed far from the host vehicle is obtained for a pair of lane marking lines specified in a captured image obtained by the imaging apparatus, and the host vehicle is within the lane. For each of the captured images obtained when moving in the left-right direction, calculating a linear approximation formula from the obtained positions of the plurality of intersections, and based on the inclination of the linear approximation formula with respect to the captured image, the imaging The inclination of the roll direction of the apparatus is detected.

  Here, when a shift in the roll direction has occurred in the imaging device, the intersection point in the captured image obtained when the host vehicle moves in the left-right direction is the size of the shift and the lane in the lane. It can be obtained as a position corresponding to the own vehicle position. Then, a linear approximation formula is calculated from the positions of such a plurality of intersections, and the roll-direction tilt of the imaging device is detected based on the slope of the linear approximation formula with respect to the captured image.

  According to the shift detection method of the imaging apparatus according to the first aspect, it is possible to accurately detect the shift in the roll direction of the imaging apparatus.

The best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described in detail with reference to the drawings.
FIG. 1 and FIG. 2 are block diagrams showing an embodiment of a white line detection device 10 to which an imaging apparatus deviation detection method and an imaging apparatus deviation correction method according to the present invention are applied, and FIGS. (C) is a side view, a plan view, and a front view showing the mounting position of the camera 11 in the host vehicle 100, and FIGS. 4 and 5 are flowcharts for explaining the operation of the white line detection device 10. FIG. 6 is an example of a captured image of the camera 11, and is a diagram illustrating an intersection, a white line model, and a white line candidate point detection area formed far from the left and right white lines obtained with respect to the captured image. It is the figure used for description of the principal part.

As shown in FIG. 1, the white line detection device 10 includes a camera 11, an image processing device 12, a microcomputer 13, a memory 14, and a sensor 15. The white line detection device 10 includes a vehicle control device 1 as an external device. And an alarm device 2 are connected.
As shown in FIG. 3, the camera 11 is attached to the vicinity of the center in the vehicle width direction on the upper part of the front window in the passenger compartment, and images the road scenery in front of the vehicle. In the camera 11, the angle formed between the optical axis of the lens and the horizontal plane (ground) in the front-rear direction is a pitch angle α, and the angle formed between the optical axis of the lens and the horizontal plane (ground) in the left-right direction is a roll angle γ. The lens is mounted so that the angle formed by the optical axis of the lens and the vehicle center line is the yaw angle β within the same horizontal plane.

The image processing device 12 processes an image captured by the camera 11 and detects a white line on the road.
The microcomputer 13 uses a plurality of parameters indicating the road shape and vehicle behavior to represent the shape of the road white line as a mathematical model, and detects the white line (lane line) obtained from the image captured by the camera 11 and the road model. By updating the parameters so that they match, the white line is detected and the road shape is recognized. Further, with respect to the left and right white lines obtained in the captured image, the position of the intersection (disappearing in some cases) formed at a distance from the host vehicle and the parameters of the horizontal line model are calculated.

  The vanishing point is a line parallel to the optical axis of the camera 11, that is, the left and right white lines, for example, the own vehicle (in this case, the optical axis of the camera 11 and the traveling direction of the own vehicle are parallel). Is the intersection formed at a distance in the captured image with respect to the left and right white lines. In other words, even if it is possible to specify an intersection formed far from the left and right white lines in the captured image, if the optical axis of the camera 11 is not parallel to the left and right white lines, for example, the own vehicle If the vehicle does not travel parallel to the left and right white lines, the intersection point cannot be said to be a vanishing point.

  The memory 14 is a storage device that stores road model parameters and the like, and the sensor 15 is a device that detects a vehicle speed, a steering angle, and the like representing the behavior of the vehicle. Moreover, the vehicle control apparatus 1 performs control of steering, an accelerator, a brake, etc. based on an output result. The warning device 2 is a device that issues a warning such as lane departure based on the output signal.

  FIG. 2 is a functional representation of the white line detection device 10. When this is compared with the hardware representation shown in FIG. 1, the camera 11 and the image processing device 12 correspond to the imaging means 21. The memory 14 corresponds to the road parameter storage means 27 and the white line intersection coordinate storage means 30, and the microcomputer 13 corresponds to the coordinate conversion means 22, the white line detection area setting means 23, the white line candidate point detection means 24, the road parameter calculation means 25, and the signal. It corresponds to the output means 26, the white line intersection calculation means 28, the running state detection means 29, and the roll angle calculation means 31.

Next, a processing procedure for white line detection performed by the microcomputer 13 will be described with reference to FIGS.
First, in step S1, various parameters are initially set.
(1) First, parameters representing road shapes and vehicle behavior (hereinafter also simply referred to as road parameters) are initially set.
Here, a white line 43 in a captured image having a screen coordinate system xy as shown in FIG. 6 is modeled, and the modeled white line (white line model) 41 is expressed by the following formula (1) using road parameters. Is expressed as a mathematical expression in the screen coordinate system xy.
x = (a + ie) (y−d) + b / (y−d) + c (1)

In the equation (1), a to e are road parameters. When the height of the camera 11 from the road surface is constant, each road parameter represents the following road and white line shape or vehicle behavior.
That is, a is the lateral displacement of the host vehicle in the travel lane, b is the curvature of the road, c is the yaw angle of the host vehicle (the optical axis of the camera 11) with respect to the road, and d is the host vehicle (the optical axis of the camera 11). The pitch angle with respect to the travel path, e, represents the lane width.

Incidentally, since the shape of the road and the white line and the vehicle behavior are unknown in the initial state, for example, a value corresponding to the median value is set as an initial value for each road parameter. That is, the center of the lane is set to the lateral deviation (position) a in the lane, and the yaw angle β of the camera 11 is set to the yaw angle c with respect to the lane. Further, a pitch angle in a stopped state is set as the pitch angle α with respect to the traveling road, and the lane width of the expressway indicated in the road structure ordinance is set as the lane width e.
The road parameter may be initially set based on a value indicating the behavior of the vehicle detected by the sensor 15. For example, when the steering is steered to the right or left in the initial state, it is determined that the vehicle is traveling on a road having a curvature corresponding to the steering angle, and a value corresponding to the steering angle is set in the parameter b. You may do it.

(2) Next, the parameters of the straight line model used for estimating the roll angle are initialized in step S21 described later.
As shown in FIG. 7, the straight line model 51 is a characteristic of a set of intersections 52 formed far away with respect to a specific pair of left and right dividing lines (dotted lines shown in the figure) in the captured image (e.g., corresponding to the horizon). ) And is expressed as a linear approximation formula in the screen coordinate system xy by the following formula (2).
y = gx + h (2)
Here, the parameter g indicates the inclination of the linear model 51 with respect to the screen coordinate system xy of the captured image, that is, the roll angle of the linear model 15 with respect to the road. The parameter h indicates the intercept of the linear model 51 in the screen coordinate system xy.

  The parameter g defined in this way is set as an initial value corresponding to the median value, assuming that the roll angle of the camera 11 with respect to the road is unknown in the initial state. That is, for example, the parameter g is initially set to a roll angle γ = 0 indicating that the camera 11 and the traveling road surface are parallel. The parameter h is also set as an initial value corresponding to the median value, assuming that the pitch angle of the camera 11 with respect to the road is unknown in the initial state. That is, for example, the parameter h is initially set based on the pitch angle α of the camera 11 when the vehicle is stopped.

  Subsequently, in step S2, as shown in FIG. 6, initial setting of the detection area 42 for detecting white line candidate points is performed. In the initial state, it is expected that there is a large gap between the white line model in which the initial value is set for the road parameter and the road white line on the actual screen. Therefore, it is desirable to set a region as large as possible. In the example shown in FIG. 6, a total of 12 white line candidate point detection areas are set for each of the left and right white lines 43.

In addition, when the white line 43 actually formed by the previous process has already been detected, it is considered that the difference between the white line 43 and the white line model 41 is small. Therefore, it is preferable to set a region as small as possible. This is preferable because the possibility of erroneous detection of things other than the white line 43 is reduced and the processing speed can be shortened.
Subsequently, in step S3, an image captured by the camera 11 and processed by the image processing device 12 is input. In subsequent step S4, a white line candidate point detection region is set on the input road image. At this time, based on the white line candidate point detection region set in step S2 or step S25 described later, and the white line model based on the road parameters calculated in step S1 or steps S12, S14, and S15 described later, A white line candidate point detection region is set so that the white line model obtained by the processing is the center of the region.

It should be noted that the white line candidate point detection region may be set at a position offset in the change direction of the white line model based on the past change of the white line model.
Subsequently, in step S5, white line candidate points are detected in the white line candidate point detection area set as described above.
The white line candidate point is detected by first generating a differential image by passing the input image through a Sobel filter or the like.
Next, for all line segments formed by connecting one point at the upper base and one point at the lower base of the white line candidate point detection region, the number of pixels having a pixel density equal to or higher than a predetermined value is measured. Furthermore, the line segment with the largest number of pixels having a density equal to or higher than a predetermined value among all the line segments is defined as a detection straight line, and the coordinate value of the start point of the line segment (one point at the upper base of the white line candidate point detection area in the image) The output value of the white line candidate point.

  At this time, if the number of pixels having a density equal to or higher than a predetermined value on the detected straight line is smaller than a predetermined ratio with respect to the length of the white line candidate point detection area, it is considered that no white line candidate point has been detected. For example, if the length of the white line candidate point detection area is 15 pixels and a pixel having a density equal to or higher than a predetermined value is detected 1/2 or more (that is, 8 pixels or more), the white line candidate point detection area is detected as a white line candidate point detected. In the case where the number of pixels on the line segment having the largest number of pixels having a density equal to or higher than a predetermined value is 7 pixels or less, it is considered that no white line candidate point has been detected in the white line candidate point detection region. On the other hand, in the case of 9 pixels, it is assumed that a white line candidate point has been detected, and the coordinate value of the start point of the line segment (one point at the top of the white line candidate point detection area in the image) is taken as the detection result, and the end point at that time The coordinates of (one point at the bottom of the white line candidate point detection area in the image) are stored.

  The above processing is sequentially executed for all white line candidate point detection areas from far to near. At this time, the predetermined ratio with respect to the length of the white line candidate point detection region for determining whether or not the white line candidate point is detected may be the same value for all the white line candidate point detection regions, or You may set for every white line candidate point detection area. Also, the predetermined value of the density may be the same value for all white line candidate point detection areas, or may be set for each white line candidate point detection area.

  In this example, the coordinates of the upper base in the white line candidate point detection area are used as the output result of the white line candidate point, and the processing order of the white line candidate point detection area is performed from a distance to a near field. The present invention is not limited to this. For example, the coordinates of the lower base of the detection region may be used as the output result of the white line candidate points, and the processing order of the detection region may be performed from near to far.

Subsequently, in step S6, it is confirmed whether or not the number of white line candidate points detected in all white line candidate point detection areas is equal to or larger than a predetermined value. The process returns to step S2 to initialize the white line candidate point detection area. On the other hand, if a white line candidate point is detected at a predetermined value or more, the process proceeds to step S7.
In addition, the process of said step S2-step S6 has implement | achieved the detection means to detect a pair of lane marking from the image imaged with the imaging means.

In step S7, a deviation amount between the detected white line candidate point and the point on the white line model obtained in the previous process is calculated for each point.
Subsequently, in step S8, the fluctuation amounts Δa to Δe of the road parameters are calculated based on the deviation amount of each point. For example, a method disclosed in JP-A-8-5388 can be used as a method for calculating the fluctuation amount.

Subsequently, in step S9, the road parameters a to e are corrected based on the calculated road parameter fluctuation amounts Δa to Δe. For example, in the case of the white line model shown in the equation (1), the road parameters a to e are corrected by the following equation (3).
a = a + Δa, b = b + Δb, c = c + Δc, d = d + Δd, e = e + Δe (3)
Subsequently, in step S10, it is confirmed whether or not the parameter representing the road shape is normal among the road parameters. If not, the process proceeds to step S14 to initialize the parameter representing the road shape.

  In the white line model expressed by the equation (1), the parameter b reflects the road curvature and the parameter e reflects the lane width. Therefore, the parameter b is initialized when the road curvature estimated from the parameter b is determined from the vehicle behavior detection value detected by the sensor 15 and becomes a curvature that cannot be the road that is currently running. Similarly, if the lane width estimated from the parameter e is determined from the vehicle behavior detection value detected by the sensor 15 and becomes a lane width that cannot be the road that is currently running, the parameter e is initialized. To do.

If the parameter representing the road shape is normal, the process proceeds to step S11. This time, it is confirmed whether the parameter representing the vehicle behavior is normal. If not, the process proceeds to step S15 to set the parameter representing the vehicle behavior. initialize.
In the white line model expressed by the equation (1), the parameter a reflects the lateral displacement in the lane, the parameter c reflects the yaw angle with respect to the road surface, and the parameter d reflects the pitch angle with respect to the road surface. Therefore, when the lateral deviation estimated from the parameter a is judged from the estimated value of the road curvature or the vehicle behavior detected value by the sensor 15, the lateral deviation which cannot be the road currently being traveled is obtained. Initialize parameter a. Similarly, when the yaw angle estimated from the parameter c is judged from the estimated value of the road curvature or the vehicle behavior detected value by the sensor 15 and becomes an angle that cannot be the road that is currently running, the parameter c Is initialized. Further, when the pitch angle estimated from the parameter d is judged from the vehicle behavior detection value by the sensor 15 and becomes an angle that cannot be the road that is currently running, the parameter d is initialized.

When the parameter representing the vehicle behavior is normal, the process proceeds to step S12, and the road parameters a to e corrected in step S9 are stored in the memory 14 as road parameters of a new white line model.
Subsequently, in step S13, the road shape is estimated based on the new road parameters, and is output to the vehicle control device 1 and the warning device 2.
Subsequently, in step S16 shown in FIG. 5, it is determined whether or not yaw rate has occurred in the host vehicle. Specifically, the curvature of the lane (particularly the lane at the current travel position) is less than a predetermined value (experimental value, experience value, etc.), and the yaw rate detected by the sensor 15 is less than a predetermined value (experimental value, experience value, etc.). It is determined whether or not. If this is satisfied, the process proceeds to step S17. Otherwise, the process proceeds to step S24.

In step S17, it is determined whether the host vehicle is accelerating or decelerating. Specifically, it is determined whether the acceleration / deceleration (absolute value) of the host vehicle is less than a predetermined value (such as an experimental value or an experience value). If the acceleration / deceleration of the host vehicle is less than the predetermined value, the process proceeds to step S18. If the acceleration / deceleration of the host vehicle is greater than or equal to the predetermined value, the process proceeds to step S24.
In step S18, it is determined whether or not learning (acquisition) of the coordinate value of the intersection of the left and right white lines is possible. Specifically, it is determined whether or not the road on which the host vehicle is traveling is a straight road and the vehicle travels continuously within a predetermined area near the center of the straight road for a predetermined time or longer. If this is satisfied, the process proceeds to step S19, and otherwise, the process proceeds to step S24.
Note that the processing in steps S16 to S18 is realized by the function of the traveling state detection means 29 of the microcomputer 13.

In step S19, the coordinate value of the intersection of the left and right white lines is learned (storage permitted).
Here, the intersection of the left and right white lines is an intersection (hereinafter referred to as a white line intersection) 44 obtained by connecting the left and right white lines in the captured image, as shown in FIG. For example, when the own vehicle moves in the left-right direction in the lane, the white line intersection 44 moves to the left and right in the captured image corresponding to the movement of the own vehicle. That is, if the own vehicle moves to the right in the lane, the white line intersection 44 moves to the left in the captured image, and if the own vehicle moves to the left in the lane, the white line intersection 44 moves to the right in the captured image. Move to.

In step S17, the coordinate value (x _i , y _i ) (i is an arbitrary integer) of the white line intersection 44 is obtained from the captured image obtained from the camera 11 at the current traveling position of the host vehicle. , It is stored in the memory 14.
In addition, the white line intersection mentioned here includes the intersection that can be estimated from the left and right white line portions in the vicinity of the own vehicle (for example, immediately before the own vehicle) in addition to those that can be directly detected as the intersection of the left and right white lines from the captured image. Shall. That is, since there may be a case where the intersection point cannot be directly detected from the captured image, in such a case, a virtual line extending far away is assumed based on the left and right white line portions in the vicinity of the own vehicle (for example, immediately before the own vehicle), The intersection point is estimated for the virtual line.

The coordinate value of the white line intersections in the memory 14 _(x i, _{y i)} as a condition to be newly stored, the y-coordinate value _{y i} is already y-coordinate value _{y i} of the white line intersection 44 which is stored in the memory 14 If not, the coordinate values (x _i , y _i ) of the white line intersection may be newly stored.
The processing of step S19 is realized by the function of the intersection calculation means, that is, the function of the intersection coordinate calculation means 28 of the microcomputer 13 and the function of the white line intersection coordinate storage means 30 of the memory 14.

  Subsequently, in step S20, it is determined whether or not to calculate the roll angle of the camera 11 (shift angle in the roll direction). Specifically, it is determined whether or not the number of coordinate values of the white line intersection stored in the memory 14 (hereinafter referred to as white line intersection coordinate value) is a predetermined number or more. Here, if the number of white line intersection coordinate values is equal to or greater than a predetermined number, the roll angle of the camera 11 is calculated. The process proceeds to step S21. If the number of white line intersection coordinate values is less than the predetermined number, the roll angle of the camera 11 is set. Assuming that no calculation has been made yet, the process proceeds to step S24.

In step S <b> 21, the roll angle (estimated value) of the camera 11 is calculated based on the white line intersection coordinate value stored in the memory 14.
Specifically, the straight line model (horizon model) of the above equation (2) is used for the calculation, and a plurality of white line intersection coordinate values (x ₁ , y ₁ ), (x ₂ , y ₂ ),..., (X _n , y _n ) (where n is the predetermined value) (for example, using the least squares method), The parameters g and h of the straight line model are calculated.

  Subsequently, in step S22, it is determined whether correction based on the roll angle of the camera 11 calculated in step S21 is possible. Specifically, the determination is made based on whether or not the absolute value of the parameter (slope) g of the linear model is equal to or less than a predetermined value. Here, when the absolute value of the parameter g of the linear model is equal to or smaller than the predetermined value, the roll angle of the camera 11 is corrected, and the process proceeds to step S23, and when the absolute value of the parameter g of the linear model is larger than the predetermined value, Assuming that the roll angle of the camera 11 is not corrected, the process proceeds to step S24.

Here, when the absolute value of the parameter (inclination) g of the linear model is equal to or larger than a predetermined value, the roll angle of the camera 11 is estimated by estimating that the error of the linear model is large and proceeding to step S24. Is limited to the case where the absolute value of the parameter (slope) g of the linear model is equal to or less than a predetermined value.
Note that the processing of steps S20 to S22 is realized by the function of the roll angle calculation means 31 of the microcomputer 13.

  In step S23, the roll angle of the camera 11 is corrected. Specifically, the screen coordinate system xy used for the captured image is rotated (rotated in the same direction) by the parameter g of the linear model. At this time, on the condition that the vehicle is traveling parallel to the left and right white lines, the screen is centered on the vanishing point (latest vanishing point) based on the left and right white lines in the captured image obtained at that time. The coordinate system xy is rotated by the parameter g.

That is, when the straight line model is obtained based on the white line intersection obtained in the captured image after this correction, the screen coordinate system used for processing the captured image so that the parameter (slope) g of the straight line model becomes zero. The xy state (roll direction inclination) is corrected.
By correcting the state of the screen coordinate system xy (the tilt in the roll direction), processing after the correction is performed such as white line candidate point detection based on the corrected screen coordinate system xy. Become so.
Note that the processing in step S23 is realized by the function of the correction means, that is, the coordinate conversion means 22 of the microcomputer 13.

  In step S24, it is determined whether or not the size of the white line candidate point detection area in the current process (the process after rotationally correcting the screen coordinate system xy) is optimal. Specifically, it is determined whether the white line candidate point detection area in the current process is the initial value set in step S2. If the white line candidate point detection area in the current process is the initial value set in step S2, the process proceeds to step S25, assuming that the size of the white line candidate point detection area in the current process is not optimal. If the white line candidate point detection area in the process is not the initial value set in step S2, the process from step S3 is started assuming that the size of the white line candidate point detection area in the current process is optimal.

  In step S25, the white line candidate point detection area is optimized. As described above, setting the white line candidate point detection area to be as small as possible reduces the possibility of erroneous detection of objects other than the white line, thereby improving the processing speed. In this step S25, the white line candidate point detection area is optimized so as to be as small as possible.

  As described above, the white line detection process is performed, and during this process, the host vehicle travels on a straight road (particularly, the current travel position is a straight road), particularly as a process for detecting and correcting a shift in the roll direction of the camera 11. In addition, no yaw rate is generated in the host vehicle (step S16), the host vehicle is not accelerated or decelerated (step S17), and the road on which the host vehicle is traveling is a straight road, and When the host vehicle continues to travel in a predetermined area near the center of the straight road for a predetermined time or longer (step S18), the white line intersection coordinate value is stored in the memory 14 (step S19). Then, at the timing when a predetermined number of white line intersection coordinate values are stored in the memory 14, the straight line model, particularly its parameter (slope) g is calculated (steps S20 and S21). And the inclination of the roll direction of the camera 11 is correct | amended based on the parameter (inclination) g. Specifically, the screen coordinate system xy used for processing the captured image of the camera 11 is corrected using the parameter (tilt) g.

Next, effects of the embodiment will be described.
As described above, a white line intersection formed far from the own vehicle is obtained for the left and right white lines in the captured image obtained by the camera 11, and the white line intersection is obtained when the own vehicle moves left and right in the lane. Each captured image is acquired (step S19). Then, a straight line model (linear approximation formula) is calculated from the obtained positions of the white line intersection points as a characteristic of the white line intersection points (step S20, step S21), and the parameters of the straight line model for the captured image ( The tilt of the camera 11 in the roll direction is corrected based on (tilt) g (steps S22 and S23).

  Here, when the camera 11 is misaligned in the roll direction, the white line intersection obtained at each position where the host vehicle moves in the lateral direction in the lane is changed according to each moved position in the captured image. It can be detected as the position. For example, when the camera 11 is shifted in the clockwise roll direction toward the vehicle traveling direction, the white line intersection can be detected as being higher as it is located on the right side in the captured image. As a result, when the camera 11 is displaced in the roll direction, a straight line approximation formula is obtained for the positions of a plurality of white line intersections obtained at the respective moved positions. Have a tilt. For example, if the camera 11 is shifted in the clockwise roll direction toward the vehicle traveling direction, as shown in FIG. 7, the linear approximation formula for the positions of the plurality of white line intersections 52 is Has a rising slope.

In the present invention, the characteristics as described above are discovered and used, and the deviation in the roll direction of the camera 11 is detected from the inclination of the straight line model at the position of the intersection of the white lines detected from each of the plurality of captured images. Furthermore, the deviation of the camera 11 in the roll direction is corrected based on the detection result.
In addition, as described above, when the curvature of the lane in which the host vehicle travels (especially the lane at the current travel position) is equal to or less than a predetermined value and the yaw rate of the vehicle is equal to or less than a predetermined value, A learning (acquisition) condition is set (see step S16).

  For example, when the host vehicle is traveling on a curved road, yaw rate is generated in the host vehicle, and as a result, a roll is generated in the host vehicle. When a roll is generated in the own vehicle in this way, as a result, the position of the white line intersection in the captured image is also affected by the roll generated in the own vehicle. For this reason, by setting the white line intersection learning (acquisition) condition when the host vehicle is traveling on a straight road or the yaw rate of the host vehicle is equal to or less than a predetermined value, the roll direction of the camera 11 can be accurately determined. The shift is detected.

Further, as described above, the fact that the host vehicle is not accelerating / decelerating is set as the learning (acquisition) condition of the white line intersection used for calculation of the straight line model (see step S17).
For example, when the host vehicle is accelerating / decelerating, a pitch is generated in the host vehicle, and as a result, the position of the intersection of the white lines in the captured image is also affected by the pitch generated in the host vehicle, and imaging is performed. The intersection of white lines in the image moves in the vertical direction. For this reason, when the host vehicle is accelerating / decelerating, the deviation (rolling direction) of the camera 11 is detected with high accuracy by using the white line intersection learning (acquisition) condition.

In addition, as described above, when the road on which the host vehicle is traveling is a straight road and the vehicle travels continuously within a predetermined area near the center of the straight road for a predetermined time or longer, a straight line model is calculated. This is a learning (acquisition) condition for the white line intersection used in the above (see step S18).
For example, if the road on which the host vehicle is traveling is a curved road, or if it is traveling on the edge of the straight road even if it is a straight road, it is difficult to identify the white line based on the captured image. As a result, it becomes difficult to specify the white line intersection accurately. For this reason, when the road on which the host vehicle is traveling is a straight road and the vehicle continues to travel within a predetermined area near the center of the left and right white lines in the straight road for more than a predetermined time, a white line intersection By using the learning (acquisition) condition, the shift in the roll direction of the camera 11 is accurately detected.

As described above, as a condition for newly storing the white line intersection coordinate value (x _i , y _i ) in the memory 14, the y coordinate value y _i is the y of the white line intersection 44 already stored in the memory 14. If the coordinate value y _i is not present, the white line intersection coordinate value (x _i , y _i ) is newly stored. As a result, a captured image obtained when the host vehicle moves in the left-right direction within the left and right white lines is selected, and the white line intersection coordinate values (x _i , y _i ) in the selected captured images are stored in the memory 14. Storing. This makes it possible to obtain many samples that are optimal for calculating a straight line model.

The embodiments of the present invention have been described above. However, the present invention is not limited to being realized as the embodiment.
That is, in the embodiment, the condition for learning (acquiring) the white line intersection is that the host vehicle is traveling on a straight road. However, it is not limited to this. That is, as a direct condition, it may be a condition that no roll is generated in the host vehicle. In this case, a roll generated in the vehicle is detected by a roll angle sensor provided in the host vehicle.

  Moreover, in the said embodiment, the case where the left-right white line which obtains a white line intersection is formed on the track of 1 lane as shown in FIG. 6 is demonstrated. However, it is not limited to this. That is, the present invention is based on the premise that a pair of lane markings is specified from the captured image, and therefore, any pair of lane markings that can be specified as being formed on the road may be used. . For example, in the case of an opposite two-lane road having a center line, the lane markings formed at both ends of the road are defined as a specific pair of lane markings, and an intersection is obtained based on the pair of lane markings. Anyway. Moreover, you may make it obtain an intersection based on a pair of lane marking formed as the thing of an adjacent lane. In addition, for this reason, it is not a condition that the host vehicle is necessarily traveling in the lane marking for obtaining the intersection.

1 is a block diagram illustrating an embodiment of a white line detection device to which a deviation detection method for an imaging apparatus and a deviation correction method for an imaging apparatus according to the present invention are applied. It is a block diagram functionally expressing the white line detection device. It is a figure which shows the attachment position of a camera. It is the first half flowchart which shows the operation | movement order in the said embodiment. It is a latter half flowchart which shows the operation | movement order in the said embodiment. It is a figure which shows a white line intersection, a white line model, and a white line candidate point detection area. It is the figure used for description of a straight line model (straight line approximation formula).

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Position detection apparatus 11 Camera 13 Microcomputer 14 Memory 21 Imaging means 22 Coordinate conversion means 28 White line intersection calculation means 29 Running state etc. detection means 30 White line intersection coordinate storage means 31 Roll angle calculation means 44 White line intersection 51 Linear model

Claims (7)

An imaging apparatus deviation detection method for detecting a deviation in a roll direction of an in-vehicle imaging apparatus attached to image the front of the host vehicle,
Obtain an intersection formed far from the own vehicle for the pair of lane markings specified in the captured image obtained by the imaging device, and obtain the intersection when the own vehicle moves in the left-right direction in the lane. Obtained for each of the captured images, calculating a linear approximation formula from the acquired positions of the plurality of intersections, and detecting the roll-direction tilt of the imaging device based on the slope of the linear approximation formula for the captured image. A method for detecting a deviation of an imaging apparatus.   The straight line approximation formula is calculated based on the intersection obtained from the captured image obtained by the imaging device when the vehicle does not tilt in the roll direction. A method for detecting deviation of an imaging apparatus.   The imaging apparatus according to claim 1, wherein the straight-line approximation formula is calculated based on the intersection obtained from a captured image obtained by the imaging apparatus while the host vehicle is traveling on a straight road. Deviation detection method.   4. The linear approximation formula is calculated based on the intersection obtained from a captured image obtained by the imaging device when no yaw rate occurs in the host vehicle. 2. A method for detecting a shift of an imaging apparatus according to item 1.   5. The linear approximation formula is calculated based on the intersection obtained from a captured image obtained by the imaging device when the host vehicle is not accelerating or decelerating. The shift detection method of the imaging device according to the item.   6. The imaging apparatus deviation detection method according to claim 1, wherein the linear approximation formula is calculated based on a predetermined number or more of the intersections.   The said imaging device or its captured image is adjusted so that the inclination of the roll direction detected with the displacement detection method of the imaging device of any one of Claims 1 thru | or 6 may become horizontal, The said imaging device A method for correcting a deviation of an imaging apparatus, wherein the inclination in the roll direction is corrected.

Images