JP6439592B2 - Falling object detection device - Google Patents

Description

  The present invention relates to a technique for detecting falling objects on a road from an image.

  Conventionally, by detecting a fallen object by comparing an image acquired by an in-vehicle camera with a background image showing a steady road condition where there is no unsteady obstacle such as another vehicle or a fallen object, A technique for obtaining three-dimensional information such as a falling object from parallax (detection direction difference) information obtained by a compound eye stereo method is known (see, for example, Patent Document 1).

JP 2012-216125 A

  However, when detecting a fallen object using the background image, the road environment changes due to secular change or construction, and if there is a difference between the road condition shown in the background image and the actual road condition, it will be detected. There was a problem that accuracy deteriorated.

In addition, there is a problem that the calculation of three-dimensional information by the stereo method becomes more difficult as the parallax is smaller, that is, as the distance to the object is longer.
The present invention has been made in view of these problems, and an object of the present invention is to provide a technique for detecting a fallen object on a road with stable accuracy regardless of changes in the road environment.

  The fallen object detection device of the present invention includes a road extraction unit, a section extraction unit, and a fallen object determination unit. The road extraction unit extracts a road area where the road is imaged from the image of the in-vehicle camera. The section extraction unit obtains a similarity with a standard road surface template for each section obtained by regularly dividing the road area extracted by the road extraction unit, and this similarity is set to a lower limit allowable Extract dissimilar partitions that are less than the value. The falling object determination unit determines the presence of a falling object from the distribution of dissimilar sections extracted by the section extraction unit.

  The standard road surface is a portion of the road surface where no fallen object or road surface sign exists, and a section where the fallen object or road surface sign exists is extracted as a dissimilar section. In addition, when the road surface marker is present, dissimilar sections are detected over a wider range than when a falling object is present. By utilizing such distribution characteristics of dissimilar sections, it is possible to determine the presence of a fallen object.

  The standard road surface template is less susceptible to changes in the road environment compared to a background image that represents a wider range of road surface conditions. Can be done with precision.

  In addition, the code | symbol in the parenthesis described in the claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: The technical scope of this invention is limited is not.

It is a block diagram which shows the structure of a driving assistance system. It is a flowchart of a falling object detection process. It is explanatory drawing which shows the outline of a white line detection and a road parameter, (a) The detection point of the edge based on a white line, (b) The definition of a pair edge, (c) The relationship between the locus | trajectory of the pair edge showing a white line, and a road parameter Show. It is explanatory drawing which shows a template area | region and a falling object detection area, (a) is the figure which projected the camera image on the horizontal surface, (b) is the image figure on an edge image. It is explanatory drawing regarding the production | generation of a template, (a) is the figure which projected the camera image on the horizontal surface, (b) is the image figure on an edge image. It is explanatory drawing which showed the determination result of a falling object detection area map and a dissimilar area, (a) is a case where a road surface marker exists, (b) is a case where a falling object etc. exist on a road surface.

Embodiments to which the present invention is applied will be described below with reference to the drawings.
[1. Constitution]
As shown in FIG. 1, the driving support system 1 includes an in-vehicle camera (hereinafter also simply referred to as “camera”) 2, a falling object detection device 3, and a vehicle control unit 4. Below, the vehicle carrying the driving assistance system 1 is called own vehicle.

  The camera 2 is a monocular camera configured using a known image sensor such as a CCD image sensor or a CMOS image sensor, and is installed so that the captured image includes at least a road surface in front of the vehicle.

  The vehicle control unit 4 executes various vehicle controls for driving support based on the size of the fallen object detected by the fallen object detection device 3 and the three-dimensional information representing the relative positional relationship with the host vehicle. . Specifically, the engine ECU includes an engine ECU that controls the engine, a brake ECU that controls the braking force, and a steering ECU that controls the steering angle.

  The falling object detection device 3 is a known computer including at least a CPU 31, a ROM 32, and a RAM 33, and various functions are realized by the CPU executing programs stored in the ROM.

[2. processing]
Next, the falling object detection process executed by the falling object detection device 3 will be described with reference to the flowchart of FIG. This process is repeatedly executed at a cycle in which the camera 2 performs imaging.

  When this process is activated, the CPU 31 acquires an image from the camera 2 in S110. In subsequent S120, a white line drawn on the road is detected from the image. In that case, as shown to Fig.3 (a), the acquired image is converted into the image projected on the horizontal surface, and it processes using this projection image. Specifically, the white line is detected by obtaining a differential value of the pixel value while scanning in the vehicle width direction, that is, both the left and right directions of the screen, with the front direction of the host vehicle as a base point, and as shown in FIG. A pair edge is detected when the interval between the edge and the falling edge is within a predetermined range corresponding to the width of the white line. A white line is detected from the locus of the pair edge obtained by repeating such detection of the pair edge while shifting the position in the vertical direction of the image shown in FIG. 3A, that is, the depth direction of the acquired image (FIG. 3). (See (c)). Since such a white line detection method is well known, further explanation is omitted. In addition, the area | region enclosed with the white line located in the outermost both ends detected is hereafter called a road area | region.

  In subsequent S130, road parameters are calculated based on the result of white line detection in S120. Specifically, as shown in FIG. 3C, the road parameters are determined by the vehicle running from the lane width, which is the width of the lane sandwiched between the white lines, the curvature of the white line (that is, the road), and the installation position of the camera 2. An offset representing the distance to the center position in the width direction of the middle lane and a yaw angle representing the angle formed by the straight line direction of the vehicle and the tangential direction of the white line (ie, road) are obtained.

  In subsequent S140, the road area extracted in S120 is converted into a straight road area using the road parameters obtained in S130, and a template extraction area and a fallen object detection area are extracted from the converted road area. Specifically, as shown in FIGS. 4 (a) and 4 (b), the template extraction region covers the entire lane width where the distance from the host vehicle is within a preset allowable range (near 20 m in the figure). It is a rectangular area. The allowable range is desirably set within this range, with the lower limit just before the host vehicle and the inter-vehicle distance from the preceding vehicle when traveling safely following the upper limit. The falling object detection area is a rectangular area over the entire lane width located at a distance that can be handled by the driver and the vehicle-mounted system when the falling object is detected (in the drawing, around 30 to 50 m of labor of the own vehicle). . Note that the fallen object detection area may be a rectangular area over the entire lane width, similarly to the template extraction area.

  In subsequent S150, a template representing a standard condition of the road surface is generated based on the image of the template extraction area. Specifically, a template is generated by cutting out a portion where the number of edges per unit area (for example, template size) to be detected is equal to or less than a preset upper limit value from the edge image of the template region. At this time, an area belonging to the own lane is preferentially searched in the template extraction area, and if there is no appropriate place, an area belonging to the adjacent lane is also searched. In addition, when an appropriate place cannot be found also in an adjacent lane, you may make it search, for example by raising an upper limit threshold value or expanding a template extraction area. In other words, as shown in FIGS. 5 (a) and 5 (b), since there are many edges detected in cracks caused by aging and tars used for repair, a comparison avoiding such parts is performed. A uniform road surface image is used as a template.

  In subsequent S160, a falling object detection area map in which the falling object detection areas are regularly divided is generated. Specifically, as shown in FIGS. 6A and 6B, the detection area map is generated so that each divided section has the same size as the template.

  In subsequent S170, the similarity with the template generated in the previous S150 is calculated for each section, and a section whose similarity is less than a preset lower limit allowable value is extracted as a dissimilar section. The degree of similarity is obtained by using a known method such as, for example, summing the differences obtained for each corresponding pixel and taking the reciprocal thereof.

  In subsequent S180, the presence of a falling object is determined based on the distribution of dissimilar sections on the falling object detection area map. Specifically, if dissimilar sections are adjacent, the number of sections that constitute each candidate area, with these combined areas and a single dissimilar section with no adjacent one as one candidate area. Is less than or equal to a preset falling object determination threshold, it is determined that there is a falling object in the candidate area. On the other hand, when the number of sections constituting the candidate area is larger than the falling object determination threshold, it is determined that the road surface marker is present in the candidate area.

  In subsequent S190, three-dimensional information representing the size of the falling object and the relative position with respect to the host vehicle is calculated for each of the candidate areas determined to have a falling object. The relative position is calculated from the position of the falling object detection area with respect to the host vehicle and the position of the dissimilar section constituting the candidate area in the falling object detection area.

In continuing S200, the calculation result of the three-dimensional information of the fallen object in S190 is output to the vehicle control part 4, and this process is complete | finished.
[3. Action / Effect]
As described above, in the falling object detection device 3, as shown in FIG. 6, the section where the road surface marker or the falling object exists, that is, the section indicated by the oblique lines in the figure has a low similarity to the template. Judged as dissimilar section. However, in the case of a road surface sign, since the road sign is drawn over a wide area of the road surface, as shown in FIG. In such a case, since the fallen object is smaller than the road sign, as shown in FIG. 6B, the number of sections constituting the candidate area is reduced. Thereby, a road surface sign and a fallen object can be distinguished and detected.

  In addition, since the fallen object detection area has a relatively wide range, the relative positional relationship with the host vehicle can be easily obtained from the image by using the road parameters obtained from the white line detection. Since each section is obtained by regularly dividing the falling object detection area, the relative position in the falling object detection area can be easily specified. That is, in the falling object detection device 3, by combining both of these, even when the distance from the own vehicle to the falling object is far enough to degrade the accuracy by the stereo method, the three-dimensional information of the falling object is stabilized. It can be determined with accuracy.

  Since the fallen object detection device 3 detects a fallen object using a template representing a standard road condition instead of a background image that covers a wide area, even if the surrounding environment changes due to construction or the like, the impact is affected. It is difficult to receive and stable detection can be realized.

  Since the fallen object detection device 3 uses a template cut out from the image each time, even when the road surface changes due to solar radiation, changes in lighting, or deterioration over time, the situation change can be dealt with at any time. In this respect, stable detection can be realized.

  In the falling object detection device 3, since the image of the monocular camera can be used, not only the driving support system 1 can be configured at low cost, but also the vehicle having only the monocular camera can be applied.

[4. Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention can take a various form, without being limited to the said embodiment.

  (1) In the above embodiment, the road parameter is obtained from the image. However, the present invention is not limited to this, and a road parameter acquired from map data or the like based on the current position of the host vehicle may be used.

  (2) The functions of one component in the above embodiment may be distributed to a plurality of components, or the functions of a plurality of components may be integrated into one component. Further, at least a part of the configuration of the above embodiment may be replaced with a known configuration having the same function. Moreover, you may abbreviate | omit a part of structure of the said embodiment. Further, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claim are embodiment of this invention.

  (3) In addition to the above-described falling object detection device, various systems including the falling object detection device as components, a program for causing a computer to function as the falling object detection device, a medium on which the program is recorded, and a falling object detection method It can also be realized in various forms.

  DESCRIPTION OF SYMBOLS 1 ... Driving assistance system, 2 ... Camera, 3 ... Falling object detection apparatus, 4 ... Vehicle control part.

Claims (9)

A road extraction unit (S120, S130) for extracting a road area where a road is imaged from the image of the in-vehicle camera (2);
For each section in which the road area extracted by the road extraction unit is regularly divided, a similarity with a template representing a standard road surface condition is obtained, and the similarity is less than a preset lower limit allowable value. A section extraction unit (S140, S160, S170) for extracting dissimilar sections that are the sections;
A fallen object determination unit (S180) that determines the presence of a fallen object from the distribution of dissimilar sections extracted by the section extraction unit;
A fallen object detection apparatus comprising:   The fallen object detection device according to claim 1, further comprising a template extraction unit (S140, S150) that extracts the template from an image of a road area extracted by the road extraction unit.   The template extracting unit extracts, as the template, a part where the number of edges per unit area detected in the road region image is equal to or less than a preset upper limit allowable value. The falling object detection device described in 1.   The template extraction unit extracts the template from the template extraction area, with the road area within a tolerance range set in advance as the distance from the host vehicle, which is a vehicle on which the in-vehicle camera is mounted, as a template extraction area. The fallen object detection device according to claim 2 or claim 3, wherein   5. The falling object detection device according to claim 4, wherein the template extraction unit preferentially extracts an area belonging to a lane in which the host vehicle is traveling in the template extraction area. .   6. The section extracting unit generates road parameters representing the shape of the road region, converts the road region into a straight road using the road parameter, and processes the road region. The falling object detection apparatus of any one of Claims 1.   The fallen object determination unit uses a combination of the dissimilar sections when the dissimilar sections are adjacent to each other and a single dissimilar section with no adjacent sections as candidate sections, and the size of the candidate section is The fallen object detection device according to any one of claims 1 to 6, wherein the fallen object is determined to be present when the fallen object determination threshold value is not more than a preset value. The fallen object detection apparatus according to claim 7 , further comprising an information calculation unit (S190) that obtains three-dimensional information of the fallen object from the position of the candidate section where it is determined that the fallen object exists.   The fallen object detection device according to any one of claims 1 to 8, wherein the on-vehicle camera is a monocular camera.