JP4033017B2 - Ambient monitoring device for vehicles - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus for monitoring the surroundings of a host vehicle using an in-vehicle camera.
[0002]
[Prior art]
2. Description of the Related Art An in-vehicle camera device that uses an in-vehicle camera to capture an image of the vicinity of the left front wheel of a vehicle that becomes a blind spot for a driver and displays the image on a display inside the vehicle is known (see Patent Document 1). According to this in-vehicle camera device, the driver can bring the vehicle closer to the road shoulder while referring to the video displayed on the display.
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 2000-222697
[Problems to be solved by the invention]
However, according to the conventional in-vehicle camera device, in a situation where there are many pedestrians and oncoming vehicles around the vehicle, the driver can afford to look at the display by being distracted by pedestrians and oncoming vehicles. There was also a possibility that the car body and tires would contact the road shoulder.
[0005]
The present invention provides a vehicle surrounding monitoring device that detects the degree of approach between a vehicle and a road shoulder and notifies the driver even under circumstances where the driver cannot afford to look at the display.
[0006]
[Means for Solving the Problems]
A vehicle periphery monitoring device according to the present invention includes an imaging unit that images a side of a vehicle including a side surface of the host vehicle, and an edge detection unit that detects an edge in a direction parallel to a road based on an image captured by the imaging unit. Based on the luminance change characteristics in the edge arrangement direction of the plurality of edges detected by the edge detection means , the white line edge on the road is excluded from the plurality of edges detected by the edge detection means. Extraction means for extracting an edge, and notification means for notifying an occupant of the degree of approach between the three-dimensional obstacle and the vehicle based on the positional relationship between the edge of the three-dimensional obstacle extracted by the extraction means and the vehicle. Features.
[0007]
【The invention's effect】
According to the vehicle periphery monitoring device of the present invention, the edge of the white line on the road is excluded based on the luminance change characteristic of the edge arrangement direction of the plurality of edges among the plurality of edges detected based on the captured image. The edge of the three-dimensional obstacle is extracted and the occupant is informed of the degree of approach between the three-dimensional obstacle and the vehicle based on the positional relationship between the extracted three-dimensional obstacle edge and the vehicle. Even when a car is present, the driver can rest on the shoulder in peace.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a diagram showing a configuration of an embodiment in which a vehicle surrounding monitoring apparatus according to the present invention is applied to a vehicle 10. The vehicle surrounding monitoring apparatus in one embodiment includes a CCD camera 1, a control unit 2, a switch device 3, a display 4, and a speaker 5.
[0009]
The CCD camera 1 is built in the left mirror 11 of the vehicle (see FIG. 2), and images the side of the vehicle including the side surface of the vehicle 10, in particular, the vicinity of the left front wheel 6. The control unit 2 detects a three-dimensional obstacle such as a road shoulder based on the video imaged by the CCD camera 1. At this time, if the distance between the detected three-dimensional obstacle and the host vehicle 10 is equal to or less than the predetermined distance, the image captured by the CCD camera 1 is displayed on the display 4 and the three-dimensional obstacle and the own vehicle approach each other. The passenger is notified using the speaker 5. A method of detecting a three-dimensional obstacle such as a road shoulder will be described later.
[0010]
The switch device 3 is provided, for example, on the dashboard of the driver's seat and is operated by a vehicle occupant. The occupant can display the video imaged by the CCD camera 1 on the display 4 installed in the vehicle by operating the switch device 3. The display 4 is provided at a position that is easy for the driver to see, and can display at least an image captured by the CCD camera 1. When the vehicle 10 includes a car navigation device, map information including route information to the destination can be displayed on the display 4. The speaker 5 notifies the driver that the three-dimensional obstacle and the vehicle 10 are approaching based on the signal from the control unit 2. The speaker 5 can also be used as a vehicle audio speaker.
[0011]
FIG. 2 is a diagram illustrating a situation in which the vehicle 10 is brought closer to the road shoulder 12 as viewed from the rear of the vehicle 10. As shown in FIG. 2, the road shoulder 12 is not on the same plane as the asphalt surface 13 of the roadway, but is a step. Therefore, if the vehicle 10 is brought too close to the road shoulder 12, the left front wheel 6, the left rear wheel 7, and a wheel (not shown) attached to the wheel may be rubbed against the road shoulder 12.
[0012]
FIG. 3 is a diagram showing a state where an image captured by the CCD camera 1 built in the left mirror 11 is displayed on the display 4 under the situation shown in FIG. As shown in FIG. 3, the left front wheel 6 and the body of the vehicle 10 are shown on the right side of the captured image, and the asphalt surface 13 and the road shoulder 12 on which the white line 14 is drawn are shown on the left side thereof. Has been.
[0013]
When the vehicle 10 is brought close to the road shoulder 12, the driver usually operates the switch device 3 to display an image captured by the CCD camera 1 on the display 4 in the vehicle. Then, by grasping the positional relationship between the host vehicle 10 and the road shoulder 12 displayed on the display 4, the vehicle 10 is shifted. Even when the switch device 3 is operated to display a captured image on the display 4, a pedestrian or the like existing around the vehicle 10 may be distracted and there is no room for viewing the image on the display 4. Therefore, in the vehicle surrounding monitoring apparatus according to the embodiment, the road shoulder 12 that is a three-dimensional obstacle is detected based on the image captured by the CCD camera 1, and based on the detected distance between the road shoulder 12 and the host vehicle 10. Thus, when it is determined that the vehicle 10 may approach the road shoulder 12 too much , the driver is notified.
[0014]
FIG. 4 is a diagram summarizing the display on the display 4 and the necessity of an alarm from the speaker 5 with respect to the presence or absence of an obstacle present near the host vehicle 10 and the operation state of the switch device 3. When a road shoulder (obstacle) 12 is detected at a position close to the host vehicle 10 by an obstacle detection process to be described later, an image captured by the CCD camera 1 is displayed on the display 4 regardless of the operation state of the switch device 3. And an alarm is issued from the speaker 5 to notify the driver. For the alarm, a beep sound or a voice announcement such as “approaching an obstacle” can be used.
[0015]
In the situation where the road shoulder 12 is not detected at a position close to the host vehicle 10, notification to the driver using the speaker 5 is not performed. In this case, an image captured by the CCD camera 1 is displayed on the display 4 in accordance with the operation state of the switch device 3. That is, when the occupant operates the switch device 3 (switch ON) in order to display the image captured by the CCD camera 1 on the display 4, the image captured by the CCD camera 1 is displayed on the display 4. . On the other hand, when the switch device 3 is not operated (switch OFF), the image captured by the CCD camera 1 is not displayed on the display 4.
[0016]
A method for detecting a three-dimensional obstacle such as a road shoulder based on an image captured by the CCD camera 1 and issuing a warning to the driver based on the detected three-dimensional obstacle will be described with reference to FIGS. To do. FIG. 5 is a flowchart showing the procedure of the obstacle detection process. The process starting from step S10 is performed by the control unit 2 at a predetermined time interval, for example. In step S <b> 10, an image of an obstacle detection area where the obstacle detection process is performed is cut out from the image captured by the CCD camera 1.
[0017]
FIG. 6A is a diagram illustrating an obstacle detection region C in an image captured by the CCD camera 1. In FIG. 6A, the horizontal direction is the X direction. As shown in FIG. 6A, the obstacle detection area C is set as a rectangular area that is long in the X direction at a position next to the left front wheel 6 of the vehicle 10 in the captured image. The position and size of the obstacle detection area C are set in advance in an area where an obstacle is desired to be detected based on the mounting position of the CCD camera 1 and the imaging range.
[0018]
In step S10, when an image in the obstacle detection area C is extracted from the captured image, the process proceeds to step S20. In step S20, an edge in the vertical direction perpendicular to the X direction is detected in the image in the obstacle detection area C by performing a differential operation in the X direction on the luminance value of each pixel. That is, since the three-dimensional obstacles such as the road shoulder 12 and the guard rail are positioned in parallel with the road, the three-dimensional obstacle such as the road shoulder 12 is detected by detecting the vertical edge. FIG. 6B shows the vertical edge detected based on the image in the obstacle detection area C of FIG.
[0019]
In step S30 subsequent to step S20, what seems to be noise is removed from the plurality of vertical edges detected in step S20. Specifically, out of a plurality of vertical edges detected in step S20, only those whose edges are continuously present from the upper end to the lower end of the obstacle detection area C are extracted. FIG. 6C shows a diagram in which noise is removed from the vertical edges shown in FIG. 6B.
[0020]
In step S40 following step S30, a density profile on the central horizontal line (AA ′) in the obstacle detection region C is created as shown in FIG. FIG. 6E shows a density profile corresponding to FIG. 6D, and the vertical axis shows the magnitude of the luminance change. As shown in FIG. 6E, a peak of luminance change appears at the paint display parallel to the road and at both ends (edges) of the obstacle.
[0021]
In step S50 following step S40, the paint display drawn on the road is removed from the paint display and obstacle detected in step S40, and only a three-dimensional obstacle having a height is extracted. This method will be described with reference to FIGS. 7 (a) and 7 (b).
[0022]
FIG. 7A is a diagram illustrating a captured image in the obstacle detection area C illustrated in FIG. In Fig. 7 (a), from left to right: (1) Roadsides such as sidewalks, (2) Road shoulder upper surface (horizontal plane), (3) Road shoulder side, (4) Asphalt (roadway), (5) Road A white line, (6) asphalt (roadway) is shown. FIG. 7B is a density profile obtained based on the image shown in FIG. A diagonal line segment in the obstacle region C shown in the upper part of FIG. 7B is a vertical edge obtained by the processing in steps S10 to S30 described above, and each region shown in FIG. This corresponds to the boundary lines 1 ▼ to 6). Further, the density profile shown in FIG. 7B is created by the process of step S40. The magnitude of the peak value of the luminance change in the density profile shown in FIG. 7B is determined according to the luminance difference between adjacent regions.
[0023]
Here, a method for distinguishing between a paint display such as a white line drawn on a road (asphalt) and a three-dimensional obstacle such as a road shoulder will be described. As described above, since the white line extends in parallel with the road like the road shoulder, the road shoulder that is a three-dimensional obstacle cannot be distinguished from the white line only by detecting the vertical edge in the captured image. . However, the white line drawn on the road is characterized in that both ends (edges) of the white line are in contact with the asphalt and the width of the line is not so large (the size is determined to some extent). Therefore, the density profile of the white line is characterized in that the peak values of the luminance change corresponding to both ends of the white line are approximately equal, and both peak values appear at close positions.
[0024]
In the density profile shown in FIG. 7B, the peak values of the luminance change are almost equal and close to each other (the boundaries between the regions (4) and (5) and the regions (5) and (6)). The boundary line is considered to be a white line drawn on the road. Even when two line segments having substantially the same peak value of luminance change are present, if both peak values appear at positions separated from each other, it is considered that these two line segments do not constitute a white line.
[0025]
In step S50 of the flowchart shown in FIG. 5, a paint display such as a white line drawn on the road is detected by such a method, and only a three-dimensional obstacle having a height is extracted.
[0026]
In step S60 following step S50, it is determined whether or not the distance between the three-dimensional obstacle (road shoulder) extracted in step S50 and the host vehicle 10 is equal to or less than a predetermined distance. First, of the plurality of edges constituting the three-dimensional obstacle extracted in step S50, the edge closest to the vehicle 10 is detected, and the detected edge and the center line A− in the obstacle detection area C described above are detected. An intersection B with A ′ is detected (see FIGS. 6A and 6E). As shown in FIG. 6A, the edge closest to the vehicle 10 corresponds to the tangent line between the side surface of the road shoulder, which is a three-dimensional obstacle, and the road surface (asphalt). Next, an intersection B ′ between the extension line of the center line AA ′ in the obstacle detection area C and the line on the side surface of the host vehicle 10 is detected. As shown in FIG. 6A, the distance d ₁ between the point B and the point B ′ is the shortest distance between the vehicle 10 and the road shoulder on the center line AA ′ in the obstacle detection area C. Corresponds to distance.
[0027]
The control unit 2 detects the point B and the point B ′ described above, thereby calculating the distance d ₁ between BB ′, and the approach danger distance d ₂ and the distance d ₁ stored in advance in a ROM (not shown). Compare the size with. If the calculated distance d ₁ is a distance d ₂ or less, it is determined that the distance between the shoulder and the vehicle 10 is short, if the distance d ₁ is greater than the distance d _2, the vehicle 10 determines that apart shoulders sufficiently .
[0028]
If it is determined in step S60 that the relationship of d ₁ ≦ d ₂ is established, the process proceeds to step S70, and if it is determined that the relationship of d ₁ > d ₂ is established, the processing according to the flowchart illustrated in FIG. In step S <b> 70, an image captured by the CCD camera 1 is displayed on the display 4, and the driver is notified that the vehicle 10 is approaching the road shoulder using the speaker 5. When the notification to the driver is given, the processing according to the flowchart shown in FIG.
[0029]
According to the vehicle surrounding monitoring apparatus in one embodiment, an edge (vertical edge) in a direction parallel to the road is detected based on an image captured by the CCD camera 1, and among a plurality of detected vertical edges, Since the edge of the three-dimensional obstacle is detected by excluding the edge corresponding to the road surface paint display such as a white line, the three-dimensional obstacle can be reliably detected by a simple method. In addition, since the passenger is informed that the vehicle is approaching too close to the three-dimensional obstacle based on the positional relationship between the detected edge of the three-dimensional obstacle and the host vehicle 10, a warning is given to the passenger under the circumstances required by the passenger. It can be performed. In other words, even when there are many pedestrians in the surrounding area such as a shopping street, or when there is a lot of traffic such as a main road, the width of the shoulder should be assured without feeling uneasy to touch the shoulder. Can do. Further, since the white line and the three-dimensional obstacle are identified, it is possible to prevent the passenger from being notified until the edge of the white line is detected.
[0030]
In the vehicle surrounding monitoring apparatus according to an embodiment, a three-dimensional obstacle is detected by detecting an edge corresponding to a paint display such as a white line based on a luminance change characteristic (density profile) of a vertical edge detected based on a captured image. Since the edge of an object is extracted, the edge of a three-dimensional obstacle can be reliably detected from a plurality of vertical edges.
[0031]
In addition, when the distance between the edge of the extracted three-dimensional obstacle and the vehicle is equal to or smaller than a predetermined distance, the passenger is notified by a warning sound or voice announcement from the speaker 5, and therefore the passenger cannot afford to see the display 4. However, it is possible to reliably notify the occupant that the vehicle 10 is too close to the three-dimensional obstacle. In this case, since the image captured by the CCD camera 1 is automatically displayed on the display 4, the switch device 3 cannot be operated in order to display the captured image on the display 4. Even when the operation is forgotten, the situation around the vehicle 10 can be recognized.
[0032]
The present invention is not limited to the embodiment described above. For example, in the above-described embodiment, when driving the right-hand drive vehicle 10, the left side of the vehicle is a blind spot for the driver, so the left side mirror 11 incorporates the CCD camera 1. In the case of a vehicle with a handle, the right side of the vehicle is a blind spot, so the CCD camera 1 can be built in the right door mirror. Further, although the CCD camera 1 is built in the door mirror 11, it may be attached to another position as long as the position that becomes the blind spot of the vehicle can be imaged.
[0033]
In the above-described embodiment, the case where the road shoulder 12 is detected as a three-dimensional obstacle has been described, but it is needless to say that a side groove or the like can be detected in addition to a three-dimensional obstacle such as a wall or a guardrail. Yes. Therefore, when the vehicle is brought close to the place where the side groove exists, it is possible to prevent anxiety that the wheel is removed from the side groove by using the vehicle surrounding monitoring device according to the embodiment. Moreover, although the paint display (white line) drawn on the road was mentioned as an example as a non-stereoscopic obstacle to distinguish from a three-dimensional obstacle, it is not limited to a paint display.
[0034]
In the above-described embodiment, the CCD camera 1 is used as the imaging device. However, a CMOS camera using a CMOS image sensor may be used, or a camera using another image sensor may be used. Further, when notifying the occupant that the vehicle is too close to the three-dimensional obstacle, it is possible to display characters such as “CAUTION!” On the display 4. Furthermore, the obstacle detection area C is not limited to the shape and size shown in FIG.
[0035]
The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the CCD camera 1 constitutes an imaging means, the control unit 2 constitutes an edge detection means, an extraction means, and a determination means, the speaker 5 constitutes a notification means, and the display 4 constitutes a display means. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an embodiment in which a vehicle periphery monitoring device according to the present invention is applied to a vehicle. FIG. 2 is a diagram showing a situation in which the vehicle is brought close to a road shoulder as viewed from the rear of the vehicle. 3 is a diagram showing an image captured by a CCD camera and displayed on a display under the situation shown in FIG. 2. FIG. 4 is the presence / absence of an obstacle present near the host vehicle and the operating state of the switch device. Fig. 5 is a flow chart showing the procedure of obstacle detection processing. Fig. 6 (a) to Fig. 6 (e) are images taken by a CCD camera. FIG. 7A is a diagram illustrating a captured image in the obstacle detection area C, and FIG. 7B is a diagram for explaining a method of detecting a three-dimensional obstacle such as a road shoulder based on the recorded image. ) A three-dimensional obstacle and a white line Diagram for explaining another methods [EXPLANATION OF SYMBOLS]
DESCRIPTION OF SYMBOLS 1 ... CCD camera, 2 ... Control unit, 3 ... Switch apparatus, 4 ... Display, 5 ... Speaker, 6 ... Left front wheel, 7 ... Left rear wheel, 10 ... Vehicle, 11 ... Left door mirror, 12 ... Road shoulder, 13 ... Roadway Asphalt surface, 14 ... white line

Claims (5)

Imaging means for imaging the side of the vehicle including the side surface of the host vehicle;
Edge detection means for detecting an edge in a direction parallel to the road based on the image picked up by the image pickup means;
Based on the luminance change characteristics in the edge arrangement direction of the plurality of edges detected by the edge detection means , the white line on the road is excluded from the plurality of edges detected by the edge detection means, and the three-dimensional obstacle Extraction means for extracting the edge of the object;
Vehicle surrounding monitoring, comprising: notification means for notifying a passenger of the degree of approach between the three-dimensional obstacle and the vehicle based on the positional relationship between the edge of the three-dimensional obstacle and the vehicle extracted by the extracting means. apparatus. The vehicle surrounding monitoring device according to claim 1,
The extraction unit excludes the edge of the white line on the road based on the peak value of the luminance change in the edge arrangement direction of the plurality of edges detected by the edge detection unit and the position where the peak value appears. A vehicle surrounding monitoring device characterized by extracting an edge of a three-dimensional obstacle . The vehicle surrounding monitoring device according to claim 1 or 2,
Of the plurality of edges of the three-dimensional obstacle extracted by the extraction means, further comprising a determination means for determining whether the distance between the edge closest to the vehicle and the vehicle is a predetermined distance or less,
The vehicle surroundings monitoring device, wherein the notification unit notifies a passenger when the determination unit determines that the distance between the edge closest to the vehicle and the vehicle is a predetermined distance or less. . In the vehicle periphery monitoring device according to claim 3,
The vehicle surrounding monitoring device, wherein the notification means notifies a passenger by a warning sound or a voice announcement. The vehicle surrounding monitoring device according to claim 3 or 4,
And further comprising display means for displaying at least the video imaged by the imaging means,
The display unit displays an image captured by the imaging unit when the determination unit determines that the distance between the edge closest to the vehicle and the vehicle is equal to or less than a predetermined distance. A vehicle surrounding monitoring device.

Images