JP7294388B2 - Driving support device - Google Patents

Description

The present disclosure relates to a driving assistance device that provides driving assistance to a driver when a vehicle enters an intersection.

Conventionally, as described in Patent Document 1, when it is detected that a vehicle is entering an intersection with poor visibility, the display image on the display is changed from the road guide image to the side images captured by the left and right blind corner cameras. A driving assistance device configured to switch to is known.

JP 2007-140992 A

In the conventional driving support system described above, it is determined whether or not the vehicle will enter an intersection with poor visibility based on an image of the front of the vehicle captured by the front camera, and obstacles and buildings on the left and right of the road in front of the vehicle are detected. is present, switch the displayed image to the lateral image.

The front camera is attached to the stay of the rearview mirror provided in the center of the interior of the vehicle, and is configured to capture an image of the area in front of the vehicle within a predetermined range of left and right angles from the top center of the windshield. The viewing angle is narrow.

Therefore, the determination of whether to switch the display image to the side image is made at a position where the vehicle is some distance away from the intersection and the front camera can image the vicinity of the intersection.
Therefore, when it is determined that the vehicle is entering an intersection with poor visibility, and the display image on the display is switched to the side image, the vehicle may be some distance from the intersection or there may be obstacles on the left and right of the road. However, the blind corner camera may not be able to capture images of intersections that intersect the road.

Then, when the display image is switched in this way, the side image showing only the obstacle is displayed on the display device. There is a problem.

One aspect of the present disclosure is a driving assistance device for a vehicle that displays an image of an intersection when the vehicle enters an intersection to provide driving assistance to the driver , and appropriately switches to the image of the intersection. It aims to make it possible .

A vehicle driving support device according to one aspect of the present disclosure includes a camera (11), an obstacle sensor (36), an approach recognition unit (22, 24, S120, S130), a movement determination unit (22, 24, S140), an obstacle determination section (22, 24, S150), and an image switching section (22, 24, S180).

The camera is configured to capture an image of the surroundings of the vehicle, the obstacle sensor is configured to detect obstacles existing on the left and right sides of the vehicle, and the approach recognition unit detects an intersection where the vehicle is currently traveling. It is configured to recognize that it has approached a road. The movement determination unit is configured to determine that the vehicle has moved to a stop position before entering the intersection after the approach recognition unit has recognized that the vehicle is approaching the intersection.

The obstacle determination unit is configured to determine whether or not the obstacle sensor detects an obstacle present on the left and right sides of the vehicle when the movement determination unit determines that the vehicle has moved to the stop position. ing.

Then, when the obstacle determination unit determines that an obstacle has been detected, the image switching unit changes the display image of the display device that displays the travel guidance of the vehicle to the image of the intersection captured by the camera. configured to switch.

As described above, in the driving support device of the present disclosure, when the vehicle approaches an intersection, the vehicle moves to a stop position in front of the intersection, and then the obstacle sensors detect whether or not the obstacles on the left and right sides are detected. judge.

Then, when an obstacle is detected at the stop position before the intersection, the display image is switched to the image of the intersection. Therefore, the driver can grasp the situation of the intersection from the switched display image, that is, the captured image of the intersection, and drive the vehicle into the intersection safely.

On the other hand, when the left and right obstacles are not detected at the stop position, that is, when the vehicle enters an intersection with good visibility, the display image cannot be switched to the image of the intersection. Therefore, when the vehicle enters an intersection with good visibility, unnecessary switching of the display image can be suppressed.

1 is a block diagram showing the configuration of a driving assistance device according to an embodiment; FIG. FIG. 3 is an explanatory diagram showing the mounting position of a front camera that captures an image of the front of the vehicle. 4 is a flowchart showing image switching processing executed by a CPU; FIG. 2 is an explanatory diagram showing an example of a stop line, a stop display, and a stop sign on a travel road; FIG. 10 is an explanatory diagram for explaining a procedure for switching display images by image switching processing;

Embodiments of the present disclosure will be described below with reference to the drawings.
[Embodiment]
[composition]
As shown in FIG. 1, the driving assistance device 1 of the present embodiment is mounted on a vehicle 2 illustrated in FIG. It provides driving assistance to a person, and includes an ECU 20 for image processing.

The surrounding cameras 10 include, for example, four cameras 11 and 12 attached to the front, rear, left, and right sides of the vehicle 2 so as to capture images of the front, rear, left, and right sides of the vehicle 2, respectively.
13 and 14.

Note that each of the cameras 11 to 14 can be configured using a CCD image sensor, a CMOS image sensor, or the like. In addition, the number of peripheral cameras 10 can be changed as appropriate, but the peripheral cameras, which are the main part of the present disclosure, are provided at the center of the front end of the vehicle 2 as shown in FIG. It is a front camera 11 that images the front of the.

A wide-angle camera having a wide-angle lens is used as the front camera 11 so that a wide range of images in front of the vehicle 2 can be captured. A display 32 as a display device is configured by a liquid crystal display, a head-up display, or the like.

The ECU 20 is an electronic control device that includes an image processing unit 22 that processes captured images captured by the cameras 11 to 14, a CPU 24 that controls the image processing unit 22, and a memory 26 such as ROM and RAM. . That is, the ECU 20 is composed of a microcomputer having an image processing function.

The ECU 20 also includes a vehicle input I/F 28 that captures detection signals from various sensors that detect the running state of the vehicle 2 and surrounding conditions of the vehicle 2, such as a vehicle speed sensor 34 and an obstacle sensor 36, and inputs them to the CPU 24. It is Note that I/F represents an interface.

In the ECU 20, the CPU 24 executes a predetermined control process according to the program stored in the memory 26, thereby grasping the vehicle state based on the detection signal input from the vehicle input I/F 28 and the information obtained from other in-vehicle devices. do.

Then, the CPU 24 causes the image processing unit 22 to execute image processing according to the grasped vehicle state or a command from the passenger, thereby obtaining a front image, a rear image, and a bird's-eye view of the surroundings of the vehicle 2 from above. Generate an image for driving support, such as an image.

In addition, the image processing unit 22 generates an image for driving support according to a command from the CPU 24 and outputs the image to the navigation device 30, so that the image displayed on the display 32 is changed from the road guide image to the image for driving support. switch to The road guidance image is an image for driving guidance generated by the navigation device 30 based on the map data.
[Image switching processing at an intersection]
In this manner, the CPU 24 causes the image processing unit 22 to generate an image for driving assistance, and switches the display image on the display 32 to the image for driving assistance. When the vehicle 2 is traveling and enters an intersection with poor visibility, the display 32 displays an image of the intersection that intersects the road on which the vehicle is currently traveling, so that the driver can grasp the situation of the intersection. Driving assistance is provided to the driver so that it becomes easier.

Image switching processing executed by the CPU 24 for driving assistance at such an intersection will be described below with reference to the flowchart shown in FIG.
The image switching process shown in FIG. 3 is one of the processes repeatedly executed by the CPU 24 when it is detected that the vehicle 2 is running based on the detection signal from the vehicle speed sensor 34 .

As shown in FIG. 3, in the image switching process, first, in S110, imaging by the front camera 11 is started. Let

Examples of the stop display include a stop line 41 drawn on the road as shown in FIG. 4A and a display 42 indicating "stop" on the road as shown in FIG. 4B. can. The stop display also includes a sign 43 indicating "Stop" placed on the shoulder or above the road, as illustrated in FIG. 4C. Then, the image processing unit 22 determines, by pattern recognition or the like, whether or not these stop displays are present in the captured image.

Next, in S130, as a result of recognition processing by the image processing unit 22, it is determined whether or not a stop display has been detected in the captured image. If not detected, the image switching process is terminated.

In S140, it is determined whether or not the vehicle 2 has moved to the stop position corresponding to the stop indication after the stop indication was detected.
That is, as shown in the left column of FIG. 5, when the stop display is imaged by the front camera 11 and the stop display is detected, the vehicle 2 has not yet reached the stop position and has left the stop position. ing. Therefore, in S140, the vehicle 2 moves after that and waits for the vehicle 2 to reach the stop position corresponding to the stop display, as shown in the center column of FIG.

Note that FIG. 5 shows a state in which it is determined that the vehicle 2 has reached the stop position when the vehicle 2 reaches the stop line on the travel road. Then, in S140, for example, it may be determined that the vehicle 2 has reached the stop position when the front camera 11 no longer captures the image of the stop display. Further, in S140, it may be determined that the vehicle 2 has reached the stop position when the vehicle 2 has moved a predetermined distance after detecting the stop indication.

When it is determined in S140 that the vehicle 2 has moved to the stop position, the process proceeds to S150, in which it is determined whether or not the obstacle sensor 36 detects an obstacle present on the left and right sides of the vehicle 2 . That is, in S150, it is determined whether or not there are other parked vehicles or structures on the left and right of the stop position of the vehicle 2, in other words, whether or not it is difficult for the driver to see the intersection due to an obstacle. to judge.

If the obstacle sensor 36 detects no left or right obstacle, it is determined that the intersection into which the vehicle 2 is about to enter has good visibility, and the image switching process ends.
On the other hand, when the obstacle sensor 36 detects the left and right obstacles, it is determined that the vehicle 2 is about to enter an intersection with poor visibility, and the process proceeds to S160.

In S160, the image processing unit 22 classifies the image captured by the front camera 11 into a road surface region and a non-road surface region other than the road surface by, for example, semantic segmentation, and causes the road surface to be recognized. Execute the recognition process.

Note that, through this processing, the image processing unit 22 functions as a road surface recognition unit of the present disclosure, and as shown in FIG. A hatched road surface area is recognized.

Semantic segmentation is a technique that uses machine learning data or the like to label which class of object each pixel that constitutes an image belongs to. It is Therefore, in this specification, the description of the procedure for recognizing a road surface by semantic segmentation is omitted.

Next, when the road surface recognition process is executed in S160, the process proceeds to S170 to determine whether or not the road surface recognized by the image processing unit 22 includes the road surface of an intersection extending in the left-right direction from the driving road. to judge whether

For this determination, for example, when the width of the recognized road surface widens in the traveling direction of the vehicle 2, it may be determined that the road surface of the intersection is included. If the road surface recognized by the image processing unit 22 does not include the road surface of the intersection, the front camera 11 cannot capture an image of the intersection due to the obstacle, so the process returns to S160 to continue road surface recognition. .

Next, as shown in the right column of FIG. 5, when the vehicle 2 moves from the stop position toward the intersection and the front camera 11 can capture an image of the intersection, the image processing unit 22 recognizes the intersection in S170. It is determined that the road surface of the cross road is included in the road surface thus obtained, and the process proceeds to S180.

Then, in S180, the image processing unit 22 is caused to extract images of left and right crossroads from the image captured by the front camera 11, and generate an image for driving support including the captured images of the left and right crossroads, The generated image for driving support is output to the navigation device 30 .

As a result, as shown in FIG. 5, the display image on the display 32 is changed from the road guide image generated by the navigation device 30 based on the map data to the driving support image including the imaged images of the left and right intersections. can be switched.

Since the image captured by the front camera 11 includes the left and right crossroads, in S180, the image captured by the front camera 11 is sent to the image processing unit 22 as it is as an image for driving support, and the navigation device 30 You may make it output to .

Next, in S180, when the image displayed on the display 32 is switched from the road guide image to the image of the intersection captured by the front camera 11, the process proceeds to S190 to determine whether the vehicle 2 has entered the intersection. By judging , it waits for the vehicle 2 to enter the intersection.

When it is determined in S190 that the vehicle 2 has entered the intersection, the image displayed on the display 32 is changed from the image of the intersection captured by the front camera 11 to the road guide image via the image processing unit 22. return to end the image switching process.
[effect]
As described above, in the vehicle driving assistance device 1 of the present embodiment, when a stop indication in front of the vehicle traveling direction is detected from an image captured by the front camera 11 while the vehicle 2 is running, the vehicle 2 is moved to the stop position. After moving, the road surface is recognized using the captured image.

Then, when the road surface of the cross road that intersects the travel road at the intersection is recognized by the road surface recognition, the display image on the display 32 is captured by the front camera 11 from the road guide image generated by the navigation device 30. switch to the image of the crossroads marked

Therefore, according to the driving assistance device 1 of the present embodiment, when the front camera 11 captures an image of the road surface of the intersection before the vehicle 2 enters the intersection, the display image on the display 32 is the intersection road surface. will be switched to the image of

Therefore, it is possible to prevent the display image on the display 32 from being switched to the image captured by the front camera 11 when the front camera 11 cannot capture the road surface of the intersection. In other words, according to the driving assistance device 1 of the present embodiment, the display image can be switched at an appropriate timing when the road surface of the intersection can be imaged by the front camera 11 .

Therefore, the driver can grasp the situation of the intersection from the switched display image, that is, the captured image of the intersection, and drive the vehicle into the intersection safely.
In addition, in the driving support device 1 of the present embodiment, the image captured by the front camera 11 is used to determine whether the display image is to be switched, and the image of the intersection to be displayed along with the image switching is generated.

Therefore, it is not necessary to separately provide a camera for judging display image switching as in the conventional device, and the configuration of the device can be simplified. Further, since the driving assistance device 1 of the present embodiment can be applied to a vehicle that is not equipped with a camera for determining display image switching, it is possible to expand the application of the driving assistance device.

Further, in this embodiment, when the vehicle has moved to the stop position and the obstacle sensor 36 has not detected the left and right obstacles, that is, when the vehicle enters an intersection with good visibility, the image of the intersection is captured. It is set not to display images.

Therefore, when the vehicle enters an intersection with good visibility, the image displayed on the display 32 is not switched from the road guide image to the image of the left and right cross roads captured by the front camera 11. Unnecessary switching can be suppressed.

In addition, in this embodiment, the ECU 20 corresponds to an approach recognition unit, a movement determination unit, an obstacle determination unit, and an image switching unit of the present disclosure. More specifically, the processing of S120 and S130 executed by the CPU 24 functions together with the image processing unit 22 as an approach recognition unit of the present disclosure, and the processing of S140 executed by the CPU 24 functions together with the image processing unit 22 of the present disclosure. function as a movement determination unit. Further, the processing of S150 executed by the CPU 24 functions as an obstacle determination unit, and the processing of S180 functions together with the image processing unit 22 as an image switching unit of the present disclosure.
[Other embodiments]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications can be made.

For example, in the above-described embodiment, in the image switching process, if it is determined in S130 that the stop display has been detected from the image captured by the front camera 11, in S140, the vehicle 2 waits for the vehicle 2 to move to the stop position, and then in S160. The description has been given assuming that the road surface recognition processing is executed.

On the other hand, if it is determined in S130 that the stop display has been detected from the image captured by the front camera 11, the road surface recognition processing of S160 may be started without performing the determination processing of S140.

Further, in the above embodiment, the road surface is recognized by semantic segmentation in order to determine whether or not the front camera 11 captures an image of the road surface of the intersection.

On the other hand, semantic segmentation may also be used for recognition of stop indication in S120. In other words, semantic segmentation can classify a captured image into various regions including stop indications, so in S120 semantic segmentation may simultaneously recognize stop indications such as stop lines and the road surface. .

On the other hand, road surface recognition using semantic segmentation cannot be performed accurately at night when the surroundings are dark or when it is raining and the weather is bad. Sometimes I can't.

Therefore, at night or when the weather is bad, the image displayed on the display 32 may be switched from the road guide image to the left and right intersection images captured by the front camera 11 without recognizing the road surface. .

That is, at night or when the weather is bad, when it is determined in S130 that the stop display has been detected, or when it is determined in S140 that the vehicle 2 has moved to the stop position, the display image on the display 32 is changed to the intersection. You may make it switch to the image of a road.

Next, in the above embodiment, the recognition result of the stop display by the image processing unit 22 is used when determining whether or not the stop display is detected in S130. However, the intersection for which the image of the intersection is displayed for driving assistance can be specified from the map data of the navigation device 30 or the like.

Therefore, in the processing of S120 and S130 as the approach recognition unit, the approach to the intersection may be detected based on the map data and the current position of the vehicle. Even in this way, when the vehicle 2 enters the intersection, the image of the intersection can be displayed on the display 32 to assist the driver in driving.

Next, in the above-described embodiment, when the display image on the display 32 is switched to the image of the intersection, the display 32 displays the road guide image generated based on the map data by the navigation device 30. described as a thing.

However, the display 32 may display, for example, an information provision image representing the vehicle state such as the operating state of the air conditioner, or an audio image such as a TV/video output or music playback screen, instead of the road guidance image.

Even when an image different from the road guidance image is displayed on the display 32 in this way, when the vehicle 2 enters the intersection, the display image on the display 32 is changed by the image switching process described above. may be switched to an image of an intersection.

Further, in the above embodiment, the functions of the approach recognition unit, the movement determination unit, the obstacle determination unit, and the image switching unit of the present disclosure are realized by the CPU 24 executing a program for image switching processing. described as being.

However, the functions of these units are not limited to software, and some or all of the elements may be implemented using hardware combining logic circuits, analog circuits, and the like.

Also, a plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or a function possessed by one component may be realized by a plurality of components. Also, a plurality of functions possessed by a plurality of components may be realized by a single component, or a function realized by a plurality of components may be realized by a single component. Also, part of the configuration of the above embodiment may be omitted. Moreover, at least part of the configuration of the above embodiment may be added or replaced with respect to the configuration of the other above embodiment.

In addition to the vehicle driving support device, the present disclosure includes a program for causing a computer to function as a driving support device, a non-transitional actual recording medium such as a semiconductor memory that records this program, and a driving support method. The present disclosure can also be implemented in a form.

DESCRIPTION OF SYMBOLS 1... Driving assistance device, 2... Vehicle, 11... Front camera, 20... ECU, 22... Image processing part,
24... CPU, 32... Display, 36... Obstacle sensor.

Claims (3)

a camera (11) configured to capture images of the surroundings of the vehicle;
an obstacle sensor (36) configured to detect obstacles to the left and right of the vehicle;
an approach recognition unit (22, 24, S120, S130) configured to recognize that the vehicle has approached an intersection that intersects the road on which the vehicle is currently traveling;
A movement determination unit (22, 24, S140) and
When the movement determination unit determines that the vehicle has moved to the stop position, the obstacle sensor detects the obstacle, and whether the obstacle makes it difficult for the driver to see the intersection. an obstacle determination unit (22, 24, S150) configured to determine whether
When the obstacle sensor detects the obstacle and it is determined that the intersection is difficult to see, the obstacle determination unit displays the driving guidance of the vehicle , the vehicle state, or an audio image. an image switching unit (22, 24, S180) configured to switch the display image of the display device to the image of the intersection captured by the camera;
A driving support device for a vehicle. A driving support device for a vehicle according to claim 1,
The driving support device for a vehicle, wherein the approach recognition unit is configured to recognize that the vehicle has approached the intersection based on an image captured by the camera. A driving support device for a vehicle according to claim 1 ,
The driving support device for a vehicle, wherein the approach recognition unit is configured to recognize that the vehicle has approached the intersection based on the current position of the vehicle and map data.

Images