JP7722134B2 - Driving assistance devices - Google Patents

Description

The present invention relates to a driving assistance device, and more particularly to a driving assistance device that assists in driving a vehicle.

Conventionally, one driving assistance device proposed for this type of vehicle displays an action plan at a point where there is a margin of error relative to the autonomous driving system's action plan when the vehicle is driving autonomously (see, for example, Patent Document 1). This display allows the driver's expected action plan to match the system's action plan, allowing the driver to understand the action plan in advance.

JP 2018-187984 A

However, with the above-mentioned driving assistance devices, the next planned course of action is displayed even when the driver is allowed to take their eyes off the road while driving. Such displays remain in the driver's field of vision even when the driver is not gazing at the display. As a result, the driver may perceive the display flickering, which can be unpleasant.

The main purpose of the driving assistance device of the present invention is to make display flicker less noticeable to the driver.

The driving assistance device of the present invention employs the following measures to achieve the above-mentioned primary objective.

The driving assistance device of the present invention comprises:
A driving assistance device that assists driving of a vehicle,
a display unit that displays information about the vehicle;
a gaze detection unit that detects the driver's gaze;
a display control unit that, when it is detected by the line-of-sight detection unit that the driver is not gazing at the display unit, reduces the visibility of at least a part of an image displayed by the display unit compared to when it is detected by the line-of-sight detection unit that the driver is gazing at the display unit;
The gist of the project is to provide the following:

In the driving assistance device of the present invention, when the line-of-sight detection unit detects that the driver is not gazing at the display unit, the visibility of at least a portion of the image displayed by the display unit is reduced compared to when the line-of-sight detection unit detects that the driver is gazing at the display unit. As a result, the driver is less likely to notice flickering in the display.

In the driving assistance device of the present invention, the display control unit may reduce the visibility of at least a portion of the image displayed by the display unit by graying out or not displaying at least a portion of the image displayed by the display unit. This makes it possible to more appropriately reduce the visibility of at least a portion of the image displayed by the display unit.

In the driving assistance device of the present invention, when the line-of-sight detection unit detects that the driver is gazing at the display unit, the display control unit may display on the display unit a first image showing information about visible objects in the surrounding area and a second image showing the driving status of the vehicle, and when the line-of-sight detection unit detects that the driver is not gazing at the display unit, the display control unit may reduce the visibility of the second image compared to when the line-of-sight detection unit detects that the driver is gazing at the display unit. The first image contributes more to maintaining safe driving than the second image. Therefore, when the line-of-sight detection unit detects that the driver is not gazing at the display unit, the visibility of the second image may be reduced compared to when the line-of-sight detection unit detects that the driver is gazing at the display unit, thereby reducing the visibility of more appropriate information. This makes it possible to more appropriately reduce the driver's perception of display flicker.

1 is a block diagram showing an example of the configuration of an automobile 20 equipped with a driving assistance device according to an embodiment of the present invention, with a main electronic control unit (hereinafter referred to as a main ECU) 30 as the central block. 4 is a flowchart showing an example of a display processing routine executed by the main electronic control unit 30. FIG. 10 is an explanatory diagram showing an example of a normal image displayed on a meter 76. FIG. 10 is an explanatory diagram showing an example of a reduction image displayed on a meter 76.

Next, we will explain how to implement the present invention using examples.

Figure 1 is a block diagram showing an example of the configuration of an automobile 20 equipped with a driving assistance device according to one embodiment of the present invention, with a main electronic control unit (hereinafter referred to as "main ECU") 30 at the center. As shown, the automobile 20 includes a drive unit 62 that outputs driving force to drive wheels (not shown), and a drive electronic control unit (hereinafter referred to as "drive ECU") 60 that controls the drive of the drive unit 62.

The drive unit 62 can be a system with an engine and automatic transmission, a hybrid system with an engine, motor, and battery, a fuel cell drive system with a fuel cell, battery, and motor, or an electric system with a battery and motor.

The drive ECU 60 is configured as a microcomputer centered around a CPU (not shown), and in addition to the CPU, is equipped with ROM, RAM, flash memory, input ports, output ports, communication ports, etc. The drive ECU 60 controls the drive of the drive unit 62 based on drive control signals from the main ECU 30.

In addition to the drive unit 62 and drive ECU 60, the automobile 20 of the embodiment includes an ignition switch 32, a shift position sensor 34, an accelerator position sensor 36, a brake position sensor 38, a vehicle speed sensor 40, an acceleration sensor 42, a gradient sensor 44, a yaw rate sensor 46, a driving assistance switch 48, an auto cruise control switch (hereinafter referred to as the ACC switch) 50, a surrounding recognition electronic control unit (hereinafter referred to as the surrounding recognition ECU) 52, a front camera 53, a rear camera 54, other surrounding recognition devices 55, an air conditioning electronic control unit (hereinafter referred to as the air conditioning ECU) 56, an air conditioning unit 58, a line of sight detection system 59, a brake electronic control unit (hereinafter referred to as the brake ECU) 64, a brake unit 66, a steering electronic control unit (hereinafter referred to as the steering ECU) 68, a steering unit 70, a center display 72, a head-up display 74, a meter 76, a GPS (Global Positioning System, It is equipped with a satellite system 78, a navigation system 80, a DCM (Data Communication Module) 86, etc.

The shift position sensor 34 detects the position of the shift lever. The accelerator position sensor 36 detects the accelerator opening amount according to the driver's accelerator pedal depression amount. The brake position sensor 38 detects the brake position according to the driver's brake pedal depression amount.

The vehicle speed sensor 40 detects the vehicle speed based on wheel speed, etc. The acceleration sensor 42 detects, for example, the vehicle's acceleration in the longitudinal direction. The gradient sensor 44 detects the road surface gradient. The yaw rate sensor 46 detects the lateral acceleration (yaw rate) in the lateral direction due to turning motion.

The driving assistance switch 48 is a switch used to select whether or not to perform autonomous driving, which is one type of driving assistance control. The ACC switch 50 is a switch used to select whether or not to perform active cruise control, which is one type of driving assistance device. The driving assistance switch 48 and the ACC switch 50 are attached to the steering wheel, the installation panel in front of the driver's seat, or nearby. The driving assistance switch 48 and the ACC switch 50 also serve as switches for setting various settings, such as the distance from the vehicle ahead and the vehicle speed.

Although not shown, the surroundings recognition ECU 52 is configured as a microprocessor centered around a CPU, and in addition to the CPU, it is equipped with ROM for storing processing programs, RAM for temporarily storing data, input/output ports, and communication ports. Images captured by a front camera 53 that captures images in front of the vehicle and a rear camera 54 that captures images behind the vehicle, as well as information about the vehicle and its surroundings from other surroundings recognition devices 55 (e.g., inter-vehicle distances D1, D2 between the vehicle and other vehicles in front and behind the vehicle, the vehicle speeds of other vehicles, and the vehicle's position in the road lane) are input to the surroundings recognition ECU 52 via the input port. Examples of other surroundings recognition devices 55 include millimeter-wave radar, quasi-millimeter-wave radar, infrared laser radar, and sonar.

The air conditioner ECU 56 is configured as a microcomputer centered around a CPU (not shown), and in addition to the CPU, is equipped with ROM, RAM, flash memory, input ports, output ports, communication ports, etc. The air conditioner ECU 56 is incorporated into the air conditioner 58 that conditions the passenger compartment, and drives and controls the air conditioner compressor in the air conditioner 58 so that the temperature in the passenger compartment is maintained at a set temperature.

The gaze detection system 59 comprises an imaging device fixed near the driver's front direction and a point light source. Using an image of the driver's eye area acquired by the imaging device, the gaze detection system 59 extracts the position of the pupil and the position of a white dot that appears on the eyeball when light from the point light source is reflected on the corneal surface. It then identifies the orientation of the eyeball based on the position of the pupil relative to the position of the white dot, and detects the driver's gaze direction from the eyeball direction. Note that, because the eyeball is not a perfect sphere and the shape of the cornea varies from person to person, the relationship between the driver's actual gaze direction and the direction of the eyeball is determined in advance, and the driver's gaze direction is detected from this relationship and the identified eyeball direction.

The brake ECU 64 is configured as a microcomputer centered around a CPU (not shown), and in addition to the CPU, is equipped with ROM, RAM, flash memory, input ports, output ports, communication ports, etc. The brake ECU 64 drives and controls a well-known hydraulically driven brake device 66. The brake device 66 is configured to generate braking force resulting from the brake depression force exerted by depressing the brake pedal, as well as braking force resulting from hydraulic pressure adjustment.

The steering ECU 68 is configured as a microcomputer centered around a CPU (not shown), and in addition to the CPU, is equipped with ROM, RAM, flash memory, input ports, output ports, communication ports, etc. The steering ECU 68 drives and controls the actuator of the steering device 70, which mechanically connects the steering wheel (not shown) and the drive wheels via a steering shaft. The steering device 70 steers the drive wheels based on the driver's steering operation, and also steers the drive wheels by driving the actuator via the steering ECU 68 based on a steering control signal from the main electronic control unit 30.

The center display 72 is located in the center in front of the driver's seat and passenger seat, and also functions as a touch panel, allowing various vehicle settings and audio and media applications to be executed, and also functions as the display unit 84 of the navigation system 80 for map navigation.

The head-up display 74 provides an image of information for the driver (as if focused at an infinite point) on the windshield, such as displaying speed or navigation guidance.

The meter 76 is incorporated, for example, into an installation panel in front of the driver's seat, and provides images of information for the driver. Details of the images provided by the meter 76 will be described later.

GPS 78 is a device that detects the vehicle's position based on signals transmitted from multiple GPS satellites.

The navigation system 80 is a system that guides the vehicle to a set destination, and is equipped with map information 82 and a display unit 84. The navigation system 80 communicates with the traffic information management center 100 via a DCM (Data Communication Module) 86, acquiring road traffic information and updating map information 82 as needed. When a destination is set, the navigation system 80 sets a route based on the destination information, current location information (current vehicle position) acquired via GPS 78, and map information 82.

The DCM (Data Communication Module) 86 transmits information about the vehicle to the traffic information management center 100 and receives road traffic information from the traffic information management center 100. Examples of vehicle information include the vehicle's position, vehicle speed V, driving power, and driving mode. Examples of road traffic information include information about current and future congestion, information about the current average vehicle speed and predicted future average vehicle speed for sections of the driving route, information about traffic regulations, information about weather, information about road surface conditions, and information about maps. The DCM 86 communicates with the traffic information management center 100 at predetermined intervals (for example, every 30 seconds, every minute, or every two minutes).

The main electronic control unit 30 is configured as a microcomputer centered around a CPU (not shown), and in addition to the CPU, it is equipped with ROM, RAM, flash memory, input ports, output ports, communication ports, and other components. Various signals are input to the main electronic control unit 30 via the input ports. Examples of information input via the input ports include an ignition switch signal from the ignition switch 32, a shift position from the shift position sensor 34, an accelerator opening from the accelerator position sensor 36, and a brake position from the brake position sensor 38. Other examples include vehicle speed V from the vehicle speed sensor 40, acceleration from the acceleration sensor 42, gradient from the gradient sensor 44, and yaw rate from the yaw rate sensor 46. Other examples include an automatic driving instruction signal from the driving assistance switch 48 and an ACC instruction signal from the ACC switch 50. Various signals are output from the main electronic control unit 30 via output ports. Examples of information output via the output ports include a display control signal to the center display 72, a display control signal to the head-up display 74, and a display control signal to the meter 76.

The main electronic control unit 30 communicates with the surroundings recognition ECU 52, air conditioning ECU 56, drive ECU 60, brake ECU 64, steering ECU 68, and navigation system 80, exchanging various types of information.

When the driving assistance switch 48 is off, the main electronic control unit 30 performs manual driving, where the driver operates the vehicle 20 to drive it. During manual driving, the main electronic control unit 30 sets the required driving force and required power based on the accelerator opening from the accelerator position sensor 36 and the vehicle speed from the vehicle speed sensor 40, and sends a drive control signal to the drive ECU 60 so that the required driving force and required power are output from the drive device 62.

When the driving assistance switch 48 is on, the main electronic control unit 30 performs autonomous driving, causing the automobile 20 to travel without driver operation. During autonomous driving, the main electronic control unit 30 sets a target vehicle speed based on the route set by the navigation system 80, the current vehicle position, map information 82 (e.g., legal speed limits), and information about the vehicle and its surroundings from the surrounding recognition ECU 52, and sends a drive control signal to the drive ECU 60 to cause the drive device 62 to output the required drive force or power so that the vehicle speed reaches the target vehicle speed.

Next, we will explain the operation of the automobile 20 configured in this manner, particularly the operation when displaying an image of information for the driver on the meter 76 while the vehicle is traveling in autonomous driving. Figure 2 is a flowchart showing an example of a display processing routine executed by the main electronic control unit 30. This display processing routine is executed repeatedly while the vehicle is traveling in autonomous driving.

When the display processing routine is executed, the CPU of the main electronic control unit 30 determines whether the driver is permitted to engage in actions other than driving (such as taking their hands off the steering wheel or looking away from the road) (step S100). Step S100 determines that the driver is permitted to engage in actions other than driving when, for example, the vehicle is operating at level 3 or higher automated driving as defined by Japan's Ministry of Land, Infrastructure, Transport and Tourism or level 3 or higher automated driving as defined by the Society of Automotive Engineers.

If, in step S100, the driver is not permitted to engage in any activity other than driving, a display control signal is sent to the meter 76 to display the normal image (step S120), and the display processing routine ends. Upon receiving the display control signal, the meter 76 displays the normal image.

Figure 3 is an explanatory diagram showing an example of a normal image displayed on the meter 76. In the center of the figure, the host vehicle (automobile 20) is displayed as image 110, and information about visible objects around the host vehicle (e.g., other vehicles traveling ahead of the host vehicle) is displayed as image 112. Images 114-124 indicating the vehicle's driving status are displayed on the top, bottom, left, and right sides of the figure. At the top of the figure, the vehicle speed V detected by the vehicle speed sensor 40 is displayed as image 114. At the left of the figure, the shift position from the shift position sensor 34 is displayed as image 116. At the right of the figure, the remaining fuel and battery charge are displayed as image 118. At the top right of the figure, the current time is displayed as image 120. At the bottom left of the figure, the outside air temperature is displayed as image 122. At the bottom right of the figure, the total mileage since the vehicle 20 was shipped is displayed as image 124. The driver can obtain a variety of information from the displays on the meter 76.

If the driver is permitted to engage in activities other than driving in step S100, it is then determined whether the driver is gazing at the meter 76 (step S110). Here, when the direction of the driver's gaze detected by the line of sight detection system 59 is directed toward the meter 76, that is, when the line of sight detection system 59 detects that the driver is gazing at the meter 76, it is determined that the driver is gazing at the meter 76.

If the driver is gazing at the meter 76 in step S110, a display control signal is sent to the meter 76 to display the normal image (step S120), and the display processing routine ends. Upon receiving the display control signal, the meter 76 displays the normal image shown in Figure 3.

If the driver is not gazing at the meter 76 in step S110, a display control signal is sent to the meter 76 to display a reduced version of the normal display (step S130), and the display processing routine ends. Upon receiving the display control signal, the meter 76 displays a reduced version of the normal image shown in Figure 3.

Figure 4 is an explanatory diagram showing an example of a reduced image displayed on the meter 76. Figure 4 shows an image in which images 110 and 112 in Figure 3 are maintained and images 114-124 are not displayed. Even when the driver is not gazing at the meter 76, if the normal image shown in Figure 3 is displayed, at least a portion of the meter 76 will be in the driver's field of view, and the driver may perceive flickering of the meter 76 display. In this embodiment, when the driver is not gazing at the meter 76, a reduced image is displayed that does not display images 114-124 of the normal image, thereby making it less likely that the driver will perceive flickering of the display. Furthermore, displaying image 110 of the vehicle and image 112 as information on visible objects around the vehicle contributes more to maintaining safe driving than displaying images 114-124. Therefore, by maintaining the display of images 110 and 112 and not displaying images 114-124 in the reduced image, the visibility of some of the content displayed by the meter 76 can be more appropriately reduced compared to the normal image.

In the automobile 20 equipped with the driving assistance device of the embodiment described above, when the line-of-sight detection system 59 detects that the driver is not gazing at the meter 76, the image displayed by the meter 76 is reduced compared to when the line-of-sight detection system 59 detects that the driver is gazing at the meter 76, making it possible to make the driver less likely to notice flickering of the display.

In a vehicle 20 equipped with the driving assistance device of the embodiment, when the driver is not gazing at the meter 76 in step S110, a reduction image is displayed on the meter 76. However, when the driver is not gazing at the meter 76 in step S110, it is sufficient to reduce the visibility of some of the images 110-124 in the normal image on the meter 76, so images 114-124 of the images 110-124 in the normal image on the meter 76 may be displayed as grayed out. Also, when the driver is not gazing at the meter 76 in step S110, none of the images 110-124 may be displayed on the meter 76.

In an automobile 20 equipped with the driving assistance device of the embodiment, as shown in FIG. 3, images 114 to 124 are displayed on the meter 76 as images showing the vehicle's driving conditions. However, a portion of images 114 to 124 may be displayed as a normal image, or an image showing the vehicle's driving conditions different from images 114 to 124 may be displayed.

The following explains the correspondence between the main elements of the embodiment and the main elements of the invention described in the "Means for Solving the Problem" section. In the embodiment, the meter 76 corresponds to the "display unit," the line-of-sight detection system 59 corresponds to the "line-of-sight detection unit," and the main ECU 30 corresponds to the "display control unit."

The correspondence between the main elements of the Examples and the main elements of the invention described in the "Means for Solving the Problem" section does not limit the elements of the invention described in the "Means for Solving the Problem" section, as the Examples are examples used to specifically explain the form for implementing the invention described in the "Means for Solving the Problem" section. In other words, the interpretation of the invention described in the "Means for Solving the Problem" section should be based on the description in that section, and the Examples are merely specific examples of the invention described in the "Means for Solving the Problem" section.

The above describes the form for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and it goes without saying that the present invention can be embodied in various forms without departing from the spirit of the present invention.

This invention can be used in the driving assistance device manufacturing industry, etc.

20 Automobile, 30 Main electronic control unit, 32 Ignition switch, 34 Shift position sensor, 36 Accelerator position sensor, 38 Brake position sensor, 40 Vehicle speed sensor, 42 Acceleration sensor, 44 Gradient sensor, 46 Yaw rate sensor, 48 Driving assistance switch, 50 Auto cruise control switch (ACC switch), 52 Surrounding recognition electronic control unit (surrounding recognition ECU), 53 Forward camera, 54 Rearward camera, 55 Other surrounding recognition devices, 56 Air conditioning system electronic control unit (air conditioning ECU), 58 Air conditioning system, 59 Line of sight detection system, 60 Drive electronic control unit (drive ECU), 62 Drive system, 64 Brake electronic control unit (brake ECU), 66 Brake system, 68 Steering electronic control unit (steering ECU), 70 Steering system, 72 Center display, 74 Head-up display, 76 Meter, 78 GPS, 80 Navigation system, 82 Map information, 84 display unit, 86 DCM, 100 traffic information management center, 110-124 images.

Claims (1)

A driving assistance device that assists driving of a vehicle,
a display unit that displays information about the vehicle;
a gaze detection unit that detects the driver's gaze;
a display control unit that, when it is detected by the line-of-sight detection unit that the driver is gazing at the display unit, displays on the display unit a first image showing information about visible objects in the surroundings and a second image showing a driving situation of the vehicle, and, when it is detected by the line-of-sight detection unit that the driver is not gazing at the display unit, reduces the visibility of the second image compared to when it is detected by the line-of-sight detection unit that the driver is gazing at the display unit;
A driving assistance device comprising: