JP7725956B2 - Character display method and driving support device - Google Patents

Description

The present invention relates to a character display method and a driving assistance device.

Driving assistance devices are known that use sensors to detect the vehicle's surrounding environment and automatically control at least one of the vehicle's acceleration, deceleration, or steering angle based on the detected surrounding environment. For example, Patent Document 1 proposes a vehicle control device with a follow-up cruise control function that adjusts the vehicle speed so that the distance between the vehicle and a leading vehicle approaches a target distance. This vehicle control device disables the follow-up cruise control function before the vehicle reaches a brake fade state.

Japanese Patent Application Laid-Open No. 2020-19455

If the driving assistance function is cancelled without the occupant's consent, the occupant may feel uncomfortable. The same applies if the results of the driving assistance function do not meet the occupant's expectations.
The present invention aims to reduce discomfort felt by occupants in response to driving support control that automatically controls at least one of the acceleration, deceleration, and steering angle of the vehicle.

One aspect of the character display method of the present invention involves having a controller perform the following processes: detecting the environment surrounding the vehicle; driving assistance control that automatically controls at least one of the acceleration, deceleration, or steering angle of the vehicle based on the detected environment; estimating the expectations of the vehicle's occupants regarding the driving assistance control; and generating and displaying on a display device within the vehicle a character image representing a character whose behavior or facial expression changes depending on the difference between the estimated expectations and the driving assistance control actually executed.

This invention can reduce the discomfort felt by occupants due to driving assistance control that automatically controls at least one of the vehicle's acceleration, deceleration, or steering angle.

1 is a diagram illustrating an example of a schematic configuration of a vehicle equipped with a driving assistance device according to an embodiment. 10A and 10B are diagrams showing examples of character images displayed according to the difference between the passenger's expectations regarding driving support control and the actual driving support control. FIG. 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1 . 10A and 10B are diagrams showing examples of character images displayed during driving support control. FIG. 10A is a diagram showing an example of a character image displayed when driving assistance control is successfully completed, and FIG. 10B is a diagram showing an example of a character image displayed during manual driving control. 10A and 10B are diagrams showing examples of character images that are displayed when there is a large difference between the temperature setting of the air conditioner and the temperature inside the vehicle cabin. FIG. 10A is a diagram showing an example of a character image that is displayed when a sensor detects an object around the vehicle, and FIG. 10B is a diagram showing an example of a character image that is displayed when the automatic brake is activated. 1 is a flowchart of a first example of a character display method according to an embodiment. 10 is a flowchart of a second example of a character display method according to an embodiment. 10 is a flowchart of a third example of a character display method according to an embodiment.

Embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, identical or similar parts will be designated by identical or similar reference numerals, and redundant explanations will be omitted. The drawings are schematic and may differ from the actual product. The embodiments shown below exemplify devices and methods that embody the technical concept of the present invention, but the technical concept of the present invention is not limited to the devices and methods exemplified in the following embodiments. The technical concept of the present invention can be modified in various ways within the technical scope described in the claims.

(composition)
1 is a diagram showing an example of a schematic configuration of a vehicle equipped with a driving assistance device according to an embodiment. The vehicle 1 is equipped with a driving assistance device 10 that assists driving of the vehicle 1. The driving assistance device 10 detects a driving environment around the vehicle 1 and automatically controls the driving of the vehicle 1 based on the detected driving environment, thereby assisting an occupant (e.g., a driver) of the vehicle 1 in driving the vehicle 1.
For example, the driving assistance provided by the driving assistance device 10 for the host vehicle 1 may include steering assistance control that automatically controls at least the steering angle. For example, the driving assistance provided by the driving assistance device 10 may be lane departure prevention assistance.

Furthermore, for example, the driving assistance of the host vehicle 1 by the driving assistance device 10 may include speed control that automatically controls at least one of the acceleration and deceleration of the host vehicle. For example, the driving assistance by the driving assistance device 10 may be a following control that causes the host vehicle to drive so as to maintain a predetermined distance from a preceding vehicle, or a constant speed driving control.
Further, for example, the driving assistance provided by the driving assistance device 10 may include autonomous driving control that automatically drives the vehicle 1 without the involvement of a passenger. Furthermore, the driving assistance provided by the driving assistance device 10 may be automatic parking assistance that automatically parks the vehicle at a target parking position by controlling the acceleration, deceleration, and steering angle of the vehicle.

The driving assistance device 10 includes a positioning device 11, a map database 12, an object sensor 13, a vehicle sensor 14, a mode selector switch 15, a human-machine interface 17, a controller 18, and an actuator 19. In the drawings, the map database is referred to as "map DB," the mode selector switch as "mode selector SW," and the human-machine interface as "HMI."

The positioning device 11 measures the current position of the vehicle 1. The positioning device 11 may include, for example, a Global Navigation System (GNSS) receiver. The GNSS receiver is, for example, a Global Positioning System (GPS) receiver, and receives radio waves from multiple navigation satellites to measure the current position of the vehicle 1. The positioning device 11 may also be an inertial navigation system.
The map database 12 stores road map data. For example, the map database 12 may store high-precision map data (hereinafter simply referred to as "high-precision map") suitable as map information for autonomous driving. The road map data stored in the map database 12 may also be map data for navigation (hereinafter simply referred to as "navigation map"). The high-precision map is map data with higher precision than the navigation map.

The object sensor 13 acquires various information (surrounding environment information) about the driving environment around the host vehicle 1. For example, the object sensor 13 detects objects around the host vehicle 1. The object sensor 13 detects the surrounding environment of the host vehicle 1, such as objects present around the host vehicle 1, the relative position between the host vehicle 1 and the objects, the distance between the host vehicle 1 and the objects, and the direction in which the objects are present. The object sensor 13 outputs the detected surrounding environment information to the controller 18.
For example, the object sensor 13 may include a camera, a LiDAR (Light Detection and Ranging), a radar, a millimeter wave radar, a laser range finder, a sonar, or the like.

The vehicle sensors 14 detect various information (vehicle information) obtained from the host vehicle 1. The vehicle sensors 14 include, for example, a vehicle speed sensor that detects the traveling speed (vehicle speed) V of the host vehicle 1, wheel speed sensors that detect the rotational speed of each tire equipped on the host vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) of the host vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle of the steering wheel, a turning angle sensor that detects the turning angle of the steered wheels, a gyro sensor that detects the angular velocity generated in the host vehicle 1, a yaw rate sensor that detects the yaw rate, an accelerator sensor that detects operation of the accelerator pedal of the host vehicle 1, a brake sensor that detects operation of the brake pedal by the driver, and a temperature sensor that detects the air temperature inside the cabin of the host vehicle 1. The vehicle sensors 14 output the vehicle information to the controller 18.

The controller 18 is an electronic control unit (ECU) that performs driving assistance control of the host vehicle 1. The controller 18 includes a processor 18a and peripheral components such as a storage device 18b. The processor 18a may be, for example, a central processing unit (CPU) or a micro-processing unit (MPU).
The storage device 18b may include a semiconductor storage device, a magnetic storage device, an optical storage device, etc. The storage device 18b may include memories such as a register, a cache memory, a ROM (Read Only Memory) used as a main memory device, and a RAM (Random Access Memory).
The functions of the controller 18 described below are realized by, for example, the processor 18a executing a computer program stored in the storage device 18b.
The controller 18 may be formed of dedicated hardware for executing the information processing described below. For example, the controller 18 may include a functional logic circuit configured in a general-purpose semiconductor integrated circuit. For example, the controller 18 may include a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

Actuator 19 operates the steering wheel, accelerator opening, and brake device of host vehicle 1 in response to control signals from controller 18 to generate vehicle behavior of host vehicle 1. Actuator 19 includes a steering actuator, an accelerator opening actuator, and a brake control actuator. The steering actuator controls the steering direction and amount of steering of host vehicle 1. The accelerator opening actuator controls the accelerator opening of host vehicle 1. The brake control actuator controls the braking operation of the brake device of host vehicle 1.

The mode changeover switch 15 is a switch that accepts operation by the occupant to switch the driving mode of the host vehicle 1. The occupant can switch the driving mode of the host vehicle 1 by operating the mode changeover switch 15.
For example, the mode changeover switch 15 may be a switch that switches the driving mode of the vehicle 1 between an automatic driving mode and a manual driving mode.
The autonomous driving mode is a mode in which the controller 18 performs autonomous driving control to automatically drive the vehicle 1 without the involvement of the occupants by driving the actuator 19 based on the surrounding environment information output from the object sensor 13 and the vehicle information output from the vehicle sensor 14.

On the other hand, the manual driving mode is a mode in which the occupant drives the vehicle 1 by manually operating the steering wheel, accelerator pedal, and brake pedal.
In the manual driving mode, the controller 18 may execute driving assistance control that automatically controls at least one of the acceleration, deceleration, or steering angle of the host vehicle 1. When executing driving assistance control, the controller 18 controls at least one of the acceleration, deceleration, or steering angle by driving the actuator 19. For example, the controller 18 may execute lane departure prevention control, following control, or constant speed cruise control. The mode selector switch 15 may be a switch for switching between activation and deactivation of the lane departure prevention control, following control, or constant speed cruise control. Furthermore, for example, the mode selector switch 15 may be a switch for switching between activation and deactivation of automatic parking control.

The human-machine interface 17 is an interface device that exchanges information between the driving assistance device 10 and the occupant. The human-machine interface 17 includes an operator that accepts an operation input by the occupant to the driving assistance device 10. The operator may be a mechanical interface device such as a button, switch, lever, dial, or keyboard, or may be a button, switch, lever, dial, keyboard, or the like displayed on a touch panel.
The human-machine interface 17 may also include a speaker or buzzer for outputting alarm sounds, notification sounds, and audio information.
Furthermore, the human-machine interface 17 includes a display device that can be viewed by the occupants of the vehicle 1 (for example, a display screen of a navigation system or a display device provided near the meter in front of the driver's seat). In the following description, the display device of the human-machine interface 17 will be simply referred to as the "display device."

The display device displays character images that characterize the driving support functions of the driving support device 10. Figures 2(a) and 2(b) are diagrams showing examples of character images 20 and 21 displayed on the display device.
As described above, the occupant may feel uncomfortable with the driving assistance control executed by the vehicle 1. For example, the occupant may feel uncomfortable if the driving assistance function is deactivated without the occupant's intention or if the results of the driving assistance function do not meet the occupant's expectations.

Therefore, the controller 18 estimates the passenger's expectations regarding the driving support control by the driving support device 10. Then, the controller 18 changes the behavior and facial expression of the character 2 displayed on the display device according to the difference between the estimated expectations and the driving support control that is actually executed.
For example, if the actually executed driving support control is likely to deviate from the passenger's expectations, a character image 20 of a character 2 driving hard is displayed on the display device as shown in FIG. 2(a).

Also, for example, if the driving support control does not meet the expectations of the occupant, a character image 21 of the character 2 with his head down as shown in Fig. 2(b) may be displayed on the display device, followed by a character image 20 of the character 2 driving hard as shown in Fig. 2(a).
Also, for example, when the driving support function is deactivated unintentionally by the driver, a character image 21 of the character 2 with its head down may be displayed as shown in FIG. 2(b).
In this way, by displaying a character 2 according to the difference between the estimated expected value and the driving support control that was actually executed, the discomfort felt by the driver regarding the results of the driving support control or the discomfort felt by the driving support control being canceled can be alleviated by the behavior and facial expression of the character 2.

The controller 18 may display, on the display device, an image of the character 2 adjusting the temperature when the temperature difference between the air temperature inside the vehicle 1 and the set temperature of the air conditioner 16 of the vehicle 1 is equal to or greater than a threshold. Information regarding the set temperature may be acquired from the air conditioner 16.
The controller 18 may also display a character image on the display device in accordance with an object around the vehicle detected by the object sensor 13 .

The function of the controller 18 to generate and display character images will be described in more detail below. FIG. 3 is a block diagram showing an example of the functional configuration of the controller 18.
The controller 18 includes a mode switching unit 30, a situation recognition unit 31, an operation input recognition unit 32, a room temperature recognition unit 33, an automatic driving control unit 34, an expected value estimation unit 35, a sensor display determination unit 36, a room temperature comparison unit 37, a comparison unit 38, and a character image generation unit 39.
The situation recognition unit 31 recognizes the current situation of the vehicle 1. The situation recognition unit 31 includes a driving state recognition unit 31a, a surrounding environment recognition unit 31b, a position recognition unit 31c, and a vehicle state recognition unit 31d.

The running state recognition unit 31a recognizes the running state of the host vehicle 1 based on vehicle information from the vehicle sensor 14. For example, the running state recognition unit 31a may recognize the speed, acceleration, deceleration, angular velocity, and yaw rate of the host vehicle 1 as the running state of the host vehicle 1, and detect the magnitude of each as a state quantity of the running state.
Further, for example, when automatic parking control is being executed as driving assistance control by the driving assistance device 10, the time elapsed since the start time of the automatic parking control or the number of turns may be detected as state quantities of the driving state.

The position recognition unit 31c measures the absolute position of the vehicle 1, i.e., the position, attitude, and speed of the vehicle 1 relative to a predetermined reference point, based on odometry using the measurement results from the positioning device 11 and the detection results from the vehicle sensor 14. The position recognition unit 31c acquires map information about the area around the vehicle 1 from the map database 12. The map information may also be acquired from an external map data server via a communication device. The position recognition unit 31c estimates the position and attitude of the vehicle 1 on the map from the absolute position of the vehicle 1 and the map information.
The position recognition unit 31c identifies the road on which the vehicle 1 is traveling and further the lane on that road in which the vehicle 1 is traveling, and detects the lateral position of the vehicle 1 within the traveling lane (vehicle width position, lateral position within the lane) and the amount of variation thereof as state quantities of the vehicle's traveling state.

The surrounding environment recognition unit 31b recognizes the surrounding environment of the vehicle 1 based on the detection results of the object sensor 13. For example, based on the detection results of the object sensor 13, it detects the positions, postures, sizes, speeds, etc. of objects around the vehicle 1, such as vehicles (cars and motorcycles), pedestrians, obstacles, etc.
The surrounding environment recognition unit 31b then integrates the multiple detection results obtained from the multiple object detection sensors and outputs a single two-dimensional position, orientation, size, speed, etc. for each object. Specifically, from the object behavior obtained from each object detection sensor, the unit calculates the most reasonable object behavior that minimizes error, taking into account the error characteristics of each object detection sensor. Specifically, by using known sensor fusion technology, the detection results obtained from multiple types of sensors are comprehensively evaluated to obtain more accurate detection results.

Furthermore, the surrounding environment recognition unit 31b verifies (corresponds) the identity of objects between different times based on the behavior of the objects output at different times based on the integrated detection results, and predicts the behavior of the object, such as its speed, based on this correspondence.
When the surrounding environment recognition unit 31b detects a vehicle ahead of the host vehicle 1, the surrounding environment recognition unit 31b may detect the inter-vehicle distance between the host vehicle 1 and the preceding vehicle as a state quantity of the traveling state.
The surrounding environment recognition unit 31b also recognizes the driving scene of the vehicle 1 based on the image captured by the camera of the object sensor 13 and map information about the area around the vehicle 1. For example, the surrounding environment recognition unit 31b may recognize whether the vehicle 1 is driving on a curved road as the driving scene. Alternatively, the surrounding environment recognition unit 31b may recognize whether the vehicle 1 is driving on a narrow road as the driving scene.
For example, the driving scene may be recognized as whether the vehicle 1 is traveling in a merging lane. For example, the driving scene may be recognized as whether there is a stopping target such as a stop line or an obstacle ahead of the vehicle 1.

The vehicle state recognition unit 31d recognizes other vehicle states of the host vehicle 1. For example, the vehicle state recognition unit 31d recognizes the soundness and detection accuracy of the object sensor 13 and the vehicle sensor 14, and any abnormalities that have occurred in the object sensor 13 and the vehicle sensor 14.
Furthermore, for example, the vehicle state recognition unit 31d may recognize whether map information necessary for driving assistance control by the driving assistance device 10 is available. For example, the vehicle state recognition unit 31d may recognize whether a high-precision map necessary for autonomous driving control of the vehicle 1 is available. For example, the vehicle state recognition unit 31d may recognize whether map information for the section in which the vehicle 1 is traveling is stored in the map database 12.

The operation input recognition unit 32 recognizes the steering angle of the steering wheel, the operation amount of the accelerator pedal, and the operation amount of the brake pedal based on the detection results of the steering angle sensor, accelerator sensor, and brake sensor of the vehicle sensor 14.
The room temperature recognition unit 33 recognizes the temperature inside the vehicle cabin of the vehicle 1 based on the detection result of the temperature sensor of the vehicle sensor 14.

The mode switching unit 30 switches the driving mode of the vehicle 1 between an automatic driving mode and a manual driving mode based on the operation of the mode switching switch 15 by the occupant.
Furthermore, when the operation input recognition unit 32 recognizes an override operation by the occupant, the mode switching unit 30 switches the driving mode of the vehicle 1 to the manual driving mode and cancels the driving assistance control by the driving assistance device 10 .
An override operation refers to an occupant operating the steering wheel, accelerator pedal, or brake pedal in autonomous driving mode to intervene in driving assistance control.
Switching from the automatic driving mode to the manual driving mode by operating the mode changeover switch 15 or by an override operation is an example of cancellation of the driving assistance control at the driver's discretion.

Furthermore, the mode switching unit 30 switches the driving mode of the host vehicle 1 from the autonomous driving mode to the manual driving mode based on the surrounding environment recognized by the surrounding environment recognition unit 31 b or the vehicle state of the host vehicle 1 recognized by the vehicle state recognition unit 31 d. In other words, the driving assist control is canceled. Cancellation of driving assist control based on the surrounding environment or the vehicle state is an example of cancellation of driving assist control unintentionally by the occupant (or cancellation of driving assist control unintentionally by the occupant).
For example, when the surrounding environment recognition unit 31b recognizes that the vehicle 1 has reached a point a predetermined distance before a section where autonomous driving is difficult (hereinafter, may be referred to as a "section where autonomous driving is difficult"), the mode switching unit 30 may switch the driving mode of the vehicle 1 from the autonomous driving mode to the manual driving mode. The section where autonomous driving is difficult may be, for example, a merging section where the vehicle merges into a congested lane.

Furthermore, for example, the driving mode of the host vehicle 1 may be switched from the autonomous driving mode to the manual driving mode when the soundness or detection accuracy of the object sensor 13 or the vehicle sensor 14 falls below an allowable limit value or when the vehicle state recognition unit 31d recognizes that an abnormality has occurred in the object sensor 13 or the vehicle sensor 14. Furthermore, for example, when the vehicle state recognition unit 31d recognizes that map information required for driving assistance control by the driving assistance device 10 is unavailable, the driving mode of the host vehicle 1 may be switched from the autonomous driving mode to the manual driving mode.
Similarly, the mode switching unit 30 may cancel lane departure prevention control, following control, constant speed driving control, and automatic parking control based on the surrounding environment recognized by the surrounding environment recognition unit 31b or the vehicle state of the vehicle 1 recognized by the vehicle state recognition unit 31d.

Furthermore, before switching the driving mode of the host vehicle 1 from the autonomous driving mode to the manual driving mode, the mode switching unit 30 may determine whether or not the driving mode of the host vehicle 1 is about to be switched from the autonomous driving mode to the manual driving mode, based on the surrounding environment recognized by the surrounding environment recognition unit 31 b or the vehicle state of the host vehicle 1 recognized by the vehicle state recognition unit 31 d. In other words, the mode switching unit 30 may determine whether or not the driving assist control is about to be released.
For example, when the vehicle 1 approaches an area where autonomous driving is difficult or an area where map information is unavailable, it may be determined that the driving mode of the vehicle 1 is likely to be switched from the autonomous driving mode to the manual driving mode. Also, when the soundness or detection accuracy of the object sensor 13 or the vehicle sensor 14 approaches an allowable limit value, it may be determined that the driving mode of the vehicle 1 is likely to be switched from the autonomous driving mode to the manual driving mode.
Similarly, the mode switching unit 30 may determine whether or not lane departure prevention control, following control, constant speed driving control, or automatic parking control is about to be released based on the surrounding environment recognized by the surrounding environment recognition unit 31b or the vehicle state of the vehicle 1 recognized by the vehicle state recognition unit 31d.

When the driving mode of the vehicle 1 is set to the autonomous driving mode, the autonomous driving control unit 34 performs autonomous driving control of the vehicle 1 based on the driving state of the vehicle 1 recognized by the driving state recognition unit 31a, the current position and attitude of the vehicle 1 recognized by the position recognition unit 31c, and the surrounding environment of the vehicle 1 recognized by the surrounding environment recognition unit 31b.
For example, the automatic driving control unit 34 calculates a target driving trajectory for the vehicle 1 based on the current position and attitude of the vehicle 1, a target route to the destination set by a navigation system or the like (not shown), and the surrounding environment of the vehicle 1.

For example, a route space map is generated that shows the route around the vehicle 1 and whether or not there are any objects, and a risk map that quantifies the danger level of the driving area, and a target driving trajectory for the vehicle 1 is generated based on the movement characteristics of the vehicle 1, the route space map, and the risk map.
The automatic driving control unit 34 drives the actuator 19 so that the vehicle 1 travels along the generated target travel trajectory.

In addition, when the driving mode of the vehicle 1 is set to manual driving mode and lane departure prevention control is activated, the automatic driving control unit 34 may automatically control the steering angle of the vehicle 1 so as to maintain the lateral position of the vehicle 1 within the lane (for example, so as to drive in the center of the lane in accordance with the lane shape) based on the driving state of the vehicle 1 and the current lateral position and attitude of the vehicle 1 within the lane.
For example, when follow-up control or constant speed driving control is activated, the automatic driving control unit 34 may automatically control the acceleration and deceleration of the vehicle based on the driving state of the vehicle 1 and the surrounding environment, so as to maintain the distance between the vehicle and the preceding vehicle, or so as to drive at a predetermined set speed.

For example, when automatic parking control is activated, the automatic driving control unit 34 calculates the relative position of the target parking position with respect to the current position of the vehicle 1 based on the surrounding environment of the vehicle 1, generates a target driving trajectory from the current position of the vehicle 1 to the target parking position, and drives the actuator 19 so that the vehicle 1 drives along the generated target driving trajectory.
For example, when the surrounding environment recognition unit 31b detects an obstacle located ahead of the vehicle 1 in the direction of travel, the automatic driving control unit 34 performs automatic braking control (e.g., emergency braking control) to stop the vehicle 1 in front of the obstacle.

The expected value estimation unit 35 estimates the expected values of the occupants of the vehicle 1 regarding the driving support control by the driving support device 10 .
For example, it is considered that the passengers expect the following (A) to (G) from the driving support control by the driving support device 10.
(A) The driving assistance control is not released unintentionally by the occupant.
(B) Driving along the lane (i.e., not changing lateral position within the lane).
(C) Do not accelerate or decelerate suddenly, or accelerate or decelerate too slowly. (D) Do not drive too fast or too slow.
(E) The distance between your vehicle and the vehicle ahead is not too short.
(F) The time required for automatic parking should not be too long.
(G) The number of turns during automatic parking should not be too many.

For this reason, the expected value estimation unit 35 estimates the occupant's expectations regarding the vehicle width position of the vehicle 1 within the driving lane, the amount of variation in the vehicle width position, the speed, acceleration, and deceleration of the vehicle 1, the distance between the vehicle 1 and the preceding vehicle, the time required for parking when automatic parking control is activated, and the number of turns required for parking.
For example, the expected value estimating unit 35 may estimate upper limits acceptable to the occupant as expected values for the vehicle width direction position, the amount of change in the vehicle width direction position, the time required for parking, and the number of turns required for parking. For example, the expected value estimating unit 35 may estimate lower limits acceptable to the occupant as expected values for the inter-vehicle distance.
For example, the expected value estimation unit 35 may estimate upper and lower limit values acceptable to the occupant as expected values for the speed, acceleration, and deceleration of the vehicle 1. In other words, the expected value estimation unit 35 may estimate the ranges of acceleration, deceleration, and speed acceptable to the occupant. These upper and lower limit values are examples of the "limit values" described in the claims.

For example, the expected value estimation unit 35 may calculate expected values for the lateral position of the host vehicle 1 within the lane in which the host vehicle 1 is traveling and the distance between the vehicle and the preceding vehicle based on driving conditions such as the width of the lane in which the host vehicle 1 is traveling, the shape of the lane (such as curvature), and the speed of the host vehicle. Alternatively, a lookup table of typical driver expectations for lateral position and distance between the vehicles depending on the width, shape of the lane, and driving conditions may be stored in advance, and the expected values may be read from this lookup table. The lookup table may also be created based on the history of lateral position and distance between the vehicles when the occupant of the host vehicle 1 was previously driving.

Furthermore, for example, the expected value estimation unit 35 may estimate expected values for the acceleration and deceleration of the host vehicle 1 based on the driving scene recognized by the surrounding environment recognition unit 31b.
For example, when the surrounding environment recognition unit 31b recognizes that the host vehicle 1 is traveling in a merging lane, the expected value for acceleration in the merging lane may be estimated. Also, when the surrounding environment recognition unit 31b recognizes a stopping target ahead of the host vehicle 1, the expected value estimation unit 35 may estimate the expected value for deceleration for each driving scene based on the history of acceleration and deceleration when the occupant of the host vehicle 1 previously drove in a similar driving scene.

Furthermore, for example, the expected value estimation unit 35 may estimate an expected value for the vehicle speed of the host vehicle 1 based on the driving scene recognized by the surrounding environment recognition unit 31b.
For example, when the surrounding environment recognition unit 31b recognizes that the vehicle 1 is traveling on a curved road or a narrow road, the expected value estimation unit 35 may estimate an expected value for the vehicle speed of the vehicle 1 traveling on these sections. For example, the expected value estimation unit 35 may estimate an expected value for the vehicle speed for each traveling scene based on a history of vehicle speeds when the occupant of the vehicle 1 drove in similar traveling scenes in the past.

For example, when automatic parking control is activated, the expected value estimation unit 35 may estimate the expected value for the time required for parking and the number of turns required for parking based on obstacles around the vehicle 1 recognized by the surrounding environment recognition unit 31b and the relative distance between the current position of the vehicle 1 and the target parking position calculated by the automatic driving control unit 34.
For example, it is possible to pre-store and read from a lookup table of typical driver expectations for the time required and the number of turns required depending on the arrangement pattern and relative distance of obstacles around the vehicle 1. The lookup table may be created based on the history of the time required and the number of turns required when the occupant of the vehicle 1 previously parked.

The comparison unit 38 compares the driving states of the host vehicle 1 recognized by the driving state recognition unit 31 a, the surrounding environment recognition unit 31 b, and the position recognition unit 31 c with the expected values estimated by the expected value estimation unit 35, and determines whether the driving states of the host vehicle generated by the driving support control of the driving support device 10 deviate from the expected values estimated by the expected value estimation unit 35. The comparison unit 38 also determines whether the driving states of the host vehicle generated by the driving support control are likely to deviate from the expected values.
For example, the comparison unit 38 may determine that the driving state of the vehicle 1 deviates from the expected value when the state quantities of the driving state of the vehicle 1 detected by the driving state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c exceed the limit values (i.e., upper and lower limit values) estimated as expected values by the expected value estimation unit 35.
In the following description, the state quantities of the running state of the vehicle 1 detected by the running state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c may be referred to as "detected state quantities."

Also, for example, when the detected state quantity approaches an estimated limit value, it may be determined that the running state of the host vehicle 1 is likely to deviate from the expected value.
For example, it may be determined whether the difference between the detected state quantity and the estimated limit value is equal to or less than a predetermined threshold, and if the difference is equal to or less than the predetermined value, it may be determined that the detected state quantity has approached the limit value.
Furthermore, for example, when an upper limit value is estimated as an expected value, it may be determined whether the ratio of the detected state quantity to the upper limit value estimated as the expected value (detected state quantity/upper limit value) is equal to or greater than a predetermined value, and when the ratio (detected state quantity/upper limit value) is equal to or greater than the predetermined value, it may be determined that the detected state quantity has approached the limit value.
Furthermore, for example, when a lower limit value is estimated as an expected value, it may be determined whether or not the ratio of the detected state quantity to the estimated lower limit value (detected state quantity/lower limit value) is equal to or less than a predetermined value. If the ratio (detected state quantity/lower limit value) is equal to or less than the predetermined value, it may be determined that the detected state quantity has approached the limit value.

The sensor display determination unit 36 determines whether an object around the vehicle has been recognized by the surrounding environment recognition unit 31b in order to generate a character image in accordance with an object around the vehicle detected by the object sensor 13. Furthermore, the sensor display determination unit 36 acquires information on the direction of the recognized object from the surrounding environment recognition unit 31b. For example, the sensor display determination unit 36 determines whether an object around the vehicle has been detected by the sonar of the object sensor 13, and acquires information on the direction of the detected object from the surrounding environment recognition unit 31b.
The room temperature comparison unit 37 detects the temperature difference between the air temperature inside the vehicle 1 recognized by the room temperature recognition unit 33 and the set temperature of the air conditioner 16. The room temperature comparison unit 37 determines whether the detected temperature difference is equal to or greater than a predetermined threshold value. In other words, it determines whether the air temperature inside the vehicle is too high or too low compared to the set temperature.

The character image generating unit 39 generates a character image to be displayed on the display device based on the driving mode of the vehicle 1, the comparison results of the comparing unit 38 and the room temperature comparing unit 37, and the determination result of the sensor display determining unit 36.
For example, when the driving mode of the host vehicle 1 is the automatic driving mode (i.e., when driving assistance control of the host vehicle 1 is being executed), the character image generation unit 39 generates a character image of a character performing an action related to driving the vehicle. For example, the character image generation unit 39 may generate a character image 22 in which the character 2 is driving the host vehicle 1 as shown in FIG. 4( a).
As indicated by arrow 22a, the character image generation unit 39 may tilt the steering wheel 3 held by the character 2 or the body of the character 2 in accordance with the actual steering angle of the host vehicle 1. Similarly, as indicated by arrows 20a and 23a, the character image 20 in Fig. 2(a) and the character image 23 in Fig. 4(b) may tilt the steering wheel 3 held by the character 2 or the body of the character 2 in accordance with the actual steering angle of the host vehicle 1.
Also, for example, when automatic parking control is activated and the vehicle is reversing, a character image 22 is generated in which the character 2 is driving the vehicle 1 while facing backward, as shown in FIG. 4(b).

The character image generation unit 39 generates a character image of the character 2 whose behavior and expression change depending on the difference between the expected value estimated by the expected value estimation unit 35 and the driving state of the vehicle 1 recognized by the driving state recognition unit 31a, the surrounding environment recognition unit 31b, and the position recognition unit 31c (i.e., the driving state of the vehicle 1 generated by the driving assistance control of the driving assistance device 10).
Specifically, when the comparison unit 38 determines that the driving state of the vehicle 1 is likely to deviate from the expected value, the character image generation unit 39 generates a character image 20 in which the character 2 is driving hard, as shown in Fig. 2(a). Alternatively, a character image in which the character 2 is impatient or sweating may be generated.

Furthermore, when the running state of the vehicle 1 deviates from the expected value, a character image 21 is generated in which the character 2 hangs its head and expresses regret, as shown in Fig. 2(b). Alternatively, a character image of the character 2 apologizing, expressing apologetic feelings, feeling regretful, or with a pale face may be generated.
Furthermore, even when the mode switching unit 30 cancels the driving support function unintentionally by the occupant, the character image generating unit 39 generates a character image 21 in which the character 2 hangs its head and expresses regret, as shown in Fig. 2(b). Alternatively, a character image in which the character 2 apologizes, expresses apologetic intent, feels regret, or has a pale face may be used.
Furthermore, even when the mode switching unit 30 determines that the driving support function is likely to be deactivated unintentionally by the occupant, the character image generating unit 39 generates a character image 20 in which the character 2 is driving hard, as shown in Fig. 2(a). Alternatively, a character image in which the character 2 is impatient or sweating may be generated.

Meanwhile, the character image generation unit 39 also determines whether the driving assistance control of the driving assistance device 10 has been successfully completed. For example, if the mode switching unit 30 cancels the driving assistance function at the driver's discretion, the character image generation unit 39 may determine that the driving assistance control has been successfully completed. Also, for example, if automatic parking control is being executed, the character image generation unit 39 may determine that the driving assistance control has been successfully completed even if parking is completed.
Furthermore, when the driving support control is completed, it may be determined whether the driving support control has been successfully completed depending on whether the occupant feels uncomfortable. For example, the facial expression of the occupant may be recognized from an image captured by an in-vehicle camera, and it may be determined that the occupant feels uncomfortable if the facial expression is classified as a "surprise" or "anger" expression.
If the driving support control is successfully completed, the character image generating unit 39 may generate a character image 24 of the character 2 looking happy, as shown in FIG. 5( a ).

On the other hand, when the driving mode of the vehicle 1 is a manual driving mode (for example, when driving assistance control of the vehicle 1 is not being executed), the character image generation unit 39 generates a character image of the character 2 performing an action that shows the immaturity or unreliability of the character 2.
As a character image of character 2 performing an action that shows immaturity or unreliability, for example, a character image of character 2 performing an action unrelated to driving a vehicle may be generated. For example, a character image of character 2 wandering around without driving a vehicle, or a character image in which images of other objects such as small animals, toys, or balls are drawn simultaneously with character 2, and character 2 is distracted by other objects may be generated.
Furthermore, a character image of a character 2 failing to perform an action unrelated to driving a vehicle may be generated as a character 2 performing an action that indicates immaturity or unreliability. For example, as shown in Fig. 5(b), a character image 24 of a character 2 wandering around and falling, a character image 24 of a character 2 failing to perform an action unrelated to driving a vehicle and therefore taking a long time to perform an action related to driving a vehicle, or a character image 24 of a character 2 becoming distracted or distracted by the surroundings while performing an action unrelated to driving a vehicle may be generated.
In this way, by presenting the occupant with the character image 24 that represents the childishness or unreliability of the character 2, it is possible to lower the occupant's expectations regarding the driving support control of the driving support device 10. This reduces the discomfort felt by the occupant when the driving support function is cancelled without the occupant's intention or when the results of the driving support function do not meet the occupant's expectations.

In addition, when the room temperature recognition unit 33 determines that the temperature difference between the air temperature inside the vehicle cabin and the set temperature of the air conditioning unit 16 is greater than or equal to a threshold value, the character image generation unit 39 generates a character image in which character 2 is adjusting the temperature inside the vehicle cabin.
For example, if the temperature inside the vehicle is higher than the set temperature, a character image of the character 2 desperately trying to cool the vehicle interior may be generated. For example, as shown in Fig. 6(a), a character image 26 of the character 2 desperately fanning himself with a fan 26a may be generated.
For example, if the temperature inside the vehicle is lower than the set temperature, a character image of the character 2 frantically trying to warm up the vehicle may be generated. For example, as shown in FIG. 6B, a character image 27 of the character 2 frantically starting a bonfire 27a may be generated.

Furthermore, when the sensor display determination unit 36 determines that an object around the vehicle has been recognized by the surrounding environment recognition unit 31b (i.e., that an object around the vehicle has been detected by the object sensor 13), the character image generation unit 39 generates a character image 28 in which the character 2 looks anxious and faces in the direction in which the object has been detected, as shown in Fig. 7(a). For example, the character image generation unit 39 generates a character image 28 in which the character looks in the direction in which the object has been detected by the sonar of the object sensor 13.
In addition, when the automatic driving control unit 34 performs automatic braking control (e.g., emergency braking control) to stop the vehicle 1 in front of an obstacle, the character image generation unit 39 generates a character image 29 in which the character 2 is scared, as shown in Figure 7 (b).
The character image generating unit 39 displays these character images on a display device.

(operation)
FIG. 8 is a flowchart of a first example of a character display method according to an embodiment.
In step S1, the character image generation unit 39 determines whether the driving mode of the vehicle 1 is the automatic driving mode. If the driving mode of the vehicle 1 is the automatic driving mode (step S: Y1), the process proceeds to step S2. If the driving mode of the vehicle 1 is not the automatic driving mode (step S1: N), the process proceeds to step S8.
In step S2, the character image generating unit 39 generates a character image 22 of the character 2 driving the vehicle 1 as shown in FIG. 4(a), and displays it on the display device.

In step S3, the comparison unit 38 determines whether the driving support control (driving control) of the driving support device 10 deviates from the expected value estimated by the expected value estimation unit 35. If the driving support control deviates from the expected value (step S3: Y), the process proceeds to step S4. If the driving support control does not deviate from the expected value (step S3: N), the process proceeds to step S6.
In step S4, the character image generating unit 39 generates a character image 21 in which the character 2 expresses regret as shown in FIG. 2(b), and displays it on the display device.
Thereafter, in step S5, the character image generating unit 39 generates a character image 20 of the character 2 driving hard as shown in Fig. 2(a) and displays it on the display device, after which the process ends.

In step S6, the comparison unit 38 determines whether the driving support control (driving control) of the driving support device 10 is likely to deviate from the expected value estimated by the expected value estimation unit 35. If the driving support control is likely to deviate from the expected value (step S6: Y), the process proceeds to step S7. If the driving support control is unlikely to deviate from the expected value (step S6: N), the process ends.
In step S7, the character image generating unit 39 generates a character image 20 of the character 2 driving hard as shown in Fig. 2(a) and displays it on the display device, after which the process ends.

In step S8, the character image generation unit 39 determines whether the driving mode of the host vehicle 1 was the automatic driving mode in the previous control cycle. If the driving mode of the host vehicle 1 was not the automatic driving mode (step S8: N), that is, if the driving mode of the host vehicle 1 was the manual driving mode in the previous control cycle, the process proceeds to step S9.
If the driving mode of the vehicle 1 was the automatic driving mode in the previous control cycle (step S8: Y), that is, if the driving mode has switched from the automatic driving mode to the manual driving mode in the current control cycle, the processing proceeds to step S10.
In step S9, the character image generating unit 39 generates an unreliable character image 24 as shown in Fig. 5(b) and displays it on the display device, after which the process ends.

In step S10, the character image generation unit 39 determines whether the driving mode of the vehicle 1 has switched from the automatic driving mode to the manual driving mode without the intention of the occupant. If the driving mode of the vehicle 1 has switched to the manual driving mode without the intention of the occupant (step S10: N), the process proceeds to step S11. If the driving mode has switched to the manual driving mode without the intention of the occupant (step S10: Y), the process proceeds to step S12.
In step S11, the character image generating unit 39 generates a character image 24 of the character 2 looking happy, as shown in Fig. 5(a), and displays it on the display device, after which the process ends.
In step S12, the character image generating unit 39 generates a character image 21 in which the character 2 expresses regret, as shown in Fig. 2(b), and displays it on the display device, after which the process ends.

FIG. 9 is a flowchart of a second example of the character display method according to the embodiment.
In step S20, the room temperature recognition unit 33 determines whether the temperature inside the vehicle cabin is significantly higher than the set temperature of the air conditioner 16. If the temperature inside the vehicle cabin is significantly higher than the set temperature (step S20: Y), the process proceeds to step S21. If the temperature inside the vehicle cabin is not significantly higher than the set temperature (step S20: N), the process proceeds to step S22.
In step S21, the character image generating unit 39 generates a character image 26 of the character 2 desperately trying to cool the interior of the vehicle, as shown in Fig. 6(a), and displays it on the display device, after which the process ends.

In step S22, the room temperature recognition unit 33 determines whether the temperature inside the vehicle cabin is significantly lower than the set temperature of the air conditioner 16. If the temperature inside the vehicle cabin is significantly lower than the set temperature (step S22: Y), the process proceeds to step S23. If the temperature inside the vehicle cabin is not significantly lower than the set temperature (step S22: N), the process ends.
In step S23, the character image generating unit 39 generates a character image 27 showing the character 2 desperately trying to warm the interior of the vehicle, as shown in Fig. 6(b), and displays it on the display device, after which the process ends.

FIG. 10 is a flowchart of a third example of the character display method according to the embodiment.
In step S30, the sensor display determination unit 36 determines whether an object has been detected around the vehicle by the object sensor 13 (e.g., sonar). If an object has been detected (step S30: Y), the process proceeds to step S31. If an object has not been detected (step S30: N), the process proceeds to step S32.
In step S31, the character image generating unit 39 generates a character image 28 in which the character 2 looks in the direction of the detected object with an anxious expression, as shown in Fig. 7(a), and displays it on the display device. Thereafter, the process ends.

In step S32, the character image generation unit 39 determines whether the automatic driving control unit 34 has performed automatic braking control (e.g., emergency braking control). If automatic braking control has been performed (step S32: Y), the process proceeds to step S33. If automatic braking control has not been performed (step S32: N), the process ends.
In step S33, the character image generating unit 39 generates a character image 29 of the character 2 that is scared, as shown in Figure 7(b), and displays it on the display device, after which the process ends.

(Effects of the embodiment)
(1) The controller 18 performs the following processes: detecting the surrounding environment of the vehicle 1; driving assistance control that automatically controls at least one of the acceleration, deceleration, or steering angle of the vehicle 1 based on the detected surrounding environment; estimating the expectations of the occupants of the vehicle 1 regarding the driving assistance control; generating a character image representing a character whose behavior or facial expression changes depending on the difference between the estimated expectations and the driving assistance control actually executed; and displaying the character image on a display device within the vehicle 1.
This allows the character's facial expressions and movements to mitigate the discomfort felt by the occupant due to the difference between the actually executed driving support control and the occupant's expectations.

(2) The controller 18 may execute a process for detecting a state quantity indicating the driving state of the vehicle 1 generated by the driving assistance control, and may generate a character image in which at least one of the behaviors or facial expressions changes depending on the difference between the estimated expected value and the detected state quantity.
This makes it possible to compare the actually executed driving support control with the passenger's expectations, and generate a character image according to the comparison result.
(3) When the detected state quantity exceeds a limit value estimated as an expected value, the controller 18 may generate a character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or turns pale.
This reduces the discomfort felt by the occupants when the driving support control deviates from the expected value.

(4) When the detected state quantity approaches a limit value estimated as an expected value, the controller 18 may generate a character image in which the character appears to be in a hurry, driving hard, or sweating.
This reduces the discomfort felt by the occupants when the driving support control is likely to deviate from the expected value.
(5) The detected state quantity may be the widthwise position of the vehicle 1 within the driving lane, the amount of change in the widthwise position, the speed, acceleration, deceleration of the vehicle 1, the distance between the vehicle 1 and the preceding vehicle, the elapsed time from the start time of automatic parking control when the driving assistance control is automatic parking control, or the number of turns that occurred during automatic parking control.
This allows the results of driving assistance control to be quantified and compared with expected values.

(6) The controller 18 may determine whether the driving assistance control has been released based on the surrounding environment or the state of the vehicle 1, and if the driving assistance control has been released based on the surrounding environment or the state of the vehicle 1, may generate a character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or looks pale.
This reduces the discomfort felt by the occupants when the driving support control is cancelled.
(7) The controller 18
It is determined whether or not the driving support control is likely to be released based on the surrounding environment or the state of the vehicle 1, and if it is determined that the driving support control is likely to be released based on the surrounding environment or the state of the vehicle 1, a character image may be generated in which the character is in a hurry, driving hard, or sweating.
This makes it possible to notify the occupants that the driving support control is about to be released, and also to reduce the discomfort felt by the occupants when the driving support control is subsequently released.

(8) When driving assistance control is not executed, the controller 18 may generate a character image of a character that performs an action related to driving the vehicle when driving assistance control is executed, and may generate a character image of a character that performs an action unrelated to driving the vehicle when driving assistance control is not executed.
For example, when the controller 18 does not execute driving assistance control, the controller 18 may generate a character image of a character that performs an action that is unrelated to driving the vehicle and fails.
This can lower the occupant's expectations regarding the driving support control of the driving support device 10. This can reduce the discomfort felt by the occupant when the driving support function is cancelled without the occupant's intention or when the results of the driving support function do not meet the occupant's expectations.
(9) When the driving assistance control is successfully completed, the controller 18 may generate a character image showing a happy character.
In this way, by expressing the joy of the passengers when the results are as intended, the passengers can empathize with the vehicle, and conversely, the passengers can be more receptive when the results are not as intended.
(10) The controller 18 may execute a process of detecting objects present around the vehicle 1 using a sensor, and when an object is detected, may generate a character image in which the character faces the direction in which the object is detected with an anxious expression.
This allows the occupant to be informed of the direction in which an object that could be an obstacle exists around the vehicle 1.
(11) The controller 18 may execute a process to detect a temperature difference between the set temperature of the air conditioner 16 and the temperature inside the vehicle 1, and if the temperature difference is equal to or greater than a threshold, may generate a character image of a character adjusting the temperature. This can reduce discomfort felt by occupants due to the temperature inside the vehicle 1 differing from the set temperature of the air conditioner 16.

1...Own vehicle, 10...Driving assistance device, 11...Positioning device, 12...Map database, 13...Object sensor, 14...Vehicle sensor, 15...Mode switch, 16...Air conditioning device, 17...Human-machine interface, 18...Controller, 18a...Processor, 18b...Storage device, 19...Actuator, 20...Character image, 30...Mode switching unit, 31...Situation recognition unit, 31a...Driving state recognition unit, 31b...Surrounding environment recognition unit, 31c...Position recognition unit, 31d...Vehicle state recognition unit, 32...Operation input recognition unit, 33...Room temperature recognition unit, 34...Autonomous driving control unit, 35...Expected value estimation unit, 36...Sensor display determination unit, 37...Room temperature comparison unit, 38...Comparison unit

Claims (13)

A process of detecting the surrounding environment of the vehicle;
a driving support control that automatically controls at least one of an acceleration, a deceleration, or a steering angle of the host vehicle based on the detected surrounding environment;
a process of estimating an expectation value of an occupant of the host vehicle regarding the driving assist control based on a width or a shape of a lane in which the host vehicle is traveling, a driving state of the host vehicle, or a result of detecting the surrounding environment;
generating a character image representing a character whose behavior or facial expression changes according to a difference between the estimated expected value and the actually executed driving support control;
a process of displaying the character image on a display device in the host vehicle;
A character display method characterized by causing a controller to execute the above. The controller
executes a process of detecting a state quantity that indicates a traveling state of the host vehicle generated by the traveling assist control;
generating the character image in which at least one of the behavior or facial expression of the character changes according to the difference between the estimated expected value and the detected state quantity;
2. The character display method according to claim 1. The character display method described in claim 2, characterized in that the controller generates the character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or becomes pale when the detected state quantity exceeds the limit value estimated as the expected value. The character display method described in claim 3, characterized in that the controller generates a character image in which the character appears impatient, driving hard, or sweating when the detected state quantity approaches a limit value estimated as the expected value. The character display method of any one of claims 2 to 4, characterized in that the detected state quantity is the vehicle width position of the host vehicle within the driving lane, the amount of change in the vehicle width position, the vehicle speed, acceleration, deceleration, the distance between the host vehicle and a preceding vehicle, the elapsed time from the start of automatic parking control when the driving assistance control is automatic parking control, or the number of turns occurring during the automatic parking control. The character display method described in any one of claims 1 to 5, characterized in that the controller determines whether the driving assistance control has been released based on the surrounding environment or the state of the host vehicle, and if the driving assistance control has been released based on the surrounding environment or the state of the host vehicle, generates the character image in which the character apologizes, expresses an apologetic attitude, hangs its head, looks disappointed, or becomes pale. The character display method of any one of claims 1 to 6, characterized in that the controller determines whether the driving assistance control is likely to be released based on the surrounding environment or the state of the host vehicle, and if the driving assistance control is likely to be released based on the surrounding environment or the state of the host vehicle, generates a character image in which the character appears impatient, driving hard, or sweating. The character display method described in any one of claims 1 to 7, characterized in that the controller generates the character image of the character that performs an action related to vehicle driving when the driving assistance control is executed, and generates the character image of the character that performs an action unrelated to vehicle driving when the driving assistance control is not executed. The character display method described in claim 8, characterized in that the controller generates the character image of the character performing an action that fails to perform the action unrelated to vehicle driving when the driving assistance control is not executed. The character display method described in any one of claims 1 to 9, characterized in that the controller generates the character image showing the character in a happy state when the driving assistance control is successfully completed. The controller
Execute a process of detecting an object present around the vehicle by a sensor;
When the object is detected, the character image is generated in which the character looks in the direction in which the object is detected with an anxious expression.
11. The character display method according to claim 1. The controller
Execute a process of detecting a temperature difference between a set temperature of an air conditioning device of the host vehicle and a temperature inside the vehicle interior;
If the temperature difference is equal to or greater than a threshold, a character image is generated in which the character is adjusting the temperature.
12. The character display method according to claim 1, wherein the character display method is a character display method for displaying a character. a sensor that detects objects around the vehicle;
a display device disposed in the vehicle;
a controller that executes the following processes: detecting the surrounding environment of the host vehicle based on an output signal of the sensor; driving support control that automatically controls at least one of the acceleration, deceleration, or steering angle of the host vehicle based on the detected surrounding environment; estimating an expectation value of an occupant of the host vehicle regarding the driving support control based on the detection result of the width or shape of the lane in which the host vehicle is traveling, the driving state of the host vehicle, or the surrounding environment ; generating a character image representing a character whose behavior or facial expression changes depending on the difference between the estimated expectation value and the driving support control actually performed; and displaying the character image on the display device;
A driving assistance device comprising: