JP7618926B2 - Driving assistance control device and driving assistance control method - Google Patents

Description

The present invention relates to a driving assistance control device and a driving assistance control method.

Technologies for estimating a user's emotions are known. For example, Patent Document 1 discloses a technology that reflects the estimation results of a user's past emotional state when estimating the user's emotional state based on information transmitted by the user.

JP 2017-29386 A

When a driver's emotions change, such as becoming irritated, they may engage in dangerous driving, such as tailgating or intentionally cutting in front of other vehicles. Dangerous driving may lead to accidents involving the driver's own vehicle or other vehicles. Therefore, it is desirable to appropriately determine changes in the driver's emotions and provide driving support in accordance with the determined emotions.

The present invention aims to provide a driving assistance control device and a driving assistance control method that can appropriately assist driving according to the driver's emotions.

The driving assistance control device according to the present invention includes a biometric information acquisition unit that acquires biometric information of a vehicle driver for a current and past predetermined period, an emotion determination unit that determines an emotion level indicating the degree of emotion of the driver for the current and past predetermined periods based on the biometric information acquired by the biometric information acquisition unit, a driving situation prediction unit that predicts the driving situation of the vehicle based on the driver's emotion levels for the current and past predetermined periods when the current emotion level of the driver is determined to be equal to or higher than a first threshold, a driving assistance determination unit that determines whether driving assistance is necessary for the vehicle based on a result of the prediction of the driving situation of the vehicle by the driving situation prediction unit and determines the content of the driving assistance for the vehicle if driving assistance is necessary, and a driving assistance execution unit that executes the driving assistance determined by the driving assistance determination unit.

The driving assistance control method according to the present invention includes the steps of acquiring biometric information of a vehicle driver for a predetermined period of time, determining an emotional level indicating the degree of the driver's emotion for the current and past predetermined periods based on the acquired biometric information, predicting the driving situation of the vehicle based on the driver's emotional levels for the current and past predetermined periods when the current emotional level of the driver is determined to be equal to or higher than a first threshold, determining whether driving assistance is necessary for the vehicle based on the predicted driving situation of the vehicle, and determining the content of the driving assistance for the vehicle if driving assistance is necessary for the vehicle, and executing the determined driving assistance.

The present invention makes it possible to provide appropriate driving assistance according to the driver's emotions.

FIG. 1 is a block diagram showing an example of the configuration of a driving assistance system according to the first embodiment. FIG. 2 is a flowchart showing an example of the flow of the driving assistance control process according to the first embodiment. FIG. 3 is a diagram for explaining a method for determining an emotion level according to the first embodiment. FIG. 4 is a diagram for explaining a method for determining a past emotion level according to the first embodiment. FIG. 5 is a diagram for explaining patterns of the contents of driving assistance according to the first embodiment. FIG. 6 is a diagram for explaining a first pattern of driving assistance content according to the first embodiment. FIG. 7 is a diagram for explaining a second pattern of driving assistance content according to the first embodiment. FIG. 8 is a diagram for explaining a third pattern of driving assistance content according to the first embodiment. FIG. 9 is a diagram for explaining a fourth pattern of driving assistance content according to the first embodiment. FIG. 10 is a diagram for explaining a fifth pattern of driving assistance content according to the first embodiment. FIG. 11 is a diagram for explaining a sixth pattern of driving assistance content according to the first embodiment. FIG. 12 is a diagram for explaining a seventh pattern of driving assistance content according to the first embodiment. FIG. 13 is a diagram for explaining an eighth pattern of driving assistance content according to the first embodiment. FIG. 14 is a block diagram showing an example of the configuration of a driving assistance system according to the second embodiment. FIG. 15 is a flowchart showing an example of the flow of a driving assistance control process according to the second embodiment. FIG. 16 is a block diagram showing an example of the configuration of a driving assistance system according to the third embodiment. FIG. 17 is a flowchart showing an example of the flow of a driving assistance control process according to the third embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. Note that the present invention is not limited to this embodiment, and in the following embodiments, the same parts are designated by the same reference numerals, and duplicated explanations will be omitted.

[First embodiment]
(Driving assistance system)
An example of the configuration of a driving assistance system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of a driving assistance system according to the first embodiment.

As shown in FIG. 1, the driving assistance system 1 includes an imaging device 10, a biosensor 12, and a driving assistance device 14. The imaging device 10 and the driving assistance device 14, and the biosensor 12 and the driving assistance device 14 are connected so that various data can be communicated.

The imaging device 10 includes a first camera 10a and a second camera 10b.

The first camera 10a is a camera that captures the surroundings of the vehicle. The first camera 10a is, for example, composed of a group of multiple cameras that capture the front, rear, left side, and right side of the vehicle. The first camera 10a may, for example, be composed of a single camera that can capture the entire sky. The first camera 10a may, for example, be composed of a single camera that captures only one direction, the front of the vehicle. The first camera 10a may, for example, have a microphone that picks up sound around the vehicle.

The second camera 10b is a camera that is positioned facing toward the interior of the vehicle cabin and captures the interior of the cabin. For example, the second camera 10b captures at least the face (facial expression, eye movement) of the driver of the vehicle. The second camera 10b is, for example, a wide-angle camera that can capture an image in a range of about 180°. The second camera 10b may be, for example, a group of multiple cameras. The second camera 10b may have, for example, a microphone that captures sound from within the cabin.

The biosensor 12 is a sensor that detects bioinformation of the vehicle driver. The biosensor 12 is, for example, a wearable device that is attached to the body of the vehicle driver. The biosensor 12 is, for example, attached to the arm, finger, torso, etc. of the vehicle driver, but may be attached to other body parts. The biosensor 12 detects the bioinformation of the vehicle driver at predetermined intervals, for example, and stores the detection results. The biosensor 12 is composed of, for example, a brain wave sensor, a pulse sensor, a blood pressure sensor, a body temperature sensor, and a sweat sensor, but may be composed of a sensor that detects other bioinformation. Specifically, the biosensor 12 is, for example, composed of a sensor that detects bioinformation that can predict the emotions of the vehicle driver.

The driving assistance device 14 is mounted on the vehicle. The driving assistance device 14 may be a device mounted on the vehicle, or may be a portable device that can be used in the vehicle. The driving assistance device 14 is a device that performs driving assistance according to the current and past emotional levels of the driver who drives the vehicle.

The driving assistance device 14 includes an input unit 20, a display unit 22, an audio output unit 24, a memory unit 26, a communication unit 28, and a control unit (driving assistance control device) 30.

The input unit 20 accepts various operations for the driving assistance device 14. The input unit 20 is composed of various input devices, such as buttons, switches, and touch panels.

The display unit 22 displays various images. For example, the display unit 22 can function as a notification unit that notifies the driver of the contents of the driving assistance by displaying the contents of the driving assistance as images. The display unit 22 is, for example, a display including a liquid crystal display (LCD), an organic electro-luminescence (EL), or a head-up display.

The audio output unit 24 outputs various sounds. For example, the audio output unit 24 can function as a notification unit that notifies the driver of the contents of the driving assistance by outputting the contents of the driving assistance by sound. The audio output unit 24 is, for example, configured with a speaker mounted on the vehicle.

The storage unit 26 is a memory that stores various types of information. The storage unit 26 stores, for example, information such as the contents of calculations performed by the control unit 30 and programs. The storage unit 26 includes, for example, at least one of a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive).

The communication unit 28 transmits and receives data to and from an external device. For example, the communication unit 28 transmits and receives data to and from the imaging device 10 and the biosensor 12. The communication unit 28 is configured to communicate using methods such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

The control unit 30 controls the operation of each part of the driving assistance device 14. The control unit 30 is realized, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program stored in the storage unit 26 or the like using a RAM or the like as a working area. The control unit 30 may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 may be realized by a combination of hardware and software.

The control unit 30 includes a video data acquisition unit 40, a biometric information acquisition unit 42, an emotion determination unit 44, a driving situation prediction unit 46, a driving assistance determination unit 48, and a driving assistance execution unit 50.

The video data acquisition unit 40 acquires video data from the imaging device 10. For example, the video data acquisition unit 40 acquires video data of the surroundings of the vehicle from the first camera 10a. For example, the video data acquisition unit 40 acquires video data of the interior of the vehicle from the second camera 10b.

The biometric information acquisition unit 42 acquires biometric information of the vehicle driver from the biometric sensor 12. The biometric information acquisition unit 42 acquires, for example, the current biometric information of the vehicle driver and the biometric information of the user for a predetermined period of time in the past. The predetermined period of time in the past also includes a period when the user was not in the vehicle.

The emotion determination unit 44 determines the emotion level of the vehicle driver. The emotion determination unit 44, for example, determines the emotion level of the vehicle driver at present and for a predetermined period in the past. The emotion determination unit 44, for example, determines the emotion level of the vehicle driver based on the biometric information of the vehicle driver acquired by the biometric information acquisition unit 42. The emotion determination unit 44, for example, determines whether the emotion of the vehicle driver is joy, anger, sadness, or happiness based on the biometric information of the vehicle driver. For example, for the emotion of joy, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "happy" or "happiness". For example, for the emotion of anger, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "anger", "worry", or "unpleasant". For example, for the emotion of sadness, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "sadness" or "depression". For example, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "joy," "satisfaction," or "relief" as the emotion of pleasure. In this embodiment, it is preferable for the emotion determination unit 44 to determine, for example, emotions such as anger and sadness, which are assumed to be highly likely to induce dangerous driving. For example, the emotion determination unit 44 may also determine emotions such as joy and pleasure, since it is assumed that excitement, joy, or enjoyment are also likely to distract attention and induce dangerous driving.

The driving condition prediction unit 46 predicts the driving condition of the vehicle after a predetermined period of time has elapsed. The driving condition prediction unit 46 predicts the driving condition of the vehicle after a predetermined period of time has elapsed, for example, based on the result of the determination of the emotion level of the driver of the vehicle by the emotion determination unit 44. In this embodiment, the driving condition includes information indicating the possibility of dangerous driving or a dangerous driving state.

The driving assistance decision unit 48 determines whether or not to provide driving assistance. If the driving assistance decision unit 48 determines to provide driving assistance, it determines the content of the driving assistance to be provided to the driver of the vehicle. The driving assistance decision unit 48 determines the content of the driving assistance based on, for example, the results of the prediction of the driving situation of the vehicle by the driving situation prediction unit 46.

The driving assistance execution unit 50 executes driving assistance for the driver of the vehicle. The driving assistance execution unit 50 executes driving assistance based on the result of the determination of the content of driving assistance by the driving assistance determination unit 48. The driving assistance execution unit 50 includes a vehicle control unit 50a and a notification control unit 50b.

The vehicle control unit 50a controls the vehicle based on the results of the determination of the content of driving assistance by the driving assistance determination unit 48. For example, the vehicle control unit 50a controls the engine to limit acceleration or controls the brake mechanism to assist in deceleration.

The notification control unit 50b issues a notification based on the result of the decision on the content of driving assistance made by the driving assistance decision unit 48. For example, the notification control unit 50b controls the display unit 22 to display the notification content, and controls the audio output unit 24 to output the notification content by audio.

(Driving assistance control process)
The flow of processing for driving assistance control according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the flow of processing for driving assistance control according to the first embodiment.

The biometric information acquisition unit 42 acquires biometric information of the vehicle driver (step S10). Specifically, the biometric information acquisition unit 42 acquires the current and past biometric information of the vehicle driver for a predetermined period of time from the biometric sensor 12 worn by the vehicle driver. Then, the process proceeds to step S12.

The emotion determination unit 44 determines the current emotion level of the driver of the vehicle (step S12). Specifically, the emotion determination unit 44 determines the emotion level of the driver based on, for example, the current biometric information of the driver acquired by the biometric information acquisition unit 42. The emotion determination unit 44 determines the current emotion level of the driver based on, for example, the difference between reference biometric information indicating the reference value of the biometric information of the driver stored in the memory unit 26 and the biometric information acquired by the biometric information acquisition unit 42. The reference value of the biometric information may be biometric information when the driver is emotionless, without any emotions such as joy, anger, sadness, or pleasure. The biometric information acquired by the biometric information acquisition unit 42 in this case may be a weak detection result. FIG. 3 is a diagram for explaining a method of determining an emotion level according to the first embodiment. For example, when the current emotion level of the driver is slightly changed from the reference value, the emotion determination unit 44 determines that the emotion level of the driver is "low". The emotion level "low" is also called the first threshold value of the emotion level. For example, when the driver's emotional level has changed moderately from normal, the emotion determination unit 44 determines that the driver's emotional level is "medium". The "medium" emotion level is also called a second emotion level threshold. The "medium" emotion level is an emotion level that may impair safe driving. For example, when the driver's emotional level has changed significantly from normal, the emotion determination unit 44 determines that the driver's emotional level is "high". In the first embodiment, the emotion determination unit 44 is described as determining whether the driver's emotional level is "normal", "low", "medium", or "high", but is not limited thereto. The emotion determination unit 44 may, for example, quantitatively calculate the driver's emotional level with a numerical value. The emotion determination unit 44 may calculate the driver's emotional level based on, for example, the driver's brain waves, pulse value, blood pressure, body temperature, and sweat amount acquired by the bioinformation acquisition unit 42. The emotion determination unit 44 may, for example, determine the driver's emotional level based on video data capturing the driver's facial expression. Then proceed to step S14.

The emotion determination unit 44 may determine the driver's emotion level based on, for example, video data of the driver while driving captured by the second camera 10b. The emotion determination unit 44 may determine the driver's current emotion level by, for example, extracting the driver's facial expression from the driver's video data. The emotion determination unit 44 may determine the driver's emotion level by, for example, analyzing the driver's gestures from the driver's video data. The emotion determination unit 44 may determine the driver's emotion level based on, for example, dictionary data that determines the emotion from the driver's gestures, etc. The emotion determination unit 44 may determine the user's emotion level using, for example, a machine learning model that predicts the user's emotion level based on the user's biometric information. The machine learning model may be a model that inputs the user's biometric information and outputs a prediction result of the user's emotion type and emotion level.

Returning to FIG. 2, the emotion determination unit 44 determines whether the current emotion level of the driver of the vehicle is equal to or greater than the first threshold (step S14). If it is determined that the current emotion level of the driver is equal to or greater than the first threshold (step S14; Yes), the process proceeds to step S16. If it is not determined that the current emotion level of the driver is equal to or greater than the first threshold (step S14; No), the process proceeds to step S30.

If step S14 is judged as Yes, the emotion judgment unit 44 judges the emotion level of the driver of the vehicle in a predetermined period in the past (step S16). Specifically, the emotion judgment unit 44 judges the emotion level of the driver based on, for example, the driver's biometric information acquired by the biometric information acquisition unit 42 for a predetermined period in the past. FIG. 4 is a diagram for explaining a method for judging a past emotion level according to the first embodiment. FIG. 4 shows the detection result of the driver's biometric information by the biometric sensor 12. FIG. 4 shows the change in the detection result of the vehicle's driver's biometric information in a predetermined period in the past. In FIG. 4, time t8 is the present, and the period from time t1 to time t8 is the predetermined period in the past. The predetermined period in the past is, for example, from the present to about 12 hours ago, but is not limited to this. The predetermined period in the past may be arbitrarily set according to, for example, the personality of the driver of the vehicle. For example, if the driver is an anxious person, the predetermined period in the past may be up to about 12 hours ago. For example, if the driver has an optimistic personality, the specified past period may be up to about 30 minutes ago.

In the example shown in FIG. 4, the emotion determination unit 44 determines, for example, that the emotional level of the vehicle driver is "low" during the period from time t1 to time t2. The emotion determination unit 44 determines, for example, that the emotional level of the vehicle driver is "normal" during the period from time t2 to time t3. The emotion determination unit 44 determines, for example, that the emotional level of the vehicle driver is "low" during the period from time t3 to time t4. The emotion determination unit 44 determines, for example, that the emotional level of the vehicle driver is "medium" during the period from time t4 to time t5. The emotion determination unit 44 determines, for example, that the emotional level of the vehicle driver is "high" during the period from time t5 to time t6. The emotion determination unit 44 determines, for example, that the emotional level of the vehicle driver is "medium" during the period from time t6 to time t7. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "low" during the period from time t7 to time t8, for example. The emotion determination unit 44 calculates the number of times the emotion level is determined to be "low", the number of times the emotion level is determined to be "medium", and the number of times the emotion level is determined to be "high" during the period from time t1 to time t8. As will be described later, the driving support determination unit 48 determines the content of the driving support based on the number of times each emotion level is determined. For example, the emotion determination unit 44 may equally divide the period from time t1 to time t8 at a predetermined time interval, and calculate the number of times the emotion level is determined to be "low", the number of times the emotion level is determined to be "medium", and the number of times the emotion level is determined to be "high" for each of the equally divided time periods. There is no particular limit to the predetermined time interval. For example, the emotion level may be determined in detail by equally dividing the period from time t1 to time t8 into 100 parts.

The emotion determination unit 44 may, for example, calculate the area of the portion exceeding the threshold lines indicating "low", "medium", and "high" for each of the emotion levels "low", "medium", and "high" to determine the emotion level. Specifically, the emotion determination unit 44 may, for example, calculate the area of the shaded portion for each of the emotion levels "low", "medium", and "high" between time t1 and time t2, between time t3 and time t4, between time t4 and time t5, between time t5 and time t6, between time t6 and time t7, and between time t7 and time t8 to determine the emotion level of the driver. The method by which the emotion determination unit 44 determines the emotion level of the driver in the past predetermined period is the same as the method in step S12, and therefore the description is omitted. Then, proceed to step S18.

Returning to FIG. 2, the emotion determination unit 44 determines whether the driver of the vehicle had an emotion level equal to or higher than the second threshold during a predetermined period in the past (step S18). Specifically, the driving assistance determination unit 48 determines, for example, whether the driver's emotion level was "medium" or higher during a predetermined period in the past. If it is determined that the driver had an emotion level equal to or higher than the second threshold during the predetermined period in the past (step S18; Yes), the process proceeds to step S20. If it is not determined that the driver had an emotion level equal to or higher than the second threshold during the predetermined period in the past (step S18; No), the process proceeds to step S22.

If step S18 is judged as Yes, the driving situation prediction unit 46 predicts the driving situation of the vehicle based on the emotional level of the driver of the vehicle at present and for a predetermined period in the past (step S20). Specifically, the driving situation prediction unit 46 predicts a driving risk indicating the possibility of compromising safety, for example, based on the emotional level of the driver of the vehicle at present and for a predetermined period in the past. More specifically, the driving situation prediction unit 46 predicts the driving risk in three stages, for example, "low", "medium", and "high". A driving risk of "low" indicates that the driver is unlikely to compromise safety. A driving risk of "medium" indicates that the driver is likely to compromise safety at a medium level. A driving risk of "high" indicates that the driver is likely to compromise safety. Then, the process proceeds to step S26.

If step S18 is determined to be No, the driving assistance decision unit 48 determines whether driving assistance is necessary (step S22). Specifically, the driving assistance decision unit 48 determines that driving assistance is necessary, for example, when the driver's current emotion level is "medium" or higher. In other words, the driving assistance decision unit 48 determines that driving assistance is not necessary when an emotion level of "medium" or higher has not been detected within a predetermined past period of time and the current emotion level is "low" or lower. If it is determined that driving assistance is necessary (step S22; Yes), the process proceeds to step S24. If it is not determined that driving assistance is necessary (step S22; No), the process proceeds to step S30.

If step S22 is judged as Yes, the driving situation prediction unit 46 predicts the driving situation based on the current emotional level of the driver of the vehicle (step S24). Specifically, the driving situation prediction unit 46 predicts the driving risk based on, for example, the current emotional level of the driver of the vehicle. Then, the process proceeds to step S26.

After step S20 or step S24, the driving assistance determination unit 48 determines the content of the driving assistance (step S26). Specifically, the driving assistance determination unit 48 determines the content of the driving assistance based on the prediction result by the driving situation prediction unit 46. The driving assistance determination unit 48 determines, for example, the notification content to be notified to the driver and the vehicle control content as the content of the driving assistance. The driving assistance determination unit 48 determines, for example, the notification content and the vehicle control content in two stages, "medium" and "high". When the notification content is "medium", an example of the notification content is, but is not limited to, a warning to the driver. When the notification content is "high", an example of the notification content is, but is not limited to, a rest warning to the driver. Examples of the vehicle control content include, but are not limited to, engine control, brake control, and warning processing control for the driver when an object is detected outside the vehicle. When the vehicle control content is "medium" and "high", the degree of control strength changes. FIG. 5 is a diagram for explaining the patterns of the content of driving assistance according to the first embodiment. As shown in the pattern information 60, in the first embodiment, there are eight possible patterns of the content of driving assistance, from the first pattern to the eighth pattern. Below, the eight patterns of the content of driving assistance are explained, but the content of driving assistance is not limited to these. In FIG. 6 to FIG. 13, the horizontal axis is time and the vertical axis is emotion level, and the number of times that the emotion level states of "low", "medium", and "high" were detected and the time are shown in a bar graph.

Figure 6 is a diagram for explaining a first pattern of driving assistance content according to the first embodiment. The first pattern shows the content of driving assistance when the determination result of the driver's current emotion level is "low" and the emotion level has been determined to be "high" once in the past within a predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk to be "medium". The driving assistance determination unit 48 determines the content of driving assistance based on the determined current emotion level, the number of times the emotion level has been determined in the past, and the predicted driving risk. In the first pattern, the driving assistance determination unit 48 determines the level of notification content to be "medium" and the level of vehicle control content to be "medium" among the driving assistance.

Figure 7 is a diagram for explaining the second pattern of the driving assistance content according to the first embodiment. The second pattern shows the driving assistance content when the judgment result of the driver's current emotion level is "low" and the emotion level has been judged to be "medium" once in the past within the specified period T. In this case, the driving situation prediction unit 46 predicts the driving risk to be "low". In the second pattern, the driving assistance decision unit 48 judges the level of notification content of the driving assistance to be "none" and the level of vehicle control content to be "none". That is, in the case of the second pattern, the driving assistance decision unit 48 judges that vehicle control will not be performed because the driver is unlikely to compromise safety.

8 is a diagram for explaining the third pattern of the driving assistance content according to the first embodiment. The third pattern shows the content of the driving assistance when the judgment result of the driver's current emotion level is "low" and the emotion level has been judged to be "medium" twice in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "medium". In the fourth pattern, the driving assistance decision unit 48 judges the level of the notification content of the driving assistance to be "medium" and the level of the vehicle control content to be "medium". That is, in the case of the third pattern, when the emotion level has been in a "medium" state multiple times within the predetermined period T, the driving assistance decision unit 48 judges that driving assistance is necessary according to the number of times the emotion level has been judged in the past. Note that the number of times the emotion level has been in a "medium" state within the predetermined period T, which is judged to be necessary for driving assistance, may be set according to the driver.

9 is a diagram for explaining the fourth pattern of the driving assistance content according to the first embodiment. The fourth pattern shows the content of the driving assistance when the judgment result of the driver's current emotion level is "low" and it is judged that the emotion level has continued to be in a "medium" state for a predetermined time or more in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk to be "medium". In the fourth pattern, the driving assistance determination unit 48 judges the level of the notification content of the driving assistance to be "medium" and the level of the vehicle control content to be "medium". That is, in the fourth pattern, the driving assistance determination unit 48 judges that driving assistance is necessary when the emotion level has continued to be in a "medium" state for a predetermined time or more within the predetermined period T. Note that the duration of the emotion level in the "medium" state within the predetermined period T, which is judged to be necessary for driving assistance, may be set according to the driver.

Figure 10 is a diagram for explaining the fifth pattern of the driving assistance content according to the first embodiment. The fifth pattern shows the content of the driving assistance when the judgment result of the driver's current emotion level is "low" and it is judged that the emotion level has continued to be in a "high" state for a predetermined time or more in the past within a predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "high". In the fifth pattern, the driving assistance determination unit 48 judges the level of the notification content of the driving assistance to be "high" and the level of the vehicle control content to be "high". That is, in the fifth pattern, the driving assistance determination unit 48 judges that driving assistance is particularly necessary when the emotion level has continued to be in a "high" state for a predetermined time or more within the predetermined period T. In the fifth pattern, the strength of the driving assistance becomes stronger. Note that the duration of the state in which the emotion level was "high" within the predetermined period T, which is judged to be particularly necessary for driving assistance, may be set according to the driver. Furthermore, the duration of the "high" emotion level state may be the same as or different from the duration of the "medium" emotion level state in the fourth pattern. For example, safety may be improved by making the duration of the "high" emotion level state shorter than the duration of the "medium" emotion level state in the fourth pattern.

Figure 11 is a diagram for explaining the sixth pattern of driving assistance content according to the first embodiment. The sixth pattern shows the content of driving assistance when the judgment result of the driver's current emotional level is "low" and the emotional level has been judged to be in a "high" state twice in the past within the specified period T. In this case, the driving situation prediction unit 46 predicts the driving risk to be "high". In the sixth pattern, the driving assistance decision unit 48 judges the level of the notification content of the driving assistance to be "high" and the level of the vehicle control content to be "high". That is, in the sixth pattern, the driving assistance decision unit 48 judges that driving assistance is particularly necessary when the emotional level has been in a "high" state multiple times within the specified period T.

Figure 12 is a diagram for explaining the seventh pattern of driving assistance content according to the first embodiment. The seventh pattern shows the driving assistance content when the judgment result of the driver's current emotion level is "medium" and the emotion level has been judged to be "high" once in the past within the specified period T. In this case, the driving situation prediction unit 46 judges the driving risk to be "high". In the seventh pattern, the driving assistance decision unit 48 judges the level of the notification content to be "high" and the level of the vehicle control content to be "high" among the driving assistance.

FIG. 13 is a diagram for explaining an eighth pattern of the driving support content according to the first embodiment. The eighth pattern shows the content of the driving support when the determination result of the driver's current emotion level is "medium" and the emotion level has not been determined to be any of "high", "medium", or "low" in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "medium". In the eighth pattern, the driving support determination unit 48 determines the level of the notification content of the driving support to be " medium " and the level of the vehicle control content to be " medium ". In other words, when the driver's current emotion level is determined to be "medium", the driving support determination unit 48 determines that driving support is necessary regardless of the emotion level in the past predetermined period.

Return to FIG. 2. After step S26, the driving assistance execution unit 50 executes driving assistance (step S28). Specifically, the driving assistance execution unit 50 executes driving assistance in accordance with the content of the driving assistance determined by the driving assistance determination unit 48 in step S26. More specifically, the vehicle control unit 50a controls the vehicle in accordance with the content of the driving assistance determined by the driving assistance determination unit 48 in step S26. The notification control unit 50b controls at least one of the display unit 22 and the audio output unit 24 in accordance with the content of the driving assistance determined by the driving assistance determination unit 48 in step S26 to output the notification content. Then, proceed to step S30.

After step S28, if step S14 is judged as No, or if step S22 is judged as No, the control unit 30 judges whether or not to end the process (step S30). Specifically, the control unit 30 judges to end the process when an operation to end the process or an operation to turn off the power is received. The control unit 30 also judges to end the process when the driver's current and past emotions for a predetermined period of time become normal after driving assistance is executed. If it is judged to end the process (step S30; Yes), the process of FIG. 2 is terminated. If it is not judged to end the process (step S30; No), the process proceeds to step S10.

As described above, the first embodiment determines the content of driving assistance based on the current emotional level of the vehicle driver and the emotional level of the driver over a predetermined period of time in the past. This allows the first embodiment to provide appropriate driving assistance according to the driver's emotions, thereby improving the safety of the vehicle.

[Modification of the first embodiment]
In the above-described first embodiment, the driving assistance system 1 has been described as providing driving assistance to the driver when the driver is driving the vehicle. The driving assistance system 1 may provide parking assistance when the driver is parking the vehicle.

The driving assistance device 14 may perform parking assistance if the driver's emotional level when parking the vehicle is equal to or greater than a first threshold and the emotional level has been determined to be equal to or greater than a second threshold during a predetermined period of time in the past.

Specifically, the emotion determination unit 44 determines the driver's current emotion level and the emotion level of the driver in the past for a predetermined period of time when the driver parks. The driving assistance determination unit 48 determines the content of parking assistance when the emotion determination unit 44 determines that the driver's current emotion level is equal to or higher than the first threshold and that the emotion level in the past for a predetermined period of time is equal to or higher than the second threshold. The driving assistance execution unit 50 executes the parking assistance determined by the driving assistance determination unit 48. As a result, the modified example of the first embodiment can provide appropriate assistance for parking according to the driver's emotion, thereby improving the safety of the vehicle when parking.

[Second embodiment]
An example of the configuration of the driving assistance system according to the second embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing an example of the configuration of the driving assistance system according to the second embodiment.

As shown in FIG. 14, the driving assistance system 1A differs from the driving assistance system 1 shown in FIG. 1 in that the control unit 30A of the driving assistance device 14A includes a cause determination unit 52.

The cause determination unit 52 determines the cause of the change in the emotion level of the vehicle driver to the first threshold or higher. The cause determination unit 52 determines the cause of the change in the emotion level of the driver based on, for example, the video data acquired by the video data acquisition unit 40 from the first camera 10a or the second camera 10b. For example, when the first camera 10a or the second camera 10b detects a sound, the cause determination unit 52 may identify the cause of the change in the emotion level to the first threshold or higher based on the detected sound. When the cause determination unit 52 determines the emotion level based on the sound, it is preferable that the driver is in the vehicle from a predetermined time in the past to the present. For example, the cause determination unit 52 associates the emotion level of the driver that is equal to or higher than the first threshold determined by the emotion determination unit 44 with the cause of the change in the emotion level, and stores the association in the storage unit 26 as cause information indicating the cause of the event in which the driver's emotion level became equal to or higher than the first threshold.

(Driving assistance control process)
The flow of processing for driving assistance control according to the second embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the flow of processing for driving assistance control according to the second embodiment.

The video data acquisition unit 40 acquires video data from the imaging device 10 (step S40). For example, the video data acquisition unit 40 acquires video data of the interior of the vehicle from the first camera 10a. For example, the video data acquisition unit 40 acquires video data of the periphery of the vehicle from the second camera 10b. Then, the process proceeds to step S42.

The processes from step S42 to step S46 are the same as the processes from step S10 to step S14 shown in FIG. 2, so their explanations are omitted.

After step S46 is judged as Yes, the cause determination unit 52 judges the cause of the driver's emotional level becoming equal to or higher than the first threshold (step S48). Specifically, the cause determination unit 52 judges the cause of the driver's emotional level becoming equal to or higher than the first threshold, for example, based on video data around the vehicle. More specifically, when the cause determination unit 52 acquires video data of another vehicle cutting in when the driver's emotional level is judged to be equal to or higher than the first threshold, the cause determination unit 52 judges that the other vehicle cutting in is the cause of the emotional level becoming equal to or higher than the first threshold. In other words, when the cause determination unit 52 acquires video data including an event that may affect the driver's emotion when the driver's emotional level becomes equal to or higher than the first threshold, the cause determination unit 52 judges that the event is the cause of the driver's emotional level becoming equal to or higher than the first threshold. Then, the process proceeds to step S50.

The processes from step S50 to step S54 are the same as the processes from step S16 to step S20 shown in FIG. 2, so their explanation is omitted.

If step S52 is judged as No, the cause judgment unit 52 judges whether or not the emotion level has become equal to or higher than the first threshold due to the same cause as the cause identified in step S48 in the past for a predetermined period of time (step S56). Specifically, the cause judgment unit 52 judges whether or not the emotion level has become equal to or higher than the first threshold due to the same cause as the cause identified in step S48, for example, based on the cause information stored in the storage unit 26. The predetermined period in step S56 may be set arbitrarily, for example, to 30 minutes, 1 hour, 6 hours, or 12 hours, depending on the driver's personality, etc. If it is judged that the emotion level has become equal to or higher than the first threshold due to the same cause as the cause identified in step S48 (step S56; Yes), proceed to step S58. If it is not judged that the emotion level has become equal to or higher than the first threshold due to the same cause as the cause identified in step S48 (step S56; No), proceed to step S62.

If step S56 returns Yes, the driving assistance determination unit 48A determines the content of the driving assistance corresponding to the cause identified in step S48 (step S58). Specifically, the driving assistance determination unit 48 determines the content of the driving assistance to calm the emotions corresponding to the cause, for example. Then, the process proceeds to step S60.

The driving assistance execution unit 50A executes driving assistance according to the cause determined in step S58 (step S60). Specifically, the driving assistance execution unit 50A executes driving assistance to calm the driver's emotions. More specifically, the notification control unit 50Ab causes the audio output unit 24 to output a voice such as, for example, "A situation has occurred that may cause emotional changes. Please be careful." Then, the process proceeds to step S70.

The processes from step S62 to step S70 are the same as the processes from step S22 to step S30 shown in FIG. 2, respectively, and therefore will not be described.

As described above, the second embodiment identifies the cause of a change in the current and past emotion levels of the vehicle driver, and performs driving assistance based on the identified cause. This allows the second embodiment to perform appropriate driving assistance according to the cause of the change in the driver's emotion, thereby improving safety.

[Modification of the second embodiment]
In the above-described second embodiment, the driving assistance system 1A has been described as providing driving assistance to the driver when the driver is driving the vehicle. The driving assistance system 1A may provide parking assistance when the driver is parking the vehicle.

The driving assistance device 14A may perform parking assistance if the driver's emotional level when parking the vehicle is equal to or higher than a first threshold and if the emotional level has been equal to or higher than the first threshold for the same reason when parking during a specified period in the past.

Specifically, the emotion determination unit 44 determines the driver's current emotion level when the driver parks the vehicle. The cause determination unit 52 determines whether the emotion level has exceeded the first threshold value due to the same cause during a predetermined period of time based on the cause information stored in the memory unit 26. The driving assistance determination unit 48A determines the content of parking assistance when the emotion determination unit 44 determines that the driver's current emotion level is at or above the first threshold value and the cause determination unit 52 determines that the emotion level has exceeded the first threshold value during a predetermined period of time due to the same cause. The driving assistance execution unit 50A executes the parking assistance determined by the driving assistance determination unit 48A. As a result, the second embodiment can appropriately execute parking assistance according to the cause of the change in the driver's emotion, thereby improving safety.

[Third embodiment]
An example of the configuration of the driving assistance system according to the third embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing an example of the configuration of the driving assistance system according to the third embodiment.

As shown in FIG. 16, the driving assistance system 1B differs from the driving assistance system 1 shown in FIG. 1 in that the driving assistance device 14B includes a GNSS (Global Navigation Satellite System) receiver 32, and the control unit 30B includes a position information acquisition unit 54 and a position information determination unit 56.

The GNSS receiver 32 receives GNSS signals including information for identifying location information from GNSS satellites (not shown). The GNSS receiver 32 can be realized, for example, by a GNSS receiver circuit or a GNSS receiver device.

The position information acquisition unit 54 acquires the GNSS signal from the GNSS receiving unit 32. The position information acquisition unit 54 calculates the current position information of the driver (driving assistance device 14B) based on the GNSS signal. For example, the position information acquisition unit 54 calculates the current position of the driver (driving assistance device 14B) in Earth coordinates. For example, the position information acquisition unit 54 associates the driver's emotion level equal to or higher than the first threshold determined by the emotion determination unit 44 with the location information where the emotion level has changed, and stores the emotion change location information indicating the location where the driver's emotion level became equal to or higher than the first threshold in the storage unit 26. Note that if the specified period in the past is before getting into the vehicle, the position information acquisition unit 54 may acquire the location information from a wearable device worn by the driver.

The position information determination unit 56 determines whether the current position is a position where the driver's emotion has changed. The position information determination unit 56 determines whether the current position is a position where the driver's emotion has changed, for example, based on the position information acquired by the position information acquisition unit 54 and the emotion change position information stored in the storage unit 26.

(Driving assistance control process)
The flow of processing for driving assistance control according to the third embodiment will be described with reference to Fig. 17. Fig. 17 is a flowchart showing an example of the flow of processing for driving assistance control according to the third embodiment.

The position information acquisition unit 54 acquires the position information of the driver (driving assistance device 14B) (step S80). Specifically, the position information acquisition unit 54 calculates the position information of the driver (driving assistance device 14B) based on the GNSS signal acquired from the GNSS receiving unit 32. Then, the process proceeds to step S82.

The processes from step S82 to step S86 are the same as the processes from step S10 to step S14 shown in FIG. 2, respectively, and therefore will not be described.

If step S86 returns Yes, the location information acquisition unit 54 identifies the current location (step S88). Specifically, the location information acquisition unit 54 identifies, for example, the location information of the point where the driver's emotion level is equal to or higher than the first threshold. Then, the process proceeds to step S90.

The processes from step S90 to step S94 are the same as the processes from step S16 to step S20 shown in FIG. 2, respectively, and therefore will not be described.

Specifically, the position information determination unit 56 determines whether the driver's emotion level at the current location has been equal to or greater than the first threshold within a predetermined period of time in the past (step S96). Specifically, the position information determination unit 56 determines whether the driver's emotion level at the current location has been equal to or greater than the first threshold within a predetermined period of time in the past, based on the current location information acquired by the position information acquisition unit 54 and the emotion change location information stored in the storage unit 26. If it is determined that the driver's emotion level at the current location has been equal to or greater than the first threshold within a predetermined period of time in the past (step S96; Yes), the process proceeds to step S98. If it is not determined that the driver's emotion level at the current location has been equal to or greater than the first threshold within a predetermined period of time in the past (step S96; No), the process proceeds to step S102.

If step S96 returns Yes, the driving assistance determination unit 48B determines the content of the driving assistance corresponding to the current location identified in step S88 (step S98). Specifically, the driving assistance determination unit 48B determines the content of the driving assistance for calming emotions corresponding to the current location, for example. Then, the process proceeds to step S100.

The driving assistance execution unit 50B executes driving assistance according to the cause determined in step S98 (step S100). Specifically, the driving assistance execution unit 50B executes driving assistance to calm the driver's emotions. More specifically, the notification control unit 50Bb causes the audio output unit 24 to output a voice such as, for example, "You are driving in an area where emotions are likely to change. Please be careful." Then, the process proceeds to step S110.

The processes from step S102 to step S110 are the same as the processes from step S22 to step S30 shown in FIG. 2, respectively, and therefore will not be described.

As described above, the third embodiment identifies location information of the location where the current and past emotion levels of the vehicle driver have changed, and performs driving assistance based on the identified location information. As a result, the third embodiment can provide appropriate driving assistance according to the location when the vehicle is traveling in a location where the driver's emotions are likely to change, thereby improving safety.

[Modification of the third embodiment]
In the third embodiment described above, the driving assistance system 1B has been described as providing driving assistance to the driver when the driver is driving the vehicle. The driving assistance system 1B may provide parking assistance when the driver is parking the vehicle.

The driving assistance device 14B may perform parking assistance if the driver's emotional level when parking the vehicle is equal to or higher than a first threshold and has been equal to or higher than the first threshold when parking in the current parking location within a specified period of time in the past.

Specifically, the emotion determination unit 44 determines the current emotion level of the driver when the driver parks. The position information determination unit 56 determines whether the emotion level of the driver has been equal to or higher than the first threshold within a predetermined period of time when parking at the current position, based on the emotion change position information stored in the storage unit 26. The driving assistance determination unit 48B determines the content of parking assistance when the emotion determination unit 44 determines that the current emotion level of the driver is equal to or higher than the first threshold and the position information determination unit 56 determines that the emotion level of the driver has been equal to or higher than the first threshold within a predetermined period of time at the current position. The driving assistance execution unit 50B executes the parking assistance determined by the driving assistance determination unit 48B. As a result, the third embodiment can appropriately execute parking assistance according to the location where the driver's emotion has changed, thereby improving safety.

Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. The above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-mentioned embodiments.

Reference Signs List 1, 1A, 1B Driving assistance system 10 Imaging device 10a First camera 10b Second camera 12 Biometric sensor 14, 14A, 14B Driving assistance device 20 Input unit 22 Display unit 24 Audio output unit 26 Storage unit 28 Communication unit 30, 30A, 30B Control unit (driving assistance control device)
32 GNSS receiving unit 40 Video data acquisition unit 42 Biometric information acquisition unit 44 Emotion determination unit 46 Driving situation prediction unit 48, 48A, 48B Driving support determination unit 50, 50A, 50B Driving support execution unit 50a Vehicle control unit 50b Notification control unit 52 Cause determination unit 54 Position information acquisition unit 56 Position information determination unit

Claims (5)

A biometric information acquisition unit that acquires current and past biometric information of a vehicle driver for a predetermined period of time;
an emotion determination unit that determines an emotion level indicating a degree of emotion of the driver in the present and the predetermined past period based on the biological information acquired by the biological information acquisition unit;
a driving situation prediction unit that predicts a driving situation of the vehicle based on the current and past emotion levels of the driver during the predetermined period when the current emotion level of the driver is determined to be equal to or higher than a first threshold;
a driving assistance determination unit that determines whether or not driving assistance of the vehicle is necessary based on a result of the prediction of the driving situation of the vehicle by the driving situation prediction unit, and determines the content of the driving assistance of the vehicle when the driving assistance of the vehicle is necessary;
a driving assistance execution unit that executes the driving assistance determined by the driving assistance determination unit;
A driving assistance control device comprising: the driving assistance decision unit changes content of the driving assistance for the vehicle depending on the number of times that the emotion level has been determined to be equal to or higher than a second threshold value, which is higher than the first threshold value, within the predetermined period in the past.
The driving assistance control device according to claim 1 . the driving assistance determination unit changes content of the driving assistance for the vehicle depending on a duration during which an emotion level equal to or higher than a second threshold continued within the predetermined period in the past.
The driving assistance control device according to claim 1 or 2. A video data acquisition unit that acquires video data of the driver,
The emotion determination unit determines an emotion level indicating a degree of emotion of the driver during the predetermined period in the past based on the video data.
The driving assistance control device according to any one of claims 1 to 3. acquiring current and past biometric information of a driver of a vehicle;
determining an emotion level indicating a degree of emotion of the driver in the present and the predetermined past period based on the acquired biological information;
When the current emotion level of the driver is determined to be equal to or higher than a first threshold, predicting a driving situation of the vehicle based on the current and past emotion levels of the driver during the predetermined period;
A step of determining whether or not driving assistance for the vehicle is necessary based on a prediction result of a driving situation of the vehicle, and determining the content of the driving assistance for the vehicle when the driving assistance for the vehicle is necessary;
Executing the determined driving assistance;
A driving assistance control method comprising:

Images