JP7831372B2 - Notification system, notification method, and computer program - Google Patents

Description

This invention relates to a notification system, a notification method, and a computer program.

Patent Document 1 discloses a vehicle alarm device configured to notify the driver inside the vehicle of the type of alarm sound, such as a siren, when such a sound is generated outside the vehicle, by vibrating a vibrating body according to a vibration rhythm pattern corresponding to the type of alarm sound.

Japanese Patent Publication No. 2002-240660

When multiple types of notifications with different content are to be transmitted via vibration, one method is to change the vibration rhythm pattern, as described in the conventional vehicle notification system mentioned earlier. However, attempting to transmit multiple types of notifications solely through changes in the vibration rhythm pattern leads to an increase in the amount of information that must be transmitted via vibration as the number of notification types increases, resulting in a more complex vibration rhythm pattern. Furthermore, the more complex the vibration rhythm pattern, the more difficult it becomes for the driver to understand what each notification is communicating.

Furthermore, when the notification content changes continuously, the vibration rhythm pattern also changes continuously. However, the more complex the vibration rhythm pattern becomes, the more difficult it is to grasp the changes, and thus the harder it becomes to detect the change in notification content.

This invention addresses these problems and aims to enable drivers to easily understand the content of multiple types of notifications with different content, as well as the transitions between them, when these notifications are transmitted via vibration.

To solve the above problems, a notification system according to one aspect of the present invention comprises a vibration unit that provides vibration stimuli to the vehicle occupants, and a control device that controls the vibration unit. The control device provides the vehicle occupants with a first notification, informing them of the distance between the vehicle and stationary objects in its vicinity, and a second notification, informing them of the presence of a moving object approaching the vehicle, by vibrating the vibration unit. When providing the first notification, the control device vibrates the vibration unit in a continuous vibration pattern, regardless of the distance to the stationary object, and changes the vibration level of the vibration unit according to the distance to the stationary object. When providing the second notification, the control device vibrates the vibration unit in an intermittent vibration pattern.

Furthermore, a notification method according to one aspect of the present invention is a notification method using a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle. Based on information about the vehicle's surroundings, the method provides a first notification to the occupants of the vehicle, informing them of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification to the occupants of the vehicle, informing them of the presence of a moving object approaching the vehicle, by vibrating the vibration unit. When providing the first notification, the vibration unit is vibrated in a continuous vibration pattern regardless of the degree of distance to the stationary object, while also changing the vibration level of the vibration unit according to the degree of distance to the stationary object. When providing the second notification, the vibration unit is vibrated in an intermittent vibration pattern.

Furthermore, a computer program according to one aspect of the present invention is a computer program for a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle. Based on information about the vehicle's surroundings, the program provides a first notification to the occupants of the vehicle, informing them of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification to the occupants of the presence of a moving object approaching the vehicle, by vibrating the vibration unit. When providing the first notification, the program causes the vibration unit to vibrate in a continuous vibration pattern, regardless of the degree of distance to the stationary object, while also changing the vibration level of the vibration unit according to the degree of distance to the stationary object. When providing the second notification, the program causes the vibration unit to vibrate in an intermittent vibration pattern.

According to these aspects of the present invention, even when multiple types of notifications with different content are transmitted by vibration, the driver can easily understand the content of the notifications and their transitions.

This is a schematic diagram of a notification system according to one embodiment of the present invention. This figure shows examples of vibration rhythm patterns and vibration levels for a notification (first notification) that indicates the distance to stationary objects around the vehicle, and for a notification (second notification) that indicates the presence of a moving object approaching the vehicle. This flowchart explains the details of the control content related to the first notification. This flowchart explains the details of the control system for the second notification.

The embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals.

Figure 1 is a schematic diagram of a notification system 100 according to one embodiment of the present invention. The notification system 100 can be installed in vehicles such as autonomous vehicles and manually driven vehicles.

The notification system 100 comprises a peripheral information acquisition device 1, a notification device 2, and a control device 3. The peripheral information acquisition device 1, notification device 2, and control device 3 are each connected via an in-vehicle network 9 compliant with standards such as a controller area network.

The surrounding information acquisition device 1 is a device for acquiring data on objects around the vehicle as vehicle surrounding information. The surrounding information acquisition device 1 according to this embodiment includes one or more external cameras 11 for photographing objects around the vehicle, and a distance measuring sensor 12 for measuring the distance to objects around the vehicle.

The external camera 11 captures images of the vehicle's surroundings at a predetermined frame rate (e.g., 10 Hz to 40 Hz) and generates camera images showing the area around the vehicle. Each time the external camera 11 generates a camera image, it transmits the generated image (vehicle surroundings information) to the control device 3 via the in-vehicle network 9.

The distance measuring sensor 12 emits multiple laser beams or radio waves with different irradiation angles horizontally and parallel to the distance measuring area around the vehicle, and receives the reflected light from each emitted laser beam or the reflected waves from each emitted radio wave. Based on the received reflected light or waves, the distance measuring sensor 12 measures the distance to various obstacles (e.g., people, vehicles, walls, etc.) within the distance measuring area. The distance to each obstacle is acquired for each reflection point (irradiation point) of the laser beam or radio wave. The distance measuring sensor 12 associates the distance to each reflection point with the coordinate information of each reflection point, and transmits this as distance measurement data (vehicle surrounding information) to the control device 3 via the in-vehicle network 9. Examples of distance measuring sensors 12 include LiDAR (Light Detection and Ranging), millimeter-wave radar sensors, and ultrasonic sensors.

The notification device 2 is a device for notifying the driver of the vehicle of various information. The notification device 2 according to this embodiment includes at least a vibration unit 21 for providing vibrational stimulation to the driver so that notifications can be made through the driver's sense of touch. The vibration unit 21 can be installed, for example, on the steering wheel or the driver's seat, and in this embodiment, it is built into the driver's seat. When the notification device 2 receives a vibration signal from the control device 3 via the in-vehicle network 9, it vibrates the vibration unit 21 based on the content of the received vibration signal.

The control device 3 is an ECU (Electronic Control Unit) comprising a communication unit 31, a storage unit 32, and a processing unit 33.

The communication unit 31 includes an interface circuit for connecting the control device 3 to the in-vehicle network 9. The communication unit 31 supplies data received from an external source (such as vehicle surrounding information) to the processing unit 33. The communication unit 31 also outputs vibration signals from the processing unit 33 to the vibration unit 21.

The memory unit 32 has a storage medium such as an HDD (Hard Disk Drive), SSD (Solid Disk Drive), or semiconductor memory, and stores various computer programs and data used for processing in the processing unit 33.

The processing unit 33 comprises one or more CPUs (Central Processing Units) and their peripheral circuits. The processing unit 33 executes various computer programs stored in the memory unit 32; for example, it is a processor.

An example of the processing performed by the processing unit 33 is that, based on vehicle surrounding information, the processing unit 33 outputs a first vibration signal to the vibration unit 21 according to the degree of distance (e.g., "long distance," "medium distance," "short distance") between the vehicle and stationary objects around it, enabling the driver to understand the distance to these objects. This allows, for example, a driver with a hearing impairment to be notified of the degree of distance between their vehicle and surrounding stationary objects through vibration.

Furthermore, the processing unit 33 outputs a second vibration signal to the vibration unit 21 when it detects the presence of a moving object approaching the vehicle based on the vehicle's surrounding information, in order to prevent collisions with moving objects that the driver may not be aware of (e.g., people or other vehicles). This allows drivers, for example, those with hearing impairments, to be notified of the presence of approaching moving objects through vibration.

By the way, when transmitting multiple types of notifications with different content via vibration, one conventional method involves changing the vibration rhythm pattern to transmit each notification. By appropriately changing the combination (sequence) of the time the vibration unit 21 vibrates (on time) and the time it does not vibrate (off time), as well as the length of each time period, any desired vibration rhythm pattern can be created.

In this embodiment, the distance between the vehicle and surrounding stationary objects is indicated to the driver in three stages: "long distance," "medium distance," and "short distance." Therefore, three types of vibration rhythm patterns are required to communicate these distances through changes in the vibration rhythm pattern. Furthermore, one more vibration rhythm pattern is needed to inform the driver of the presence of a moving object approaching the vehicle.

Thus, attempting to implement multiple types of notifications with different content solely through changes in the vibration rhythm pattern results in a complex vibration rhythm pattern. The more complex the vibration rhythm pattern, the more difficult it becomes to understand what each currently running notification represents.

Furthermore, when the notification content changes continuously, the vibration rhythm pattern also changes continuously. However, the more complex the vibration rhythm pattern, the more difficult it becomes to perceive the change. In other words, there is a risk that the driver may not be able to immediately perceive the change in notification content.

Therefore, in this embodiment, as shown in Figure 2, when providing notification (first notification) indicating the distance between the vehicle and surrounding stationary objects, the vibration unit 21 is vibrated in a vibration rhythm pattern that causes continuous vibration regardless of the distance to the stationary objects, and the vibration level (vibration intensity) of the vibration unit 21 is changed according to the distance to the stationary objects. Specifically, in this embodiment, the vibration level is set to "weak" when the distance between the vehicle and surrounding stationary objects is "long distance," to "medium" when it is "medium distance," and to "strong" when it is "short distance."

Furthermore, when issuing a notification (second notification) to alert the vehicle of the presence of a moving object approaching it, the vibration unit 21 is vibrated in a vibration rhythm pattern that causes it to vibrate intermittently. Specifically, in this embodiment, three short vibrations are repeated periodically. Note that in the example shown in Figure 2, the vibration level for the second notification is "medium," but the vibration level for the second notification is not particularly limited.

Thus, according to this embodiment, the vibration rhythm pattern is common for the first notification that indicates the distance between the vehicle and surrounding stationary objects, making it easy to recognize that the currently running notification is the first notification. Furthermore, since the degree of distance to stationary objects can be communicated to the driver through the strength of the vibration level, the driver can intuitively and immediately grasp whether the distance to the stationary object is close or far. Even when the notification content switches continuously, the change in vibration level makes it easy to understand the transition.

Furthermore, regarding the second notification, which alerts the vehicle to the presence of an approaching moving object, the vibration rhythm pattern is set to intermittent vibration. This clearly distinguishes it from the first notification, which uses continuous vibration, making it easy to recognize that the currently running notification is the second notification. Additionally, if the notification content switches seamlessly from, for example, the first notification to the second notification, this switch can also be easily detected.

Figure 3 is a flowchart illustrating the details of the control content related to the first notification. The control device 3 repeatedly executes this routine at a predetermined calculation cycle.

In step S11, the control device 3 determines the distance to stationary objects around the vehicle based on the vehicle's surrounding information. In this embodiment, the control device 3 recognizes stationary objects around the vehicle based on the distance measurement data and determines the distance to the recognized stationary objects. Stationary object recognition can be performed, for example, by grouping reflection points that satisfy predetermined conditions as reflection points of laser light, etc., reflected from the same object, based on the distance to each reflection point and the coordinate information of each reflection point. Camera images can also be used for stationary object recognition.

In step S12, the control device 3 determines whether the distance to the stationary object is less than a predetermined proximity threshold D1. If the distance to the stationary object is less than the proximity threshold D1, the control device 3 proceeds to step S13. On the other hand, if the distance to the stationary object is equal to or greater than the proximity threshold D1, the control device 3 proceeds to step S14.

In step S13, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "continuous vibration" and the vibration level becomes "strong," causing the vibration unit 21 to vibrate.

In step S14, the control device 3 determines whether the distance to the stationary object is less than a predetermined medium-distance threshold D2. The medium-distance threshold D2 is greater than the short-distance threshold D1. If the distance to the stationary object is less than the medium-distance threshold D2, the control device 3 proceeds to step S15. Conversely, if the distance to the stationary object is equal to or greater than the medium-distance threshold D2, the control device 3 proceeds to step S16.

In step S15, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "continuous vibration" and the vibration level becomes "medium," causing the vibration unit 21 to vibrate.

In step S16, the control device 3 determines whether the distance to the stationary object is less than a predetermined long-distance threshold D3. The long-distance threshold D3 is greater than the medium-distance threshold D2. If the distance to the stationary object is less than the long-distance threshold D3, the control device 3 proceeds to step S17. On the other hand, if the distance to the stationary object is greater than or equal to the long-distance threshold D3, the control device 3 terminates the process without providing vibration-based notification.

In step S17, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "continuous vibration" and the vibration level becomes "weak," causing the vibration unit 21 to vibrate.

Figure 4 is a flowchart illustrating the details of the control process for the second notification. The control device 3 repeatedly executes this routine at predetermined calculation cycles.

In step S21, the control device 3 detects targets around the vehicle based on the vehicle's surrounding information. In this embodiment, the control device 3 sequentially inputs camera images received from the external camera 11 into a classifier to detect the region in the camera image where a target is represented and the type of target represented in that region. The classifier can be, for example, a convolutional neural network (CNN) having multiple convolutional layers connected in series from the input side to the output side. The control device 3 then estimates the distance from the external camera 11 to the target using the standard size of each target stored in the storage unit 32 for each target type and the size of the target detected in the camera image, and calculates the position and velocity of the target by tracking the target detected in the camera image over time. Note that the target detection method is not limited to this method, and various known methods may be used for detection.

In step S22, the control device 3 determines whether any moving objects, such as other vehicles or pedestrians approaching the vehicle, are within a predetermined range around the vehicle. If the control device 3 determines that an approaching moving object is within the predetermined range, it proceeds to step S23 to prevent a collision. On the other hand, if the control device 3 determines that there is no risk of collision, it terminates the process without issuing a vibration notification.

In step S23, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "intermittent vibration" and the vibration level becomes "medium," causing the vibration unit 21 to vibrate.

The notification system 100 according to this embodiment, as described above, comprises a vibration unit 21 that provides vibration stimuli to the vehicle occupants and a control device 3 that controls the vibration unit 21. The control device 3, based on information about the vehicle's surroundings, provides a first notification to the vehicle occupants by vibrating the vibration unit 21, informing them of the distance between the vehicle and stationary objects in its vicinity, and a second notification to the vehicle occupants of the presence of a moving object approaching the vehicle. When providing the first notification, the vibration unit 21 is vibrated in a continuous vibration rhythm pattern, regardless of the distance to the stationary object, while also changing the vibration level of the vibration unit 21 according to the distance to the stationary object. When providing the second notification, the vibration unit 21 is vibrated in an intermittent vibration rhythm pattern.

In this embodiment, the control device 3, in its first notification, is configured to increase the vibration level when the distance to the stationary object is close compared to when it is far away. The vehicle surrounding information includes camera images (image data) from an external camera 11 that photographs objects around the vehicle, and distance measurement data from a distance measuring sensor 12 that measures the distance to objects around the vehicle.

Thus, according to this embodiment, the vibration rhythm pattern is common for the first notification that indicates the distance between the vehicle and surrounding stationary objects, making it easy to recognize that the currently running notification is the first notification. Furthermore, since the degree of distance to stationary objects can be communicated to the driver through the strength of the vibration level, the driver can intuitively and immediately grasp whether the distance to the stationary object is close or far. Even when the notification content switches continuously, the change in vibration level makes it easy to understand the transition.

Furthermore, the second notification, which alerts the vehicle to the presence of an approaching moving object, uses a vibration rhythm pattern. This clearly distinguishes it from the first notification, which uses continuous vibration, making it easy to recognize that the currently active notification is the second notification. Additionally, if the notification content switches seamlessly from the first to the second notification, the switch can be easily detected.

Therefore, according to the notification system 100 of this embodiment, even when multiple types of notifications with different content (the "strong," "medium," and "weak" levels of the first notification, and the second notification) are transmitted by vibration, the driver can easily grasp the content of the notifications and their transitions through vibration. For example, even a driver with a hearing impairment can easily grasp the content of multiple types of notifications through vibration.

Although embodiments of the present invention have been described above, these embodiments represent only a portion of the applications of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

For example, in the above embodiment, the notification device 2 was equipped with a vibration unit 21, but it may also be equipped with a display for providing notification through the driver's sight, or a speaker for providing notification through the driver's hearing. In that case, in addition to notification by vibration from the vibration unit 21, notification may also be provided by displaying text information on the display or by providing audio notification from the speaker.

Furthermore, in the above embodiment, the computer program executed in the control device 3 may be provided in the form of a computer-readable, portable recording medium, such as a semiconductor memory, magnetic recording medium, or optical recording medium.

3 Control device 21 Vibration unit 100 Notification system

Claims (5)

A vibration unit that provides vibration stimulation to the vehicle occupants,
A control device for controlling the vibration unit,
A notification system comprising,
The control device is
Based on the surrounding information of the vehicle, a first notification informing the occupants of the vehicle of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification informing the occupants of the vehicle of the presence of a moving object approaching the vehicle, are provided by vibrating the vibration unit.
When the first notification is made, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate continuously, regardless of the distance from the stationary object, and the vibration level of the vibration unit is changed according to the distance from the stationary object.
When the second notification is performed, the vibration unit is configured to vibrate with a vibration pattern that causes the vibration unit to vibrate intermittently.
Notification system. The control device is
In the first notification, the system is configured to indicate the degree of distance between the vehicle and stationary objects in its vicinity, and to increase the vibration level when the distance to the stationary objects is close compared to when they are far away.
The notification system according to claim 1. The surrounding information includes image data from a camera that photographs objects around the vehicle and distance measurement data from a distance measuring sensor that measures the distance to objects around the vehicle.
The notification system according to claim 1 or claim 2. A notification method using a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle,
Based on the surrounding information of the vehicle, a first notification informing the occupants of the vehicle of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification informing the occupants of the vehicle of the presence of a moving object approaching the vehicle, are provided by vibrating the vibration unit.
When the first notification is made, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate continuously, regardless of the distance from the stationary object, and the vibration level of the vibration unit is changed according to the distance from the stationary object.
When the second notification is performed, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate intermittently.
Notification method. A computer program for a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle,
Based on the surrounding information of the vehicle, a first notification informing the occupants of the vehicle of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification informing the occupants of the vehicle of the presence of a moving object approaching the vehicle, are provided by vibrating the vibration unit.
When the first notification is made, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate continuously, regardless of the distance from the stationary object, and the vibration level of the vibration unit is changed according to the distance from the stationary object.
When the second notification is performed, the control device is instructed to vibrate the vibration unit in a vibration pattern that causes the vibration unit to vibrate intermittently.
Computer program.