JP7790139B2 - Driving Lesson System - Google Patents

Description

The present invention relates to a driving training system.

One known example of a driving training system for automobiles is one that provides training based on, for example, video of model driving or video of the student driving (see, for example, Patent Document 1).

Patent No. 6415851

The technology described in Patent Document 1 allows the student to objectively understand their driving by reviewing the video, but does not evaluate their driving.

The present invention was made in consideration of the above, and aims to provide a driving training system that can appropriately evaluate a driver's driving.

The driving training system of the present invention comprises a vehicle capable of running on a training course, and a control device that acquires driving information when a driver drives the vehicle on the training course, and extracts inappropriate driving behavior of the driver based on the acquired driving information and model information that serves as a model for driving the vehicle on the training course.

The present invention provides a driving training system that can appropriately evaluate a driver's driving skills.

FIG. 1 is a diagram schematically illustrating an example of a driving training system according to this embodiment. FIG. 2 is a functional block diagram showing an example of a driving training system according to this embodiment. FIG. 3 is a diagram showing an example of the operation of the driving training system. FIG. 4 is a diagram showing an example of the operation of the driving training system. FIG. 5 is a diagram showing an example of the operation of the driving training system. FIG. 6 is a diagram showing an example of the operation of the driving training system. FIG. 7 is a diagram showing an example of the operation of the driving training system. FIG. 8 is a diagram showing an example of the operation of the driving training system. FIG. 9 is a flowchart showing an example of the operation of the driving lesson system. FIG. 10 is a flowchart showing another example of the operation of the driving training system.

The following describes an embodiment of a driving training system according to the present invention, with reference to the drawings. Note that the present invention is not limited to this embodiment. Furthermore, the components in the following embodiments include those that are easily replaceable by those skilled in the art, or those that are substantially identical.

Figure 1 is a diagram schematically illustrating an example of a driving training system 100 according to this embodiment. Figure 2 is a functional block diagram illustrating an example of a driving training system 100 according to this embodiment. As shown in Figures 1 and 2, the driving training system 100 includes a vehicle 10 and a control device 20.

The automobile 10 is capable of traveling on a predetermined training course. Predetermined training courses include various training courses such as those set up at driving schools, such as turning courses, parallel parking, S-curves, and crank courses. The automobile 10 travels under the control of the driver. In this embodiment, the automobile 10 is capable of both normal driving, in which the automobile travels under the control of the driver, and autonomous driving, in which the automobile travels automatically without the need for driver operation.

The automobile 10 has an onboard camera 11, a peripheral camera 12, a distance sensor 13, an operation data acquisition unit 14, an automatic driving control unit 15, and a communication unit 16. The automobile 10 constituting the driving training system 100 may be one or more automobiles 10.

The vehicle-mounted camera 11 captures an image of the driver. The vehicle-mounted camera 11 generates driver image data associated with time information such as the time of capture.

The peripheral camera 12 captures images of the surroundings of the automobile 10, such as the front, sides, and rear of the automobile 10. The peripheral camera 12 generates peripheral image data associated with time information such as the time of capture.

The distance sensor 13 measures the distance between the automobile 10 and surrounding objects. Examples of distance sensors 13 include radar and LiDAR (Laser Imaging Detection and Ranging). The distance sensor 13 generates distance data associated with time information such as the measurement time.

The operation data acquisition unit 14 acquires operation data related to the operation of the automobile 10, such as the amount of operation of operating devices such as the steering wheel, accelerator, and brake, and the timing of operation. The operation data acquisition unit 14 generates operation data by associating the acquired operation information with time information such as the time of operation.

The autonomous driving control unit 15 controls the driving of the automobile 10 during autonomous driving. The autonomous driving control unit 15 can control the driving of the automobile 10 based on, for example, driving instruction information (described below) transmitted from the control device 20.

The communication unit 16 transmits and receives information between the vehicle 10 and the control device 20. It transmits the driver image data, surrounding image data, distance data, and operation data described above to the control device 20. The communication unit 16 receives driving instruction information (described below) related to the content of autonomous driving instructions from the control device 20.

The control device 20 has a communication unit 21, an acquisition unit 22, an extraction unit 23, a training processing unit 24, a presentation unit 25, and a memory unit 26.

The communication unit 21 transmits and receives information between the control device 20 and the vehicle 10. The communication unit 21 receives driver image data, surrounding image data, distance data, and operation data transmitted from the communication unit 16 of the vehicle 10. The communication unit 21 transmits driving instruction information (described below) related to the content of autonomous driving instructions.

The acquisition unit 22 extracts or generates driving information for when the driver drives a vehicle on a training course based on the driver image data, surrounding image data, distance data, and operation data received by the communication unit 21, and acquires the driving information. Examples of driving information include the driver's line of sight and posture while driving, the amount and timing of operation of the operation device, the driving route of the vehicle 10, and the distance between the vehicle 10 and surrounding objects.

The extraction unit 23 extracts inappropriate driving behaviors of the driver based on the acquired driving information and the model information stored in the memory unit 26. The model information is model information for driving a vehicle on a training course and corresponds to driving information provided by the driver. The model information includes information on model behaviors that the driver should perform, such as line of sight and posture while driving, and the amount and timing of operating the control devices. The model information also includes information such as the driving route of the vehicle 10 when performing the model behaviors, and the distance between the vehicle 10 and surrounding objects. The model information may be information extracted when an instructor at a driving school actually drives the vehicle 10, or information generated based on that information, for example.

The extraction unit 23 detects the difference between the driving information and the model information, and extracts inappropriate behavior based on the detected difference. When detecting the difference, the extraction unit 23 associates the driving information with the model information. That is, it determines which behavior of the driver corresponds to which behavior in the model behavior. When associating the driving information with the model information, the extraction unit 23 can perform the association based on, for example, the elapsed time since the start of driving, the relative positional relationship between the automobile 10 and surrounding objects 30, the distance traveled by the automobile 10, etc. When the difference between the driving information and the model information in the associated portion is greater than a threshold, the extraction unit 23 extracts the portion related to the difference as an inappropriate behavior.

For example, if a predetermined time has passed since the start of driving and necessary actions such as checking the mirrors or operating the steering wheel, accelerator, or brakes have not been performed, the extraction unit 23 can extract the driver's actions as inappropriate actions. When the vehicle 10 is instructed to drive a predetermined route on a driving course, the extraction unit 23 can extract the driver's actions as inappropriate actions when it detects that the difference between the route driven by the driver and the route in the model information is greater than a threshold. Furthermore, when the driver has driven the vehicle 10 a certain distance, the extraction unit 23 compares the distance between the vehicle 10 and surrounding objects with the model information, and can extract the driver's actions as inappropriate actions when it detects that the difference is greater than a threshold. In these cases, the threshold can be set, for example, to a value that indicates a situation in which continuing driving would result in contact with surrounding objects and would be difficult to correct without stopping the vehicle 10 and starting over.

The training processing unit 24 provides training to the driver based on the extraction results of the extraction unit 23. For example, if an inappropriate behavior is extracted, the training processing unit 24 generates driving instruction information to stop the vehicle 10 and causes the communication unit 21 to transmit the generated driving instruction information to the vehicle 10. If the vehicle 10 stops, the training processing unit 24 generates driving instruction information to move the stopped vehicle 10 back to the corresponding position before the inappropriate behavior occurred and causes the communication unit 21 to transmit the generated driving instruction information to the vehicle 10. Here, the corresponding position may be, for example, a position where the state of the vehicle 10 due to the inappropriate behavior can be corrected. Note that the corresponding position may also be a starting point on the above-mentioned predetermined route.

When the vehicle 10 moves to the corresponding location, the training processing unit 24 generates driving instruction information to drive the vehicle 10 so that the driver's driving from the corresponding location is reproduced, and causes the communication unit 21 to transmit the generated driving instruction information to the vehicle 10. When the driver's driving from the corresponding location is reproduced, the training processing unit 24 presents the driver with training information including details on how to correct the inappropriate behavior at the time that the driver's driving from the corresponding location is extracted. For example, the training information can be presented to the driver by displaying images, text, etc. on the display unit or outputting audio from the audio output unit.

The training processing unit 24 may also determine whether the driver's driving is pass or fail based on the results of the extraction of inappropriate behavior by the extraction unit 23. For example, the training processing unit 24 may assign a score depending on whether or not there was inappropriate behavior, and determine whether the driver passed or failed if the score is equal to or exceeds a threshold. In cases where the driver drives again after being presented with training information regarding inappropriate behavior, the training processing unit 24 may determine that the driver has corrected the inappropriate behavior, and may not count the previous inappropriate behavior or may reduce the impact of the inappropriate behavior.

The training processing unit 24 may also cause the automobile 10 to drive automatically based on the model information. In this case, the training processing unit 24 generates driving instruction information for causing the automobile 10 to drive based on the model information, and transmits the generated driving instruction information to the automobile 10 from the communication unit 21. The training processing unit 24 may also present training information to the driver, including model actions to be performed while driving the automobile 10, while causing the automobile 10 to drive automatically. In this case, the training processing unit 24 can present to the driver, for example, by video, text, audio, etc., what actions to perform and when to perform them when driving the automobile 10, based on the model information. Furthermore, when causing the automobile 10 to drive automatically based on the model information after the driver has driven, the training processing unit 24 may present model actions corresponding to the extracted inappropriate actions as training information, and may not present model actions for actions not extracted as inappropriate actions.

The presentation unit 25 presents the above-mentioned training information to the driver. Examples of the presentation unit 25 include a display unit that displays images, text, etc., such as a display, and an audio output unit that outputs audio, such as a speaker. The presentation unit 25 is located, for example, inside the automobile 10. The presentation unit 25 may also be located outside the automobile 10.

The memory unit 26 stores the driver image data, surrounding image data, distance data, and operation data received by the communication unit 21. The memory unit 26 stores the driving information acquired by the acquisition unit 22. The memory unit 26 stores pre-generated example information. When an inappropriate behavior is detected by the extraction unit 23, the memory unit 26 stores the inappropriate behavior in association with the detection time.

Next, we will explain an example of the operation of the driving training system 100 configured as described above. Below, we will explain the case of making a turn (parking) as training course C as an example. Note that a similar explanation can be applied to other training courses.

Before the driver drives, model information is generated and stored in the memory unit 26. With the model information stored in the memory unit 26, the driver drives the vehicle 10 on the training course C. In the vehicle 10, the onboard camera 11, peripheral camera 12, distance sensor 13, and operation data acquisition unit 14 generate driver image data, peripheral image data, distance data, and operation data, respectively. Each generated data is transmitted to the control device 20 by the communication unit 16. The generation and transmission of each data can be performed in real time at a predetermined cycle, such as a cycle corresponding to the frame rate of the onboard camera 11 and peripheral camera 12.

The control device 20 receives each piece of data transmitted from the vehicle 10 via the communication unit 21. The received data is stored in the memory unit 26. The acquisition unit 22 acquires driving information based on the received data. The acquisition unit 22 stores the acquired driving information in the memory unit 26.

The extraction unit 23 extracts inappropriate driving behaviors of the driver based on the acquired driving information and the model information stored in the memory unit 26. The extraction unit 23 detects differences between the driving information and the model information, and extracts inappropriate behaviors based on the detected differences. Figures 3 to 8 are diagrams that schematically show an example of operation in the driving training system 100.

For example, as shown in Figure 3, the extraction unit 23 compares the driver's gaze E1 in the driving information with the gaze E2 in the model information to detect the difference. In Figure 3, the driver's gaze E1 is directed forward, while the gaze E2 in the model information is directed toward the mirror. In other words, the model information shows the driver checking the mirror, but the driver does not check the mirror. Based on this difference, the extraction unit 23 extracts the driver's behavior as inappropriate if a required action, such as checking the mirror, has not been taken a predetermined time after the start of driving. In this case, the training processing unit 24 can present training information to the driver by voice, etc., via the presentation unit 25, instructing the driver to take the required action.

Furthermore, as shown in Figure 4, when the automobile 10 is instructed to travel backward from the driving start position P1 to the stopping position P2 on the driving course C, the extraction unit 23 can extract the driver's behavior as an inappropriate behavior when it detects that the difference D1 between the route R1 traveled by the automobile 10 driven by the driver and the route R2 in the model information is greater than a threshold value.

Furthermore, as shown in FIG. 5, when the driver drives the automobile 10 a certain distance L from the driving start position P1, the extraction unit 23 compares the distance L1 between the automobile 10 and the surrounding object 30 with the case in the example information (distance L2). If the extraction unit 23 detects that the difference between distance L1 and distance L2 is greater than a threshold, it can extract the driver's action as an inappropriate action.

In the cases shown in Figures 4 and 5, the training processing unit 24 generates driving instruction information to stop the automobile 10 and transmits the generated driving instruction information from the communication unit 21 to the automobile 10. The automobile 10 receives the driving instruction information from the control device 20 via the communication unit 16. When the automatic driving control unit 15 receives the driving instruction information, it stops the automobile 10 by, for example, automatically operating the brakes based on the driving instruction information. The automatic driving control unit 15 transmits information that the automobile 10 has stopped from the communication unit 16 to the control device 20.

When the control device 20 receives information that the vehicle 10 has stopped, the training processing unit 24 generates driving instruction information to move the stopped vehicle 10 back to the corresponding position before the inappropriate behavior occurred, and transmits the generated driving instruction information from the communication unit 21 to the vehicle 10. The vehicle 10 receives the driving instruction information from the control device 20. The autonomous driving control unit 15 automatically operates the steering, accelerator, brake, and other operating devices based on the received driving instruction information. Through this control, the vehicle 10 moves to the corresponding position P3, as shown in Figure 6. The autonomous driving control unit 15 transmits information that the vehicle 10 has moved to the corresponding position P3 to the control device 20 from the communication unit 16.

When the control device 20 receives information that the automobile 10 has moved to the corresponding position P3, the training processing unit 24 generates driving instruction information for driving the automobile 10 so as to reproduce the driver's driving from the corresponding position P3, and transmits the generated driving instruction information from the communication unit 21 to the automobile 10. The automobile 10 receives the driving instruction information from the control device 20. The autonomous driving control unit 15 automatically operates the steering, accelerator, brake, and other operating devices based on the received driving instruction information. Through this control, the automobile 10 automatically drives so as to reproduce the driver's driving from the corresponding position P3, as shown in FIG. 7. When the driver's driving from the corresponding position P3 is reproduced, the training processing unit 24 can present the driver with training information including details on how to correct the inappropriate behavior via video, text, audio, etc., as shown in FIG. 7, at the time an inappropriate behavior is extracted. The training information includes, for example, the direction to look, the amount of steering operation, the timing of operation, the amount of accelerator operation, the timing of operation, and the timing of brake operation, which are actions that the driver should correct. Additionally, the training information may include, for example, the position of the vehicle 10, the distance to surrounding objects 30, and images reflected in each mirror of the vehicle 10. When the training information is presented to the driver, the vehicle 10 may be stopped as appropriate.

The training processing unit 24 may also automatically drive the automobile 10 based on the model information before or after the driver drives. The training processing unit 24 generates driving instruction information to drive the automobile 10 so that driving based on the model information is performed, and causes the communication unit 21 to transmit the generated driving instruction information to the automobile 10. The automobile 10 receives the driving instruction information from the control device 20. The automatic driving control unit 15 automatically operates operating devices such as the steering wheel, accelerator, and brake based on the received driving instruction information. Through this control, the automobile 10 travels along route R2 based on the model information, as shown in Figure 8. When the automobile 10 travels along route R2, the training processing unit 24 can present training information including model actions to be performed while driving to the driver using images, text, audio, etc. When the vehicle 10 is driven automatically based on model information after the driver has driven, the training processing unit 24 may present model actions corresponding to the extracted inappropriate actions as training information, and may not present model actions for actions that are not extracted as inappropriate actions.

Figure 9 is a flowchart showing an example of the operation of the driving training system 100. As shown in Figure 9, when the driver drives the vehicle 10, driver image data, surrounding image data, distance data, and operation data are generated in the vehicle 10 (step S101). Each of the generated data is transmitted to the control device 20 by the communication unit 16 (step S102).

In the control device 20, the communication unit 21 receives each piece of data transmitted from the vehicle 10 (step S103). The received data is stored in the memory unit 26. The acquisition unit 22 acquires driving information based on the received data (step S104). The acquisition unit 22 stores the acquired driving information in the memory unit 26. The extraction unit 23 extracts inappropriate driving behaviors of the driver based on the acquired driving information and the model information stored in the memory unit 26 (step S105). If an inappropriate behavior is extracted (Yes in step S106), the training processing unit 24 performs training processing according to the extracted inappropriate behavior (step S107). Examples of training processing include stopping the vehicle 10 and returning it to a corresponding position, having the driver reproduce the driver's driving from the corresponding position, and presenting the driver with training information including details on how to correct the inappropriate behavior. In this case, the training processing unit 24 generates driving instruction information based on the training processing and causes the communication unit 21 to transmit the generated driving instruction information to the vehicle 10. Additionally, the training process includes determining whether the driver passes or fails based on the extracted inappropriate behavior.

On the other hand, if no inappropriate behavior is detected (No in step S106), it is checked whether driving on the training course has been completed (step S108). If it has been completed (Yes in step S108), the process ends; if it has not been completed (No in step S108), the operations from step S103 onwards are repeated.

When driving instruction information is generated and transmitted in the vehicle 10 during the training process in step S107, the vehicle 10 receives the driving instruction information (Yes in step S109). The autonomous driving control unit 15 automatically drives the vehicle 10 based on the received driving instruction information while presenting the training information to the driver (step S110). If driving instruction information is not received (No in step S109), the system checks whether driving along the training course has been completed (step S111). If completed (Yes in step S111), the process ends; if not completed (No in step S111), the operations from step S101 onwards are repeated.

Figure 10 is a flowchart showing another example of the operation of the driving training system 100. As shown in Figure 10, in the control device 20, the training processing unit 24 generates driving instruction information to drive the automobile 10 so that driving based on the model information is performed, and causes the communication unit 21 to transmit the generated driving instruction information to the automobile 10 (step S201). The automobile 10 receives the driving instruction information from the control device 20 (step S202). The automatic driving control unit 15 automatically operates operating devices such as the steering wheel, accelerator, and brake based on the received driving instruction information (step S203). When the automobile 10 is driving automatically, the training processing unit 24 presents training information including model actions to be performed while driving to the driver using images, text, audio, etc. via the presentation unit 25 (step S204). In step S204, when the vehicle 10 is driven automatically based on the model information after the driver has driven, the training processing unit 24 may present model actions corresponding to the extracted inappropriate actions as training information, and may not present model actions for actions that were not extracted as inappropriate actions.

As described above, the driving training system 100 according to this embodiment includes a vehicle 10 capable of running on a training course, and a control device 20 that acquires driving information when a driver drives the vehicle 10 on the training course, and extracts inappropriate driving behaviors of the driver based on the acquired driving information and model information that serves as a model for driving the vehicle 10 on the training course.

With this configuration, the control device 20 extracts inappropriate driving behaviors by the driver based on the driving information and model information, and by checking the extracted inappropriate behaviors, it becomes possible to appropriately evaluate the driver's driving.

In the driving training system 100 according to this embodiment, the automobile 10 is capable of autonomous driving, and the control device 20 stops the automobile 10 if an inappropriate behavior is detected. This configuration prevents the automobile 10 from continuing to drive due to an inappropriate behavior, thereby reducing the likelihood of the automobile 10 coming into contact with surrounding objects.

In the driving training system 100 according to this embodiment, the control device 20 moves the stopped vehicle 10 back to the corresponding position before the inappropriate behavior occurred based on the acquired driving information, and automatically drives the vehicle 10 so as to reproduce the driver's driving from the corresponding position, while presenting the driver with training information including details on how to correct the inappropriate behavior. This configuration allows the driver to objectively view their own driving while receiving training on inappropriate behavior.

In the driving training system 100 according to this embodiment, the control device 20 judges whether the driver's driving is pass or fail based on the results of the extraction of inappropriate behavior. This configuration makes it possible to efficiently judge whether the driver's driving is pass or fail.

In the driving training system 100 according to this embodiment, the automobile 10 is capable of autonomous driving, and the control device 20 automatically drives the automobile 10 along a training course based on model information while presenting the driver with training information including details of actions to be performed while driving. If an inappropriate behavior is detected, the control device 20 presents the driver with training information including details of actions corresponding to the detected inappropriate behavior. This configuration allows the driver to objectively view exemplary driving while receiving training on inappropriate behavior.

In the driving training system 100 according to this embodiment, the control device 20 detects the difference between the driving information and the model information based on the distance between the vehicle 10 and objects around the vehicle 10, and extracts inappropriate behaviors based on the difference. This configuration enables inappropriate behaviors to be extracted with high accuracy.

The driving lesson system 100 according to this embodiment includes a vehicle 10 capable of autonomous driving on a lesson course, and a control device 20 that automatically drives the vehicle 10 on the lesson course based on model information that serves as a model for driving the vehicle 10 on the lesson course.

This configuration allows the vehicle 10 to be driven automatically on a training course based on model information, allowing the driver to objectively observe model driving while undergoing training.

In the driving training system 100 according to this embodiment, the control device 20 automatically drives the automobile 10 while presenting the driver with training information, including the actions to be performed while driving. This configuration allows the driver to understand in real time the actions to be performed while driving while being shown exemplary driving.

In the driving training system 100 according to this embodiment, the automobile 10 can be driven by the driver, and the control device 20 acquires driving information when the driver drives the automobile 10 on a training course, extracts inappropriate driving behaviors by the driver based on the acquired driving information and model information, and presents the details of the behaviors corresponding to the extracted inappropriate behaviors. This configuration enables training that reflects the driver's driving behavior.

The technical scope of the present invention is not limited to the above-described embodiments, and modifications can be made as appropriate without departing from the spirit of the present invention. For example, the above-described embodiments have been described using an example configuration in which the peripheral camera 12 and distance sensor 13 are disposed on the automobile 10, but this is not limited to this. For example, a configuration corresponding to the peripheral camera and distance sensor may be disposed on the training course side. Furthermore, peripheral cameras and distance sensors may be used on both the automobile 10 side and the training course side.

In addition, in the above embodiment, an example was described in which the control device 20 provides training to each individual vehicle 10, but this is not limited to this. The control device 20 may also provide training to multiple vehicles 10 in cooperation with each other. In this case, it may be possible to detect common weak actions among drivers from the driving information of the multiple vehicles 10, and to reinforce training for weak actions.

Furthermore, in the above embodiment, the vehicle 10 was described as an example of a vehicle that drives on an actual driving course, but this is not limited to this. For example, driving information obtained by driving the vehicle 10 may be used in the simulator.

E1, E2...line of sight, L, L1, L2...distance, P1...starting position, P2...stopping position, P3...corresponding position, R1, R2...route, 10...vehicle, 11...onboard camera, 12...surrounding camera, 13...distance sensor, 14...operation data acquisition unit, 15...autonomous driving control unit, 16, 21...communication unit, 20...control device, 22...acquisition unit, 23...extraction unit, 24...training processing unit, 25...presentation unit, 26...storage unit, 30...object, 100...driving training system

Claims (3)

A car that can drive autonomously on a training course,
a control device that acquires driving information when a driver drives the vehicle on the training course, extracts inappropriate driving behavior of the driver based on the acquired driving information and model information that serves as a model for driving the vehicle on the training course , and transmits driving instruction information to the vehicle that instructs the vehicle to drive automatically ;
Equipped with
The control device instructs the vehicle to stop when the inappropriate behavior is extracted, instructs the vehicle to return to a corresponding position before the inappropriate behavior occurred based on the acquired driving information, instructs the vehicle to automatically drive so that the driving of the driver from the corresponding position is reproduced when the vehicle moves to the corresponding position, and presents the driver with training information including details to correct the inappropriate behavior when the vehicle is automatically driven based on the instructions.
Driving instruction system. The driving training system of claim 1, wherein the control device presents the driver with training information including the details of the actions that should be performed when driving the vehicle when the vehicle is driven autonomously based on the example information, and when an inappropriate action is extracted, the control device presents the driver with training information including the details of the action that corresponds to the extracted inappropriate action. The driving training system according to claim 1 or 2, wherein the control device detects a difference between the driving information and the example information based on a distance between the vehicle and an object around the vehicle, and extracts the inappropriate behavior based on the difference.