JP7613129B2 - Pedal misapplication prevention device - Google Patents

Description

The present invention relates to a pedal misapplication prevention device.

Patent Document 1 (Utility Model Registration No. 3227177) discloses a technology to prevent accidents caused by accidentally stepping on the accelerator pedal instead of the brake pedal. Specifically, a display device is installed that displays images of the accelerator pedal, the brake pedal, and the driver's feet.

However, the configuration of Patent Document 1 above has the problem that the display device constantly displays images, making it difficult to concentrate on driving.

The objective of this disclosure is to provide a pedal misapplication prevention device that prevents pedal misapplication while being mindful of not interfering with the driver's ability to concentrate on driving.

According to the present disclosure, a pedal misapplication prevention device is provided that includes an image information acquisition unit that acquires image information from an imaging unit that images an accelerator pedal, a brake pedal, and a driver's feet, an indication detection unit that detects indications of pedal misapplication by the driver, and an image information output unit that outputs the image information in a manner that allows the driver to view the image information when the indication detection unit detects the indication.

This disclosure provides a pedal misapplication prevention device that prevents pedal misapplication while being mindful not to interfere with the driver's concentration on driving.

FIG. 2 is a functional block diagram of a suppression device according to the first embodiment; 1 is a control flow of a suppression device (first embodiment). FIG. 11 is a diagram illustrating an example of an output from a captured image information output unit (first embodiment). FIG. 11 is a diagram illustrating an example of an output from a captured image information output unit (first embodiment). FIG. 11 is a diagram illustrating an example of an output from a captured image information output unit (first embodiment). FIG. 11 is a functional block diagram of a suppression device according to a second embodiment.

First Embodiment
Hereinafter, a first embodiment of the present disclosure will be described with reference to Fig. 1 to Fig. 5. The suppression device 1 is a specific example of a pedal erroneous depression suppression device. The suppression device 1 suppresses a driver of a vehicle from erroneously depressing the accelerator pedal when the driver intends to depress the brake pedal.

The suppression device 1 is equipped with a CPU 1a (Central Processing Unit) as a central processing unit, a readable and writable RAM 1b (Random Access Memory), and a read-only ROM 1c (Read Only Memory). The CPU 1a reads and executes a control program stored in the ROM 1c, which causes hardware such as the CPU 1a to function as various functional units.

The various functional units include an image information acquisition unit 2, a precursor detection unit 3, an image information output unit 4, a pedal operation detection unit 5, a foot position information generation unit 6, and a distribution information generation unit 7.

Furthermore, a foot camera 8 and a display 9 are connected to the restraint device 1.

The foot camera 8 is a specific example of an imaging unit. The foot camera 8 captures the accelerator pedal, brake pedal, and driver's feet of the vehicle. The driver's feet are typically the right foot of the driver involved in pedal operation, and are the part from the ankle onwards. In the embodiment, the operation of the right foot is described, but it can also be applied to brake pedal operation with the left foot. The foot camera 8 is typically attached facing downward around the dashboard so that the accelerator pedal, brake pedal, and driver's feet can be simultaneously captured. However, instead of this, a camera mounted on a drive recorder may be used as the foot camera 8. The camera mounted on a drive recorder is typically installed on the windshield or dashboard, and captures the front of the vehicle. Therefore, when a camera mounted on a drive recorder is used as the foot camera 8, multiple mirrors may be installed in the vehicle cabin so that the accelerator pedal, brake pedal, and driver's feet are within the angle of view of the camera mounted on the drive recorder, and mirror reflection may be used. Similarly, a camera for a driver monitoring system or a camera for capturing images inside the vehicle cabin may be used.

The display 9 is a HUD (Head-Up Display) that displays an image as a virtual image, or a projector that displays an image as a real image. The display 9 may be a display mounted on a navigation device, a display mounted on an instrument panel, or a display unit of a mobile terminal owned by a passenger that is used via wireless communication, etc.

The captured image information acquisition unit 2 acquires captured image information from the foot camera 8. Hereinafter, the captured image information acquired by the captured image information acquisition unit 2 from the foot camera 8 is also referred to as foot image information.

The sign detection unit 3 detects signs of pedal misapplication by the driver. "Pedal misapplication by the driver" means that the driver mistakenly presses the accelerator pedal while intending to press the brake pedal. Hereinafter, "signs of pedal misapplication by the driver" will be simply referred to as pedal misapplication signs.

When the indication detection unit 3 detects an indication of pedal misapplication, the captured image information output unit 4 outputs the foot image information in a manner that allows the driver to visually recognize the foot image information. The captured image information output unit 4 typically outputs the foot image information to a display 9. Figure 3 shows a specific example of the manner in which the captured image information output unit 4 outputs the foot image information.

Returning to FIG. 1, the pedal operation detection unit 5 detects the pedal operation by the driver via an in-vehicle network (not shown) or the like. Here, "pedal operation" includes the driver's operation of depressing the brake pedal with his/her foot and the driver's operation of depressing the accelerator pedal with his/her foot. The pedal operation detection unit 5 typically detects the driver's operation of depressing the brake pedal with his/her foot using a mechanical switch or a brake pedal position sensor mounted on the brake pedal. The pedal operation detection unit 5 also typically detects the driver's operation of depressing the accelerator pedal with his/her foot using an accelerator position sensor mounted on the accelerator pedal.

The foot position information generating unit 6 generates foot position information when the driver operates the pedal based on the foot image information. Specifically, the foot position information generating unit 6 detects the driver's feet using machine learning object detection techniques such as Faster R-CNN (Region Based Convolutional Neural Networks), YOLO (You only look once), and SSD (Single Shot Multibox Detector) based on the foot image information acquired by the captured image information acquiring unit 2 when the pedal operation detecting unit 5 detects the pedal operation, generates foot position information when the driver operates the pedal, and stores the generated foot position information in RAM 1b. The foot position information is coordinate information corresponding to the position of the foot in the foot image information. The foot position information is typically coordinate information of a specific reference point of a bounding box generated when the foot position information generating unit 6 detects the driver's feet using object detection technology. The specific reference point is typically the upper left corner of the bounding box. The foot position information generating unit 6 generates accelerator foot position information, which is foot position information when the driver operates the accelerator pedal, and brake foot position information, which is foot position information when the driver operates the brake pedal.

The distribution information generating unit 7 generates distribution information of the foot position information and stores the generated distribution information in RAM 1b. The distribution information includes accelerator distribution information, which is distribution information of the accelerator foot position information, and brake distribution information, which is distribution information of the brake foot position information.

The accelerator distribution information includes the mean and standard deviation of the accelerator foot position information. When the accelerator foot position information is coordinate information composed of ordinate values and abscissa values, the accelerator distribution information includes the mean and standard deviation of the ordinate values, and the mean and standard deviation of the abscissa values.

Similarly, the brake distribution information includes the mean and standard deviation of the brake foot position information. If the brake foot position information is coordinate information consisting of ordinate values and abscissa values, the brake distribution information includes the mean and standard deviation of the ordinate values and the mean and standard deviation of the abscissa values.

The distribution information generation unit 7 stores the generated accelerator distribution information and brake distribution information in RAM 1b.

Then, the sign detection unit 3 detects signs of misstepping based on the distribution information.

Specifically, when the pedal operation detection unit 5 detects the operation of the accelerator pedal by the driver, the sign detection unit 3 detects a sign of misapplication based on the accelerator foot position information generated by the foot position information generation unit 6 and the accelerator distribution information stored in RAM 1b. For example, the sign detection unit 3 detects a sign of misapplication when the abscissa value of the accelerator foot position information deviates from the average of the abscissa values of the accelerator distribution information by more than twice the standard deviation.

Similarly, when the pedal operation detection unit 5 detects the driver's operation of the brake pedal, the sign detection unit 3 detects a sign of misapplication based on the brake foot position information generated by the foot position information generation unit 6 and the brake distribution information stored in RAM 1b. For example, the sign detection unit 3 detects a sign of misapplication when the abscissa value of the brake foot position information deviates from the average of the abscissa values of the brake distribution information by more than twice the standard deviation.

In short, the sign detection unit 3 detects a sign of misapplication if the position of the driver's foot when operating the pedal is significantly different from the usual position. The position of the driver's foot and the degree of deviation may be determined and updated using known techniques such as machine learning. Also, for example, if the ordinate value of the foot position is significantly different from the average of the ordinate values of the accelerator or brake distribution information, it is possible that the driver may miss pressing the pedal because he or she is not familiar with the vehicle, and such a case may also be detected as a sign of misapplication.

Next, the operation of the suppression device 1 will be explained with reference to Figure 2.

S100:
First, the captured image information acquisition unit 2 acquires foot image information from the foot camera 8 .

S110:
Next, the pedal operation detection unit 5 detects the pedal operation by the driver.

S120:
Next, the foot position information generating unit 6 generates foot position information when the driver operates the pedal based on the foot image information, and stores the generated foot position information in the RAM 1b.

S130:
Next, the suppression device 1 determines whether the foot position information generating unit 6 has repeated the generation of foot position information a predetermined number of times. If the result of step S130 is NO, the suppression device 1 returns the process to step S100. If the result of step S130 is YES, the suppression device 1 advances the process to step S140.

S140:
Next, the distribution information generating unit 7 generates distribution information of the foot position information and stores the generated distribution information in the RAM 1b. The generation of the distribution information of the foot position information is not limited to the above, and may be performed by continuously detecting the pedal operation of the driver and periodically updating the distribution information, or may be performed by identifying the driver using a publicly known person authentication technology (not shown) and storing and using information for each driver.

S150:
Next, the captured image information acquisition unit 2 acquires foot image information from the foot camera 8 .

S160:
Next, the pedal operation detection unit 5 detects the pedal operation by the driver.

S170:
Next, the foot position information generating unit 6 generates foot position information when the driver operates the pedal based on the foot image information, and stores the generated foot position information in the RAM 1b.

S180:
Next, the sign detection unit 3 detects a sign of misstepping based on the distribution information. If a sign of misstepping is detected, the sign detection unit 3 proceeds to step S190. On the other hand, if the sign detection unit 3 does not detect a sign of misstepping, the sign detection unit 3 returns the process to step S150.

S190:
Next, the captured image information output unit 4 outputs the feet image information to the display 9 as shown in FIG.

S200:
Next, the suppression device 1 determines whether the accelerator pedal is operated. If the answer is YES in step S200, the suppression device 1 advances the process to step S210. On the other hand, if the answer is NO in step S200, the suppression device 1 advances the process to step S230.

S210:
Next, the suppression device 1 determines whether the brake pedal is being operated. If the result of step S210 is YES, the captured image information output unit 4 stops outputting the foot image information (S220). On the other hand, if the result of step S210 is NO, the suppression device 1 repeats step S210.

S230:
Next, the suppression device 1 judges whether a predetermined time has elapsed since the judgment in step S200. If the answer is YES in step S230, the captured image information output unit 4 stops outputting the foot image information (S220). On the other hand, if the answer is NO in step S230, the suppression device 1 repeats step S230.

The first embodiment has been described above, and the above embodiment has the following features.

That is, the suppression device 1 (pedal misapplication suppression device) includes an image information acquisition unit 2 that acquires image information from a foot camera 8 (imaging unit) that captures images of the accelerator pedal, the brake pedal, and the driver's feet, a sign detection unit 3 that detects signs of misapplication, and an image information output unit 4 that outputs foot image information in a manner that allows the driver to visually recognize the foot image information when the sign detection unit 3 detects signs of misapplication. With the above configuration, a suppression device 1 is realized that suppresses pedal misapplication while being mindful not to interfere with the driver's concentration on driving.

The suppression device 1 further includes a foot position information generating unit 6 that generates foot position information when the driver operates the pedal based on foot image information, and a distribution information generating unit 7 that generates distribution information of the foot position information. The sign detecting unit 3 detects signs of misapplication based on the foot position information and the distribution information. That is, for example, when the driver twists his/her body to directly check the rear with his/her eyes, when there is a foreign object at the driver's feet that prevents the driver from operating the pedal as usual, or when the driver's clothes are disheveled that prevents the driver from operating the pedal as usual, the driver operates the pedal with a foot position different from usual, and therefore there is a high possibility that the driver will misapplication the pedal. Therefore, according to the above configuration, for example, when the driver operates the pedal with a foot position different from usual, it is possible to detect signs of misapplication.

In addition, in step S160, the pedal operation detection unit 5 detects the pedal operation by the driver. Specifically, the pedal operation detection unit 5 detects the operation of the accelerator pedal by the driver and the operation of the brake pedal by the driver. However, instead of this, the pedal operation detection unit 5 may detect only the operation of the accelerator pedal by the driver in step S160.

The captured image information output unit 4 also outputs foot image information until the driver changes his/her foot from the accelerator pedal to the brake pedal (steps S200, S210, S220). With the above configuration, foot image information continues to be output until the pedal misapplication is resolved, so that the misapplication can be effectively suppressed.

The first embodiment described above can be modified as follows:

In other words, the captured image information output unit 4 may process the foot image information by doubling the R value and halving the G and B values of the RGB component information of the foot image information so that the entire foot image has a reddish hue, and then output the foot image information. This makes it possible to effectively alert the driver.

Also, as shown in FIG. 4, the captured image information output unit 4 may output the foot image information after image processing the foot image information so as to emphasize the driver's right foot. Specifically, the captured image information output unit 4 may image process the foot image information so that a colored transparent figure 4a is displayed superimposed on the position of the right foot that operates the pedal. The captured image information output unit 4 may also image process the foot image information so as to relatively increase the brightness of the area of the right foot that operates the pedal. The captured image information output unit 4 may also image process the foot image information so as to superimpose a message, symbol, figure, etc. indicating that a sign of pedal misapplication has been detected around the area of the right foot that operates the pedal. This makes it possible to effectively alert the driver.

Also, as shown in FIG. 5, when the driver is operating the accelerator pedal, the captured image information output unit 4 may output the foot image information after image processing the foot image information so that a figure 4b is displayed to prompt the driver to shift from the accelerator pedal to the brake pedal. This makes it possible to effectively alert the driver. The captured image information output unit 4 may also process the foot image information so that the figure 4b is displayed in a flashing manner.

Second Embodiment
Next, a second embodiment of the present disclosure will be described with reference to FIG.

As shown in FIG. 6, a front camera 10 is connected to the suppression device 1. The front camera 10 is a camera that captures an image in front of the vehicle. The captured image information acquisition unit 2 acquires front image information as captured image information from the front camera 10. The front camera 10 is not limited to this, and may be a combination of a front camera and a rear camera that capture images in the direction in which the vehicle is traveling, or may be composed of multiple cameras facing the front, rear, left and right around the vehicle, or a camera that captures an image of the entire circumference using a fisheye lens, etc.

In this embodiment, the sign detection unit 3 detects a sign of misstepping when the driver does not depress the accelerator pedal even after a predetermined time has elapsed since the vehicle driven by the driver is stopped as the lead vehicle at an intersection where traffic lights are installed and the traffic light state changes from a no-go state that prohibits the vehicle from starting to a go-allowing state that allows the vehicle to start.

Specifically, the sign detection unit 3 detects that the vehicle is stopped as the lead vehicle at an intersection where traffic lights are installed, based on the forward image information and a publicly known image recognition method. The sign detection unit 3 also detects, based on the forward image information, that the traffic light's indication state has switched from a no-go state that prohibits the vehicle from starting to a proceeding allowance state that allows the vehicle to start. Then, when the vehicle driven by the driver is stopped as the lead vehicle at an intersection where traffic lights are installed, and the driver does not press the accelerator pedal even after a predetermined time has elapsed since the traffic light's indication state switched from a no-go state that prohibits the vehicle from starting to a proceeding allowance state that allows the vehicle to start, the sign detection unit 3 detects a sign of misstepping if the driver does not press the accelerator pedal.

In other words, the risk of misstepping increases when the timing of an operation that would normally be performed is delayed, for example, if the driver drops an object at their feet or becomes confused about operating the shift lever and does not press the accelerator pedal for a specified time. The specified time is, for example, about a few seconds, but it is not limited to this and may be a time set by learning the tendency of the time it takes to press the pedal based on the driver's daily operation timing and surrounding conditions. Therefore, with the above configuration, the risk of misstepping can be effectively reduced.

The above embodiments may be implemented in combination with each other, and may be modified, for example, as follows:

For example, the sign detection unit 3 may detect a sign of misstepping when it detects that clothing around the feet is loosened to a predetermined extent based on foot image information. "Clothing around the feet is loosened to a predetermined extent" refers to a state in which the excess part of the shoelace is not fastened, the shoelace is untied, the hook-and-loop fastener of the shoe is not fastened, or a large area (e.g., several centimeters or more) protrudes from the outline of the shoe body. Also, "clothing around the feet is loosened to a predetermined extent" refers to a state in which clothing such as pants or a skirt is spread out to a predetermined extent or more around the pedal, the outline of the shoe body cannot be recognized, or the shoe body cannot be detected by object detection technology.

The sign detection unit 3 may also quantitatively measure the driver's level of fatigue and driving posture, and detect signs of pedal misapplication based on the measurement results.

The sign detection unit 3 may also detect the position of the floor mat of the driver's seat based on the foot image information, and detect a sign of misstepping when the position of the floor mat is not within a predetermined range.

The sign detection unit 3 may also detect a sign of misapplication when it detects that there is a foreign object at the driver's feet based on the foot image information.

The sign detection unit 3 may also detect a sign of mispeduction when the distance from the vehicle to an obstacle ahead when the vehicle starts moving forward is less than or equal to a predetermined value.

The warning detection unit 3 may also detect warnings of misapplication when the distance from the vehicle to an obstacle behind the vehicle is less than or equal to a predetermined value when the vehicle starts moving backwards.

The sign detection unit 3 may also be configured to detect signs of misapplication when the vehicle starts moving forward while parked facing forward.

The sign detection unit 3 may also be configured to detect signs of misapplication when the vehicle is parked facing backwards and starts moving backwards.

The captured image information output unit 4 may also detect the accelerator pedal, brake pedal, and the driver's feet based on the foot image information, and output the foot image information after image processing so that an accelerator pedal figure is displayed at the accelerator pedal position, a brake pedal figure is displayed at the brake pedal position, and foot figures are displayed at the driver's foot position.

The suppression device 1 may further include a warning sound output unit that outputs a warning sound when the sign detection unit 3 detects a sign of misstepping.

The suppression device 1 may further include a vibration unit that vibrates the steering wheel and the driver's seat when the sign detection unit 3 detects a sign of pedal misapplication. The vibration unit includes a vibration motor and a vibration control unit. When the sign detection unit 3 detects a sign of pedal misapplication, the vibration control unit drives the vibration motor to vibrate the steering wheel and the driver's seat.

In the above example, the program can be stored and supplied to the computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). Examples of non-transitory computer readable media further include CD-ROM (Read Only Memory), CD-R, CD-R/W, and semiconductor memory (e.g., mask ROM. Examples of non-transitory computer readable media further include PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (random access memory). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path, such as an electric wire or optical fiber, or via a wireless communication path.

(Appendix 1)
an image information acquiring step of acquiring image information from an imaging unit that captures images of an accelerator pedal, a brake pedal, and a driver's feet;
a sign detection step of detecting a sign of pedal misapplication by the driver;
an image information output step of outputting the image information in a manner that allows the driver to visually recognize the image information when the sign is detected in the sign detection step;
Including,
A method for preventing pedal misapplication.

(Appendix 2)
A program that causes a computer to execute the pedal misapplication prevention method described in appendix 1.

REFERENCE SIGNS LIST 1 Suppression device 2 Captured image information acquisition unit 3 Premonition detection unit 4 Captured image information output unit 4a Colored transparent figure 4b Figure 5 Pedal operation detection unit 6 Foot position information generation unit 7 Distribution information generation unit 8 Foot camera 9 Display 10 Front camera

Claims (5)

an image information acquiring unit that acquires image information from an imaging unit that captures images of an accelerator pedal, a brake pedal, and a driver's feet;
a foot position information generating unit that generates foot position information, which is a position of the foot of the driver when operating a pedal, within the captured image information;
a distribution information generating unit that generates distribution information including an average value of the foot position information;
a sign detection unit that detects a sign of pedal erroneous operation by the driver based on the foot position information and an average value of the distribution information ;
an image information output unit that outputs the image information in a manner that the driver can visually recognize the image information when the sign detection unit detects the sign;
Equipped with
Pedal misapplication prevention device. an image information acquiring unit that acquires image information from an imaging unit that captures images of an accelerator pedal, a brake pedal, and a driver's feet;
a foot position information generating unit that generates foot position information, which is a position of the foot of the driver when operating a pedal, within the captured image information;
a distribution information generating unit that generates distribution information including an average value of the foot position information;
a sign detection unit that detects a sign of pedal erroneous operation by the driver based on the foot position information and an average value of the distribution information;
at least one of a warning sound output unit that outputs a warning sound when the sign detection unit detects the sign, and a vibration unit that vibrates a predetermined location when the sign detection unit detects the sign;
Equipped with
Pedal misapplication prevention device. an image information acquiring unit that acquires image information from an imaging unit that captures images of an accelerator pedal, a brake pedal, and a driver's feet;
a sign detection unit that detects a sign of pedal misapplication by the driver;
an image information output unit that outputs the image information in a manner that the driver can visually recognize the image information when the sign detection unit detects the sign;
Equipped with
the sign detection unit detects this as a sign of pedal misapplication when the driver does not depress the accelerator pedal even after a predetermined time has elapsed since the indication state of the traffic light changes from a no-go state that prohibits the vehicle from starting to a go-permit state that permits the vehicle to start, while the vehicle driven by the driver is stopped as a leading vehicle at an intersection where a traffic light is installed.
Pedal misapplication prevention device. the captured image information output unit outputs the captured image information so as to emphasize a foot position of the driver.
The pedal misapplication prevention device according to claim 1 or 3 . the captured image information output unit outputs the captured image information until the driver changes his/her foot position from the accelerator pedal to the brake pedal.
The pedal misapplication prevention device according to claim 1 or 3 .

Images