JP7528776B2 - Apparatus and method for determining inattentive behavior - Google Patents

Description

The present invention relates to an inattentive driving detection device and an inattentive driving detection method.

There is a known device that photographs the driver of a vehicle and judges whether the driver is looking away from the road based on the direction of the driver's face and gaze (see, for example, Patent Document 1). The technology described in Patent Document 1 judges the driver to be looking away from the road if the driver faces a specific on-board component for a reference period of time or longer.

JP 2020-24532 A

When a driver of a vehicle looks away, they tend to neglect to check the direction of travel if there is a particularly eye-catching object in a direction other than the direction of travel. In general, devices such as those described in Patent Document 1 warn the driver when they are looking away, within a time period that will prevent a dangerous situation from occurring.

However, for particularly eye-catching objects, other passengers in the vehicle as well as the driver tend to focus their gaze on the object. When other passengers in the vehicle as well as the driver of the vehicle focus their gaze on the object, the driver is more likely to be negligent in checking the direction in which the vehicle is moving.

The present invention has been made in consideration of the above, and aims to provide an appropriate warning to vehicle drivers when they are looking away from the road.

To solve the above-mentioned problems and achieve the objective, the inattentive behavior judgment device of the present invention includes a detection unit that detects the face or gaze direction of the driver of the vehicle and passengers other than the driver, and a judgment unit that, based on the detection result of the detection unit, judges that the driver is looking aside when facing a direction other than the traveling direction of the vehicle for a first period or more, and, if the passengers other than the driver are facing the same direction as the driver, judges that the driver is looking aside when facing a direction other than the traveling direction of the vehicle for a second period or more that is shorter than the first period.

The inattentive behavior determination method executed by the inattentive behavior determination device of the present invention includes the steps of detecting the face or gaze direction of the driver of the vehicle and passengers other than the driver, determining that the driver is looking away when facing a direction other than the traveling direction of the vehicle for a first period or more, and, if the passengers other than the driver are facing the same direction as the driver, determining that the driver is looking away when facing a direction other than the traveling direction of the vehicle for a second period or more that is shorter than the first period.

The present invention has the effect of being able to provide an appropriate warning to the driver of a vehicle when he or she is looking away from the road.

FIG. 1 is a block diagram showing an example of the configuration of an inattentive driving determination device according to the first embodiment. FIG. 2 is a schematic diagram illustrating an example of a detection range. FIG. 3 is a schematic diagram illustrating the direction of a passenger's face or line of sight. FIG. 4 is a flowchart showing a process flow in the inattentive driving determination device according to the first embodiment. FIG. 5 is a flowchart showing a process flow in the inattentive driving determination device according to the second embodiment. FIG. 6 is a flowchart showing a process flow in the inattentive driving determination device according to the third embodiment. FIG. 7 is a flowchart showing a process flow in the inattentive driving determination device according to the fourth embodiment.

The following describes in detail an embodiment of an inattentive driving detection device and an inattentive driving detection method according to the present invention with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiment.

[First embodiment]
<Warning device>
1 is a block diagram showing an example of the configuration of an inattentive driving determination device 20 according to a first embodiment. The warning device 10 determines inattentive driving and issues a warning based on the face or line of sight of the driver and passengers other than the driver of the vehicle.

The warning device 10 has an imaging unit 11, a notification unit 12, and an inattentive driving judgment device 20. The warning device 10 may be implemented, for example, as a device with a safe driving support function that is pre-installed in the vehicle, a navigation device, a drive recorder, etc., or as a function of an infotainment system. The warning device 10 may also be implemented as a function of a portable device that is brought into the vehicle.

The imaging unit 11 captures video including audio from within the vehicle cabin. The imaging unit 11 is, for example, an in-vehicle camera. The imaging unit 11 outputs imaging data, which is in-vehicle video data captured from within the vehicle cabin, to the imaging data acquisition unit 21 of the inattentive driving judgment device 20.

The function of the imaging unit 11 as an in-vehicle camera will be described. The imaging unit 11 is a camera that captures images of the interior of the vehicle. The imaging unit 11 is disposed in a position that allows it to capture images of at least the faces of passengers, including the driver, seated in all the vehicle seats. The imaging unit 11 is disposed, for example, inside or around the vehicle's rearview mirror. The imaging unit 11 has a fixed or nearly fixed imaging range and imaging direction. The imaging unit 11 is, for example, composed of a visible light camera or a near-infrared camera. The imaging unit 11 may be, for example, composed of a combination of a visible light camera and a near-infrared camera. The imaging unit 11 constantly captures images from when the engine starts until it stops, that is, while the vehicle is operating. The imaging data is, for example, a moving image composed of images at 30 frames per second.

The notification unit 12 notifies the driver of the vehicle of a warning. The notification unit 12 includes at least one of a speaker and a display unit. The notification unit 12 controls to output a notification based on a control signal from the warning control unit 24 of the inattentive driving judgment device 20. In this embodiment, the notification unit 12 controls to output a notification warning about inattentive driving based on a control signal from the warning control unit 24 of the inattentive driving judgment device 20.

The function of the notification unit 12 as a speaker will be described. As an example, the notification unit 12 is an audio output device specific to the warning device 10, or an audio output device shared with other systems including a navigation system. The notification unit 12 outputs audio based on an audio signal output from the warning control unit 24 of the inattentive driving judgment device 20. In this embodiment, the notification unit 12 controls the speaker to output an audio warning against inattentive driving based on an audio signal from the warning control unit 24 of the inattentive driving judgment device 20.

The function of the notification unit 12 as a display unit will be described. As an example, the notification unit 12 is a display unit specific to the warning device 10, or a display unit shared with other systems including a navigation system. The notification unit 12 is a display unit including, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display. In this embodiment, the notification unit 12 is disposed in front of the driver of the vehicle on a dashboard, an instrument panel, a center console, or the like. When the notification unit 12 is a display unit, it displays an image based on a video signal output from the warning control unit 24 of the inattentive driving judgment device 20. In this embodiment, the notification unit 12 controls the display unit to display an image warning about inattentive driving based on a video signal from the warning control unit 24 of the inattentive driving judgment device 20.

<Inattentiveness Detection Device>
The inattentive behavior judgment device 20 judges whether a driver of a vehicle is looking at something inattentive. The inattentive behavior judgment device 20 is an arithmetic processing device (control device) constituted by, for example, a CPU (Central Processing Unit) and the like. The inattentive behavior judgment device 20 loads a stored program into memory and executes instructions contained in the program. The inattentive behavior judgment device 20 includes an internal memory (not shown) that is used for temporarily storing data in the inattentive behavior judgment device 20, etc. The inattentive behavior judgment device 20 is a computer that runs the program according to the present invention. The inattentive behavior judgment device 20 has an imaging data acquisition unit 21, a detection unit 22, a judgment unit 23, and a warning control unit 24, which are connected to a bus 20X.

The imaging data acquisition unit 21 acquires imaging data captured by the imaging unit 11. More specifically, the imaging data acquisition unit 21 acquires imaging data that is in-vehicle video data output by the imaging unit 11, and that captures the faces of the driver of the vehicle and passengers other than the driver.

The detection unit 22 detects the face or gaze direction of the driver of the vehicle and passengers other than the driver. More specifically, the detection unit 22 recognizes people from the imaging data acquired by the imaging data acquisition unit 21. The method of recognizing people from the imaging data can be any known method and is not limited. The people to be detected are the driver of the vehicle and passengers other than the driver.

The detection unit 22 detects the face or gaze direction of the driver and passengers other than the driver of the vehicle. In this embodiment, the detection unit 22 performs image processing on the in-vehicle video data to recognize the faces of the driver and passengers other than the driver and detect the face or gaze direction. More specifically, the detection unit 22 recognizes, for example, the eyes of the driver and passengers other than the driver from the in-vehicle video data and obtains information indicating the face or gaze direction. Any method can be applied for gaze detection based on the image of the eyes of the driver and passengers other than the driver, such as gaze detection based on the positional relationship between the inner corner of the eye and the iris detected from the image of the eyes, or gaze detection based on the positional relationship between the corneal reflex and the pupil.

Using FIG. 2, a method for detecting a driver and passengers other than the driver from the in-vehicle video data 100 will be described. FIG. 2 is a schematic diagram for explaining an example of a detection range, and in-vehicle video data 100 of a vehicle with a passenger capacity of five will be explained as an example. The face of a person sitting in each seat is photographed within a range defined by the coordinates of the in-vehicle video data 100. As a result, a person or face recognized from a range defined by the coordinates of the in-vehicle video data 100 corresponding to each seat is a passenger in that seat. In this embodiment, for example, when a person or face is recognized from range 102 of the in-vehicle video data 100 in a vehicle with a steering position on the right side, the driver is seated in the right front seat. For example, when a person or face is recognized from range 101, range 103, range 104, or range 105 of the in-vehicle video data 100, the passenger is a passenger other than the driver. In this way, it is possible to detect the presence or absence of a passenger in each seat, and the direction of the passenger's face and eyes.

Figure 3 is a schematic diagram illustrating the direction of the passenger's face or gaze. A method for detecting the direction of the passenger's face or eyes will be described with reference to Figure 3. The detection unit 22 sets an offset angle with the face or eyes facing forward based on the position of the imaging unit 11 and the position of each seat. Figure 3 shows the detection of a passenger M (e.g., a driver) sitting in the right front seat (e.g., the driver's seat) when the imaging unit 11 is provided inside or near the inner mirror of the vehicle. Figure 3 shows the head of the passenger M. The front direction of the passenger M is the front of the vehicle, indicated by f. An offset angle b is set around a position assumed to be the center of the passenger M's head. The direction of the passenger M's face or gaze detected by the detection unit 22 is corrected by the offset angle b from the actually detected direction. The angle a shows an example of a range in which the direction of the passenger M's face or gaze is determined to be facing the front of the vehicle, in other words, not to be looking aside. In other words, if the direction of the passenger M's face or gaze is other than angle a when the vehicle is traveling straight, it is determined that the passenger is facing away from the vehicle, in other words, that the passenger is looking aside. Angle a is the range in which a normal driver would turn their face or gaze when the vehicle is traveling straight, that is, when the vehicle is not turning right or left or reversing.

Based on the detection result of the detection unit 22, the determination unit 23 determines that the driver is looking aside when the driver faces a direction other than the traveling direction of the vehicle for a first period t1 or more, and when the driver and passengers other than the driver face the same direction as the driver, the determination unit 23 determines that the driver is looking aside when the driver faces a direction other than the traveling direction of the vehicle for a second period t2 (t2<t1) or more that is shorter than the first period t1. In other words, based on the detection result of the detection unit 22, when the driver faces a direction other than the traveling direction of the vehicle, the determination unit 23 sets the period until the driver is determined to be looking aside to the first period t1. Also, when the driver and passengers other than the driver face the same direction as the driver, the determination unit 23 sets the period until the driver is determined to be looking aside to the second period t2 (t2<t1) that is shorter than the first period t1. For example, the first period t1 is set to 3 seconds, and the second period t2 is set to 2 seconds.

When a passenger other than the driver faces the same direction as the driver, it does not necessarily mean that the angles are completely the same, but it may be determined that the directions are the same within a specified angle range including the same direction. For example, if the driver faces in the direction of angle a, the directions are determined to be the same within a specified angle range including angle a.

The timing when passengers other than the driver are looking in the same direction as the driver may be such that the period when the driver is looking in a direction other than the direction of travel partially overlaps with the period when passengers other than the driver are looking in the same direction as the driver.

When passengers other than the driver are facing the same direction as the driver, there is a high possibility that there is a particularly eye-catching object in the direction they are facing, and the driver may end up staring at the object. Furthermore, passengers other than the driver will also be less conscious of checking the direction the vehicle is traveling, making it more likely that the driver will become even less vigilant in checking the direction the vehicle is traveling. Therefore, the system determines that the driver is looking away at an earlier point in time than when only the driver is facing in a direction other than the direction the vehicle is traveling.

The warning control unit 24 controls the output of a warning to the driver. More specifically, when the determination unit 23 determines that the driver is looking aside, the warning control unit 24 warns the driver of looking aside by using a display by the notification unit 12 or an audio output (not shown). The warning control unit 24 outputs a video signal and controls the notification unit 12 to display an image. The warning control unit 24 outputs an audio signal and controls the notification unit 12 to output an audio. A warning against looking aside is appropriately issued by outputting an audio signal, but a warning may also be issued by displaying an image in addition to an audio signal.

<Processing in the inattentiveness detection device>
Next, the flow of processing in the inattentive driving judgment device 20 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing in the inattentive driving judgment device 20 according to the first embodiment. When the warning device 10 is activated, the processing of the flowchart shown in Fig. 4 is started. The processing of Fig. 4 is started under any condition. For example, this is when the engine of the vehicle equipped with the warning device 10 is started, making the vehicle available for use, or when the operation of the warning device 10 is started by a user operation. Furthermore, the processing of step S101 may be executed on the condition that the vehicle is traveling.

When the process of FIG. 4 starts, the inattentiveness judgment device 20 starts the process of detecting passengers and detecting the direction of their faces or gaze (step S101). More specifically, the inattentiveness judgment device 20 starts capturing images of the driver and passengers other than the driver in the vehicle cabin using the imaging unit 11. The inattentiveness judgment device 20 acquires the imaging data output by the imaging unit 11 using the imaging data acquisition unit 21. The inattentiveness judgment device 20 recognizes the driver and passengers other than the driver in the vehicle cabin from the imaging data acquired by the imaging data acquisition unit 21 using the detection unit 22. Then, the inattentiveness judgment device 20 detects the direction of the faces or gaze of the driver and passengers other than the driver using the detection unit 22. The inattentiveness judgment device 20 proceeds to step S102.

The inattentiveness judgment device 20 judges whether the driver is facing in a direction other than the traveling direction (step S102). More specifically, the inattentiveness judgment device 20 judges that the driver is facing in a direction other than the traveling direction when the direction of the driver's face or gaze detected from the range 102 of the in-vehicle video data 100 shown in FIG. 2 is a direction other than angle a when the vehicle is traveling straight, based on the detection result of the detection unit 22, by the judgment unit 23. When the inattentiveness judgment device 20 judges that the driver is facing in a direction other than the traveling direction based on the detection result of the detection unit 22 (Yes in step S102), it proceeds to step S103. When the inattentiveness judgment device 20 judges that the driver is not facing in a direction other than the traveling direction based on the detection result of the detection unit 22, by the judgment unit 23 (No in step S102), it proceeds to step S109.

When it is determined that the driver is facing a direction other than the traveling direction (Yes in step S102), the inattentiveness judgment device 20 determines whether passengers other than the driver are facing the same direction as the driver (step S103). More specifically, the inattentiveness judgment device 20 determines, based on the detection result of the detection unit 22, by the judgment unit 23, that passengers other than the driver, in other words, passengers detected outside the range 102 of the in-vehicle video data 100 shown in FIG. 2, are facing the same direction. When the inattentiveness judgment device 20 determines, based on the detection result of the detection unit 22, by the judgment unit 23, that passengers other than the driver are facing the same direction as the driver (Yes in step S103), the device proceeds to step S104. When the inattentiveness judgment device 20 determines, based on the detection result of the detection unit 22, that passengers other than the driver are not facing the same direction as the driver (No in step S103), the device proceeds to step S105.

When it is determined that a passenger other than the driver is facing the same direction as the driver (Yes in step S103), the inattentive driving judgment device 20 sets the period t until a warning is issued to the driver to a second period t2 by the judgment unit 23 (step S104). More specifically, the inattentive driving judgment device 20 sets the period for issuing a warning when the second period t2 has elapsed since the judgment unit 23 detected that the driver has faced in a direction other than the traveling direction, and proceeds to step S106.

When it is determined that passengers other than the driver are not facing the same direction as the driver (No in step S103), the inattentive behavior judgment device 20 sets the period t until a warning is issued to the driver to a first period t1 by the judgment unit 23 (step S105). More specifically, the inattentive behavior judgment device 20 sets the period for issuing a warning when the first period t1 has elapsed since the judgment unit 23 detected that the driver has faced in a direction other than the traveling direction, and proceeds to step S106.

The inattentive behavior judgment device 20 judges whether the period t has elapsed (step S106). More specifically, the inattentive behavior judgment device 20 judges by the judgment unit 23 whether the period t set in step S104 or step S105 has elapsed. If it is judged that the period t has elapsed (Yes in step S106), the process proceeds to step S108. In this case, the second period t2 or the first period t1 has elapsed since it was detected that the driver was facing in a direction other than the traveling direction. If it is not judged that the period t has elapsed, that is, if the period t has not elapsed (No in step S106), the process proceeds to step S107.

If the period t has not elapsed (No in step S106), the inattentive behavior judgment device 20 judges whether the driver is still facing in a direction other than the traveling direction (step S107). More specifically, the inattentive behavior judgment device 20 judges, by the judgment unit 23, whether the driver is continuing to face in a direction other than the traveling direction. If it is judged that the driver is still facing in a direction other than the traveling direction (Yes in step S107), the process proceeds to step S106, where it is judged whether the second period t2 or the first period t1 based on the judgment in step S103 has elapsed. If it is judged that the driver is not facing in a direction other than the traveling direction (No in step S107), the process proceeds to step S109.

The inattentive driving judgment device 20 determines that the driver is looking aside and issues a warning (step S108). More specifically, the inattentive driving judgment device 20 warns the driver of inattentive driving via the warning control unit 24. For example, the inattentive driving judgment device 20 may output an audio signal via the warning control unit 24 to the notification unit 12 to output an audio indicating a warning from the speaker. For example, the inattentive driving judgment device 20 may output a video signal via the warning control unit 24 to the notification unit 12 to display an image indicating a warning on the display unit. The inattentive driving judgment device 20 proceeds to step S109.

The inattentiveness judgment device 20 determines whether or not to end the occupant detection and face or gaze direction detection process (step S109). For example, when an operation to end the process is detected, or when the vehicle is stopped and the engine is turned off, the inattentiveness judgment device 20 determines to end the occupant detection and face or gaze direction detection process. When the inattentiveness judgment device 20 determines to end the occupant detection and face or gaze direction detection process (Yes in step S109), it ends the process. When the inattentiveness judgment device 20 does not determine to end the occupant detection and face or gaze direction detection process (No in step S109), it executes the process of step S102 again.

The process shown in FIG. 4 and described above is the process when the driver and passengers other than the driver are in the vehicle. When there are no passengers other than the driver in the vehicle, in other words, when the driver is the only driver in the vehicle, if the driver's face or line of sight is directed in a direction other than the direction of travel for a first period of time t1, it is determined that the driver is looking away, and a warning is issued.

＜Effects＞
As described above, in this embodiment, it is determined that the driver is looking aside when facing a direction other than the traveling direction of the vehicle for a first period t1 or more. In this embodiment, when a passenger other than the driver faces the same direction as the driver, it is determined that the driver is looking aside when the passenger faces a direction other than the traveling direction of the vehicle for a second period t2 or more, which is shorter than the first period t1. According to this embodiment, it is possible to determine whether the driver is looking aside based on the face or line of sight of the driver and the passenger other than the driver. In this embodiment, when the passenger other than the driver and the driver are facing the same direction, it is possible to issue a warning at an earlier timing. According to this embodiment, it is possible to appropriately issue a warning against the driver of the vehicle looking aside.

[Second embodiment]
The warning device 10 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of processing in an inattentive driving determination device according to a second embodiment. The configuration of the warning device 10 according to the second embodiment is the same as that of the warning device 10 according to the first embodiment, but the processing is different, so a description of the configuration will be omitted.

The detection unit 22 further detects the seating positions of passengers other than the driver. In this embodiment, the detection unit 22 distinguishes between passengers other than the driver who are seated in the front seat and the rear seat. More specifically, as shown in FIG. 2, the imaging range and imaging direction of the imaging unit 11 are fixed or almost fixed, so that the detection unit 22 can distinguish between passengers who are seated in the front seat and the rear seat. In the example shown in FIG. 2, the seating positions of the front seat correspond to the range 101 and the range 102, and the seating positions of the rear seat correspond to the range 103, the range 104, and the range 105. For example, when a person or face is recognized from the range 101 of the in-vehicle image data 100, the passenger is in the left front seat. For example, when a person or face is recognized from the range 102 of the in-vehicle image data 100, the passenger is in the right front seat. For example, when a person or face is recognized from the range 103 of the in-vehicle image data 100, the passenger is in the left rear seat. For example, if a person or face is recognized from range 104 of the in-vehicle video data 100, the passenger is in the center rear seat. For example, if a person or face is recognized from range 105 of the in-vehicle video data 100, the passenger is in the right rear seat.

When a passenger in the front seat of the vehicle faces the same direction as the driver, the determination unit 23 sets the second period to be shorter than when a passenger in the rear seat of the vehicle faces the same direction as the driver. In this embodiment, when a passenger detected from range 101 of the in-vehicle video data 100 shown in FIG. 2 faces the same direction as the driver detected from range 102, the determination unit 23 sets the second period to be shorter than when a passenger detected from ranges 103, 104, and 105 faces the same direction as the driver.

Front seat passengers have the same field of vision as the driver, and can therefore see the same objects as the driver. In addition, front seat passengers are more likely to communicate with the driver, so their looking away is more likely to affect the driver. For this reason, if a front seat passenger is facing the same direction as the driver, the driver is more likely to be less vigilant in checking the direction the vehicle is traveling. As such, the system determines that a rear seat passenger is looking away at an earlier point in time than if they are facing the same direction as the driver.

Next, the flow of processing in the inattentive driving judgment device 20 will be described with reference to FIG. 5. The processing in steps S111 to S113, S116, S117, S120, and S121 shown in FIG. 5 is the same as that in steps S101 to S103, S105, S104, S108, and S109 shown in FIG. 4.

When it is determined that the passengers other than the driver are facing the same direction as the driver (Yes in step S113), the inattentiveness judgment device 20 determines whether the front seat passengers other than the driver are facing the same direction as the driver (step S114). More specifically, the inattentiveness judgment device 20 determines whether the passengers detected from the range 101 of the in-vehicle video data 100 shown in FIG. 2 are facing the same direction as the driver detected from the range 102 based on the detection result of the detection unit 22 by the judgment unit 23. When the inattentiveness judgment device 20 determines that the front seat passengers other than the driver are facing the same direction as the driver based on the detection result of the detection unit 22 (Yes in step S114), the device proceeds to step S115. When the inattentiveness judgment device 20 determines that the front seat passengers other than the driver are not facing the same direction as the driver based on the detection result of the detection unit 22 by the judgment unit 23 (No in step S114), the device proceeds to step S117.

When it is determined that a front seat passenger other than the driver is facing the same direction as the driver (Yes in step S114), the inattentive driving judgment device 20 sets the period t until a warning is issued to the driver to a third period t3 (t3<t2) that is shorter than the second period t2 by the judgment unit 23 (step S115). More specifically, the inattentive driving judgment device 20 sets the period for issuing a warning when the third period t3 has elapsed since the judgment unit 23 detected that the driver has faced in a direction other than the traveling direction), and proceeds to step S118. The third period t3 is a value that is a shorter value than the second period, and is set to, for example, 1 second.

The inattentive behavior judgment device 20 judges whether the period t has elapsed (step S118). More specifically, the inattentive behavior judgment device 20 judges by the judgment unit 23 whether the period t set in step S115, step S116, or step S117 has elapsed. If it is judged that the period t has elapsed (Yes in step S118), the process proceeds to step S120. In this case, the third period t3, the second period t2, or the first period t1 has elapsed since it was detected that the driver was facing in a direction other than the traveling direction. If it is not judged that the period t has elapsed, that is, if the period t has not elapsed (No in step S118), the process proceeds to step S119.

If the period t has not elapsed (No in step S118), the inattentive behavior judgment device 20 judges whether the driver is still facing in a direction other than the traveling direction (step S119). More specifically, the inattentive behavior judgment device 20 judges, by the judgment unit 23, whether the driver is continuing to face in a direction other than the traveling direction. If it is judged that the driver is still facing in a direction other than the traveling direction (Yes in step S119), the process proceeds to step S118, and it is judged whether the third period t3, the second period t2, or the first period t1 based on the judgment in step S113 or step S114 has elapsed. If it is judged that the driver is not facing in a direction other than the traveling direction (No in step S119), the process proceeds to step S121.

As described above, in this embodiment, when a passenger in the front seat of the vehicle is facing the same direction as the driver, a third period is set that is shorter than the second period, compared to when a passenger in the rear seat of the vehicle is facing the same direction as the driver. According to this embodiment, when a passenger in the front seat is facing the same direction as the driver, a warning can be issued at an earlier timing than when a passenger in the rear seat is facing the same direction as the driver.

[Third embodiment]
The warning device 10 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of processing in an inattentive driving determination device according to a third embodiment. The configuration of the warning device 10 according to the third embodiment is the same as that of the warning device 10 according to the first embodiment, but the processing is different, so a description of the configuration will be omitted.

The determination unit 23 sets the second period to be shorter the more passengers are facing the same direction as the driver. In this embodiment, the determination unit 23 sets a third period that is shorter than the second period when the number of passengers facing the same direction as the driver is a predetermined ratio. The predetermined ratio is, for example, 50%. In other words, when there are two passengers other than the driver, it can be determined that the predetermined ratio is met when one passenger is facing the same direction as the driver. Similarly, when there are three passengers other than the driver, it can be determined that the predetermined ratio is met when two passengers are facing the same direction as the driver.

The more passengers who are facing the same direction as the driver, the more likely it is that the vehicle passengers will be paying more attention to the object, and the more likely it is that the driver will be less careful about checking the direction the vehicle is traveling. Therefore, the more passengers who are facing the same direction as the driver, the earlier it will be determined that they are looking away.

Next, the flow of processing in the inattentive driving judgment device 20 will be described with reference to FIG. 6. The processing in steps S131 to S133 and steps S135 to S141 shown in FIG. 6 is the same as the processing in steps S111 to S113 and steps S115 to S121 shown in FIG. 5.

When it is determined that passengers other than the driver are facing the same direction as the driver (Yes in step S133), the inattentive behavior judgment device 20 determines whether or not a predetermined percentage or more of passengers other than the driver are facing the same direction as the driver (step S134). More specifically, when the inattentive behavior judgment device 20 determines, based on the detection result of the detection unit 22, by the judgment unit 23, that a predetermined percentage of passengers other than the driver are facing the same direction as the driver (Yes in step S134), the device proceeds to step S135. When the inattentive behavior judgment device 20 determines, based on the detection result of the detection unit 22, by the judgment unit 23, that a predetermined percentage of passengers other than the driver are not facing the same direction as the driver (No in step S134), the device proceeds to step S137.

As described above, in this embodiment, the greater the number of passengers facing the same direction as the driver, the shorter the third period is set, which is shorter than the second period. According to this embodiment, the greater the number of passengers facing the same direction as the driver, the earlier the warning can be issued.

[Fourth embodiment]
The warning device 10 according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of processing in an inattentive driving determination device according to a fourth embodiment. The configuration of the warning device 10 according to the fourth embodiment is the same as that of the warning device 10 according to the first embodiment, but the processing is different, so a description of the configuration will be omitted.

The imaging unit 11 also functions as a microphone that picks up audio data of the sound inside the vehicle cabin. In this embodiment, the imaging unit 11 includes an in-vehicle camera and a microphone. The imaging unit 11 outputs imaging data including the captured video data and audio data of the inside of the vehicle cabin to the imaging data acquisition unit 21 of the inattentive driving judgment device 20.

The imaging data acquisition unit 21 further acquires in-vehicle video data including audio data output by the imaging unit 11.

The detection unit 22 further detects voices uttered by the driver of the vehicle and passengers other than the driver. More specifically, the detection unit 22 recognizes voices from the audio data of the imaging data acquired by the imaging data acquisition unit 21. The method of recognizing voices from audio data can be any known method and is not limited to this method. The voices uttered by the driver of the vehicle and passengers other than the driver detected by the detection unit 22 may simply be detected as having been uttered, or the content of the utterance may also be detected by speech analysis. For example, when words such as "look" or "accident" are spoken, it may be determined that a voice has been uttered.

When a passenger other than the driver is facing the same direction as the driver and is uttering a voice, the determination unit 23 sets a third period that is shorter than the second period.

If a passenger other than the driver is facing the same direction as the driver and is speaking, the driver's attention will be drawn to the voice, increasing the likelihood that the driver's attention will be diverted to a direction other than the direction of travel. For this reason, it is preferable to determine whether the passenger is looking away at an earlier stage.

The timing of the voice being emitted only needs to overlap partially with the period during which passengers other than the driver are looking in the same direction as the driver.

Next, the flow of processing in the inattentive driving judgment device 20 will be described with reference to FIG. 7. The processing in steps S151 to S153 and steps S155 to S161 shown in FIG. 7 is the same as that in steps S111 to S113 and steps S115 to S121 shown in FIG. 5.

When it is determined that a passenger other than the driver is facing the same direction as the driver (Yes in step S153), the inattentive behavior judgment device 20 determines whether or not the passenger is speaking (step S154). More specifically, when the inattentive behavior judgment device 20 determines by the judgment unit 23 based on the detection result of the detection unit 22 that the passenger is speaking (Yes in step S154), the device proceeds to step S155. When the inattentive behavior judgment device 20 determines by the judgment unit 23 based on the detection result of the detection unit 22 that the passenger is not speaking (No in step S154), the device proceeds to step S157.

As described above, in this embodiment, if a passenger other than the driver is facing the same direction as the driver and is uttering a voice, a third period that is shorter than the second period is set. According to this embodiment, a warning can be issued at an earlier timing when there is a high possibility that the driver's attention will be diverted to a direction other than the traveling direction.

So far, we have described the warning device 10 according to the present invention, but it may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated warning device 10 is a functional concept, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to that shown in the figure, and all or part of it may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.

The configuration of the warning device 10 is realized, for example, as software, such as a program loaded into memory. In the above embodiment, these functional blocks are described as being realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms, using only hardware, only software, or a combination of both.

The above-mentioned components include those that a person skilled in the art would easily imagine and those that are substantially the same. Furthermore, the above-mentioned configurations can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the configurations are possible without departing from the spirit of the present invention.

REFERENCE SIGNS LIST 10 Warning device 11 Imaging unit 12 Notification unit 20 Inattentive driving determination device 21 Imaging data acquisition unit 22 Detection unit 23 Determination unit 24 Warning control unit

Claims (5)

A detection unit that detects the face or gaze direction of a driver of a vehicle and a passenger other than the driver;
a determination unit that determines that the driver is looking aside when the driver faces a direction other than the traveling direction of the vehicle for a first period or more based on a detection result of the detection unit, and determines that the driver is looking aside when the driver faces a direction other than the traveling direction of the vehicle for a second period or more that is shorter than the first period when a passenger other than the driver faces the same direction as the driver;
a warning control unit that issues a warning to the driver when the determination unit determines that the driver is looking away;
An inattentiveness judgment device comprising: The detection unit further detects the seating positions of passengers other than the driver,
The determination unit determines that the driver is looking away from the vehicle when the passenger in the front seat of the vehicle faces the same direction as the driver and the passenger faces a direction other than the traveling direction of the vehicle for a second period or more that is shorter than the first period.
The inattentive driving detection device according to claim 1 . The determination unit sets the second period to be shorter as the number of passengers facing the same direction as the driver increases.
The inattentive driving detection device according to claim 1 or 2. The detection unit further detects voices uttered by a driver of the vehicle and a passenger other than the driver,
The determination unit sets the second period to be short when a voice is uttered while a passenger other than the driver is facing the same direction as the driver.
The inattentive driving detection device according to any one of claims 1 to 3. Detecting the face or gaze direction of a driver of a vehicle and a passenger other than the driver;
determining that the driver is looking aside when facing a direction other than the traveling direction of the vehicle for a first period or more, and, if a passenger other than the driver is facing the same direction as the driver, determining that the driver is looking aside when facing a direction other than the traveling direction of the vehicle for a second period or more that is shorter than the first period;
and when it is determined that the driver is looking away, issuing a warning to the driver.
An inattentive driving judgment method executed by an inattentive driving judgment device.

Images