JP6507015B2 - Driver status determination device - Google Patents

Description

  One aspect of the present invention relates to a driver state determination device.

  For example, the technology described in Patent Document 1 is known as a technology related to a conventional driver state determination device. In the technology described in Patent Document 1, a camera provided in front of the driver's seat captures an image of the driver's face position with a camera, and the driver's face does not exist or is blinded from the captured image. Is continuously detected for a predetermined period, it is determined that the driver's condition is abnormal.

JP 2007-331652 A

  In recent years, we have introduced a system that determines the driver's condition while riding in a vehicle and, if the driver's condition is abnormal, for example reduces the damage at the time of contact with an obstacle by automatically controlling the traveling of the vehicle. It is desired. In order to realize such a system, in the above-described driver state determination device, it is important to accurately determine whether the state of the driver is abnormal.

  One aspect of the present invention is made in view of the above situation, and it is an object of the present invention to provide a driver state determination device capable of accurately determining whether a driver's state is abnormal.

  A driver state determination device according to one aspect of the present invention recognizes a driving posture including a relative positional relationship of a plurality of skeletal parts in a driver of a vehicle based on a detection result by a detection unit including at least a distance sensor. A driving posture recognized by the recognition unit with respect to the recognition unit, a storage unit storing a plurality of normal driving postures, which are the driving postures of the driver in a normal driving state, and a plurality of normal driving postures stored in the storage unit And a determination unit that determines that the driver's condition is abnormal when the driving posture does not correspond to any of a plurality of normal driving postures.

  In this driver state determination device, the driving posture of the driver in the normal driving state is stored in the storage unit as the normal driving posture. If the recognition unit recognizes a driving posture that does not correspond to the normal driving posture, it is determined that the state of the driver is abnormal. As described above, it is possible to determine based on the normal driving posture related to the normal driving state of the driver instead of using only the predetermined threshold as the reference of determination, and therefore it is not influenced by the driver's body shape etc. It becomes possible to determine the driver's abnormality. Further, since the detection unit includes the distance sensor, for example, even when the interior of the vehicle is dark, it is possible to accurately detect the driving posture of the driver. Therefore, according to one aspect of the present invention, it is possible to accurately determine whether the driver's condition is abnormal.

  In the driver condition determination device according to one aspect of the present invention, the determination unit may store the driving posture recognized by the recognition unit in the storage unit as a normal driving posture, when it is determined that the driver's condition is not abnormal. According to this configuration, it is possible to make the storage unit learn the normal driving posture along with the determination by the determination unit.

  In the driver state determination device according to one aspect of the present invention, the driving posture is the size of the coordinates of the other skeletal portion with respect to the coordinates of one skeletal portion in each item consisting of a combination of two of the plurality of skeletal portions. The relative position information on the relationship may be included as a relative position relationship. According to this configuration, it is possible to reduce the data volume related to the driving posture and to increase the processing speed.

  In the driver state determination device according to one aspect of the present invention, the determination unit calculates, for each of the plurality of normal driving postures stored in the storage unit, the similarity similar to the driving posture recognized by the recognition unit. When the maximum value of the similarity in the normal driving posture is equal to or less than a preset threshold value, it may be determined that the driver's condition is abnormal. According to this configuration, matching of the driving posture with respect to the plurality of stored normal driving postures can be suitably realized.

  The driver condition determination device according to one aspect of the present invention includes a learning unit that stores the driving posture of the driver recognized by the recognition unit as a normal driving posture, except when determining by the determination unit. May be According to this configuration, it is possible to cause the storage unit to learn the driving posture of the driver at times other than the determination by the determination unit as the normal driving posture.

  According to one aspect of the present invention, it is possible to provide a driver state determination device capable of accurately determining whether the state of the driver is abnormal.

It is a block diagram showing composition of a driver state judging device concerning one embodiment. It is a figure explaining a plurality of skeletal parts. It is a figure which shows an example of the driving posture recognized by the recognition part. FIG. 4A is a flow chart for explaining the processing of the determination mode performed in the driver state determination ECU. FIG. 4B is a flow chart for explaining the processing of the learning mode executed by the driver state determination ECU. FIG. 5A is a graph showing the number of registered dictionary with respect to the threshold value. FIG. 5B is a graph showing the false alarm rate and the recall rate with respect to the threshold value. FIG. 6A is a graph showing the number of dictionary registrations with respect to the number of dictionary creation cycles. FIG. 6B is a graph showing the false alarm rate and the recall rate with respect to the number of cycles for creating a dictionary. FIG. 7A is a graph showing the recall rate and false positive rate by another person's dictionary. FIG. 7B is a graph showing the gesture recognition rate.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the same or corresponding elements are denoted by the same reference numerals and redundant description will be omitted.

  FIG. 1 is a block diagram showing a configuration of a driver state determination device 100 according to an embodiment. FIG. 2 is a diagram for explaining a plurality of skeletal parts 2 of the driver 1. As shown in FIG. 1, the driver state determination device 100 is a device that determines whether or not the state of the driver 1 riding in a vehicle such as a car is abnormal. The vehicle is not particularly limited, and may be, for example, any of commercial vehicles such as trucks and buses, ordinary passenger cars, large vehicles, medium vehicles, small vehicles, and light vehicles.

  The driver state determination device 100 includes an RGB-D sensor (detection unit) 10, a driver state determination ECU 20, and an output unit 30. The RGB-D sensor 10 is a sensor that acquires both a color image and depth information. The RGB-D sensor 10 is configured to include an RGB camera that acquires a color image of an object, and an infrared sensor that is a depth sensor that acquires depth information (distance image) to the object. The RGB-D sensor 10 is disposed between the window on the passenger seat side of the vehicle and the windshield. The RGB-D sensor 10 is capable of shooting in a direction from above the driver 1 toward the driver 1 side. Note that other distance sensors may be included instead of or in addition to the infrared sensor.

  As shown in FIG. 1 and FIG. 2, the RGB-D sensor 10 acquires a color image and depth information of the upper body of the driver 1, and based on the acquired color image and depth information, a plurality of skeleton parts 2 in the driver 1 Three-dimensional coordinates (X-axis coordinate, Y-axis coordinate and Z-axis coordinate) are acquired. The RGB-D sensor 10 outputs information on the three-dimensional coordinates to the driver state determination ECU 20 as sensor information. As the RGB-D sensor 10, for example, the obtainable range of depth is 0.5 to 8.0 [m], the detectable range of the driver 1 is 0.5 to 4.5 [m], and the skeletal portion One having a detectable number of 2 of 20 or 25 [locations / person] is used. The method for acquiring the three-dimensional coordinates of the plurality of skeletal parts 2 is not particularly limited, and a known method can be adopted.

  Skeletal site 2 is a structural site constituted by one or more bones and cartilage. The backbone site 2 is a site having one or more grouped functions and / or structures on the backbone. Skeletal site 2 is also referred to as a skeletal point. The skeletal part 2 of the present embodiment is a skeletal part of the upper body of the driver 1 and includes, for example, “head”, “left shoulder”, “left elbow”, “left arm”, “left hand”, “right shoulder”, The “right elbow”, the “right arm”, the “right hand”, the “spinal cord”, the “spine center”, the “hip” and the like can be mentioned. The upper body is the part of the body from the head to the waist.

  The driver state determination ECU 20 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The driver state determination ECU 20 loads a program stored in the ROM into the RAM and executes the program by the CPU to execute various controls. The driver state determination ECU 20 may be configured of a plurality of electronic control units. The driver state determination ECU 20 includes a recognition unit 21, a storage unit 22, a determination unit 23, and a learning unit 24.

  The recognition unit 21 recognizes, based on sensor information of the RGB-D sensor 10, a driving posture including relative three-dimensional positional relationships among the plurality of skeletal parts 2 of the driver 1. Specifically, first, a plurality of items consisting of two combinations of a plurality of skeletal parts 2 are extracted. For each of the plurality of extracted items, the magnitude relationship of the three-dimensional coordinates of the other skeletal part 2 with the coordinates of one skeletal part 2 is calculated. For example, the magnitude relationship is bit-patterned according to whether the coordinate of the other skeletal part 2 is larger, smaller or equal to the coordinate of one skeletal part 2 in each of the X axis, Y axis and Z axis. Be expressed. Then, the magnitude relationship is referred to as relative position information (also referred to as relative check point). A group of information having such relative position information for each of a plurality of items is recognized as a driving posture (also referred to as a posture model).

  FIG. 3 is a view showing an example of the driving posture recognized by the recognition unit 21. As shown in FIG. In the example shown in FIG. 3, the driving posture has, for example, the combination (item) of 66 skeletal parts 2 where the number of skeletal parts 2 is twelve. Each item has relative position information on the relative magnitude of the three-dimensional coordinates of the other skeletal part 2 with respect to one skeletal part 2 in the XYZ coordinate system. As illustrated, for example, when the item is "head-left shoulder", the X-axis coordinate of the head is "<" smaller than the X-axis coordinate of the left shoulder, and the Y-axis coordinate of the head is equal to the Y-axis coordinate of the left shoulder "= The Z-axis coordinate of the head is ">" which is larger than the Z-axis coordinate of the left shoulder. That is, one driving posture is described as a vector formed by comparing the relative magnitude relationship of coordinates with respect to combinations of a plurality of skeletal parts 2 in three axes.

  As shown in FIG. 1, the storage unit 22 stores and accumulates a plurality of normal driving postures. The normal driving posture is the driving posture of the driver 1 in the normal driving state. As described later, the storage unit 22 recognizes the driving posture when the determination unit 23 determines that the state of the driver 1 is normal and / or the learning unit 24 recognizes normal driving (during normal driving), as described later. The driving posture is stored as a normal driving posture. The plurality of normal driving postures stored in the storage unit 22 constructs a “dictionary”, that is, an aggregate that acquires a large number of normal driving postures. The plurality of normal driving postures stored in the storage unit 22 are used as teacher data. The plurality of normal driving postures are configured in the form of a list of relative position information. Hereinafter, the plurality of normal driving postures stored in the storage unit 22 will be simply referred to as “dictionary”.

  The determination unit 23 determines the next determination mode (detection mode) based on the plurality of normal driving postures stored in the storage unit 22 and the current driving posture recognized by the recognition unit 21 (hereinafter referred to as “current driving posture”). (Also referred to as That is, the determination unit 23 performs matching of the current driving posture with respect to a plurality of normal driving postures, and determines that the state of the driver 1 is abnormal when the current driving posture does not correspond to any of the plurality of normal driving postures. Do.

  Specifically, the determination unit 23 calculates, for each of the plurality of normal driving postures stored in the storage unit 22, a similarity degree similar to the current driving posture recognized by the recognition unit 21. If the maximum value among the similarities of the plurality of normal driving postures is equal to or less than the threshold value, it is determined that the driver's condition is abnormal. The threshold is a value set and stored in advance in order to appropriately determine whether the driver's condition is abnormal. The threshold can be determined based on experiments, past results and experiences, and simulations. The threshold may be a fixed value or a variable value.

  More specifically, for one normal driving posture, the determination unit 23 first collates the relative position information of the plurality of items with the relative position information of the plurality of items included in the current driving posture, and matches each other. The number of relative position information is calculated as the degree of similarity. At this time, it is assumed that the relative position information matches when all of the relative magnitudes of the X-axis coordinate, the magnitudes of the Y-axis coordinate, and the magnitudes of the Z-axis coordinate included in the relative position information are the same. Subsequently, such calculation of the degree of similarity is performed for all of the plurality of normal driving postures. Then, when the maximum value among the similarities of the plurality of normal driving postures is equal to or less than the threshold, it is determined that the plurality of normal driving postures do not correspond to the current driving posture, and the driver 1 is determined to be abnormal. On the other hand, if the maximum value among the similarities of the plurality of normal driving postures is larger than the threshold, it is determined that the normal driving posture having the maximum similarity is the current driving posture, and the driver 1's state is normal. Determine that there is.

  When the determination unit 23 determines that the state of the driver 1 is not abnormal, the determination unit 23 stores the current driving posture recognized by the recognition unit 21 in the storage unit 22 as a normal driving posture. Thus, the plurality of normal driving postures stored in the storage unit 22 are updated.

  The learning unit 24 executes a learning mode (also referred to as a dictionary registration mode or a dictionary reinforcement mode) in which the current driving posture recognized by the recognition unit 21 is learned as a normal driving posture, except at the time of determination by the determination unit 23. Specifically, the learning unit 24 stores and stores the current driving posture of the driver 1 recognized by the recognition unit 21 in the storage unit 22 as a normal driving posture, for example, during normal driving other than the mode determination execution time.

  The output unit 30 notifies the driver 1 of information according to the determination result by the determination unit 23. The output unit 30 has, for example, at least one of a display for displaying image information and a speaker for audio output. When the determination unit 23 determines that the driver's condition is abnormal, the output unit 30 displays, for example, a warning display on the display panel and / or outputs a warning voice such as a warning message from the speaker. On the other hand, when the determination unit 23 determines that the driver's condition is normal, the output unit 30 displays, for example, that the current driving posture corresponds to the normal driving posture on the display panel, and / or the current The speaker outputs a sound indicating that the driving posture corresponds to the normal driving posture.

  In the present embodiment, the driver state determination device 100 may be applied to a driving assistance device (not shown), and the traveling of the vehicle may be automatically controlled based on the determination result of the determination unit 23. For example, when the determination unit 23 determines that the state of the driver 1 is abnormal, the vehicle control ECU (not shown) may control the steering actuator and the brake actuator to move the vehicle to the road shoulder and stop. The driver state determination device 100 can be adopted as various driving support devices, driving support devices, driving safety devices, and the like.

  In the present embodiment, the execution of the determination mode by the determination unit 23 and the execution of the learning mode by the learning unit 24 may be switchable as desired according to the operation of the mode switching unit (not shown) by the driver 1, for example. . In the present embodiment, acquisition of three-dimensional coordinates of the skeletal part 2, conversion to a representation (driving posture) to be a relative positional relationship, and matching with a dictionary are performed with the number of processed frames per second as several tens of frames. Real-time processing is performed.

  Next, the processing of the driver state determination device 100 will be described with reference to the flowchart shown in FIG.

  FIG. 4A is a flow chart for explaining the processing of the determination mode executed by the driver state determination ECU 20. As shown in FIG. 4A, the driver state determination ECU 20 repeatedly executes the following processing relating to the determination mode in a predetermined cycle.

  First, sensor information is acquired from the RGB-D sensor 10 (S1). The recognition unit 21 recognizes the current driving posture of the driver 1 based on the sensor information of the RGB-D sensor 10 (S2). The determination unit 23 matches the current driving posture with respect to a plurality of normal driving postures. That is, the similarity of the current driving posture is calculated for each of the plurality of normal driving postures (S3).

  Subsequently, the determination unit 23 determines whether the calculated maximum value of the degree of similarity is larger than a threshold (S4). In the case of Yes in the above-mentioned S4, the determination unit 23 determines that the state of the driver 1 is normal, and stores the current driving posture as the normal driving posture in the storage unit 22 (S5 and S6). Thereafter, the output unit 30 outputs that the current driving posture corresponds to the normal driving posture, and shifts to processing of the next cycle.

  On the other hand, when the above-mentioned S3 is No, the judging section 23 judges that the state of the driver 1 is abnormal (S7). Thereafter, the output unit 30 outputs a warning voice and / or a warning display, and shifts to processing of the next cycle. Alternatively or additionally, the traveling of the vehicle is automatically controlled by the vehicle control ECU, for example, the vehicle is brought close to the road shoulder and stopped.

  FIG. 4B is a flow chart for explaining the processing of the learning mode executed by the driver state determination ECU 20. As shown in FIG. 4B, the driver state determination ECU 20 repeatedly executes the following processing relating to the determination mode at a predetermined cycle, for example, during normal operation other than the time of mode determination execution.

  First, sensor information is acquired from the RGB-D sensor 10 (S11). The recognition unit 21 recognizes the current driving posture of the driver 1 based on the sensor information of the RGB-D sensor 10 (S12). The learning unit 24 stores the current driving posture as the normal driving posture in the storage unit 22 (S13). Thereafter, the process proceeds to the next cycle of processing.

  As described above, in the driver state determination device 100, the driving posture of the driver 1 in the normal driving state is stored in the storage unit 22 as a normal driving posture (that is, as a dictionary). When the current driving posture not corresponding to the normal driving posture is recognized by the recognition unit 21, it is determined that the state of the driver 1 is abnormal. As described above, it is possible to determine based on the normal driving posture related to the normal driving state of the driver 1 instead of using only the predetermined threshold as the reference of determination, so Instead of this (robust to the physique of the driver 1), it is possible to determine an abnormality of the driver 1. Further, since the RGB-D sensor 10 including the infrared sensor is provided as the detection unit, the driving posture of the driver 1 can be detected with high accuracy even when the interior of the vehicle is dark, for example. Therefore, according to the driver state determination device 100, it can be accurately determined whether the state of the driver 1 is abnormal. Also, for example, even when the driver 1 switches driving to another driver 1, it is possible to create a dictionary for each driver 1 or share a dictionary to make a determination, and versatility It can be enhanced.

  In the driver state determination device 100, when it is determined that the state of the driver 1 is not abnormal, the determination unit 23 stores the current driving posture recognized by the recognition unit 21 in the storage unit 22 as a normal driving posture. According to this configuration, it becomes possible to make the storage unit 22 learn the normal driving posture along with the determination by the determination unit 23. As a result, it is possible to improve the determination accuracy (suppress the erroneous determination).

  In the driver state determination device 100, the driving posture is relative to the coordinate of one skeleton part 2 with respect to the coordinate of one skeleton part 2 with respect to the coordinate of one skeleton part 2 in each item consisting of two combinations of a plurality of skeleton parts 2. It consists of position information. According to this configuration, it is possible to reduce the data volume related to the driving posture and to increase the processing speed. For example, the data capacity of one driving posture can be suppressed to about several hundred bits (66 (number of items) × 3 (XYZ axes) × 2 (magnitude relationship) = 396 bits). Abnormality of the driver 1 can be determined with a small number of calculations. When the moving image information of the driver 1 is stored in the storage unit 22, a capacity of megabyte order or gigabyte order is required for the storage unit 22. Can be stored in the storage unit 22.

  Further, since the relative position information is expressed by the three-dimensional relative positional relationship among the plurality of skeletal parts 2, the posture of the driver 1 can be expressed simply and flexibly. Such relative position information can robustly describe subtle movements and postures such as tilt and twist of the body of the driver 1.

  In the driver state determination device 100, the determination unit 23 calculates, for each of the plurality of normal driving postures stored in the storage unit 22, a similarity similar to the driving posture recognized by the recognition unit 21. And when the maximum value of the degree of similarity in a plurality of normal driving postures is below a threshold, it judges with a state of driver 1 being abnormal. According to this configuration, matching of the driving posture with respect to a plurality of normal driving postures can be suitably realized.

  The driver state determination device 100 includes a learning unit 24 that executes a learning mode. In the learning mode, the driving posture of the driver 1 recognized by the recognition unit 21 is stored in the storage unit 22 during normal driving other than the determination mode execution time, for example. Thus, for example, the storage unit 22 can learn the driving posture of the driver 1 during normal driving as the normal driving posture. By repeatedly learning the normal driving posture and strengthening the dictionary, the number of normal driving postures stored in the storage unit 22 can be increased, and the erroneous determination rate can be reduced.

  The driving posture recognized by the driver state determination device 100 is clear driver information as compared to the driver's 1's habit or exhalation, etc., and is clear driver information even compared to the driver 1's moving image information is there. The driving posture recognized by the driver state determination device 100 is robust to the wearing of the sunglasses of the driver 1, the sunshine, and the like.

  The driver state determination device 100 adopts a white list format that determines that the state of the driver 1 is abnormal when the current driving attitude does not correspond to the normal driving attitude. In the white list format, although there is a possibility that the information stored in the storage unit 22 may be increased, the unsearched information is searched. Therefore, if the recognition of the current driving posture is accurate, the judgment is missed. Can be suppressed.

  In the driver state determination device 100, by making the threshold value large, it can be suppressed that the state of the driver 1 is erroneously determined to be abnormal. However, the threshold may be about 90% of the maximum value of the similarity (the total number of relative position information) in order to reduce the influence of noise.

  In the driver state determination device 100, the traveling of the vehicle can be controlled based on the determination as to whether or not the state of the driver 1 is abnormal. For example, by determining that the state of the driver 1 is abnormal, it is possible to grasp the occurrence of mental loss or improper operation due to poor health of the driver 1 and perform driving support that contributes to traveling safety. The driver state determination device 100 has a high degree of certainty as a trigger such as a safe stop.

  In the driver state determination device 100, a plurality of drivers 1 may share a dictionary, that is, a plurality of normal driving postures stored in the storage unit 22. Even with a dictionary that learns the posture and motion of another person different from the driver 1, it is possible to determine the abnormality of the driver 1 with sufficient accuracy.

  The driver state determination device 100 according to the present embodiment recognizes a gesture that is a driving posture at the time of a specific operation based on the current driving posture of the driver 1 and controls the operation of the in-vehicle device based on the recognized gesture. May be Specifically, a gesture is stored in the storage unit 22 for each specific operation. The gestures to be stored include, for example, a gesture at the time of air conditioner operation, a gesture at the time of operation of a car navigation system, a gesture at the time of audio operation, and the like. Subsequently, for each of the plurality of gestures stored in the storage unit 22, the similarity with the current driving posture recognized by the recognition unit 21 is calculated. A gesture whose similarity is greater than a threshold and has a maximum value is recognized as the current gesture. And operation control of in-vehicle apparatus corresponding to the recognized present gesture is performed. In this way, driving assistance can be performed according to the current gesture. In addition, the operation to be performed by the driver 1 can be recognized with higher accuracy than voice recognition.

[Example]
Hereinafter, the driver state determination device 100 will be described more specifically based on the measurement results.

(Evaluation test with changing threshold)
In this evaluation test, the reproduction rate and the false detection rate when the threshold value is changed in the driver state determination device 100 are investigated. The subjects were 5 males. The subject operates the vehicle in the learning mode (dictionary reinforcement mode) while performing six normal motions: straight motion, left turn motion, right turn motion, car navigation operation, air conditioner operation The motion and the free motion were performed in order. This was defined as one cycle of dictionary creation cycle, and the dictionary creation cycle was repeated five cycles to create a dictionary. Thereafter, the mode was switched to the determination mode (detection mode), and the above six operations in the normal operation were repeated five times. In this process, since the subject performs only the operation during normal driving, the determination result that the subject's state is abnormal is false detection.

Next, while driving the vehicle in the judgment mode, the test subject performs four incorrect operations that are not performed during normal operation, that is, operations that tilt the body forward, to the left, and to the right of the vehicle, and the waist falls from the seat I did something like Then, it was confirmed whether the subject's condition was determined to be abnormal. As the following formula, the number of false positives for all acquired frames was taken as false positive rate.
False detection rate = false detection number / total number of frames Further, as shown in the following equation, the number of recognition for the number of incorrect operations (the number of correctly determined as abnormal at the time of incorrect operations) was taken as the recall ratio.
Reproduction rate = recognition number / incorrect operation number

  FIG. 5A is a graph showing the number of registered dictionary with respect to the threshold value. The dictionary registration number is the number (the number of data) of the plurality of normal driving postures stored in the storage unit 22. As shown in FIG. 5A, it can be seen that when the threshold value is increased, the number of dictionary registrations increases and the data capacity of the dictionary increases.

  FIG. 5B is a graph showing the false alarm rate and the recall rate with respect to the threshold value. In the figure, the false alarm rate and the recall rate indicate the average value of a plurality of subjects. In the figure, each of the action A, the action B and the action C is an action of inclining the body forward, left and right of the vehicle, and the action D is an action such that the waist falls from the seat. The same applies to the following false detection rates, recall rates and operations A to D. As shown in FIG. 5B, as the threshold value is increased, the recall rate is increased and the false alarm rate is also slightly increased. When the threshold value is "33", the false alarm rate is 4% or less, and the recall rate can be increased while maintaining a low false alarm rate.

(Evaluation test of dictionary reinforcement)
In this evaluation test, the driver state determination device 100 increases the number of dictionary reinforcement attempts to increase the amount of data in the dictionary, and investigates the influence of the recall rate, false detection rate, and number of dictionary registrations. The subjects were 3 males. The threshold was set to “33” based on the results of the above evaluation test.

  FIG. 6A is a graph showing the number of dictionary registrations with respect to the number of dictionary creation cycles. The dictionary creation cycle here is the same as the dictionary creation cycle in the evaluation test. As shown in FIG. 6A, it can be seen that as the number of cycles for creating a dictionary is increased, the number of dictionary registrations increases and the amount of data in the dictionary increases. It can be seen that as the number of cycles for creating a dictionary is increased, the rate of increase in the number of registered dictionary is lower. Thus, it is considered that the increase in the number of dictionary registrations converges as the number of cycles for creating a dictionary is increased.

  FIG. 6B is a graph showing the false alarm rate and the recall rate with respect to the number of cycles for creating a dictionary. As shown in FIG. 6B, as the number of dictionary creation cycles is increased, the number of dictionary registrations increases, which indicates that the false detection rate decreases. When the number of dictionary cycles is "5", the false alarm rate is less than 2%. It can be seen that by repeatedly strengthening the dictionary, the number of dictionary registrations increases and the false detection rate can be reduced. By repeatedly strengthening the dictionary, it can be seen that the recall rate is kept high even if the number of dictionary registrations increases.

(An evaluation test with another person's dictionary)
In this evaluation test, the driver state determination device 100 investigates the false detection rate and the recall rate when using another person's dictionary. The subjects were 4 males. A dictionary of one subject was prepared in the same manner as the above evaluation test. Another three subjects repeated the above operations A to D three times as four incorrect operations under the execution of the determination mode using the created dictionary. Then, it was confirmed whether the condition of another three subjects was determined to be abnormal.

  FIG. 7A is a graph showing the recall rate and false positive rate by another person's dictionary. As shown in FIG. 7A, in the operation A, the operation B, and the operation C, it can be seen that the recall ratio is as high as 100%, and the false detection rate is also kept low. As a result, it can be understood that the dictionary can be shared and operated except for the characteristic operation of each individual.

(Gesture recognition evaluation test)
In this evaluation test, the driver state determination device 100 is used to evaluate whether the in-vehicle apparatus can be operated (operation control) by a gesture. The subject placed the right hand on the steering wheel while performing the learning mode, and performed a specific action in which the left hand was projected upward, forward, and front obliquely upward. Thereby, the gesture regarding the said specific operation | movement was memorize | stored in the memory | storage part 22, and the dictionary was created. After that, the subject performed the particular operation under the execution of the determination mode, and confirmed whether the particular operation was correctly recognized as a gesture. The rate at which the gesture could be correctly recognized was determined as the gesture recognition rate. The subjects were 3 males. The threshold was set to “33” based on the results of the evaluation test.

  FIG. 7B is a graph showing the gesture recognition rate. In the figure, "upper" is a gesture according to the above specific operation in which the left hand is projected upward, "front" is a gesture according to the above specific operation in which the left hand is projected forward, "diagonal" is projecting the left left diagonally upward It is a gesture concerning the above-mentioned specific operation. As shown in FIG. 7 (B), it can be seen that all gestures can be correctly recognized. Therefore, by allocating the gesture performed in the evaluation test this time to the operation gesture of the air conditioner, the car navigation system, or the like, it is possible to perform the driving support for operating the in-vehicle device by the gesture.

  The preferred embodiments of the present invention have been described above, but the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the scope of the invention as set forth in the claims. May be

  For example, although the RGB-D sensor 10 is used as a detection part in the said embodiment, you may use the other sensor containing a distance sensor. The point is that it is only necessary to be able to recognize the driving posture including the relative positional relationship between the plurality of skeletal parts 2 of the driver 1 based on the detection result of the detection unit.

  DESCRIPTION OF SYMBOLS 1 driver | operator 2 frame | skeleton part 10 RGB-D sensor (detection part) 21 recognition part 22 memory | storage part 23 determination part 24 learning part 100 driver state determination apparatus.

Claims (5)

A recognition unit that recognizes a driving posture including a relative positional relationship of a plurality of skeletal parts of a driver of the vehicle based on a detection result by at least a detection unit including a distance sensor;
A storage unit that stores a plurality of normal driving postures that are the driving postures of the driver in a normal driving state;
The matching of the driving posture recognized by the recognition unit is performed on the plurality of normal driving postures stored in the storage unit, and the driving posture does not correspond to any of the plurality of normal driving postures. A driver state determination device comprising: a determination unit that determines that a driver's state is abnormal.   The driver state determination device according to claim 1, wherein the determination unit stores the driving posture recognized by the recognition unit in the storage unit as the normal driving posture, when it is not determined that the state of the driver is abnormal. .   In the driving posture, relative position information on the relative magnitude of the coordinate of the other skeletal part with respect to the coordinate of one skeletal part with respect to the coordinate of one skeletal part in each item consisting of a combination of two of the plurality of skeletal parts; The driver state determination device according to claim 1 or 2, which is included as a relation. The determination unit is
The similarity degree similar to the driving posture recognized by the recognition unit is calculated for each of the plurality of normal driving postures stored in the storage unit,
The driving according to any one of claims 1 to 3, wherein when the maximum value of the similarity in a plurality of the normal driving postures is equal to or less than a preset threshold value, it is determined that the state of the driver is abnormal. Person status determination device.   The learning unit according to any one of claims 1 to 4, further comprising: a learning unit that stores the driving posture of the driver recognized by the recognition unit in the storage unit as the normal driving posture, except at the time of determination by the determination unit. The driver state determination device according to item 5.

Images