JP4123050B2 - Arousal level judgment device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a wakefulness level determination device.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there is known a wakefulness level determination device that determines a driver's wakefulness level based on a frequency distribution for each blinking time in a predetermined time zone, based on the time during which the closed eye state continues. ing. In this apparatus, the reliability of blinking is determined, the frequency weight is changed according to the reliability, and the frequency distribution in a predetermined time zone is obtained. In other words, a state in which the eyes are not actually closed, such as a state of looking down when looking at the meter, is not reflected so much in the frequency distribution, and an erroneous arousal level is not determined (see, for example, Patent Document 1).
[0003]
[Patent Document 1]
JP 2003-000571 A
[0004]
[Problems to be solved by the invention]
In the apparatus of the above-mentioned patent document 1, attention is paid to the fact that the duration time of the bind-off looking at the meter is longer than the duration time of the closed eye state due to normal blinking. When a closed eye state is detected for a longer time than a normal blink, the weighting coefficient of “0.2 to 0.8” is not used so much in the frequency distribution, and the blinded eye state is not determined as the closed eye state. I am doing so.
[0005]
However, in an actual dozing state, the driver may keep his eyes closed for a longer period of time than he is looking at the meter. For this reason, if the above-mentioned coefficient is used when the closed eye state continues for a longer time than a normal blink, there is a possibility that the determination accuracy of the arousal level is lowered.
[0006]
Thus, in the conventional arousal level determination apparatus, it is still desired to improve the determination accuracy.
[0007]
[Means for Solving the Problems]
According to the present invention, a wakefulness level determination device that processes a face image to determine a driver's wakefulness level, images a driver's face with an image capturing unit, and is obtained by the image capturing unit. The position detection means detects the position of the driver's eyes from the entire image, and the tracking means sets a predetermined area on the image including the driver's eyes on the basis of the eye position detected by the position detection means. The eye is tracked from within the predetermined area, the open / close eye determination means determines the open / closed eye state for the eye within the predetermined area tracked by the tracking means, and the closed eye ratio detection means is the open / close eye determination means. Based on the open / closed eye state of the eye determined by the above, the ratio of the appearance time of the closed eye state occupying during a predetermined time is detected, and the arousal level determination means determines the arousal level from the ratio detected by the closed eye ratio detection means. While judging, variation The degree detection unit detects the degree of variation in the eye coordinate position based on the eye position tracked by the tracking unit, and the arousal level correction unit detects the variation in the eye coordinate position detected by the variation degree detection unit. When the degree exceeds a predetermined threshold, the wakefulness level determined by the wakefulness level determining means is corrected to the wakefulness side.
[0008]
【The invention's effect】
According to the present invention, when the degree of variation in the coordinate position of the eye exceeds a predetermined threshold, the arousal level is corrected to the arousal side. For this reason, even when the arousal level is low and it is determined to be a doze state, when the degree of variation in the coordinate position exceeds a predetermined threshold, the arousal level is corrected to the arousal side to be accurate. be able to. Therefore, the determination accuracy can be improved.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the drawings.
[0010]
FIG. 1 is a configuration diagram of a wakefulness level determination device according to a first embodiment of the present invention. As shown in the figure, the arousal level determination device 1 of the present embodiment determines a driver's arousal level by processing an image including a driver's face, and An image photographing unit (image photographing means) 10 for photographing a face is provided. The image capturing unit 10 is, for example, a CCD camera for capturing visible light, and is installed below the front of the driver.
[0011]
The arousal level determination device 1 includes a processing device 20 that determines a driver's arousal level based on an image captured by the image capturing unit 10. Furthermore, the wakefulness level determination device 1 includes a wakefulness level display unit (wakefulness level display means) 30 that displays the wakefulness level detected by the processing device 20. The arousal level display unit 30 is provided in a position that is visible to the driver, for example, in the vicinity of the dashboard, and presents the arousal level detected by the processing device 20 to the driver. Note that the awakening level display unit 30 may be configured as an awakening level notifying unit that notifies the driver of the awakening level, for example, by voice.
[0012]
Next, a detailed configuration of the processing device 20 will be described with reference to FIG. FIG. 2 is a functional block diagram of the arousal level determination device 1 according to the first embodiment of the present invention. 2 also illustrates the image capturing unit 10 and the arousal level display unit 30 in order to clarify the connection relationship.
[0013]
The processing device 20 includes a position detection unit (position detection unit) 21 that detects the position of the driver's eyes based on image data from the image capturing unit 10 and a tracking unit that tracks the position of the driver's eyes. (Tracking means) 22.
[0014]
The position detection unit 21 detects the position of the eye from the entire image input from the image photographing unit 10. The tracking unit 22 determines a tracking region (predetermined region) including the eyes on the image based on the position of the driver's eyes detected by the position detection unit 21, and detects the position of the eyes from the image captured later. In this case, the driver's eyes are detected from the tracking area. Furthermore, the tracking unit 22 has a function of sequentially storing the coordinate values of the eye position detected by the position detection unit 21 and the eye itself.
[0015]
The processing device 20 includes an open / closed eye determination unit (open / closed eye determination unit) 23 that analyzes the eyes in the tracking region set by the tracking unit 22 and determines the open / closed eye state of the eyes. The open / close eye determination unit 23 determines the open / closed eye state by, for example, obtaining an eye opening corresponding to the vertical width of the driver's eye and comparing the eye opening with an opening threshold. The open / closed eye determination unit 23 is not limited to the configuration that determines the open / closed eye state based on the vertical width of the eye, and may determine the open / closed eye state by other methods.
[0016]
Further, the processing device 20 detects the ratio of the appearance time of the closed eye state in the predetermined time (hereinafter referred to as the “closed eye ratio”) based on the open / closed eye state of the eye determined by the open / closed eye determination unit 23. A detection unit (closed eye ratio detection means) 24 is provided. The closed eye ratio detection unit 24 has a function of integrating the closed eye time, thereby obtaining the integrated time. Then, the closed eye ratio is obtained based on the ratio of the accumulated time in the predetermined time.
[0017]
In addition, the processing device 20 includes a wakefulness level determination unit (wakefulness level determination unit) 25 that determines a wakefulness level from the closed eye rate detected by the closed eye rate detection unit 24. The arousal level determination unit 25 has, for example, a preliminarily stored arousal level determination table, and compares the detection result from the closed eye ratio detection unit 24 with the determination table to determine the arousal level.
[0018]
With the above configuration, the arousal level is determined based on the captured image. However, with only the above configuration, the arousal level that is the result of the determination may be inaccurate. Therefore, the wakefulness level determination device 1 according to the present embodiment further includes a variation degree detection unit (variation degree detection unit) 26 and a wakefulness level correction unit (wakefulness level correction unit) 27 for determination. The level of arousal level is made accurate.
[0019]
The variation degree detection unit 26 detects the degree of variation in the coordinate position of the eye based on the eye position detected by the tracking unit 22. For example, the variation degree detection unit 26 inputs the eye coordinate values sequentially stored by the tracking unit 22 and detects the variation degree from the set of input coordinate values.
[0020]
The wakefulness level correction unit 27 corrects the wakefulness level determined by the wakefulness level determination unit 25 to the wakefulness side when the degree of variation in the coordinate position of the eye exceeds a predetermined threshold. Usually, when the driver is in an awake state, the driver performs various operations such as entrainment confirmation. For this reason, when the coordinate positions of the eyes vary, that is, when the degree of variation exceeds a predetermined threshold, it can be said that the driver is performing various actions and is in an awake state. Therefore, when the degree of variation exceeds a predetermined threshold, the wakefulness level determined by the wakefulness level determination unit 25 is corrected to the wakefulness side.
[0021]
Next, an outline of the operation of the arousal level determination device 1 will be described. First, the image capturing unit 10 captures an area including a driver's face and sends the obtained image data to the processing device 20.
[0022]
The position detection unit 21 of the processing device 20 receives image data and detects the position of the driver's eyes. The eye position is detected as follows, for example. FIG. 3 is an explanatory diagram of an initial process performed when the position detection unit 21 detects an eye position. In FIG. 3, an example of an image having 480 pixels vertically and 512 pixels horizontally will be described.
[0023]
First, the position detection unit 21 acquires density value data for all pixels in the vertical direction of the image. Thereafter, the position detection unit 21 extracts pixels in which the change in density value satisfies a predetermined condition, and obtains a pixel group as shown in FIG. FIG. 4 is an explanatory diagram showing a state when the position detection unit 21 extracts a predetermined pixel. As shown in the figure, the extracted pixels correspond to the positions of the driver's eyebrows, eyes, nose and mouth. After the pixels are extracted, the position detection unit 21 groups the pixels that are close to each other in the horizontal direction of the image.
[0024]
FIG. 5 is an explanatory diagram showing a state when pixels adjacent in the horizontal direction of the image are grouped. As shown in the figure, by grouping, the position detection unit 21 forms continuous data G1 to G6 corresponding to the right eyebrow, left eyebrow, right eye, left eye, nose and mouth of the driver.
[0025]
Thereafter, the position detection unit 21 performs zoning processing. FIG. 6 is an explanatory diagram showing a state after zoning by the position detection unit 21. The position detection unit 21 zones the positions where the continuous data G1 to G6 exist in the vertical direction of the image. At this time, the position detection unit 21 forms three zones (ZONE: L, ZONE: C, and ZONE: R). Then, the position detection unit 21 detects the position of the eye by determining the relative positional relationship.
[0026]
After detecting the eye position, the tracking unit 22 stores the coordinate value of the detected eye position, and sets a tracking area smaller than the entire image with the stored position as a reference. Thereafter, the tracking unit 22 detects the position of the eye in the tracking region every time an image is input.
[0027]
7A and 7B are explanatory diagrams showing how the tracking unit 22 tracks the eye position. FIG. 7A shows the initial tracking area, and FIG. 7B shows the detected eye after setting the initial tracking area. (C) shows the newly set tracking area based on the detected eye position, and (d) shows the detected eye position from within the newly set tracking area. Yes. In FIG. 7, it is assumed that the eye indicated by the broken line has been detected last time, and the eye indicated by the solid line has been detected this time.
[0028]
When the position of the eye is detected by the position detection unit 21, the tracking unit 22 sets a tracking region around the detected eye position (FIG. 7A). The coordinate position of the eye at this time is (xk1, yk1). Thereafter, when an image is input, the tracking unit 22 detects the position of the eye from the tracking region centered at (xk1, yk1) (FIG. 7B). The detected coordinate position of the eye is (xk2, yk2).
[0029]
Then, the tracking unit 22 sets a tracking region centered on the newly detected eye coordinate position (xk2, yk2) (FIG. 7C). Thereafter, when an image is input again, the tracking unit 22 detects the position of the eye from the tracking region centered at (xk2, yk2). The coordinate position of the eye detected at this time is assumed to be (xk3, yk3).
[0030]
Then, the tracking unit 22 sets again a tracking region centered on the new eye coordinate position (xk3, yk3) (FIG. 7D). Thereafter, similarly, the tracking unit 22 detects the position of the eye from the tracking region.
[0031]
Then, the tracking unit 22 sends the eye position data to be tracked to the open / closed eye determination unit 23 as shown in FIG. For example, the opening / closing eye determination unit 23 obtains the opening degree of the eye according to the vertical width of the driver's eye, and determines the opening / closing eye state by comparing the opening degree of the eye with an opening degree threshold value.
[0032]
FIG. 8 is an explanatory view showing the opening of the vertical width of the eye, (a) showing when the eye is open, and (b) showing when the eye is closed. As shown in the figure, the opening degree is determined by the vertical width of the eye. In this case, since the size of the vertical width of the eye varies from person to person, the open / close eye determination unit 23 does not take the opening as an absolute value, but obtains a reference opening by photographing and calculates a relative value with respect to the reference opening. It is desirable to capture as
[0033]
That is, the open / closed eye determination unit 23 acquires the vertical width of the eye when opening the eye shown in FIG. 8A as a reference value. Thereafter, when the vertical width of the eye at the time of opening shown in FIG. 8B is detected, the open / close eye determination unit 23 acquires the eye opening in this state as a value with respect to the reference value. Then, the open / closed eye state is determined by comparing the value with respect to the reference value and the opening threshold.
[0034]
After the determination, the open / close eye determination unit 23 sends the open / closed eye state data to the closed eye ratio detection unit 24. Receiving this, when the eye state determined by the open / close eye determination unit 23 indicates the closed eye, the closed eye ratio detection unit 24 starts to accumulate the appearance time of the closed eye state. Then, integration is performed each time an image is input and an eye-closed state is detected by the eye-closing ratio detection unit 24, and the eye-closing ratio is obtained after a predetermined time has elapsed. After the determination, the closed eye rate detection unit 24 sends the closed eye rate data to the arousal level determination unit 25.
[0035]
Then, the arousal level determination unit 25 compares the detection result from the closed eye ratio detection unit 24 with the determination table to determine the arousal level.
[0036]
On the other hand, the variation degree detection unit 26 inputs the coordinate values sequentially stored by the tracking unit 22 and detects the variation degree from the set of input coordinate values. After the detection, the variation degree data is sent to the arousal level correction unit 27.
[0037]
Receiving this, the arousal level correction unit 27 determines whether or not the degree of variation in the coordinate position of the eye exceeds a predetermined threshold value. And when it is judged that it exceeds, the alertness level determined by the alertness level determination unit 25 is corrected to the alertness side. Here, the reason for correcting the arousal level to the awakening side when the variation is large is as described above.
[0038]
After the correction, the processing device 20 sends the information on the arousal level to the arousal level display unit 30, and the arousal level display unit 30 displays the arousal level to present the information to the driver.
[0039]
Next, the operation of the arousal level determination device 1 according to the present embodiment will be described in detail. First, the image capturing unit 10 of the arousal level determination device 1 captures an area including the driver's face and sends the obtained image data to the processing device 20.
[0040]
And the processing apparatus 20 performs the process according to FIG.9 and FIG.10. 9 and 10 are flowcharts showing detailed operations of the processing apparatus 20 shown in FIG. FIG. 9 shows the first half of the process, and FIG. 10 shows the second half of the process.
[0041]
As shown in the figure, first, the position detection unit 21 inputs image data (ST1). Thereafter, the position detection unit 21 determines whether or not an eye tracking region is set (ST2). If the eye tracking area is set (ST2: YES), the process proceeds to step ST5.
[0042]
On the other hand, when the eye tracking area is not set, such as immediately after activation of the present apparatus 1 (ST2: NO), the position detection unit 21 detects the eye position from the entire image (ST3). Here, as described with reference to FIGS. 3 to 6, the position of the eye is detected. Then, the position detection unit 21 sends the eye position data to the tracking unit 22.
[0043]
Thereafter, the tracking unit 22 sets a tracking region based on the eye position data from the position detection unit 21 (ST4). At this time, the tracking unit 22 sets the size of the tracking region as small as possible in consideration of the eye movement amount and the image capture interval. Thereby, the process which was excellent in the followable | trackability of eyes can be performed faster.
[0044]
Thereafter, the tracking unit 22 detects an eye from within the tracking region based on newly input image data (ST5). Thereafter, the tracking unit 22 determines whether or not eye tracking is performed correctly (ST6).
[0045]
When it is determined that the eye tracking is not performed correctly (ST6: NO), the tracking unit 22 clears the eye tracking area (ST7). Furthermore, when the timer mentioned later is operating, the processing apparatus 20 stops the timer (ST8). Thereafter, the process returns to step ST1. Then, the tracking area setting process is performed again.
[0046]
On the other hand, when it is determined that the eye tracking is correctly performed (ST6: YES), the open / close eye determination unit 23 determines the open / closed eye state as described with reference to FIG. 8 (ST9).
[0047]
Thereafter, when the eye state detected by the open / closed eye determination unit 23 indicates closed eyes, the closed eye ratio detection unit 24 starts to accumulate the appearance time of the closed eye state (ST10). In addition, when the eye state detected by the open / closed eye determination unit 23 indicates that the eye is open, the integration is not started.
[0048]
Further, while steps ST9 and ST10 are being performed, the tracking unit 22 stores the detected coordinate position of the eye (ST11). Then, the processing device 20 determines whether or not the timer is operating (ST12), and when determining that the timer is not operating (ST12: NO), starts the timer (ST13). Thereafter, the process proceeds to step ST14. On the other hand, if it is determined that the timer is operating (ST12: YES), the process proceeds to step ST14 without starting the timer.
[0049]
In step ST14, the tracking unit 22 updates the eye tracking area (ST14). At this time, for example, the tracking unit 22 sets and updates a tracking region centered on the detected eye position (ST14). And a process transfers to step ST15 of FIG.
[0050]
In step ST15, the processing device 20 determines whether or not the time measured by the timer has reached a predetermined time (ST15). If it is determined that the predetermined time has not been reached (ST15: NO), the process returns to step ST1 in FIG.
[0051]
On the other hand, when it is determined that the predetermined time has been reached (ST15: YES), the processing device 20 stops the timer and clears the measured time (ST16). Then, the closed eye ratio detection unit 24 obtains the closed eye ratio (ST17). At this time, the closed eye ratio detection unit 24 obtains the closed eye ratio based on the integration time at which integration is started in step ST10 shown in FIG. Thereafter, the arousal level determination unit 25 determines the arousal level based on the closed eye ratio (ST18).
[0052]
Thereafter, the variation degree detection unit 26 calculates the variation degree of the coordinate position of the eye during a predetermined time (ST19). At this time, the variation degree detection unit 26 inputs the coordinate value data of the eye position stored by the tracking unit 22 during a predetermined time, and obtains the variation degree based on the input coordinate value.
[0053]
And the arousal level correction | amendment part 27 compares the calculated variation degree with an opening degree threshold value. As a result of the comparison, when the variation degree exceeds a predetermined threshold value, the wakefulness level correction unit 27 corrects the wakefulness level determined by the wakefulness level determination unit 25 in step ST18 to the wakefulness side (ST20). On the other hand, if the degree of variation does not exceed a predetermined threshold as a result of the comparison, the wakefulness level correcting unit 27 does not correct the wakefulness level.
[0054]
Then, when the arousal level is corrected, the arousal level determination unit 25 gives the corrected arousal level to the arousal level display unit 30. On the other hand, when the arousal level is not corrected, the arousal level determination unit 25 provides the arousal level level determined in step ST18 to the arousal level display unit 30. Thereby, the arousal level display unit 30 presents information related to the current arousal level to the driver (ST21).
[0055]
Thereafter, the open / close eye determination unit 23 clears the accumulated time in the predetermined time accumulated so far (ST22). Further, the tracking unit 22 clears the coordinate position data of the eye during the predetermined time that has been stored so far (ST23). Then, the process returns to step ST1.
[0056]
The above processing is repeated until, for example, the apparatus 1 is turned off.
[0057]
Next, the degree of variation of the eye coordinate position and the correction by the wakefulness level correcting unit 27 will be described in more detail. Prior to this, a wakefulness index for evaluating the wakefulness level will be described.
[0058]
FIG. 11 is an explanatory diagram of a wakefulness index. The arousal level index indicates a driver's arousal level that is actually obtained based on the driver's facial expression, brain waves, or the like without using the device 1, and the arousal level obtained as a detection result of the device 1 Are different.
[0059]
The awakening degree index is indicated by a value of “3 to 9” so that a higher value is obtained when the person is awake. As shown in FIG. 11, the numerical values “3 to 9” are determined by three evaluation items such as the number of blinks, time, brain waves, and facial expression. Each of these items is divided into 3 to 5 levels, and a grade is assigned to each level.
[0060]
More specifically, the items of the electroencephalogram are divided into five stages, and scores of “1”, “1.5”, “2”, “2.5”, and “3” are assigned respectively. The state of the score “3” for the electroencephalogram means a state where no α2 wave appears. Moreover, the state of the score “2.5” means a state where a small amplitude α2 wave appears (a state where it appears about half of the time when the eyes are closed), and the state of the score “2” means that many small amplitude α2 waves appear. The state to do. Moreover, the state of the score “1.5” means a state in which a large amplitude α2 wave appears (a state that appears in the same degree as when the eyes are closed), and the state of the score “1” means that a large amplitude α2 wave continues. A state that appears.
[0061]
Similarly, the item of blinking is also divided into five stages, and “1”, “1.5”, “2”, “2.5”, and “3” are assigned respectively. Specifically, the state of the score “3” for blinking means a state in which a sharp blink continues quickly. Moreover, the state of the score “2.5” refers to a state where the number of blinks increases compared to the case of the score “3”, and the state of “2” refers to a state where a late blink appears. Moreover, the state of the score “1.5” means a state where a short closed eye (longer than blinking) appears, and the state of the score “1” means a state where a long closed eye appears.
[0062]
Furthermore, the facial expression items are divided into three levels, and “1”, “2”, and “3” are assigned respectively. Specifically, the state of the score “3” for the facial expression means a state in which the facial muscles are tightened and a clean expression is maintained. Moreover, the state of the score “2” means a state where the upper eyelid is lowered and has a slightly blurred expression, and the state of the score “1” means a thin and dull state.
[0063]
When these items are evaluated, for example, when there are many α2 waves with small amplitude for brain waves, slow blinks for blinks, and upper facial expressions for facial expressions and faint expressions, “2” “2” “2”. Therefore, the wakefulness index in this case is “6”.
[0064]
Here, the case where the wakefulness level is accurately determined in the present apparatus 1 is a case where the wakefulness level and the wakefulness index match. This is because the wakefulness index is detected without relying on image processing and is extremely accurate. Therefore, when the wakefulness index and the wakefulness level match, it can be said that an accurate determination is made. Further, the arousal level indicates a lower value as the proportion of eyes closed during a predetermined time is higher. Therefore, it can be said that the arousal level is accurately determined when the relationship between the value of the arousal level index and the closed eye integration time is matched.
[0065]
12 and 13 are explanatory diagrams showing the arousal index obtained by actually measuring the driver, the closed eye integration time detected by the device 1, and the variation in the coordinate position of the eyes. FIG. 12 shows an example in which the arousal level is determined satisfactorily, and FIG. 13 shows an example in which the arousal level is determined with some errors. In the description of FIG. 12 and FIG. 13, the state of the wakefulness index “3” is the dozing state, the state of the wakefulness index “6” is the light dozing state, and the state of the wakefulness index “9” is the wakefulness state. To do.
[0066]
First, FIG. 12 will be described. Here, for convenience of explanation, the first half of the elapsed time will be described. In FIG. 12, it can be seen that the numerical value of the wakefulness index 101 corresponds to the closed eye integration time 102. That is, the part where the closed eye integration time 102 exceeds the threshold Th1 and the part where the wakefulness index shows a low value (“6”) generally coincide.
[0067]
As described above, the state in which the relationship between the value of the wakefulness index and the closed eye integration time is a state in which the determination of the wakefulness level by the device 1 is accurately performed. In the example of FIG. 12, the above requirement is satisfied, and the arousal level is accurately determined.
[0068]
In FIG. 13, the threshold value Th2 is drawn at the same graph position as the threshold value Th1 shown in FIG. In the example illustrated in FIG. 13, the actually measured wakefulness index 201 indicates the wakefulness state although the closed eye integration time 202 exceeds the threshold Th2 and should indicate a light dozing state. That is, in the example of FIG. 13, it can be seen that there is some deviation. In the present embodiment, correction is performed in the case shown in FIG. 13 so that the arousal level is accurately determined.
[0069]
Reference is again made to FIG. In FIG. 12, the standard deviation 103 indicating variation in the X-axis direction and the standard deviation 104 indicating variation in the Y-axis direction are small values. As described above, when the coordinate position of the driver's eyes does not vary, the driver is likely to be dozing. That is, it can be said that there is no movement of the driver due to dozing and the coordinate position of the eyes does not vary.
[0070]
On the other hand, in FIG. 13, the standard deviation 203 indicating the variation in the X-axis direction and the standard deviation 204 indicating the variation in the Y-axis direction are larger than those in FIG. As described above, when the coordinate position of the driver's eyes varies, the driver is likely to be in an awake state. That is, it can be said that the coordinate position of the eye varies due to the driver's action.
[0071]
In the present embodiment, attention is paid to the variation in the coordinate position, and when the variation exceeds a predetermined threshold value, the arousal level is corrected to the awakening side. The state of the correction is shown in FIGS.
[0072]
14 and 15 are explanatory diagrams showing the state before and after the correction of the arousal level obtained on the basis of the closed eye integration time shown in FIG. FIG. 15 shows an example in which the data shown in FIG. 14 is graphed. In the description of FIGS. 14 and 15, the accumulated time is represented by the number of image frames. For this reason, the actual integration time is a value obtained by multiplying the processing speed of one frame.
[0073]
First, as shown in FIG. 14, in each of the arousal level determination sections “1 to 40”, the respective closed eye integration times are obtained. At this time, if the closed eye accumulated time is 30 frames or more and the arousal level is “2” indicating a light doze state, and the closed eye accumulated time is 60 frames or more is awakeness level “1” indicating a doze state, As shown in FIG. 14, the arousal level is determined.
[0074]
In the conventional wakefulness level determination device, detection of doze is performed based on this wakefulness level, but in this embodiment, correction is performed based on the degree of variation in the coordinate position of the eyes. Here, assuming that the predetermined threshold value in the horizontal direction and the vertical direction of the image is “20.0”, the degree of variation exceeding the threshold value is detected in a plurality of arousal level determination sections.
[0075]
When the degree of variation exceeding this threshold is detected, correction to the wakefulness side is performed for those whose wakefulness level obtained by the closed eye integration time is “1” or “2”. That is, in each of the arousal level determination interval “3, 8, 12, 14, 16, 24 to 26, 28, 29”, the arousal level is changed from “2” to “3”.
[0076]
FIG. 15 shows the graph. In the figure, the arousal level index “3” corresponds to the arousal level “1”. Also, the arousal level index “9” corresponds to the arousal level “3”, and the arousal level index “6” corresponds to the arousal level “2”.
[0077]
As shown in the figure, it can be seen that the rate of coincidence with the wakefulness index is higher after the correction than before the correction. In particular, it can be seen that the coincidence rate is significantly improved in the first half of the elapsed time. As described above, it is clear that the determination accuracy of the arousal level is improved by the correction based on the degree of variation in the coordinate position of the eye.
[0078]
Further, in the above example, the variation degree detection unit 26 is configured to obtain the standard deviation of the eye coordinate position, but is not limited to the standard deviation, and the eye relative position relationship with respect to the reference position may be obtained.
[0079]
Hereinafter, an example in which the relative positional relationship is set as the degree of variation will be described. FIG. 16 is an explanatory diagram showing the reference position of the eye and the distribution of the coordinate position of the eye in the case shown in FIG. FIG. 17 is an explanatory diagram showing the reference position of the eye and the distribution of the coordinate position of the eye in the case shown in FIG.
[0080]
First, the variation degree detection unit 26 obtains a reference position. This reference position is determined, for example, by obtaining a highly-established eye position where the driver is gazing forward by pattern matching or the like. Reference positions are shown in FIGS. 16 and 17, respectively.
[0081]
After obtaining the reference position, the variation degree detection unit 26 obtains an average coordinate value. The average coordinate value is obtained by averaging the coordinate positions of the eyes shown in FIG. 16 and FIG.
[0082]
Then, the variation degree detection unit 26 sets the relative positional relationship between the average coordinate value and the reference position, for example, the distance or the like as the variation degree.
[0083]
Here, comparing the distribution of eye coordinate positions shown in FIGS. 16 and 17, it can be seen that FIG. 16 is more concentrated on the reference position. For this reason, it can be said that the example shown in FIG. 17 has a larger degree of variation than the example shown in FIG. And when the variation degree obtained here exceeds a predetermined threshold value, it correct | amends similarly to what was shown in FIGS. Thus, even when the degree of variation is the relative positional relationship of the eyes, it is possible to improve the determination accuracy of the wakefulness level as in the case of using the standard deviation.
[0084]
In the second half of the elapsed time in FIG. 12, since there is a large variation in the coordinate position of the eye in the X-axis direction, that portion is also reflected in FIG. However, such a large variation is, for example, when the side mirror is visually recognized. Except for the latter half, it can be seen that the coordinate position of the eye is concentrated at the reference position as shown in FIG.
[0085]
In this way, according to the arousal level determination device 1 according to the present embodiment, the arousal level is corrected to the arousal side when the degree of variation in the coordinate position of the eye exceeds a predetermined threshold.
[0086]
Usually, when the driver is in an awake state, the driver performs various operations such as entrainment confirmation. For this reason, when the coordinate positions of the eyes vary, that is, when the degree of variation exceeds a predetermined threshold, it can be said that the driver is performing various actions and is in an awake state. Therefore, when the degree of variation exceeds a predetermined threshold, the wakefulness level determined by the wakefulness level determining means is corrected to the wakefulness side. Thereby, even if it is determined that the arousal level is low and the state is a dozing state, the awakening level can be corrected to the awakening side to be accurate.
[0087]
Therefore, the determination accuracy can be improved.
[0088]
The corrected arousal level is displayed by the arousal level display unit 30. For this reason, the driver can clearly indicate the arousal level and can take a rest or the like based on the driver's own judgment.
[0089]
Here, according to the applicant's research, if the driver's arousal level is significantly reduced, the driver will be drowsed even if he takes some countermeasures, such as listening to music. The result is obtained. However, some conventional determination apparatuses are configured such that an alarm or the like is issued only when the closed eye appears for a fairly long time. In this case, information is performed when the arousal level is significantly lowered, and the original information function is impaired.
[0090]
However, in this embodiment, since the determination accuracy can be improved, there is no need to provide information or the like under severe conditions, which can contribute to effective sleepiness elimination.
[0091]
Furthermore, in this embodiment, as shown in FIG. 15, the determination accuracy between the awake state and the light doze state is remarkably improved. That is, the state before the wakefulness level is significantly reduced can be determined with high accuracy, and information can be effectively provided before the wakefulness level is significantly lowered. Therefore, a further drowsiness relieving effect can be brought about.
[0092]
Next, a second embodiment of the present invention will be described. The arousal level determination device 2 according to the second embodiment is the same as that of the first embodiment, but the configuration of the open / close eye determination unit 23 is different from that of the first embodiment. Moreover, the arousal level determination device 2 according to the second embodiment does not include the variation degree detection unit 26 of the first embodiment.
[0093]
Hereinafter, differences from the first embodiment will be described. FIG. 18 is a functional block diagram of the arousal level determination device 2 according to the second embodiment. As shown in the figure, the open / close eye determination unit 23 of the second embodiment includes a longitudinal open / close eye determination unit 23a, a shape open / close eye determination unit 23b, and an interval open / close eye determination unit 23c.
[0094]
The vertical open / closed eye determination unit 23a determines the open / closed eye state based on the vertical width of the eyes existing in the tracking area. The shape open / close eye determination unit 23b determines the open / close eye state based on the shape of the upper eyelid in the tracking region. The interval opening / closing eye determination unit 23c determines the opening / closing eye state based on the peak point of the upper eyelid existing in the tracking region and the interval between the eyebrows above the eye. Next, these will be described in more detail with reference to FIG.
[0095]
FIG. 19 is an explanatory diagram illustrating a determination method by the vertical opening / closing eye determination unit 23a, the shape opening / closing eye determination unit 23b, and the interval opening / closing eye determination unit 23c. The vertical opening / closing eye determination unit 23a obtains a reference opening based on the vertical width of the eye, and then obtains a relative value with respect to the reference opening and compares the relative value with a threshold value to determine the open / closed eye state. To do. Specifically, the open / closed eye state is determined in the same manner as in the description of the first embodiment (description of FIG. 8).
[0096]
In addition, the shape opening / closing eye determination unit 23b generates data indicating the upper eyelid line by performing edge detection processing within the tracking region. Then, the open / closed eye state is determined by both the three-point method and the radius-of-curvature method.
[0097]
Here, as shown in FIG. 19, the three-point method uses the upper end point 50 as the left end point of the data indicating the upper eyelid line, the right end point as the eye corner point 51, and the upper eyelid peak point 52 with respect to a straight line connecting both end points. This is a method for obtaining the length of the perpendicular line. Thereby, the shape open / close eye determination unit 23b obtains the perpendicular length. Then, the shape open / close eye determination unit 23b compares the obtained perpendicular length with a preset opening degree threshold to determine the closed eye closed state.
[0098]
In addition, the curvature radius method is a method of calculating the curvature radius of the upper eyelid by calculating an approximate expression of a quadratic curve for each edge extraction point 53 of the upper eyelid as shown in FIG. Then, the shape opening / closing eye determination unit 23b compares the calculated curvature radius with a preset opening degree threshold value, and determines that the eye is in a closed / closed eye state.
[0099]
Note that the shape opening / closing eye determination unit 23b is not limited to the configuration in which the opening threshold value is set in advance, and may calculate the opening threshold value from the captured image in order to take into account individual differences.
[0100]
The interval open / close eye determination unit 23c binarizes the tracking region portion in the image to cut out the shape of the eye, and obtains the peak point of the upper eyelid from the shape. The interval open / close eye determination unit 23c then obtains, as the representative coordinate point, the data corresponding to the eyebrows, the point having the same coordinate value in the X-axis direction as the obtained upper eyelid peak point. After that, the interval opening / closing eye determination unit 23c subtracts the Y-axis coordinate of the representative eyebrow coordinate point from the Y-axis coordinate of the upper eyelid peak point to calculate the upper eyelid peak point and the eyebrow vertical distance.
[0101]
Then, the interval opening / closing eye determination unit 23c compares the obtained distance between the upper eyelid and the eyebrows with a preset opening degree threshold value to determine the closed / closed eye state. The interval opening / closing eye determination unit 23c is not limited to the configuration in which the opening threshold value is set in advance, similarly to the shape opening / closing eye determination unit 23b, and the opening threshold value may be calculated from a captured image. .
[0102]
In this way, they perform open / closed eye determinations that do not interfere with each other. In other words, even if any one of the open / closed eye determinations is erroneously detected, the other open / closed eye determinations are affected by the erroneous detection, and both are not erroneously detected.
[0103]
The open / close eye determination unit 23 includes a general open / close eye determination unit 23d. The overall open / close eye determination unit 23d finally determines the open / close eye state based on at least one of the three detection results of the open / close eye determination units 23a to 23c. Here, which of the three detection results is used for the determination by the overall opening / closing eye determination unit 23d is determined depending on whether each of the three detection results is usable.
[0104]
Specifically, the integrated open / close eye determination unit 23d determines which one to use based on the result of self-diagnosis by each of the three open / close eye determination units 23a to 23c. Next, self-diagnosis performed by each of the three open / close eye determination units 23a to 23c will be described.
[0105]
First, the vertical opening / closing eye determination unit 23a performs self-diagnosis based on the vertical width data of a plurality of eyes obtained in time series. Specifically, the self-diagnosis method is performed by comparing the average value and standard deviation of the vertical width data of a plurality of eyes with a predetermined condition.
[0106]
More specifically, as long as the target is an eye, the range in which the vertical width changes depending on the opening and closing eyes and the operation pattern when opening and closing the eyes are limited. Therefore, the vertical width open / closed eye determination unit 23a obtains a vertical width change amount and an operation pattern from the vertical width data of a plurality of eyes, determines whether or not this matches the open / closed eye change, and the detection result is Determine whether it is available.
[0107]
In addition, the shape open / close eye determination unit 23b performs self-diagnosis based on the lateral width data of the eye that is image-recognized in the three-point method, and approximates a quadratic curve for each edge extraction point 51 of the upper eyelid in the curvature radius method. Based on the self-diagnosis.
[0108]
First, the three-point method will be described. The shape opening / closing eye determination unit 23b detects the distance between both the points of the eyes and the corners of the eyes. These can be detected from the difference between the left end point of the data indicating the upper eyelid line obtained when calculating the length of the perpendicular and the upper end point 50 and the right end point of the eye end point 51. Next, the obtained value is compared with the distance between both points of the eye and the corner of the eye when the eye is correctly detected as the image feature amount, and it is determined whether or not the detection result is usable. .
[0109]
Specifically, the shape opening / closing eye determination unit 23b obtains an average value and a standard deviation for the eye top point 50 and the eye corner point 51 obtained in time series, and compares them with values when the eye is correctly detected. Then, it is determined whether or not the detection result is usable.
[0110]
As a result, even if a part of the eye is exposed to direct sunlight from the sun and the distance between both the eyes and the corners of the eye is detected small, the use of the open / closed eye determination result by the shape change of the upper eyelid Can be limited.
[0111]
Further, in the curvature radius method, the shape opening / closing eye determination unit 23b determines that the edge line composed of the edge extraction points 53 does not accurately follow the upper eyelid line when the driver is a double eyelid due to individual differences in the upper eyelid. There is a case. Therefore, the shape opening / closing eye determination unit 23b obtains a correlation between the approximate curve for obtaining the radius of curvature and the actual line on the image connecting a plurality of extraction points 53 on the image. Then, based on the obtained correlation value, it is determined whether or not the detection result is usable.
[0112]
Specifically, the shape opening / closing eye determination unit 23b obtains an average value and a standard deviation of correlation values obtained from eye data obtained in time series, and compares these with a predetermined threshold value so that a detection result can be used. Determine whether it is a thing. For example, when the average value is small or the standard deviation is large, it is determined that the detection result cannot be used.
[0113]
The interval open / close eye determination unit 23c performs self-diagnosis based on the appearance rate of the eyebrow data, the stability of the appearance position of the eyebrow data, and the interval data between the plurality of upper eyebrows and the eyebrows obtained in time series. I do.
[0114]
For example, it is difficult to detect eyebrow data for people with originally thin eyebrows or people with long bangs and hidden eyebrows. In spite of such a difficult situation, if the open / close eye is determined, the detection result may vary. Accordingly, these appearance rates are obtained, and it is determined that the detection result cannot be used for those that do not satisfy the condition of the appearance rate.
[0115]
In addition, when the driver frequently moves his / her face, it is difficult to detect eyebrow data, and the detection result may be inaccurate if the open / close eye is determined based on the data in such a case. There is sex. Therefore, the interval opening / closing eye determination unit 23c determines the stability of the appearance position, that is, whether the positions of a predetermined number of eyebrows are stable. Specifically, the interval open / close eye determination unit 23c calculates an average value and a standard deviation of the appearance position of the eyebrows, compares them with a predetermined reference value, and determines whether or not the detection result is usable. .
[0116]
Furthermore, there is a possibility that the face part that is currently determined to be an eye is not an eye. Therefore, since the amount of change in the upper eyelid, eyebrow, and vertical width accompanying opening and closing of the eyes is limited to some extent, it is based on the interval data between a plurality of upper eyelids and eyebrows obtained in time series. The amount of change is obtained, and it is determined whether or not a predetermined condition is satisfied. Specifically, the interval data between the upper eyebrows and the eyebrows obtained in time series is sequentially stored, and whether the average value and the standard deviation of the intervals between the upper eyebrows and the eyebrows satisfy a predetermined condition. And whether or not the detection result is usable is determined.
[0117]
After determining whether or not it can be used as described above, the overall open / close eye determination unit 23d determines the current open / closed eye state based on the available eye.
[0118]
Moreover, since it is comprised as mentioned above, the arousal level correction | amendment part 27 changes arousal level as the number of the detection results used when the integrated open / close eye determination part 23d determined the open / closed eye state decreases. It is configured to correct to the awakening side. Further, the wakefulness level correction unit 27 corrects the wakefulness level so as to reduce, for example, the closed eye integration time when correcting the wakefulness level to the wakefulness side. For example, X% (X is 100 or less) is multiplied by the closed eye integration time.
[0119]
Here, this X% is determined according to the number of detection results used when the overall open / close eye determination unit 23d determines the open / closed eye state. For example, when the open / closed eye state is determined based on one detection result such as the vertical width open / closed eye determination unit 23a alone, the numerical value X is “80”. When the open / closed eye state is determined based on the two detection results, the numerical value X is “90”, and when the open / closed eye state is determined based on the three detection results, the numerical value X is “100”. . Furthermore, when it is not based on any detection result, the numerical value X is not set, and the wakefulness level correcting unit 27 forcibly corrects the wakefulness level to the one-stage wakefulness side.
[0120]
Thus, the arousal level correction unit 27 is configured to correct the arousal level according to the number of detection results used when the overall open / close eye determination unit 23d determines the open / closed eye state.
[0121]
Next, the operation of the processing apparatus 20 having the above configuration will be described. 20 to 23 are flowcharts showing detailed operations of the processing apparatus 20 shown in FIG.
[0122]
First, the processes shown in steps ST31 to ST38 shown in FIG. 18 are the same as steps ST1 to ST8 shown in FIG. In step ST39, the tracking unit 22 stores the detected coordinate position of the eye (ST39). Thereafter, the vertical width open / close eye determination unit 23a calculates the vertical width value of the data detected as the eye (ST40).
[0123]
Then, the vertical opening / closing eye determination unit 23a stores the obtained vertical width value and holds a plurality of data in time series (ST41). Thereafter, the vertical width open / close eye determination unit 23a performs open / close eye determination based on the vertical width of the eye (FIG. 21: ST42). Here, as described with reference to FIG. 19, for example, the eye width is compared with a threshold value to determine whether the eye is open or closed.
[0124]
After the open / closed eye determination, the vertical width open / closed eye determination unit 23a determines whether or not the vertical width value of the eye satisfies a predetermined condition (ST43). Specifically, the vertical width opening / closing eye determination unit 23a obtains a vertical width change amount or the like from a plurality of vertical width value data within a predetermined time range, and determines whether this matches the eye change amount or the like. .
[0125]
When it is determined that the eye vertical width value does not satisfy the predetermined condition (ST43: NO), the open / closed eye determination result based on the eye vertical width value is determined to be unused in the subsequent processing (ST44). That is, it is determined that the result of the open / closed eye determination by the vertical width open / closed eye determination unit 23a is not usable. Then, the process proceeds to step ST46.
[0126]
On the other hand, when it is determined that the eye vertical width value satisfies the predetermined condition (ST43: YES), it is determined that the open / closed eye determination result based on the eye vertical width value is used in the subsequent processing (ST45). That is, it is determined that the result of the open / closed eye determination by the vertical width open / closed eye determination unit 23a can be used.
[0127]
Thereafter, the open / close eye determination unit 23c performs open / close eye determination based on the distance between the peak point of the upper eyelid and the eyebrows (ST46). Here, as described above, the obtained interval is compared with a preset opening degree threshold value to determine the closed eye closed state.
[0128]
The interval open / close eye determination unit 23c determines whether the appearance rate of the eyebrow data satisfies a predetermined condition (ST47). Here, as described above, when the driver is originally a person with a thin eyebrow or the like, the detection result is prevented from varying.
[0129]
When it is determined that the appearance rate of the eyebrow data does not satisfy the predetermined condition (ST47: NO), it is determined that the open / closed eye determination result based on the distance between the peak point of the upper eyelid and the eyebrows is not used in later processing. (ST48). That is, it is determined that the result of the open / close eye determination by the interval open / close eye determination unit 23c is not usable. And a process transfers to step ST54 of FIG.
[0130]
On the other hand, when it is determined that the appearance rate of the eyebrow data satisfies a predetermined condition (ST47: YES), the interval open / close eye determination unit 23c stores the appearance position of the eyebrows and stores a plurality of data in time series. Hold (ST49).
[0131]
Thereafter, the interval opening / closing eye determination unit 23c determines whether or not the average value and the standard deviation of the appearance positions of a predetermined number of eyebrows satisfy a predetermined condition (ST50). Here, as described above, when the driver frequently moves the face, the detection result is prevented from being inaccurate.
[0132]
If it is determined that the predetermined condition is not satisfied (ST50: NO), the process proceeds to step ST48. On the other hand, when it is determined that the predetermined condition is satisfied (ST50: YES), the interval opening / closing eye determination unit 23c stores the interval value between the upper eyelid and the eyebrows and holds a plurality of data in time series ( ST51).
[0133]
Thereafter, the interval opening / closing eye determination unit 23c determines whether the average value and the standard deviation of the predetermined number of interval values satisfy the predetermined condition (ST52). As described above, this process is performed based on the idea that the amount of change in the upper eyelid, the eyebrows, and the vertical width associated with the opening and closing of the eyes is limited to some extent.
[0134]
If it is determined that the predetermined condition is not satisfied (ST52: NO), the process proceeds to step ST48. On the other hand, when it is determined that the predetermined condition is satisfied (ST52: YES), the interval opening / closing eye determination unit 23c uses the opening / closing eye determination result based on the peak point of the upper eyelid and the interval between the eyelids in a later process. Determine (ST53). That is, it is determined that the result of the open / close eye determination by the interval open / close eye determination unit 23c can be used.
[0135]
Then, the shape open / close eye determination unit 23b performs open / close eye determination based on the shape change of the upper eyelid (FIG. 22: ST54). First, the open / closed eye determination by the three-point method is performed here.
[0136]
After the open / close eye determination, the shape open / close eye determination unit 23b stores a plurality of data in a time series by storing the distance between the points of the eyes and the corners of the eyes (ST55). Then, the shape opening / closing eye determination unit 23b determines whether or not the average value and standard deviation of the predetermined number of distances between the two points satisfy the predetermined condition (ST56). Here, the predetermined condition is a distance between both the points of the eyes and the corners of the eyes when the eye is correctly detected as the image feature amount described above.
[0137]
When it is determined that the predetermined condition is not satisfied (ST56: NO), it is determined that the open / closed eye determination result due to the shape change of the upper eyelid is not used in the subsequent processing (ST57). In other words, it is determined that the result of the open / close eye determination by the shape open / close eye determination unit 23b is not usable. And a process transfers to step ST64.
[0138]
On the other hand, when it is determined that the predetermined condition is satisfied (ST56: YES), it is determined that the result of the open / closed eye determination based on the shape change of the upper eyelid (limited to the three-point method) is used in later processing. (ST58). That is, the open / closed eye determination result (three-point method) by the vertical width open / closed eye determination unit 23a is determined to be usable.
[0139]
Thereafter, the shape open / close eye determination unit 23b performs open / close eye determination based on the shape change of the upper eyelid (ST59). Here, open / close eye determination is performed by the curvature radius method. After the open / close eye determination, the shape open / close eye determination unit 23b stores the correlation value between the approximate curve for obtaining the radius of curvature and the actual line on the image connecting the plurality of extraction points 53 on the image. A plurality of data are held in series (ST60).
[0140]
Then, the shape opening / closing eye determination unit 23b determines whether the average value and the standard deviation of the predetermined number of correlation values satisfy a predetermined condition (ST61). Here, the shape opening / closing eye determination unit 23b determines that the condition is not satisfied, for example, when the average value is small or the standard deviation is large.
[0141]
When it is determined that the predetermined condition is not satisfied (ST61: NO), it is determined that the open / closed eye determination result (limited to the curvature radius method) due to the shape change of the upper eyelid is not used in later processing. (ST62). That is, it is determined that the open / close eye determination result (curvature radius method) by the shape open / close eye determination unit 23b is not usable. And a process transfers to step ST64.
[0142]
On the other hand, when it is determined that the predetermined condition is satisfied (ST61: YES), it is determined that the opening / closing eye determination result (curvature radius method) based on the shape change of the upper eyelid is used in later processing (ST63). That is, it is determined that the open / closed eye determination result (both the three-point method and the radius of curvature method) by the vertical width open / closed eye determination unit 23a can be used.
[0143]
Then, the integrated open / close eye determination unit 23d comprehensively determines the result of the open / close eye determination made available by the above processing (ST64). Thereafter, when the eye state detected by the open / closed eye determination unit 23 indicates closed eyes, the closed eye ratio detection unit 24 starts to accumulate the appearance time of the closed eye state (ST65). In addition, when the eye state detected by the open / closed eye determination unit 23 indicates that the eye is open, the integration is not started.
[0144]
Thereafter, in steps ST66 to ST72 in FIG. 23, processing similar to steps ST12 to ST18 described with reference to FIGS. 9 and 10 is executed, and step ST73 is reached.
[0145]
In step ST73, the arousal level correction unit 27 calculates the number of detection results used when the overall opening / closing eye determination unit 23d determines (ST73). Then, the awakening level correcting unit 27 corrects the awakening level to the awakening side as the number decreases (ST74). Specifically, as the number of detection results decreases, the awakening level is corrected to the awakening side so as to reduce the closed eye integration time.
[0146]
Thereafter, the processes of steps ST75 and ST76 are performed in the same manner as steps ST21 and ST22 shown in FIG. 10, and the process returns to step ST31 of FIG.
[0147]
In this way, according to the arousal level determination device 2 according to the present embodiment, the overall opening / closing eye determination unit 23d is finally based on the available determination results of the three opening / closing eye determination units 23a to 23c. Thus, the open / closed eye state is determined.
[0148]
And the arousal level correction | amendment part 27 correct | amends the arousal level to the awakening side as the number of the detection results used when the general open / close eye determination unit 23d determines the open / closed eye state decreases.
[0149]
Here, the three open / close eye determination units 23a to 23c are configured not to affect each other. For this reason, when the number of detection results used when the integrated open / close eye determination unit 23d determines the open / closed eye state is large, the determination accuracy is increased.
[0150]
On the other hand, when the number of detection results used when the general opening / closing eye determination unit 23d determines the open / closed eye state is small, the possibility that the general opening / closing eye determination unit 23d makes an erroneous determination increases. Therefore, the wakefulness level correcting unit 27 corrects the wakefulness level to the wakefulness side when the number of available detection results is small. In other words, the detection of dozing by mistake is avoided.
[0151]
Therefore, the determination accuracy can be improved.
[0152]
Further, as in the first embodiment, the driver can take a rest or the like based on his / her own judgment, and can contribute to effective sleepiness cancellation.
[0153]
Next, a third embodiment of the present invention will be described. The arousal level determination device 3 according to the third embodiment is similar to that of the first embodiment, but includes a vehicle interior light environment detection unit (light environment detection means) 28 instead of the variation degree detection unit 26. This is different from the first embodiment. In the arousal level determination device 2 according to the third embodiment, the image capturing unit 10 is connected not only to the position detection unit 21 but also to the vehicle interior light environment detection unit 28.
[0154]
Hereinafter, differences from the first embodiment will be described. FIG. 24 is a functional block diagram of the arousal level determination device 3 according to the third embodiment. The vehicle interior light environment detection unit 28 detects the light environment in the vehicle, and detects the vehicle interior environment based on the density data of the photographed image.
[0155]
Further, the present apparatus 3 does not have the variation degree detection unit 26. Therefore, the wakefulness level correction unit 27 is configured to correct the wakefulness level determined by the wakefulness level determination unit 25 based on the light environment detected by the vehicle interior light environment detection unit 28. Specifically, the arousal level correction unit 27 first stores a predefined light environment at a good time, and determines the light environment detected by the vehicle interior light environment detection unit 28 and the stored light environment. Compare. If the detected light environment is worse than the predefined good light environment, the wakefulness level determined by the wakefulness level determination unit 25 is corrected to the wakefulness side.
[0156]
FIG. 25 is an explanatory diagram of the light environment detected by the vehicle interior light environment detection unit 28. In FIG. 25, the light environment in the square area around the eye shown in (a1), (b1), and (c1) is measured, and the light environment is shown in (a2), (b2), and (c2), respectively. Represented as a density histogram.
[0157]
First, FIG. 25 (a1) shows a case where the light environment around the eye is good. The density histogram at that time is shown in FIG. The arousal level correction unit 27 stores, for example, a density histogram shown in FIG. 25 (a2) as data when the light environment is good.
[0158]
Next, in FIG. 25 (b1), the vicinity of the eye is covered with a shadow. At this time, as shown in FIG. 25 (b2), the density histogram has a higher frequency at a lower density value. In FIG. 25 (c1), sunlight is irradiated near the eyes. At this time, as shown in FIG. 25 (b2), the density histogram has a higher frequency at a higher density value.
[0159]
When these are compared with the density histogram shown in FIG. 25 (a2), it can be said that neither of them clearly shows a medium density value and the light environment is deteriorated. In such a situation, even if an attempt is made to detect the open / closed eye state of the driver's eyes, there is a possibility that it cannot be detected accurately due to the shadow or sunlight irradiation near the driver's eyes.
[0160]
In this embodiment, in such a case, by correcting the arousal level to the awakening side, it is possible to prevent erroneously determining that the patient is in the dozing state despite the awakening state.
[0161]
Next, the operation of the processing apparatus 20 having the above configuration will be described. 26 and 27 are flowcharts showing the detailed operation of the processing apparatus 20 shown in FIG. FIG. 26 shows the first half of the process, and FIG. 27 shows the second half of the process.
[0162]
First, the processes shown in steps ST81 to ST90 shown in FIG. 26 are the same as steps ST1 to ST10 shown in FIG. In step ST91, the vehicle interior light environment detection unit 28 calculates density information around the eyes (ST91).
[0163]
Thereafter, similarly to steps ST12 to ST18 of FIG. 9, the processing of steps ST92 to ST98 is followed to step ST99 of FIG. In step ST99, the vehicle interior light environment detection unit 28 estimates the light environment state based on the density information around the eye during a predetermined time (ST99).
[0164]
Then, the awakening level correction unit 27 compares the light environment obtained by the vehicle interior light environment detection unit 28 with the good light environment stored in advance. As a result of the comparison, when it is determined that the light environment is worse than when it is good, the arousal level correcting unit 27 corrects the arousal level to the awake side (ST100). Then, the process proceeds to step ST101.
[0165]
On the other hand, if it is determined that the light environment has not deteriorated compared to when the light environment is good, the process proceeds to step ST101 without correcting the arousal level. Thereafter, similarly to steps ST21 to ST23 of FIG. 9, the processes of steps ST101 to ST103 are performed. Then, the process returns to step ST81 in FIG.
[0166]
Thus, according to the arousal level determination device 3 according to the present embodiment, when the detected light environment is worse than the predefined good light environment, the arousal level Is corrected to the awakening side.
[0167]
That is, the awakening level correction unit 27 corrects the awakening level to the awakening side because the possibility of misjudging the awakening level to the dozing side increases when the light environment is deteriorated. And it is made not to detect a dozing accidentally. Conversely, if the lighting environment has not deteriorated, the arousal level is not corrected, and in an environment where the arousal level is accurately determined, unnecessary correction is performed to improve the accuracy of the arousal level. We do not invite a decline.
[0168]
Therefore, the determination accuracy can be improved.
[0169]
Further, as in the first embodiment, the driver can take a rest or the like based on his / her own judgment, and can contribute to effective sleepiness cancellation.
[0170]
In the present embodiment, when the light environment detected by the vehicle interior light environment detection unit 28 is worse than the light environment when the vehicle is in good condition, the wakefulness level is corrected to the wakefulness side. . However, the configuration of the present embodiment is not limited to this, and even when the detected light environment is worse than the good light environment, the arousal level is low when there is a low possibility of erroneous detection due to the deterioration. May not be corrected. Thereby, the determination accuracy can be further improved.
[0171]
Next, a fourth embodiment of the present invention will be described. The arousal level determination device 4 according to the fourth embodiment is the same as that of the first embodiment, but includes a glasses presence / absence detection unit (glasses detection means) 29 instead of the variation degree detection unit 26. It differs from that of the first embodiment. In the arousal level determination device 4 according to the fourth embodiment, the position detection unit 21 is connected not only to the tracking unit 22 but also to the eyeglass presence / absence detection unit 29.
[0172]
Hereinafter, differences from the first embodiment will be described. FIG. 28 is a functional block diagram of the arousal level determination device 4 according to the fourth embodiment. First, as described above, the arousal level determination device 4 includes the glasses presence / absence detection unit 29. The eyeglass presence / absence detection unit 29 detects whether the driver wears eyeglasses, and detects presence / absence of wearing based on specific continuous data detected when wearing eyeglasses. For this reason, the spectacle presence / absence detecting unit 29 is connected to the position detecting unit 21 in order to obtain continuous data, and is configured to input continuous data from the position detecting unit 21.
[0173]
In addition, because of the above-described configuration, the arousal level correcting unit 27 determines the arousal level determined by the arousal level determining unit 25 when the spectacle presence / absence detecting unit 29 detects the wearing of the driver's glasses. The level is corrected.
[0174]
Next, a captured image and continuous data when wearing glasses will be described. FIG. 29 is an explanatory diagram illustrating an example of a captured image when wearing glasses.
[0175]
As shown in the figure, when wearing spectacles, a scene or the like may be reflected in the lens portion. When the image is captured with the eye and the portion that is reflected overlapping, the eye opening degree cannot be accurately captured, which hinders determination of the arousal level.
[0176]
Specifically, when determining the arousal level by obtaining the opening of the eye based on the vertical width of the eye, since the eye and the portion that is reflected overlap, the open / close eye determination unit 23 determines whether the actual eye There is a possibility of recognizing a small opening. And when the opening degree of an eye is detected small, the determination result that it is a dozing state will increase.
[0177]
Therefore, in the present embodiment, the presence / absence of wearing of the driver's glasses is detected, and when worn, the arousal level is corrected to remove the increase. FIG. 30 is an explanatory diagram illustrating an example of continuous data obtained from a captured image when wearing glasses.
[0178]
As shown in the figure, when glasses are worn, continuous data tends to be detected more than when glasses are not worn. That is, the position detection unit 21 of the apparatus 4 extracts a feature portion similar to the eye from the image. At this time, all the continuous data continuously extending in the horizontal direction of the image is extracted as the feature amount similar to the eye. Since the frame portion of the glasses has a portion extending in the lateral direction of the image in shape, more continuous data is detected when wearing glasses than when glasses are not worn.
[0179]
The spectacle presence / absence detection unit 29 detects the presence / absence of wearing spectacles based on the number of continuous data, the position where the continuous data is extracted, and the like.
[0180]
Next, the operation of the processing apparatus 20 having the above configuration will be described. 31 and 32 are flowcharts showing detailed operations of the processing apparatus 20 shown in FIG. FIG. 31 shows the first half of the process, and FIG. 32 shows the second half of the process.
[0181]
First, the position detection unit 21 inputs image data (ST111). Thereafter, the position detection unit 21 determines whether or not an eye tracking region is set (ST112). When it is determined that the eye tracking area is set (ST112: YES), the process proceeds to step ST117.
[0182]
On the other hand, when it is determined that the eye tracking area is not set (ST112: NO), the position detection unit 21 detects the eye position from the entire image (ST113). At this time, the position detection unit 21 extracts a plurality of continuous data as eye candidates.
[0183]
Thereafter, the presence / absence detection unit 29 receives a plurality of pieces of continuous data from the position detection unit 21 and detects whether or not the driver is wearing glasses based on the plurality of pieces of continuous data (ST114). Then, the detection result here is sent to the arousal level correction unit 27.
[0184]
The arousal level correction unit 27 sets an arousal level correction coefficient when wearing spectacles is detected by the spectacle presence / absence detection unit 29 (ST115). Then, this correction coefficient is given to the arousal level determination unit 25. Meanwhile, during this time, the tracking unit 22 sets a tracking region based on the eye position data from the position detection unit 21 (ST116).
[0185]
Thereafter, similarly to steps ST6 to ST14 shown in FIG. 9, the processes shown in steps ST117 to ST125 shown in FIG. 31 are executed. And a process transfers to step ST126 of FIG. Steps ST126 to ST128 are the same as steps ST15 to ST17 in FIG.
[0186]
In step ST129, the arousal level determination unit 25 determines the arousal level by using the correction coefficient given from the arousal level correction unit 27 in step ST115 (ST129). Then, similarly to steps ST21 and ST22 of FIG. 10, the processes of steps ST130 and ST131 are executed. Thereafter, the processing returns to step ST111 in FIG.
[0187]
As described above, the arousal level correction unit 27 substantially corrects the arousal level by giving a correction coefficient to the arousal level determination unit 25. That is, in the first to third embodiments described above, the wakefulness level correcting unit 27 corrects the wakefulness level after being determined by the wakefulness level determining unit 25. On the other hand, in the present embodiment, a correction coefficient is given before the determination by the arousal level determination unit 25, and the arousal level after the determination is not corrected.
[0188]
However, in the case of the present embodiment, it can be said that the arousal level is substantially corrected because an arousal level different from the arousal level when the arousal level correction unit 27 does not act is obtained.
[0189]
Thus, according to the arousal level determination device 4 according to the present embodiment, the presence or absence of the driver's glasses is detected, and when the driver's spectacles are detected, the arousal level is set to the awakening side. It is corrected to.
[0190]
In other words, the arousal level correction unit 27 is likely to cause a misjudgment of the arousal level to the dozing side due to occurrence of reflection in the spectacles when the spectacles are worn. It has been corrected. That is, the opening degree of the eye is detected as a result of reflection, and the number of determination results that the patient is dozing increases, but the determination accuracy is improved by removing the increase. Conversely, when the glasses are not worn, the arousal level is not corrected, and in an environment where the arousal level is accurately determined, unnecessary correction is made so as not to reduce the accuracy of the arousal level. I have to.
[0191]
Therefore, the determination accuracy can be improved.
[0192]
Further, as in the first embodiment, the driver can take a rest or the like based on his / her own judgment, and can contribute to effective sleepiness cancellation.
[0193]
In the first to fourth embodiments, either the awakening level is determined by either method, whether the correction coefficient is given before the determination by the awakening level determination unit 25 or the corrected awakening level is corrected. The degree level may be corrected.
[0194]
Next, a fifth embodiment of the present invention will be described. The arousal level determination device 5 according to the fifth embodiment is the same as that of the first embodiment, except that a dimension determination unit (dimension determination means) 31 is provided instead of the variation degree detection unit 26. Different from that of one embodiment. Further, in the arousal level determination device 5 according to the fifth embodiment, the tracking unit 22 is connected only to the open / closed eye determination unit 23, and the open / closed eye determination unit 23 is not only the closed eye ratio detection unit 24 but also the size determination unit 31. Also connected to.
[0195]
Hereinafter, differences from the first embodiment will be described. FIG. 33 is a functional block diagram of the arousal level determination device 5 according to the fifth embodiment. First, as described above, the arousal level determination device 5 includes the dimension determination unit 31. The size determination unit 31 learns the size of the driver's eyes on the image and determines whether the size is smaller than a predetermined value.
[0196]
The size determination unit 31 is connected to the open / close eye determination unit 23 and is configured to input information on the eye opening from the open / close eye determination unit 23 to learn the size of the eye.
[0197]
Here, the learning is to learn the size of the eyes of the driver in a normal state other than snoozing or narrowing eyes. For example, when determining the driver's doze state or the like based on the vertical width of the eyes, a threshold value for determination is usually provided, and whether or not the driver's eye size obtained from the image is larger than the threshold value. to decide. However, when the driver's eyes are smaller than that of a general person, the vertical width of the eyes is close to the threshold value, and it may be erroneously determined that the driver is dozing just by narrowing the eyes slightly. There is. And in such a case, the determination result that it is a dozing state will increase, and a determination result will become inaccurate.
[0198]
Therefore, in the present embodiment, the size of the driver's eyes is learned, and when the size is smaller than a predetermined value, the level of wakefulness is corrected to the wakefulness side, and the determination of being a doze state is performed. The increase is removed.
[0199]
In addition, since it is configured as described above, the arousal level correction unit 27 is determined by the arousal level determination unit 25 when the size determination unit 31 determines that the eye size is smaller than a predetermined value. It is configured to correct the arousal level.
[0200]
Next, the operation of the processing apparatus 20 having the above configuration will be described. 34 and 35 are flowcharts showing the detailed operation of the processing apparatus 20 shown in FIG. FIG. 34 shows the first half of the process, and FIG. 35 shows the second half of the process.
[0201]
First, the processes shown in steps ST141 to ST148 shown in FIG. 34 are the same as steps ST1 to ST8 shown in FIG. In step ST149, the dimension determining unit 31 determines whether or not the learning of the eye size is completed (ST149).
[0202]
When it is determined that the learning of the eye size has not ended (ST149: NO), the dimension determining unit 31 continues the learning of the eye size (ST150). Then, the process returns to step ST141.
[0203]
On the other hand, when it is determined that the learning of the eye size has ended (ST149: YES), the process proceeds to step ST151. In step ST151, the wakefulness level correction unit 27 sets the wakefulness level correction coefficient in accordance with the eye size learned by the size determination unit 31 (ST151).
[0204]
Here, it is determined whether or not the size of the learned eye is smaller than a predetermined value, and if it is smaller than the predetermined value, a correction coefficient for correcting the arousal level to the awakening side is set. On the other hand, if the learned eye size is not smaller than the predetermined value, the correction coefficient is not set. Thereafter, the wakefulness level correcting unit 27 gives this correction coefficient to the wakefulness level determining unit 25.
[0205]
Then, similarly to steps ST9, ST10, and ST12 to ST17 in FIG. 9, the process goes to step ST160 in FIG. 35 through the processes in steps ST152 to ST156. In step ST160, the wakefulness level determination unit 25 determines the wakefulness level using the correction coefficient given from the wakefulness level correction unit 27 in step ST1515 (ST160). Then, similarly to steps ST21 and ST22 of FIG. 10, the processes of steps ST161 and ST162 are executed. Thereafter, the process returns to step ST141 in FIG.
[0206]
Here, the wakefulness level correction unit 27 substantially corrects the wakefulness level in the same manner as in the fourth embodiment, but is not limited thereto, and is corrected before the determination by the wakefulness level determination unit 25. A coefficient may be given.
[0207]
Thus, according to the arousal level determination device 5 according to the present embodiment, the size of the driver's eyes on the image is determined, and the determined size of the eyes is smaller than a predetermined size. In some cases, the arousal level is corrected to the awakening side.
[0208]
In other words, the arousal level correction unit 27 corrects the arousal level to the awakening side because there is a high possibility that the awakening level is erroneously determined to be a nap when the driver's eyes are small. . That is, when a person who originally has small eyes is targeted, it is prevented that the person is more likely to be determined to be dozing than a person with large eyes. Conversely, when the eye size is not small, the arousal level is not corrected, and in an environment where the arousal level is accurately determined, unnecessary correction is performed to improve the accuracy of the arousal level. We do not invite a decline.
[0209]
Therefore, the determination accuracy can be improved.
[0210]
Further, as in the first embodiment, the driver can take a rest or the like based on his / her own judgment, and can contribute to effective sleepiness cancellation.
[0211]
Next, a sixth embodiment of the present invention will be described. The arousal level determination device 6 according to the sixth embodiment is the same as that of the first embodiment, but differs from that of the first embodiment in that a correction content display unit (correction content display means) 40 is provided. ing.
[0212]
Hereinafter, differences from the first embodiment will be described. FIG. 36 is a configuration diagram of the arousal level determination device 6 according to the sixth embodiment, and FIG. 37 is a functional block diagram of the arousal level determination device 6 according to the sixth embodiment. As shown in FIG. 36, the correction content display unit 40 is provided in a position where the driver can visually recognize, for example, in the vicinity of the arousal level display unit 30.
[0213]
As shown in FIG. 37, the correction content display unit 40 is connected to the arousal level correction unit 27 and inputs the correction content performed by the arousal level level correction unit 27 to the arousal level determination unit 25. It is said that.
[0214]
Here, the correction content display unit 40 may be configured to present the correction content as it is, but it is desirable to present it to the driver in a graph or the like. Thereby, the correction content can be presented to the driver more clearly.
[0215]
Further, it is desirable that the correction content display unit 40 and the arousal level display unit 30 are integrally configured. Thereby, space saving can be achieved. Furthermore, since it is configured integrally, it is possible to display the correction content and the corrected arousal level in parallel, and display in consideration of ease of viewing.
[0216]
Next, the operation of the arousal level determination device 6 according to the sixth embodiment is substantially the same as that according to the first embodiment. However, in the sixth embodiment, since the correction content display unit 40 is provided, the correction content display processing is performed. For example, this processing may be performed simultaneously with the display of the arousal level (ST21) shown in FIG. 10, or may be performed after clearing the coordinate position (ST23).
[0217]
Thus, according to the arousal level determination device 6 according to the present embodiment, it is possible to improve the determination accuracy as in the first embodiment, and it is possible to take a rest or the like based on the driver's own determination. Can contribute to effective drowsiness elimination.
[0218]
Furthermore, since the wakefulness level correcting unit 27 includes the correction content display unit 40 that displays the correction content when the wakefulness level is corrected, the reliability of the wakefulness level that is currently displayed to the driver is based on reliability. You can specify that it is a thing. In addition, this clarification can give the driver a sense of security regarding the operating state of the device 6.
[0219]
In this embodiment, the configuration in which the correction content display unit 40 is added to that of the first embodiment has been described. However, the present invention is not limited to this, and the correction content display unit 40 is added to any of the other embodiments described above. You may do it.
[0220]
Next, a seventh embodiment of the present invention will be described. The configuration of the arousal level determination device 7 according to the seventh embodiment is the same as that of the first embodiment. However, in the wakefulness level determination device 7 according to the seventh embodiment, the correction method of the wakefulness level correction unit 27 is different from that of the first embodiment.
[0221]
Hereinafter, differences from the first embodiment will be described. FIG. 38 is an explanatory diagram showing a correction method performed by the arousal level determination device 7 according to the seventh embodiment. In FIG. 38, the accumulated time is represented by the number of image frames. For this reason, the actual integration time is a value obtained by multiplying the processing speed of one frame.
[0222]
As shown in the figure, in each of the arousal level determination sections “13 to 52”, the respective closed eye integration times are obtained. In the first embodiment, when the degree of variation exceeding a predetermined threshold “20.0” in the horizontal and vertical directions of the image is detected, the arousal level is corrected to the one-stage awakening side.
[0223]
However, in this embodiment, instead of correcting the wakefulness level to the wakefulness side, a process of holding the wakefulness level determined by the previous wakefulness level determining unit 25 as the current wakefulness level is performed.
[0224]
More specifically, first, in the arousal level determination section “14”, a degree of variation exceeding a predetermined threshold “20.0” is detected. For this reason, the arousal level correction unit 27 holds the arousal level of the determination section “13” that is the previous awakening level. That is, the awakening level “3” of the determination section “13” is directly used as the awakening level of the determination section “14”.
[0225]
At this time, in the wakefulness level determination section “14”, the wakefulness level is set to the previous value without setting the wakefulness level “2” indicating a light doze state, even though the closed eye accumulated time is 30 frames or more. It remains at “3”.
[0226]
Also, the degree of variation exceeding the predetermined threshold “20.0” is detected for the arousal level determination section “15”. Therefore, the wakefulness level correction unit 27 holds the wakefulness level of the determination section “14”, which is the previous wakefulness level, and sets the wakefulness level to “3”.
[0227]
Similarly, for the other determination sections “16-18, 21, 25-30, 42, 43, 46”, the previous wakefulness level is held as the current wakefulness level. Thereby, the influence by disturbance is reduced. That is, by maintaining the previous arousal level, it is possible to prevent a situation in which the arousal level changes drastically due to disturbance.
[0228]
Thus, according to the arousal level determination device 7 according to the present embodiment, when the degree of variation in the coordinate position exceeds a predetermined threshold, the previous awakening level is maintained. For this reason, an arousal level does not change abruptly by disturbance, and the troublesomeness given to a driver can be reduced.
[0229]
Further, as in the first embodiment, the driver can take a rest or the like based on his / her own judgment, and can contribute to effective sleepiness cancellation.
[0230]
In addition, although the example which changed the processing content of the arousal level determination apparatus 1 of 1st Embodiment was demonstrated above, not only this but the processing method of this embodiment about other 2nd-6th embodiment. You may make it apply.
[0231]
Although the first to seventh embodiments have been described above, the present invention is not limited to these. For example, the first to seventh embodiments may be combined. For example, when the configurations of the first to fifth embodiments are combined, two or more methods are combined, so that the determination accuracy can be further improved.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a wakefulness level determination device according to a first embodiment of the present invention.
FIG. 2 is a functional block diagram of the arousal level determination device according to the first embodiment of the present invention.
FIG. 3 is an explanatory diagram of an initial process performed when a position detection unit detects an eye position.
FIG. 4 is an explanatory diagram showing a state when a position detection unit extracts a predetermined pixel.
FIG. 5 is an explanatory diagram showing a state when pixels adjacent in the horizontal direction of the image are grouped.
FIG. 6 is an explanatory diagram showing a state after zoning by a position detection unit.
FIGS. 7A and 7B are explanatory diagrams showing the tracking of the eye position by the tracking unit, where FIG. 7A shows an initial tracking area, and FIG. 7B shows the detected eye after setting the initial tracking area; (C) shows a tracking area newly set based on the detected eye position, and (d) shows a position of the eye detected from within the newly set tracking area. .
FIGS. 8A and 8B are explanatory views showing the opening degree of the vertical width of the eye, in which FIG. 8A shows when the eye is open and FIG. 8B shows the time when the eye is closed.
9 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 1, and shows the first half of the processing.
10 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 1, and shows the latter half of the processing. FIG.
FIG. 11 is an explanatory diagram of a wakefulness index.
FIG. 12 is an explanatory diagram showing an arousal index obtained by actually measuring a driver, an eye-closed integrated time detected by the present apparatus, and a variation in eye coordinate position, and a favorable arousal level. An example in the case of being determined is shown.
FIG. 13 is an explanatory diagram showing a wakefulness index actually obtained by measuring a driver, an eye-closed integrated time detected by the present device, and a variation in eye coordinate position, and a wakefulness level has a slight error. The example in the case of determining including is shown.
14 is an explanatory diagram showing a state before and after correction of the arousal level obtained based on the closed eye integration time shown in FIG. 13; FIG.
15 is an explanatory diagram showing the state before and after the correction of the arousal level obtained based on the integrated eye closure time shown in FIG. 13, and shows an example in the case of graphing the data shown in FIG. ing.
16 is an explanatory diagram showing a distribution of eye reference positions and eye coordinate positions in the case shown in FIG. 12. FIG.
17 is an explanatory diagram showing a distribution of eye reference positions and eye coordinate positions in the case shown in FIG. 13; FIG.
FIG. 18 is a functional block diagram of a wakefulness level determination device according to a second embodiment.
FIG. 19 is an explanatory diagram illustrating a determination method by a vertical width open / close eye determination unit, a shape open / close eye determination unit, and an interval open / close eye determination unit;
20 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 18, and shows an initial part of the processing.
FIG. 21 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 18, showing the first half of the intermediate processing.
FIG. 22 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 18, showing the latter half of the intermediate processing.
FIG. 23 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 18, showing the final part of the processing.
FIG. 24 is a functional block diagram of a wakefulness level determination device according to a third embodiment.
FIG. 25 is an explanatory diagram of a light environment detected by a vehicle interior light environment detection unit.
FIG. 26 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 24, showing the first half of the processing.
FIG. 27 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 24, showing the latter half of the processing.
FIG. 28 is a functional block diagram of a wakefulness level determination device according to a fourth embodiment.
FIG. 29 is an explanatory diagram illustrating an example of a captured image when wearing glasses.
FIG. 30 is an explanatory diagram showing an example of continuous data obtained from a photographed image when wearing glasses.
31 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 28, showing the first half of the process.
32 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 28, showing the latter half of the processing.
FIG. 33 is a functional block diagram of a wakefulness level determination device according to a fifth embodiment.
34 is a flowchart showing detailed operations of the processing apparatus shown in FIG. 33, showing the first half of the processing.
35 is a flowchart showing a detailed operation of the processing apparatus shown in FIG. 33, showing the latter half of the processing.
FIG. 36 is a configuration diagram of a wakefulness level determination device according to a sixth embodiment.
FIG. 37 is a functional block diagram of a wakefulness level determination device according to a sixth embodiment.
FIG. 38 is an explanatory diagram showing a correction method by the arousal level determination device according to the seventh embodiment.
[Explanation of symbols]
1-7 ... Arousal level determination device
10. Image capturing unit (image capturing means)
21... Position detection unit (position detection means)
22 ... Tracking unit (tracking means)
23: Open / close eye determination unit (open / close eye determination means)
23a ... Vertical width open / close eye determination unit
23b ... Shape open / close eye determination unit
23c: Interval open / close eye determination unit
23d ... General open / close eye determination unit
24: Eye-closed ratio detection unit (eye-closed ratio detection means)
25. Arousal level determination unit (awakening level determination means)
26: Variation degree detection unit (variation degree detection means)
27. Arousal level correction unit (awakening level correction means)
28 ... Light environment detection part in the vehicle interior (light environment detection means)
29 ... Glasses presence / absence detecting unit (glasses detecting means)
30 ... arousal level display section (awakening level display means)
31 ... Dimension determination unit (dimension determination means)
40. Correction content display section (correction content display means)

Claims (9)

In a wakefulness level determination device that processes a face image and determines a driver's wakefulness level,
Image photographing means for photographing the driver's face;
Position detecting means for detecting the position of the eyes of the driver from the entire image obtained by the image photographing means;
A tracking unit configured to set a predetermined area on the image including the driver's eye on the basis of the position of the eye detected by the position detection unit, and to track the eye from within the predetermined area;
Open / closed eye determination means for determining an open / closed eye state for an eye in a predetermined area tracked by the tracking means;
Based on the open / closed eye state of the eye determined by the open / closed eye determination unit, the closed eye rate detection unit that detects the ratio of the appearance time of the closed eye state in a predetermined time period;
Arousal level determination means for determining arousal level from the ratio detected by the closed eye ratio detection means;
A variation degree detecting means for detecting a degree of variation in the coordinate position of the eye based on the position of the eye tracked by the tracking means;
Wakefulness level correcting means for correcting the wakefulness level determined by the wakefulness level determining means to the wakefulness side when the degree of variation in the coordinate position of the eye detected by the unevenness degree detecting means exceeds a predetermined threshold; ,
A wakefulness level determination device comprising: In a wakefulness level determination device that processes a face image and determines a driver's wakefulness level,
Image photographing means for photographing the driver's face;
Position detecting means for detecting the position of the eyes of the driver from the entire image obtained by the image photographing means;
A tracking unit configured to set a predetermined area on the image including the driver's eye on the basis of the position of the eye detected by the position detection unit, and to track the eye from within the predetermined area;
Open / closed eye determination means for determining an open / closed eye state for an eye in a predetermined area tracked by the tracking means;
Based on the open / closed eye state of the eye determined by the open / closed eye determination means, the appearance time of the closed eye state that occupies during a predetermined time is integrated to calculate the closed eye integrated time, and the appearance time of the closed eye state that occupies during the predetermined time is calculated . A closed eye ratio detecting means for detecting a ratio;
Arousal level determination means for determining arousal level from the ratio detected by the closed eye ratio detection means;
Wakefulness level correcting means for correcting the wakefulness level determined by the wakefulness level determining means,
The opening / closing eye determination means includes
A vertical open / closed eye determination unit that determines an open / closed eye state based on the vertical width of an eye existing in the predetermined region;
A shape opening / closing eye determination unit that determines an opening / closing eye state based on the shape of the upper eyelid in the predetermined region;
An interval open / close eye determination unit that determines an open / close eye state based on an interval between a peak point of the upper eyelid existing in the predetermined region and an eyebrow existing above the eye;
An overall open / close eye determination unit that finally determines an open / close eye state based on at least one of the three determination results of the open / close eye determination unit;
The wakefulness level correction unit is configured to detect the closed eye calculated by the closed eye ratio detection unit as the number of determination results of the open / closed eye determination unit used when the integrated open / closed eye determination unit determines the open / closed eye state decreases. A wakefulness level determination apparatus, wherein the wakefulness level is corrected to the wakefulness side so as to reduce the accumulated time . In a wakefulness level determination device that processes a face image and determines a driver's wakefulness level,
Image photographing means for photographing the driver's face;
Position detecting means for detecting the position of the eyes of the driver from the entire image obtained by the image photographing means;
A tracking unit configured to set a predetermined area on the image including the driver's eye on the basis of the position of the eye detected by the position detection unit, and to track the eye from within the predetermined area;
Open / closed eye determination means for determining an open / closed eye state for an eye in a predetermined area tracked by the tracking means;
Based on the open / closed eye state of the eye determined by the open / closed eye determination unit, the closed eye rate detection unit that detects the ratio of the appearance time of the closed eye state in a predetermined time period;
Arousal level determination means for determining arousal level from the ratio detected by the closed eye ratio detection means;
Glasses detecting means for detecting whether or not the driver wears glasses;
An arousal level correction unit that corrects the arousal level determined by the arousal level determination unit to the arousal side when wearing of the driver's glasses is detected by the glasses detection unit;
A wakefulness level determination device comprising: The wakefulness level determination device according to any one of claims 1 to 3 , further comprising wakefulness level display means for displaying the wakefulness level corrected by the wakefulness level correction means. 5. The wakefulness level determination apparatus according to claim 4 , further comprising a correction content display means for displaying correction content when the wakefulness level correction means corrects the wakefulness level. The wakefulness level correcting means replaces the wakefulness level determined by the wakefulness level determining means with the wakefulness side instead of correcting the wakefulness level determined by the previous wakefulness level determining means. alertness level determination apparatus according to any one of claims 1 to 2, characterized in that the holding as. A wakefulness level determination device that shoots the driver's face, determines the open / closed eye state of the driver's eyes from the obtained image, and determines the driver's wakefulness level from the open / closed eye state,
The eye position is sequentially detected from the obtained image, and the degree of variation of the coordinate position of the eye is detected from the sequentially detected eye position. An arousal level determination device characterized by correcting to the side. A wakefulness level determination device that shoots the driver's face, determines the open / closed eye state of the driver's eyes from the obtained image, and determines the driver's wakefulness level from the open / closed eye state,
In determining the open / closed eye state, the detection result of the open / closed eye state based on the vertical width of the eye, the detection result of the open / closed eye state based on the shape of the upper eyelid of the eye, and the peak point of the upper eyelid and above the eye A final opening / closing eye state is determined based on at least one detection result among the three detection results of the detection result of the opening / closing eye state based on an interval between the eyebrow and the eyebrow
As the number of detection results used in determining the final open / closed eye state decreases, the awakening level is set so as to reduce the closed eye accumulated time obtained by integrating the appearance time of the closed eye occupying the predetermined time. A wakefulness level determination device characterized by correcting to A wakefulness level determination device that shoots the driver's face, determines the open / closed eye state of the driver's eyes from the obtained image, and determines the driver's wakefulness level from the open / closed eye state,
A wakefulness level determination device that detects whether or not a driver wears eyeglasses, and corrects the wakefulness level to the wakefulness side when the driver wears glasses.

Images