JP2018136848A - Driver monitoring device - Google Patents

Abstract

An object of the present invention is to suppress deterioration in accuracy in recognizing a driver. A system control unit (6) acquires image data representing a captured image captured by a camera (2) from a camera (2) that continuously captures a face (DF) of a driver sitting in a driver's seat of a vehicle. The system control unit 6 sets the recognition range so as to include the face area in which the driver's face is captured in the captured image indicated by the acquired image data. The system control unit 6 acquires a vibration voltage signal indicating the magnitude of vibration from the vibration sensor 4 that detects the magnitude of vibration generated in the vehicle. The system control unit 6 expands the set recognition range from before the vibration sensor 4 detects the vibration according to the magnitude of the vibration indicated by the acquired vibration voltage signal. [Selection drawing] Fig. 1

Description

  The present disclosure relates to a driver monitoring device that monitors a driver of a vehicle.

  Patent Document 1 describes a driver monitoring device that captures a face of a driver of a vehicle with a camera and detects a driver's dozing or looking aside.

JP 2000-60799 A

  The processing load is limited by limiting only the area around the area where the driver's face is captured in the most recently captured image captured by the camera as the monitoring area for monitoring the driver. A technique for reducing the above is known.

  However, if the monitoring area is limited in this way, the driver's face may be removed from the monitoring area when vibration occurs while the vehicle is traveling. For this reason, there is a possibility that the accuracy of recognizing the driver is lowered.

  An object of the present disclosure is to suppress a decrease in accuracy of recognizing a driver.

  One aspect of the present disclosure includes an image acquisition unit (S210), a region setting unit (S330), a vibration acquisition unit (S10), an expansion unit (S260 to S280), and a monitoring unit (S310, S320, S340, S350). A driver monitoring device (6).

An image acquisition part is comprised so that the image data which show the picked-up image which the imaging | photography part image | photographed from the imaging | photography part (2) which continuously image | photographs the driver | operator's face sitting in the driver's seat of a vehicle may be acquired.
The region setting unit is configured to set a monitoring region for monitoring the driver so as to include a face region in which the driver's face is reflected in the captured image indicated by the image data acquired by the image acquisition unit. Is done.

The vibration acquisition unit is configured to acquire vibration amount information indicating the magnitude of vibration from a vibration detection unit (4) that detects the magnitude of vibration generated in the vehicle.
The expansion unit expands the monitoring region set by the region setting unit according to the magnitude of the vibration indicated by the vibration amount information acquired by the vibration acquisition unit than before the vibration detection unit detects vibration. Composed.

The monitoring unit is configured to execute a monitoring process for monitoring a driver's face included in the monitoring area.
The driver monitoring device of the present disclosure configured in this manner extends the monitoring area according to the magnitude of vibration generated in the vehicle. For this reason, the driver monitoring device according to the present disclosure has a situation in which the driver's face is out of the monitoring area due to the vibration of the driver's face due to vibration, and the driver's face cannot be recognized. Can be suppressed. Thereby, the driver | operator monitoring apparatus of this indication can suppress the fall of the precision which recognizes a driver | operator.

  Note that the reference numerals in parentheses described in this column and in the claims indicate the correspondence with the specific means described in the embodiment described later as one aspect, and the technical scope of the present disclosure It is not limited.

1 is a block diagram showing a configuration of a driver monitoring system 1. FIG. It is a figure which shows the imaging | photography range SR of the camera. It is a flowchart which shows an adjustment amount calculation process. It is a figure which shows imaging | photography timing and the time change of the X-axis direction vibration amount. It is a flowchart which shows a driver | operator monitoring process. It is a figure which shows the setting method of an X-axis recognition range and a Y-axis recognition range. It is a figure which shows the method of setting a recognition range based on Z-axis direction vibration amount. It is a figure which shows that the camera 2 is installed so that a driver | operator may be image | photographed from the front. It is a figure which shows that the camera 2 is installed so that a driver | operator may be imaged from diagonally.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
The driver monitoring system 1 according to this embodiment is mounted on a vehicle and includes a camera 2, a frame memory 3, a vibration sensor 4, a speaker 5, and a system control unit 6, as shown in FIG.

  The camera 2 includes an imaging unit 11, an irradiation unit 12, an image generation unit 13, an exposure control unit 14, and a controller 15. The camera 2 is, for example, installed on the upper part of the steering column and photographs the face DF of an occupant (ie, driver) sitting in the driver's seat.

  In the camera 2, as shown in FIG. 2, a rectangular shooting range SR is set. The longitudinal direction of the imaging range SR is the X-axis direction, and the short direction of the imaging range SR is the Y-axis direction. The camera 2 is installed so that the vehicle width direction of the vehicle is parallel to the X-axis direction of the shooting range SR, and the vehicle height direction of the vehicle is parallel to the Y-axis direction of the shooting range SR.

  As illustrated in FIG. 1, the imaging unit 11 includes an optical element unit 21 and an image sensor 22. The optical element unit 21 includes at least a condenser lens that collects light and an optical filter that transmits a predetermined light component (for example, light having a wavelength in the near infrared region). The image sensor 22 is a well-known device configured by two-dimensionally arranging CCD or CMOS elements, and outputs a signal corresponding to the intensity of light incident through the optical element unit 21.

  The irradiation unit 12 includes one or more light sources that irradiate light including a component that passes through the optical filter, and irradiates light toward the driver's face. As the light source, for example, a light emitting diode can be used.

The image generation unit 13 generates image data indicating a two-dimensional image captured by the camera 2 by converting a signal output from the image sensor 22 from analog to digital.
The exposure control unit 14 has a dynamic signal in which an average signal intensity of a signal (hereinafter referred to as an image signal) acquired by the image generation unit 13 is set in advance according to various parameters representing characteristics of the imaging unit 11 and the irradiation unit 12. The brightness of the light irradiated by the irradiation unit 12 (hereinafter referred to as light source brightness) and the shutter time (that is, the exposure time) of the image sensor 22 are set so as to be near the middle of the range.

  The controller 15 controls the imaging unit 11 and the irradiation unit 12 so that the light source luminance and the shutter time are set according to the setting in the exposure control unit 14. The control of the imaging unit 11 and the irradiation unit 12 by the controller 15 is executed according to an instruction from the system control unit 6 to the controller 15.

  The frame memory 3 is a memory that temporarily stores the image data acquired from the image generation unit 13 by the system control unit 6 and has a storage capacity capable of storing image data for a plurality of screens.

  The vibration sensor 4 is, for example, an acceleration sensor, detects the magnitude of vibration along each of the X-axis, Y-axis, and Z-axis constituting the three-dimensional orthogonal coordinate system, and according to the detected magnitude of vibration. A voltage signal (hereinafter referred to as an oscillating voltage signal) indicating the detected voltage is output to the system controller 6. For example, the vibration sensor 4 is installed in the seat portion of the driver's seat so as to be directly below the driver when the driver sits on the driver's seat. The vibration sensor 4 is installed such that the X-axis direction of the vibration sensor 4 is parallel to the vehicle width direction of the vehicle and the Z-axis direction of the vibration sensor 4 is parallel to the traveling direction of the vehicle. Thereby, the Y-axis direction of the vibration sensor 4 is parallel to the vehicle height direction of the vehicle.

The speaker 5 is installed in the vehicle interior, converts the audio electric signal input from the system control unit 6 into sound, and outputs the sound.
The system control unit 6 is an electronic control device that is configured around a known microcomputer including a CPU 31, a ROM 32, a RAM 33, and the like. Various functions of the microcomputer are realized by the CPU 31 executing a program stored in a non-transitional tangible recording medium. In this example, the ROM 32 corresponds to a non-transitional tangible recording medium that stores a program. Further, by executing this program, a method corresponding to the program is executed. Note that some or all of the functions executed by the CPU may be configured by hardware using one or a plurality of ICs. Further, the number of microcomputers constituting the system control unit 6 may be one or plural.

In the driver monitoring system 1 configured as described above, the system control unit 6 executes a driver monitoring process and an adjustment amount calculation process.
First, the procedure of the adjustment amount calculation process will be described. The adjustment amount calculation process is a process that is executed every time a preset calculation execution time elapses during the operation of the system control unit 6. This calculation execution time is set to a time sufficiently shorter than the period of vibration generated in the vehicle.

  When this adjustment amount calculation process is executed, the system control unit 6 first acquires a vibration voltage signal from the vibration sensor 4 in S10 as shown in FIG. 3, and based on the acquired vibration voltage signal, X The amount of vibration along each of the axis, the Y axis, and the Z axis is calculated. Hereinafter, the vibration amounts in the X-axis, Y-axis, and Z-axis directions are referred to as the X-axis direction vibration amount, the Y-axis direction vibration amount, and the Z-axis direction vibration amount, respectively. Further, in S <b> 10, the system control unit 6 stores the calculated X-axis direction vibration amount, Y-axis direction vibration amount, and Z-axis direction vibration amount in the RAM 33.

  Next, in S <b> 20, the system control unit 6 determines whether or not a shooting timing for causing the camera 2 to execute shooting has come. The shooting timing comes every time a preset shooting execution time elapses.

Here, when the photographing timing has not arrived, the system control unit 6 once ends the adjustment amount calculation process.
On the other hand, when the shooting timing has arrived, in S30, the system control unit 6 determines the previous shooting timing and the current shooting for each of the X-axis direction vibration amount, the Y-axis direction vibration amount, and the Z-axis direction vibration amount. The maximum value and the minimum value between the timings are calculated, and the calculated maximum value and minimum value are stored in the RAM 33. Note that the shooting timing determined in S20 is the current shooting timing, and the shooting timing one time before the current shooting timing is the previous shooting timing.

In S40, the system control unit 6 calculates the X-axis vibration average value Bx _{ave according} to the following equation (1).
A time range between the previous shooting timing and the current shooting timing is referred to as an n-th shooting time range. n is an integer of 2 or more. The current shooting timing is set to the (n + 1) th shooting timing, and the shooting timing n times before the current shooting timing is set to the first shooting timing.

Therefore, the k-th shooting time range is a time range between the shooting timing before k times and the shooting timing before (k + 1) times. k is an integer that is 1 or more and n or less.
Bx (k) max in the following expression (1) is the maximum value of the X-axis direction vibration amount in the k-th imaging time range. Bx (k) min in the following formula (1) is the minimum value of the X-axis direction vibration amount in the k-th imaging time range.

  FIG. 4 shows the maximum value Bx (1) max and the minimum value Bx (1) min in the first shooting time range that is the time range between the first shooting timing t1 and the second shooting timing t2. Yes. Similarly, FIG. 4 shows the maximum value Bx (2) max and the minimum value Bx (2) min in the second shooting time range that is the time range between the second shooting timing t2 and the third shooting timing t3. Is shown. FIG. 4 shows the maximum value Bx (3) max and the minimum value Bx (3) min in the third shooting time range that is the time range between the third shooting timing t3 and the fourth shooting timing t4. Show.

The X-axis vibration average value Bx _ave calculated by the equation (1) is an average value of the difference between the maximum value and the minimum value of the X-axis direction vibration amount in the first imaging time range to the n-th imaging time range. However, the sign of the minimum value of the X-axis direction vibration amount is negative. For this reason, the X-axis vibration average value Bx _ave is the average value of the addition values obtained by adding the absolute value of the maximum value and the absolute value of the minimum value of the vibration amount in the X-axis direction in the first shooting time range to the n-th shooting time range. It is. That is, the X-axis vibration average value Bx _ave is a simple moving average between the current shooting timing and the n-th previous shooting timing for time-series data of the difference between the maximum value and the minimum value of the X-axis direction vibration amount. This is a value obtained by calculation.

When the process of S40 ends, as shown in FIG. 3, in S50, the system control unit 6 calculates the Y-axis vibration average value By _ave by the following equation (2).
By (k) max in the following expression (2) is the maximum value of the Y-axis direction vibration amount in the k-th imaging time range. By (k) min in the following expression (2) is the minimum value of the Y-axis direction vibration amount in the k-th imaging time range. The Y-axis vibration average value By _ave calculated by Expression (2) is an average value of the difference between the maximum value and the minimum value of the Y-axis direction vibration amount in the first imaging time range to the n-th imaging time range.

Further, in S60, the system control unit 6 calculates the Z-axis vibration average value Bz _ave by the following equation (3), and temporarily ends the adjustment amount calculation process.
Bz (k) max in the following expression (3) is the maximum value of the Z-axis direction vibration amount in the k-th imaging time range. Bz (k) min in the following expression (3) is the minimum value of the Z-axis direction vibration amount in the k-th imaging time range. The Z-axis vibration average value By _ave calculated by Expression (3) is an average value of the difference between the maximum value and the minimum value of the Z-axis direction vibration amount in the first imaging time range to the n-th imaging time range.

Next, the procedure of the driver monitoring process will be described. The driver monitoring process is a process that is executed every time the shooting execution time elapses during the operation of the system control unit 6.
When the driver monitoring process is executed, the system control unit 6 first causes the camera 2 to perform shooting in S210 and acquires image data generated by the shooting from the camera 2 as shown in FIG. .

  In S220, the system control unit 6 determines whether or not an area in which the driver's face is captured (hereinafter referred to as a face area) can be extracted by the latest process in S310 described below. If the driver's face area could not be extracted, the system control unit 6 sets the entire area of the image indicated by the image data acquired in S210 as a recognition range in S230, and S290. Migrate to

  On the other hand, if the driver's face area can be extracted, the system control unit 6 sets the recognition range set in the latest processing in S330 described later as the recognition range in S240.

  In S250, the system control unit 6 determines that at least one of the X-axis direction vibration amount, the Y-axis direction vibration amount, and the Z-axis direction vibration amount stored in the RAM 33 in the process of S10 is set in advance. It is determined whether or not the value is greater than or equal to the value. Here, when all of the X-axis direction vibration amount, the Y-axis direction vibration amount, and the Z-axis direction vibration amount are less than the set determination value, the system control unit 6 proceeds to S290.

  On the other hand, if at least one of the X-axis direction vibration amount, the Y-axis direction vibration amount, and the Z-axis direction vibration amount is equal to or greater than the set determination value, the system control unit 6 determines based on the X-axis direction vibration amount in S260. To set the recognition range.

Specifically, the system control unit 6 first calculates the X-axis pixel adjustment amount APx by the following equation (4). Α in the following equation (4) is an adjustment coefficient for converting the X-axis vibration average value Bx _ave into the number of pixels for adjusting the recognition range.

APx = α × Bx _ave (4)
As shown in FIG. 6, the lower limit value and the upper limit value of the X axis recognition range in the recognition range set in S240 are set as an X axis lower limit value Px1 and an X axis upper limit value Px2, respectively. Further, the lower limit value and the upper limit value of the adjusted X-axis recognition range are set as an X-axis lower limit value Px1 ′ and an X-axis upper limit value Px2 ′, respectively. The X axis recognition range is a range in the X axis direction in the recognition range.

In S260, the system control unit 6 sets the X-axis recognition range by setting the X-axis lower limit value Px1 ′ and the X-axis upper limit value Px2 ′ by the following expressions (5) and (6).
Px1 ′ = Px1−APx (5)
Px2 ′ = Px2 + APx (6)
When the process of S260 ends, as shown in FIG. 5, the system control unit 6 sets a recognition range based on the Y-axis direction vibration amount in S270.

Specifically, the system control unit 6 first calculates the Y-axis pixel adjustment amount APy by the following equation (7). Β in the following equation (7) is an adjustment coefficient for converting the Y-axis vibration average value By _ave into the number of pixels for adjusting the recognition range.

APy = β × By _ave (7)
As shown in FIG. 6, the lower limit value and the upper limit value of the Y-axis recognition range in the recognition range set in S240 are set as a Y-axis lower limit value Py1 and a Y-axis upper limit value Py2, respectively. Further, the lower limit value and the upper limit value of the adjusted Y-axis recognition range are set as an X-axis lower limit value Py1 ′ and an X-axis upper limit value Py2 ′, respectively. The Y-axis recognition range is a range in the Y-axis direction in the recognition range.

In S270, the system control unit 6 sets the Y-axis recognition range by setting the Y-axis lower limit value Py1 ′ and the Y-axis upper limit value Py2 ′ using the following equations (8) and (9).
Py1 ′ = Py1−APy (8)
Py2 ′ = Py2 + APy (9)
When the processing of S270 ends, as shown in FIG. 5, the system control unit 6 sets a recognition range based on the Z-axis direction vibration amount in S280.

Specifically, the system control unit 6 first calculates the Z-axis pixel adjustment amount APz by the following equation (10). Γ in the following equation (10) is an adjustment coefficient for converting the Z-axis vibration average value Bz _ave into the number of pixels for adjusting the recognition range.

APz = γ × Bz _ave (10)
As shown in FIG. 7, the four vertices in the recognition range adjusted in S260 and S270 are designated as vertices VX1, VX2, VX3, and VX4, respectively. Note that the side connecting the vertex VX1 and the vertex VX2 is the lower side, the side connecting the vertex VX2 and the vertex VX3 is the right side, the side connecting the vertex VX3 and the vertex VX4 is the upper side, and the side connecting the vertex VX4 and the vertex VX1 is the left side To do. A line segment connecting the vertex VX1 and the vertex VX3 is a diagonal line DL1, and a line segment connecting the vertex VX2 and the vertex VX4 is a diagonal line DL2.

  A line segment extending from the vertex VX1 to the outside of the recognition range so as to be parallel to the diagonal line DL1 and having a length of APz is defined as an extension line EL1. A line segment extending outward from the recognition range so as to be parallel to the diagonal line DL2 from the vertex VX2 and having a length of APz is defined as an extension line EL2. A line segment extending outward from the recognition range so as to be parallel to the diagonal line DL1 from the vertex VX3 and having a length of APz is defined as an extension line EL3. A line segment extending outward from the recognition range so as to be parallel to the diagonal line DL2 from the vertex VX4 and having a length of APz is defined as an extension line EL4.

  Further, a point different from the vertex VX1 among both ends of the extension line EL1 is defined as a vertex VX1 '. Of both ends of the extension line EL2, a point different from the vertex VX2 is defined as a vertex VX2 '. Of both ends of the extension line EL3, a point different from the vertex VX3 is defined as a vertex VX3 '. Of both ends of the extension line EL4, a point different from the vertex VX4 is defined as a vertex VX4 '.

In S280, the system control unit 6 sets the rectangles having the vertices VX1 ′, VX2 ′, VX3 ′, and VX4 ′ as vertices as the recognition range.
When the process of S280 ends, the system control unit 6 proceeds to S290 as shown in FIG. In S290, the system control unit 6 determines whether or not the number of pixels included in the recognition range set in the processes of S230 to S280 is equal to or greater than a preset reduction determination value. Here, when the number of pixels included in the recognition range is less than the reduction determination value, the system control unit 6 proceeds to S310.

  On the other hand, if the number of pixels included in the recognition range is equal to or greater than the reduction determination value, the system control unit 6 uses pixels in the processing of S310 and S340 described later among the pixels included in the recognition range in S300. The number of pixels (hereinafter, recognition target pixels) is reduced, and the process proceeds to S310. For example, it is assumed that the recognition range includes 640 pixels along the X-axis direction and 480 pixels along the Y-axis direction. In this case, the system control unit 6 includes, for example, 120 recognition target pixels along the X-axis direction and 120 recognition target pixels along the Y-axis direction by thinning out pixels included in the recognition range. To include.

  When the process proceeds to S310, the system control unit 6 performs well-known image recognition for all areas of the image indicated by the image data acquired in S210 and included in the recognition range set in the processes of S230 to S280. By performing the processing, the driver's face area is extracted.

  Furthermore, in S320, the system control unit 6 determines whether or not the driver's face area has been extracted in S310. Here, if the driver's face area cannot be extracted, the system control unit 6 once ends the driver monitoring process.

  On the other hand, if the driver's face area can be extracted, the system control unit 6 resets the recognition range in S330. Specifically, the system control unit 6 first calculates a rectangular area that circumscribes the face area extracted in S310. A rectangular area Rr1 shown in FIG. 2 is a rectangular area that circumscribes the face area of the driver's face DF. Next, the system control unit 6 sets a rectangular area including a rectangular area circumscribing the face area as a recognition range. A rectangular area Rr2 shown in FIG. 2 is a rectangular area including a rectangular area Rr1 circumscribing the face area.

  When the processing of S330 is completed, as shown in FIG. 5, the system control unit 6 performs image analysis on the extracted facial region image in S340, thereby enabling the feature amount of the driver's face (for example, Eyebrow, eyelid and mouth movements).

  Next, in S350, the system control unit 6 determines the degree of sleepiness (hereinafter, sleepiness level) of the driver based on the feature amount extracted in S340. Specifically, in S350, the system control unit 6 first determines a facial expression pattern corresponding to the combination of feature amounts extracted in S340 by referring to the facial expression setting table stored in the ROM 32. In the facial expression setting table, a correspondence relationship between the combination of the facial feature amount of the driver and the facial expression pattern is set. In S350, the system control unit 6 further determines the sleepiness level corresponding to the determined facial expression pattern by referring to the sleepiness level setting table stored in the ROM 32. In the sleepiness level setting table, the correspondence between the expression pattern and the sleepiness level is set. In the present embodiment, the sleepiness level is set as an integer of 0 or more. The greater the sleepiness level, the stronger the sleepiness.

  Thereafter, in S360, the system control unit 6 determines whether or not the drowsiness level determined in S350 is equal to or higher than a preset warning determination value. Here, when it is determined that the drowsiness level is less than the warning determination value, the system control unit 6 once ends the driver monitoring process. On the other hand, if it is determined that the drowsiness level is equal to or higher than the warning determination value, the system control unit 6 outputs a warning sound from the speaker 5 in S370 and temporarily ends the driver monitoring process.

  The system control unit 6 configured as described above first acquires image data indicating a captured image captured by the camera 2 from the camera 2 that continuously captures the face DF of the driver sitting in the driver's seat of the vehicle. To do. Then, the system control unit 6 sets the recognition range so as to include a face area in which the driver's face is shown in the captured image indicated by the acquired image data.

  Further, the system control unit 6 acquires a vibration voltage signal indicating the magnitude of vibration from the vibration sensor 4 that detects the magnitude of vibration generated in the vehicle. Furthermore, the system control unit 6 expands the set recognition range according to the magnitude of vibration indicated by the acquired vibration voltage signal than before the vibration sensor 4 detects vibration.

And the system control part 6 performs the process which determines a drowsiness level based on the driver | operator's face included in the recognition range.
Thus, the system control unit 6 extends the recognition range according to the magnitude of vibration generated in the vehicle. For this reason, the system control unit 6 suppresses the occurrence of a situation in which the driver's face is out of the recognition range due to the vibration of the driver's face due to vibration and the driver's face cannot be recognized. Can do. Thereby, the system control part 6 can suppress the fall of the precision which recognizes a driver | operator.

  The system control unit 6 extends the recognition range along the X-axis direction of the recognition range according to the magnitude of the component in the X-axis direction of the vibration. Similarly, the system control unit 6 extends the recognition range along the Y-axis direction of the recognition range according to the magnitude of the component of the vibration in the Y-axis direction. Thereby, the system control part 6 can expand a recognition range only in the direction where a driver | operator's face shakes and moves due to a vibration. For this reason, the system control unit 6 avoids unnecessarily extending the recognition range in a direction in which the driver's face does not move due to vibration, and extracts the driver's face region within the recognition range. An increase in processing load of processing can be suppressed to the minimum necessary.

Further, the system control unit 6 uses the imaging timing n times before for the time-series data of the difference between the maximum value Bx (k) max and the minimum value Bx (k) min of the X-axis direction vibration amount in the k-th imaging time range. A moving average within a period from the current imaging timing is calculated as an X-axis vibration average value Bx _ave . k is an integer that is 1 or more and n or less.

In addition, the system control unit 6 sets the imaging timing of n times before the time-series data of the difference between the maximum value By (k) max and the minimum value By (k) min of the Y-axis direction vibration amount in the k-th imaging time range. The moving average within the period from the current imaging timing is calculated as the Y-axis vibration average value By _ave .

In addition, the system control unit 6 uses the imaging timing n times before for the time series data of the difference between the maximum value Bz (k) max and the minimum value Bz (k) min of the Z-axis direction vibration amount in the k-th imaging time range. A moving average within a period between the current photographing timing and the Z-axis vibration average value Bz _ave is calculated.

Then, the system control unit 6 extends the recognition range according to the calculated X-axis vibration average value Bx _ave , Y-axis vibration average value By _ave, and Z-axis vibration average value Bz _ave .
In this way, the system control unit 6 extends the recognition range using a value obtained by averaging the magnitude of vibration over a plurality of imaging timings. For this reason, the system control unit 6 can smooth the time variation of the parameter for determining the expansion amount of the recognition range so that the recognition range does not change abruptly.

  Further, the system control unit 6 determines whether or not the number of pixels included in the recognition range is equal to or greater than a preset reduction determination value. When the system control unit 6 determines that the number of pixels included in the recognition range is equal to or greater than the reduction determination value, the system control unit 6 reduces the number of recognition target pixels in the recognition range. Thereby, the system control part 6 can suppress that the processing load of the process which extracts a driver | operator's face area within a recognition range increases due to extending a recognition range.

  The vibration sensor 4 is installed in the driver's seat. Thereby, the vibration sensor 4 is installed in the vicinity of the driver. Therefore, the system control unit 6 can accurately estimate the shake of the driver's face based on the detection result of the vibration sensor 4, and appropriately expand the recognition range based on the detection result of the vibration sensor 4. can do.

In the embodiment described above, the system control unit 6 corresponds to a driver monitoring device, the camera 2 corresponds to a photographing unit, and the vibration sensor 4 corresponds to a vibration detection unit.
S210 corresponds to processing as an image acquisition unit, S330 corresponds to processing as an area setting unit, and S10 corresponds to processing as a vibration acquisition unit.

S260 to S280 correspond to processing as an extension unit, and S310, S320, S340, and S350 correspond to processing as a monitoring unit.
The recognition range corresponds to a monitoring area, the vibration voltage signal corresponds to vibration amount information, and S310, S320, S340, and S350 correspond to monitoring processing.

Further, the X-axis direction and the Y-axis direction in the recognition range correspond to the extension direction, S20 to S60 correspond to processing as an average calculation unit, and the period between two adjacent photographing timings corresponds to a sample period. The period between the imaging timing n times before and the current imaging timing corresponds to an average calculation period, and the X-axis vibration average value Bx _ave , the Y-axis vibration average value By _ave, and the Z-axis vibration average value Bz _ave are vibrations. Corresponds to the average value.

  Further, S290 corresponds to processing as a high load determination unit, S300 corresponds to processing as a pixel reduction unit, a determination condition in S290 corresponds to a high load condition, and a recognition target pixel is a pixel used for monitoring processing. Equivalent to.

As mentioned above, although one embodiment of this indication was described, this indication is not limited to the above-mentioned embodiment, and can carry out various modifications.
[Modification 1]
For example, in the above embodiment, as shown in FIG. 8, the X-axis pixel adjustment amount APx is the same on the right side and the left side of the recognition range. That is, the adjustment coefficient α is the same on the right side and the left side of the recognition range. As shown in FIG. 8, since the camera 2 is installed so as to photograph the driver's face DF from the front, the camera photographing angle θ ₁ corresponding to the vibration range on the left side of the recognition range, This is because the shooting angle θ _{2 of the} camera corresponding to the vibration range on the right side of the recognition range becomes equal.

On the other hand, when the camera 2 is installed on the right or left side with respect to the driver's face DF, the X-axis pixel adjustment amount APx may be different between the left side and the right side of the recognition range. FIG. 9 shows that the camera 2 is installed on the left side with respect to the driver's face DF. As shown in FIG. 9, when the camera 2 is installed on the left side of the driver's face DF, the shooting angle θ ₁ of the camera 2 corresponding to the vibration range on the left side of the recognition range is the right side of the recognition range. It becomes larger than the shooting angle θ ₂ of the camera 2 corresponding to the vibration range. Therefore, if the adjustment coefficient corresponding to the left side of the recognition range is α ₁ and the adjustment coefficient corresponding to the right side of the recognition range is α ₂ , the adjustment coefficient may be set so that α ₁ > α ₂ . Thereby, the X-axis pixel adjustment amount APx on the left side of the recognition range becomes larger than the X-axis pixel adjustment amount APx on the right side of the recognition range. In this way, the system control unit 6 recognizes with an appropriate pixel adjustment amount according to the amount of movement of the driver's face DF left and right within the recognition range due to vibration in the X-axis direction. The range can be expanded.

Similarly, regarding the upper side and the lower side of the recognition range, the adjustment coefficient β may be made different between the upper side and the lower side of the recognition range according to the installation position of the camera 2. That is, when the camera 2 is installed so as to be biased upward or downward with respect to the driver's face DF, the Y-axis pixel adjustment amount APy may be different between the upper side and the lower side of the recognition range. For example, when the camera 2 is installed below the driver's face DF, the shooting angle of the camera 2 corresponding to the vibration range below the recognition range corresponds to the vibration range above the recognition range. It becomes larger than the shooting angle of the camera 2. Therefore, if the adjustment coefficient corresponding to the lower side of the recognition range is β ₁ and the adjustment coefficient corresponding to the upper side of the recognition range is β ₂ , the adjustment coefficient may be set so that β ₁ > β ₂ . Thereby, the Y-axis pixel adjustment amount APy on the lower side of the recognition range becomes larger than the Y-axis pixel adjustment amount APy on the upper side of the recognition range. In this way, the system control unit 6 can adjust the pixel adjustment amount according to the amount of movement of the driver's face DF downward and upward within the recognition range due to vibration in the Y-axis direction. The recognition range can be expanded.

[Modification 2]
In the said embodiment, the form in which the vibration sensor 4 was installed in the seat part of the driver's seat was shown. However, the installation position of the vibration sensor 4 is not limited to the seat. However, it is desirable to install in the vicinity of the driver.

[Modification 3]
In the above-described embodiment, the mode in which it is determined whether or not the number of pixels included in the recognition range is equal to or greater than a preset reduction determination value is not limited to this, and the processing load is high. Any condition may be used. For example, the time required to extract the driver's face area within the recognition range is estimated, and when the estimated time is longer than a preset reduction determination time, the number of recognition target pixels may be reduced. Good.

[Modification 4]
In the above embodiment, the controller 15 is included in the camera 2. However, the controller 15 may be installed outside the camera 2.

[Modification 5]
In the above embodiment, the mode in which the period between two adjacent imaging timings is the sample period is shown. However, the present invention is not limited to this. For example, the sample period is set at a predetermined time interval. There may be.

[Modification 6]
In the above-described embodiment, the warning sound is output from the speaker 5. However, the present invention is not limited to this. For example, an actuator that gives the driver some kind of stimulus to promote awakening is operated. Also good.

  In addition, the function of one component in the above embodiment may be shared by a plurality of components, or the function of a plurality of components may be exhibited by one component. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified from the wording described in the claims are embodiments of the present disclosure.

  In addition to the system control unit 6 described above, a system including the system control unit 6 as a constituent element, a program for causing a computer to function as the system control unit 6, a medium on which the program is recorded, a driver monitoring method, and the like The present disclosure can also be realized in the form.

  2 ... Camera, 4 ... Vibration sensor, 6 ... System controller

Claims (7)

An image acquisition unit (S210) configured to acquire image data indicating a captured image captured by the imaging unit from an imaging unit (2) that continuously captures the face of the driver sitting in the driver's seat of the vehicle. )When,
The monitor image for monitoring the driver is set so as to include a face area in which the driver's face is reflected in the captured image indicated by the image data acquired by the image acquisition unit. The area setting unit (S330),
A vibration acquisition unit (S10) configured to acquire vibration amount information indicating the magnitude of the vibration from a vibration detection unit (4) that detects the magnitude of vibration generated in the vehicle;
According to the magnitude of the vibration indicated by the vibration amount information acquired by the vibration acquisition unit, the monitoring region set by the region setting unit is expanded more than before the vibration detection unit detects the vibration. An extension unit (S260 to S280) configured to:
A driver monitoring device (6) comprising: a monitoring unit (S310, S320, S340, S350) configured to execute a monitoring process for monitoring the driver's face included in the monitoring region. The driver monitoring device according to claim 1,
The extension is
A driver monitoring device that expands the monitoring region in accordance with a magnitude of a component of the vibration in the expansion direction along a predetermined expansion direction. The driver monitoring device according to claim 1 or 2, wherein
The difference between the maximum value and the minimum value of the vibration magnitude within a preset sample period is calculated as a vibration fluctuation amount, and the time series data of the vibration fluctuation amount is set to be longer than the sample period. An average calculation unit (S20 to S60) configured to calculate a moving average of the vibration fluctuation amount within the average calculation period as a vibration average value;
The extension is
A driver monitoring device that expands the monitoring area according to the magnitude of the vibration average value calculated by the average calculation unit. It is a driver | operator monitoring apparatus of any one of Claims 1-3,
The extension is
When the photographing unit is installed on the right side or the left side with respect to the driver's face, the right side expansion amount expands toward the right side of the monitoring area, and expands toward the left side of the monitoring area. A driver monitoring device that expands the monitoring region so that the left side expansion amount is different. It is a driver | operator monitoring apparatus of any one of Claims 1-3,
The extension is
When the photographing unit is installed on the upper side or the lower side with respect to the driver's face, the upper extension amount that extends toward the upper side of the monitoring area, and the lower side of the monitoring area. A driver monitoring device that extends the monitoring area so that a lower expansion amount to be expanded is different. It is a driver | operator monitoring apparatus of any one of Claims 1-5,
The monitoring unit
A high load determination unit (S290) configured to determine whether or not a preset high load condition indicating that the processing load of the monitoring process is high is satisfied;
A pixel reduction unit (S300) configured to reduce the number of pixels used for the monitoring process in the monitoring region when the high load determination unit determines that the high load condition is satisfied; Driver monitoring device provided. The driver monitoring device according to any one of claims 1 to 6,
The vibration detection unit is a driver monitoring device installed in the driver's seat.

Images