JP6945775B2 - In-vehicle image processing device and in-vehicle image processing method - Google Patents

Description

The present invention relates to an in-vehicle image processing device that detects a driver's face based on an image, and an in-vehicle image processing method.

Conventionally, the driver's face is detected based on the image captured by the driver acquired from the image pickup device, and the image becomes more suitable for detecting the driver's face based on the detection result. A technique for controlling the optical settings of an image pickup device is known. Controlling the optical settings of the imaging device includes exposure control, gain adjustment, and the like.
For example, Patent Document 1 has a function of detecting a driver's face area from an image received from a camera, calculating an exposure time so as to obtain an image with optimum brightness in the detected face area, and instructing the CCD controller. An in-vehicle image processing apparatus having the same is disclosed.

Japanese Unexamined Patent Publication No. 2009-116742

In the conventional in-vehicle image processing device as disclosed in Patent Document 1, there is a problem that the possibility that the face of an occupant other than the driver is imaged in the image cannot be taken into consideration. When the face of an occupant other than the driver is imaged in the image, the conventional in-vehicle image processing device may erroneously detect the face of the occupant as the face of the driver. The control of the optical settings based on the falsely detected driver's face is not an appropriate control.

The present invention has been made to solve the above problems, and an object of the present invention is to prevent control of inappropriate optical settings based on a falsely detected driver's face.

The in-vehicle image processing device according to the present invention includes an image acquisition unit that acquires an image obtained by capturing an image of a range in the vehicle including a range in which the face of a driver sitting in the driver's seat should exist from the image pickup device. Based on the image acquired by the image acquisition unit, the face detection unit that detects the driver's face and the driver's face area on the image, and the average brightness of the pixels in the driver's face area detected by the face detection unit. The brightness calculation unit, the brightness determination unit that determines whether or not the average brightness calculated by the brightness calculation unit is equal to or greater than the brightness determination threshold, and the brightness determination unit have an average brightness that is less than the brightness determination threshold. When it is determined that, the face detection unit mistakes the driver's face based on the result of the control performed by the optical setting control unit that controls the optical setting of the image pickup device according to the average brightness and the optical setting control unit. When the erroneous detection determination unit that determines whether or not the detection is detected and the erroneous detection determination unit determines that the face detection unit erroneously detects the driver's face, the optical setting of the image pickup device is set with respect to the optical setting control unit. It is provided with a recontrol instruction unit that outputs a recontrol instruction for performing control to return the value to an appropriate value.

According to the present invention, it is possible to prevent improper control of optical settings based on a falsely detected driver's face.

It is a figure which shows the configuration example of the vehicle-mounted image processing apparatus which concerns on Embodiment 1. FIG. 2A and 2B are diagrams for explaining an example of an image of the installation position of the image pickup apparatus and the image pickup range in the first embodiment. FIG. 5 is a diagram for explaining an image of a method in which a face detection unit determines a driver's face in the first embodiment. It is a figure for demonstrating the image of an example of the state in the vehicle when the face detection part erroneously detects a driver's face in Embodiment 1. FIG. In the first embodiment, in the first embodiment, it is a diagram showing an image in which overexposure occurs around the driver's face region on an image. It is a figure for demonstrating the image of the face existence estimation area in Embodiment 1. FIG. 7A and 7B are diagrams for explaining an image of an example of the face detection region reduced by the region reduction portion in the first embodiment. It is a flowchart for demonstrating the operation of the vehicle-mounted image processing apparatus which concerns on Embodiment 1. FIG. In Embodiment 1, it is a flowchart for demonstrating the operation of step ST804 to step ST807 of FIG. 8 in more detail. It is a flowchart for demonstrating the operation of step ST804-step ST807 of FIG. 8 in more detail in the vehicle-mounted image processing apparatus which concerns on Embodiment 2. FIG. It is a figure which shows the configuration example of the vehicle-mounted image processing apparatus which concerns on Embodiment 3. It is a flowchart for demonstrating the operation of step ST804-step ST807 of FIG. 8 in more detail in the vehicle-mounted image processing apparatus which concerns on Embodiment 3. FIG. 13A and 13B are diagrams showing an example of the hardware configuration of the vehicle-mounted image processing device according to the first to third embodiments.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing a configuration example of an in-vehicle image processing device 10 according to the first embodiment.
The in-vehicle image processing device 10 is mounted on a vehicle (not shown).
The in-vehicle image processing device 10 is connected to the image pickup device 20 and the lighting device 30.
In the first embodiment, it is assumed that the image pickup device 20 is shared with the image pickup device included in the so-called "driver monitoring system" mounted on the vehicle for monitoring the state of the driver in the vehicle. Note that this is only an example, and the image pickup device 20 is a camera or the like installed in the vehicle for the purpose of monitoring the inside of the vehicle, and at least the face of the driver sitting in the driver's seat should be present. It suffices as long as it is installed so that the range in the vehicle including the range can be imaged. The range in which the face of the driver seated in the driver's seat should exist is, for example, a range corresponding to the space near the front of the headrest of the driver's seat. The image pickup apparatus 20 may be, for example, a visible light camera or an infrared camera.
2A and 2B are diagrams for explaining an example of an image of the installation position and the imaging range of the imaging device 20 in the first embodiment. Note that FIGS. 2A and 2B are views showing an image of the interior of the vehicle as viewed from above. In the first embodiment, as shown in FIGS. 2A and 2B, it is assumed that an occupant other than the driver is also in the vehicle, and in FIGS. 2A and 2B, the driver is 21a and the rear seat occupant. Is shown by 21b, and the passenger in the passenger seat is shown by 21c.
In the first embodiment, for example, as shown in FIG. 2A, the image pickup apparatus 20 is installed in the center of the instrument panel of the vehicle (hereinafter referred to as “instrument panel”) and images the driver's seat side from the center of the instrument panel. Is assumed.
Further, in the first embodiment, it is assumed that the image pickup device 20 is installed in the center of the instrument panel of the vehicle and images the driver's seat and the passenger seat from the center of the instrument panel, for example, as shown in FIG. 2B. ..
In the following description, it is assumed that the imaging device 20 is installed so as to have an installation position and an imaging range as shown in FIG. 2A.
The lighting device 30 is a light emitter that emits illumination light, such as an LED (Light Emitting Diode). The lighting device 30 is installed in the vicinity of the image pickup device 20. In the first embodiment, it is assumed that the lighting device 30 is mounted on the image pickup device 20. In addition, in FIG. 2A and FIG. 2B, the illustration of the lighting device 30 is omitted.

The in-vehicle image processing device 10 acquires an image from the image pickup device 20, detects the driver's face and the driver's face area on the image, and takes an image according to the average brightness of the pixels in the face area. Controls the optical settings of the device 20. The face area is a predetermined area on the image that is set based on the detection position on the image of the driver's face part. The details of the face area will be described later.
When the in-vehicle image processing device 10 controls the optical settings of the image pickup device 20, it determines whether or not the driver's face is erroneously detected based on the result of the control (hereinafter, "erroneous detection determination"). That.). When it is determined that the driver's face is erroneously detected in the erroneous detection determination, the in-vehicle image processing device 10 controls to set the optical setting of the image pickup apparatus 20 to an appropriate value. The control for setting the optical setting of the image pickup apparatus 20 to an appropriate value is, for example, a control for setting the optical setting of the image pickup apparatus 20 to a default state. In the first embodiment, when it is determined that the driver's face is erroneously detected, the in-vehicle image processing device 10 controls to set the optical setting of the image pickup device 20 to the default state. The in-vehicle image processing device 10 then re-detects the driver's face based on the image acquired from the image pickup device 20 in which the optical setting is set to the initial state.
In the first embodiment, the optical setting of the image pickup device 20 means, for example, the exposure time of the image pickup device 20, the lighting time of the lighting device 30, or the gain value of the image pickup device 20. To set the optical setting to the default state, set the exposure time to a preset initial value, set the lighting time to a preset initial value, or set the gain value to a preset initial value. Means. Each initial value is set to a value that is assumed to be sufficient to clearly image the driver's face under certain predetermined conditions.
Further, when the vehicle-mounted image processing device 10 determines that the driver's face is not erroneously detected in the erroneous detection determination, the vehicle-mounted image processing device 10 monitors the driver's state based on the detected driver's face and monitors the driver's condition. Processes such as outputting an alarm are performed according to the state of.
As a result, the vehicle-mounted image processing device 10 prevents the optical setting of the image pickup device 20 from being controlled according to the average brightness of the pixels in the face region including the face of the occupant other than the driver. Alternatively, the vehicle-mounted image processing device 10 prevents the output of an alarm or the like that does not depend on the state of the driver.

As shown in FIG. 1, the in-vehicle image processing device 10 includes an image acquisition unit 101, a face detection unit 102, a driver monitoring unit 103, a brightness calculation unit 104, a brightness determination unit 105, an optical setting control unit 106, and an erroneous detection determination. A unit 107, a recontrol instruction unit 108, an area reduction unit 109, and a redetection instruction unit 110 are provided.
The optical setting control unit 106 includes an exposure control unit 1061 and an image processing unit 1062.

The image acquisition unit 101 acquires an image obtained by capturing an image of a range in the vehicle including a range in which the face of the driver sitting in the driver's seat should exist from the image pickup device 20.
The image pickup apparatus 20 converts the light acquired from the inside of the vehicle into an electric signal, and generates image data based on the electric signal. The image pickup apparatus 20 outputs the generated image data to the image acquisition unit 101. The image acquisition unit 101 acquires the image data output from the image pickup apparatus 20.
Alternatively, the image pickup device 20 may convert the light acquired from the inside of the vehicle into an electric signal and output it to the image acquisition unit 101. In this case, the image acquisition unit 101 converts the electric signal output from the image pickup apparatus 20 to acquire image data.
Here, for example, as described above, the image acquisition unit 101 acquires the image data output from the image pickup device 20, or the image acquisition unit 101 converts the electrical signal output from the image pickup device 20 to form an image. The image acquisition unit 101 "acquires an image from the image pickup apparatus 20" including acquiring data.
The image acquisition unit 101 outputs the acquired image to the face detection unit 102 and the brightness calculation unit 104.

The face detection unit 102 detects the driver's face and the driver's face region on the image based on the image acquired by the image acquisition unit 101. Specifically, for example, the face detection unit 102 detects the driver's face by detecting the face parts in the face detection area set on the image. The face detection unit 102 may detect a face part by using a known image recognition technique. The face detection area is an area on the image where the face detection unit 102 detects the driver's face, and the initial value of the face detection area is set in advance by, for example, the user. ing. Then, information regarding the face detection area is stored in a storage unit (not shown). The face detection area is defined in the information about the face detection area, and the information about the face detection area includes, for example, coordinates for specifying a frame indicating the face detection area on the image. Information that can identify the face detection area is included. The face detection unit 102 identifies the face detection area with reference to the storage unit, and detects the driver's face.
The face detection unit 102 also detects the driver's face area. In the first embodiment, for example, among the facial parts, the position of the eyebrows is the upper end of the face, the position of the mouth is the lower end of the face, and the ends of the left and right eyes on the outer corners of the eyes are the left and right edges of the face, respectively. It is set as a rectangle passing through the upper end of the face, the lower end of the face, the left end of the face, and the right end of the face. Further, the face area may be set as a rectangle obtained by expanding the rectangle passing through the upper end of the face, the lower end of the face, the left end of the face, and the right end of the face at a preset ratio or the like as described above.
In the following description, when the face detection unit 102 detects the driver's face, detecting the driver's face also includes detecting the driver's face area. ..

When a plurality of faces are detected in the face detection area, the face detection unit 102 determines the face with the highest reliability as the driver's face based on the reliability of the detected faces.
In the first embodiment, the face detection unit 102 sets, for example, the face having the largest face region as the face with the highest reliability.

Here, FIG. 3 is a diagram for explaining an image of a method in which the face detection unit 102 determines the driver's face in the first embodiment.
In FIG. 3, the image acquired by the image acquisition unit 101 is shown by 200, and the face detection area on the image 200 is shown by 201.

Now, it is assumed that the face detection unit 102 has detected two faces, the first face 202a and the second face 202b, in the face detection region 201 on the image 200. Further, the face detection unit 102 includes a face region of the first face 202a (hereinafter referred to as "first face region 203a") and a face region of the second face 202b (hereinafter referred to as "second face region 203b"). ) Is detected.
In this case, the face detection unit 102 has the highest reliability of the first face 202a corresponding to the first face area 203a, which is the larger of the first face area 203a and the second face area 203b. Judge it as a face and decide on the driver's face. That is, the face detection unit 102 detects the first face 202a as the driver's face among the detected first face 202a and the second face 202b.

For example, if the face detection area is set wide enough to detect the driver's face even if the driver's face moves to various positions, the face detection area is shown in, for example, FIG. As such, the face of an occupant sitting in the back seat may also be included. When the face detection area includes the face of an occupant sitting in the back seat, the face detection unit 102 sets the second face 202b, which is the occupant's face, as the first face 202a, which is the driver's face. It may also be detected.
Normally, the driver is sitting closer to the image pickup device 20, and the driver's face is larger in the image. Therefore, even if the face detection unit 102 detects both the second face 202b, which is the face of the occupant, and the first face 202a, which is the face of the driver, the face detection unit 102 has a larger face area. The first face 202a is determined to be the most reliable face, and the first face 202a is determined to be the driver's face.

Here, as an example, the face detection unit 102 determines the reliability of the detected face based on the size of the face region, but this is only an example. The face detection unit 102 may determine the reliability of the face according to the high reliability of the detected face parts, for example. The reliability of facial parts is, for example, the degree to which the eyes are as eye-catching or the nose is as nose-like. The face detection unit 102 may calculate the reliability of the face parts by using a known image recognition technique such as pattern matching.

The face detection unit 102 outputs the detected information about the driver's face (hereinafter referred to as “face information”) to the driver monitoring unit 103, the brightness calculation unit 104, and the erroneous detection determination unit 107. The driver's face information includes information about the driver's face parts and information about the driver's face area. More specifically, the information about the driver's face part is, for example, information in which the information indicating the driver's face part is associated with the coordinates on the image indicating the position of the part. .. Further, the information regarding the driver's face area is, for example, coordinates on an image for specifying a frame indicating the driver's face area. The face detection unit 102 may combine the images output from the image acquisition unit 101 and output the images to the driver monitoring unit 103, the brightness calculation unit 104, and the erroneous detection determination unit 107.
Further, the face detection unit 102 stores the detected face information of the driver in the storage unit.

Returning to the description of FIG.
When the face detection presence information is output from the erroneous detection determination unit 107, the driver monitoring unit 103 monitors the driver's state based on the face information output from the face detection unit 102. In the first embodiment, the state of the driver monitored by the driver monitoring unit 103 refers to the state of the driver, such as falling asleep or looking aside, which is not concentrated on driving and may interfere with driving. The erroneous detection determination unit 107 will be described later.

The driver monitoring unit 103 monitors whether the driver is dozing or looking aside, for example, based on the driver's eye opening rate or the angle of the nose. Since the technique for calculating the eye opening rate or the angle of the nose muscle based on the information on the facial parts is a known technique, detailed description thereof will be omitted. The driver monitoring unit 103 may monitor whether the driver is in a state where he / she may interfere with driving.
When the driver monitoring unit 103 determines that the driver is in a state where the driving may be hindered, the driver monitoring unit 103 outputs the alarm output instruction information to the alarm output control unit (not shown).
When the alarm output instruction information is output from the driver monitoring unit 103, the alarm output control unit causes an output device (not shown) to output an alarm. The output device is, for example, an audio output device provided in a vehicle and is connected to an in-vehicle image processing device 10. The alarm output control unit causes the output device to output, for example, an alarm sound.

The brightness calculation unit 104 calculates the average brightness of the pixels in the driver's face region (hereinafter referred to as “driver's face region pixels”) detected by the face detection unit 102 in the image acquired by the image acquisition unit 101.
The brightness calculation unit 104 outputs the calculated information on the average brightness of the driver's face area pixels (hereinafter referred to as “face area brightness information”) to the brightness determination unit 105.

The brightness determination unit 105 determines whether or not the average brightness of the driver's face area pixels is equal to or greater than a threshold value (hereinafter referred to as “luminance determination threshold”) based on the face area brightness information output from the brightness calculation unit 104. do. The luminance determination threshold is assumed to be the operation when the driver's face is appropriately detected from an image captured in advance under standard light, for example, at the time of product shipment. Based on the average brightness of the pixels in the face area of the person, for example, the lowest average brightness value that can be assumed as the average brightness of the pixels in the face area of the driver is set.
When the average brightness of the driver's face area pixels is less than the brightness determination threshold value, the brightness determination unit 105 provides information to the effect that the average brightness of the driver's face area pixels is less than the brightness determination threshold value. The output is output to the exposure control unit 1061 of the 106 or the image processing unit 1062 of the optical setting control unit 106.
Specifically, the luminance determination unit 105 first outputs information to the exposure control unit 1061 that the average luminance of the driver's face region pixels is less than the luminance determination threshold. In response to this, the exposure control unit 1061 performs exposure control. The details of the exposure control unit 1061 will be described later. When the exposure control unit 1061 performs the exposure control, the brightness determination unit 105 determines whether or not the average brightness of the driver's face region pixels after the exposure control is equal to or greater than the brightness determination threshold value. The brightness determination unit 105 may acquire the average brightness of the driver region pixels after exposure control by having the brightness calculation unit 104 calculate the average brightness.
When the average brightness of the driver's face area pixels after exposure control is less than the brightness determination threshold value, the brightness determination unit 105 provides information that the average brightness of the driver's face area pixels is less than the brightness determination threshold value. Output to the image processing unit 1062. The image processing unit 1062 adjusts the gain. The details of the image processing unit 1062 will be described later. When the image processing unit 1062 adjusts the gain, the brightness determination unit 105 determines whether or not the average brightness of the driver's face region pixels after the gain adjustment is equal to or greater than the brightness determination threshold value. The brightness determination unit 105 may acquire the average brightness of the driver region pixels after the gain adjustment by having the brightness calculation unit 104 calculate the average brightness.
In this way, the brightness determination unit 105 changes the optical settings in the order of the exposure control unit 1061 and the image processing unit 1062.

When the brightness determination unit 105 determines that the average brightness of the driver's face area pixels is equal to or greater than the brightness determination threshold value, the brightness determination unit 105 determines that the average brightness of the driver's face area pixels is the brightness with respect to the optical setting control unit 106. Does not output information that the value is less than the threshold value.

When the brightness determination unit 105 determines that the average brightness of the driver's face region pixels is less than the brightness determination threshold value, the optical setting control unit 106 sets the average brightness of the driver's face region pixels calculated by the brightness calculation unit 104. Accordingly, the optical setting of the image pickup apparatus 20 is controlled.
The exposure control unit 1061 of the optical setting control unit 106 executes exposure control when the brightness determination unit 105 outputs information that the average brightness of the driver's face region pixels is less than the brightness determination threshold value. Specifically, the exposure control unit 1061 changes the exposure time of the image pickup apparatus 20 to be longer. Further, the exposure control unit 1061 changes the lighting time of the lighting device 30 to be longer according to the changed exposure time, if necessary. How long the exposure control unit 1061 changes the exposure time or the lighting time is determined in advance according to the average brightness of the driver's face region pixels.

When the brightness determination unit 105 outputs information that the average brightness of the driver's face region pixels after exposure control is less than the brightness determination threshold value, the image processing unit 1062 of the optical setting control unit 106 adjusts the gain. conduct. Specifically, the image processing unit 1062 raises the gain value of the image pickup apparatus 20 to brighten the image. How much the image processing unit 1062 raises the gain value is determined in advance according to the average brightness of the driver's face region pixels.

Further, the optical setting control unit 106 controls to return the optical setting of the image pickup apparatus 20 to the initial value when the recontrol instruction is output from the recontrol instruction unit 108. The details of the recontrol instruction unit 108 will be described later.
When the recontrol instruction is output, the exposure control unit 1061 performs exposure control by returning the exposure time or the exposure time and the lighting time to the initial values and setting the default state.
When the recontrol instruction is output, the image processing unit 1062 adjusts the gain value by returning it to the initial value and setting it as the default state.

The erroneous detection determination unit 107 determines whether or not the face detection unit 102 erroneously detects the driver's face, that is, erroneous detection determination, based on the result of the control performed by the optical setting control unit 106. Specifically, when the false detection determination unit 107 performs the exposure control of the exposure control unit 1061 of the optical setting control unit 106, the information about the image pickup device 20 obtained as a result of the exposure control is the face false detection. Determine if the condition is met. Further, in the false detection determination unit 107, when the image processing unit 1062 of the optical setting control unit 106 adjusts the gain, the information about the image pickup device 20 obtained as a result of the gain adjustment satisfies the face false detection condition. Judge whether or not. In the first embodiment, the information about the image pickup apparatus 20 obtained as a result of the exposure control unit 1061 performing the exposure control or the result of the image processing unit 1062 performing the gain adjustment is the information of the image acquired from the image pickup apparatus 20. It includes information on the brightness or information on the gain value of the image pickup apparatus 20.

The erroneous detection determination unit 107 determines that the face detection unit 102 erroneously detects the driver's face when the information regarding the imaging device 20 satisfies the face erroneous detection condition.
On the other hand, the erroneous detection determination unit 107 determines that the face detection unit 102 has correctly detected the driver's face when the information regarding the imaging device 20 does not satisfy the face erroneous detection condition.
That is, in the first embodiment, the face erroneous detection condition is a condition for determining whether or not the face detection unit 102 erroneously detects the driver's face.

Here, FIG. 4 is a diagram for explaining an image of an example of the state inside the vehicle when the face detection unit 102 erroneously detects the driver's face in the first embodiment.
For example, when the occupant in the vehicle is in a state as captured in the image shown in FIG. 3, the face detection unit 102 detects the first face 202a as the driver's face. Actually, since the first face 202a is the face of the driver, the face detection unit 102 can correctly detect the face of the driver.
After that, it is assumed that the driver moves his / her hand to a position in front of the driver's face when viewed from the image pickup device 20 from the state of being imaged in the image shown in FIG.
Then, as shown in FIG. 4, the face detected in the face detection area 201 is only the second face 202b, which is the face of the occupant in the rear seat. The face detection unit 102 cannot detect the part of the first face 202a, which is the driver's face that could be detected until then, but the face of the second face 202b in the face detection area 201 on the image 200. Parts can be detected. Therefore, the face detection unit 102 detects the second face 202b as the driver's face. In this way, the face detection unit 102 can erroneously detect the driver's face.

When the face detection unit 102 erroneously detects the driver's face, the optical setting control unit 106 can control the optical setting according to the average brightness of the erroneously detected driver face region pixels. Specifically, when the face detection unit 102 erroneously detects the driver's face, the brightness calculation unit 104 calculates the average brightness of the erroneously detected driver face region pixels. Then, when the brightness determination unit 105 determines that the average brightness of the erroneously detected driver face region pixels is less than the brightness determination threshold value, the optical setting control unit 106 responds to the erroneously detected driver face region pixels. Therefore, the optical setting of the image pickup apparatus 20 is controlled.
In the example described with reference to FIG. 4 above, when the face detection unit 102 erroneously detects the second face 202b as the driver's face, the optical setting control unit 106 causes the pixels of the second face region 203b. The optical setting of the image pickup apparatus 20 is controlled according to the average brightness of.

Normally, the light of the lighting device 30, for example, is harder to reach the rear seats than the driver's seat, so that the occupants of the rear seats are imaged darker in the image. Therefore, when the face detection unit 102 erroneously detects the face of the occupant in the rear seat as the driver's face, the second face 202b is darkly imaged, and therefore the second face area 203b calculated by the brightness calculation unit 104. The average brightness of is less than the brightness determination threshold. Then, the optical setting control unit 106 controls the optical setting according to the second face area 203b, which is the face area of the occupant other than the driver.
When the optical setting is controlled according to the second face region 203b, the first face region 203a, which should be originally detected as the driver's face region, is originally imaged brighter than the second face region 203b. Because it is a region, the image will be imaged brighter than necessary. Specifically, for example, when the exposure control unit 1061 of the optical setting control unit 106 performs exposure control, the average brightness of the pixels in the first face region 203a is assumed to be the average brightness of the pixels in the driver's face region. It is much larger than the average brightness. Further, for example, when the image processing unit 1062 of the optical setting control unit 106 adjusts the gain, the gain value of the first face region 203a is assumed to be sufficient to clearly image the driver's face. Will be significantly larger than. Further, for example, when the image processing unit 1062 adjusts the gain, the gain value of the first face region 203a increases, so that so-called “overexposure” occurs in the first face region. FIG. 5 is a diagram showing an image in which overexposure occurs around the driver's face region on the image acquired from the image pickup apparatus 20 in the first embodiment.

As a result, for example, even if the state in which the driver's face part cannot be detected temporarily (see FIG. 4) returns to the state in which the driver's face part can be detected (see FIG. 3), the image shows. Since the brightness of the pixels of the first face region 203a, which is the driver's face region, is not optimized, the face detection unit 102 re-detects the first face 202a, which is the driver's face, as a face. It becomes difficult. After that, the face detection unit 102 will continue to detect the second face 202b, which is the face of the occupant in the rear seat, as the driver's face.
When the face detection unit 102 continues to detect the second face 202b as the driver's face, the driver monitoring unit 103 continues to monitor the state of the second face 202b. As a result, if the driver monitoring unit 103 determines that the occupant in the rear seat is in a dozing state, an alarm is output from the output device. This can be a useless alarm, as it is not really an alarm for the driver being asleep.

Therefore, in consideration of the possibility that the face detection unit 102 may detect the face of an occupant other than the driver as the driver's face (see FIG. 4), the erroneous detection determination unit 107 causes the optical setting control unit 106 to detect the face. False positive determination is made based on the result of the controlled control.
As described above, in the false detection determination unit 107, as a result of the exposure control unit 1061 performing the exposure control or the image processing unit 1062 performing the gain adjustment, whether the information regarding the image pickup apparatus 20 satisfies the facial false positive detection condition. By determining whether or not it is, a false detection determination is made.

When the exposure control unit 1061 performs exposure control, the face erroneous detection condition is, for example, "a region in which it is more likely that the driver's face is imaged in the face detection region (hereinafter," face ". Whether or not the average brightness of the pixels in the "existence estimation region") satisfies a preset condition (hereinafter referred to as "first determination condition").

Here, the face presence estimation region in the first embodiment will be described.
FIG. 6 is a diagram for explaining an image of a face presence estimation region in the first embodiment.
As an example, FIG. 6 shows a second face 202b in which the face detection unit 102 is the face of an occupant in the rear seat because the driver's face cannot be detected temporarily as shown in FIG. Is erroneously detected as the driver's face, and the image processing unit 1062 shows the image of the driver's face range on the image 200 captured after adjusting the gain according to the second face region 203b. In FIG. 6, the face detection area 201 is not shown for the sake of simplicity. Further, in FIG. 6, the face existence estimation region is shown by 601. Further, in FIG. 6, 600 indicates a headrest.
As described above, the face presence estimation region 601 is a region in which the driver's face is more likely to be imaged in the face detection region. The face detection area 201 may be set wide enough to detect the driver's face even if the driver's face moves to various positions, for example. On the other hand, the posture of the driver during driving is often in a substantially constant state, and therefore the position of the driver's face often stays at a constant position. Therefore, when the face detection area 201 is set widely, there are a portion in the face detection region 201 in which the driver's face is relatively likely to be imaged and a portion in which the driver's face is relatively low. It becomes a state. The face presence estimation area 601 is set in consideration of such a state, and is set as a portion of the face detection area 201 where the driver's face is more likely to be imaged. Is.
The face presence estimation area 601 may be estimated by the exposure control unit 1061 from the position of the headrest 600, for example. In this case, the exposure control unit 1061 may first detect the position of the headrest by using a known image processing technique such as pattern matching. Further, in the storage unit provided in the in-vehicle image processing device 10, a driver having a standard physique sits in a standard position in the driver's seat and looks at the front in a standard posture in advance. It is assumed that information regarding the relationship between the position of the driver's face and the position of the headrest is stored. The exposure control unit 1061 can estimate the face presence estimation area 601 based on the detected position of the headrest and the information on the relationship between the position of the driver's face and the position of the headrest stored in the storage unit. ..

Further, for example, the exposure control unit 1061 divides the face detection area 201 into left and right on the image 200, and the area of the bisected area near the steering wheel may be used as the face existence estimation area. good. For example, in the case of a right-hand drive vehicle as assumed in the first embodiment, the area on the left side of the bisected range is set as the face presence estimation area 601.

Further, the exposure control unit 1061 refers to the storage unit, for example, and based on the stored face information, the exposure control unit 1061 detects a face immediately before the driver's face area currently detected by the face detection unit 102. The region may be a face presence estimation region 601. For example, when the inside of the vehicle changes from the state captured by the image shown in FIG. 3 to the state captured by the image shown in FIG. 4, the face detection unit 102 has detected it until then. The driver's face (first face 202a) is no longer detected, and the rear seat occupant's face (second face 202b) is detected as the driver's face. The exposure control unit 1061 may use the first face region 203a of the first face 202a that has been detected so far as the face presence estimation region 601.

Further, since the driver's face does not exist behind the headrest, the exposure control unit 1061 may calculate the position of the headrest based on the seat reclining amount and estimate the face presence estimation area 601.
Further, the face presence estimation area 601 considers the position of the driver's face when a driver having a standard physique sits at a standard position in the driver's seat and looks at the front in a standard posture. It may be a region on the image that is fixedly defined in advance with respect to the image 200 captured by the image pickup apparatus 20.

The exposure control unit 1061 may be capable of identifying the face presence estimation region 601 on the image by some method.

The first determination condition is, for example, that the average brightness of the pixels in the face presence estimation region 601 (hereinafter referred to as "estimated region pixels") is equal to or higher than a preset value (hereinafter referred to as "driver's seat brightness determination threshold value"). It may or may not be present. For the driver's seat brightness determination threshold value, for example, the minimum value of the brightness value that is assumed to be too bright to detect the driver's face is set.
For example, the false detection determination unit 107 erroneously detects the driver's face by assuming that the face false detection condition is satisfied when the average brightness of the estimated region pixels is equal to or higher than the driver's seat brightness determination threshold value. Judge that there is.
When the average brightness of the estimated area pixels is equal to or greater than the driver's seat brightness determination threshold value, the exposure control of the imaging device 20 depends on the average brightness of the pixels in the face area of the occupant other than the driver, who is in a darker place than the driver's seat. It is highly probable that this is done.

When the image processing unit 1062 adjusts the gain, the face erroneous detection condition is, for example, "whether or not the gain value of the image pickup apparatus 20 satisfies a preset condition (hereinafter referred to as" second determination condition "). ". The erroneous detection determination unit 107 may acquire information on the gain value of the image pickup apparatus 20 from, for example, the image processing unit 1062.
The second determination condition may be, for example, whether or not the gain value of the image pickup apparatus 20 is equal to or greater than a preset threshold value (hereinafter referred to as “gain determination threshold value”). The gain determination threshold is set to a gain value that is assumed to be too bright for a clear image of the driver's face.
For example, the false detection determination unit 107 states that when the gain value of the imaging device 20 is equal to or greater than the gain determination threshold value, the face false detection requirement is satisfied, and the face detection unit 102 falsely detects the driver's face. judge.
When the gain value of the image pickup device 20 is equal to or greater than the gain determination threshold value, the gain adjustment of the image pickup device 20 is performed according to the average brightness of the pixels in the face area of the occupant other than the driver, which is darker than the driver's seat. It is assumed that there is a high possibility that it has been damaged.

When the image processing unit 1062 adjusts the gain, the face erroneous detection condition is, for example, "a condition in which the overexposed area of the driver's face range is preset (hereinafter referred to as" third determination condition "). Whether or not it is satisfied. "
The image processing unit 1062 may determine the face presence estimation area 601 in the same manner as the method in which the exposure control unit 1061 determines the face presence estimation area 601. Since the method for determining the face presence estimation region 601 has already been described, detailed description thereof will be omitted.

The third determination condition may be, for example, whether or not the overexposure area of the face existence estimation region 601 is equal to or larger than a preset threshold value (hereinafter referred to as "overexposure determination threshold value"), or the face existence estimation. Whether or not the overexposure area of the region 601 is large may be determined. The overexposure area, which is assumed that the driver's face part cannot be detected, is set in advance in the overexposure determination threshold value.
For example, when the overexposure area of the face presence estimation region 601 is equal to or larger than the overexposure determination threshold value, the erroneous detection determination unit 107 satisfies the face erroneous detection condition, and the face detection unit 102 erroneously detects the driver's face. It is determined that it is.
When the overexposure area of the driver's face range is equal to or greater than the overexposure determination threshold value, the gain adjustment of the image pickup apparatus 20 is performed on the average brightness of the pixels of the face area of the occupant other than the driver who is in a darker place than the driver's seat. It is highly probable that this is done accordingly.

The erroneous detection determination unit 107 combines a plurality of erroneous face detection conditions as described above to determine whether or not the information regarding the image pickup apparatus 20 satisfies the erroneous face detection condition. It may be.
For example, when the exposure control unit 1061 controls the exposure and the image processing unit 1062 also adjusts the gain, the false detection determination unit 107 states that "the average brightness of the estimated area pixels satisfies the first determination condition. "Whether or not" may be determined, and "whether or not the overexposed area of the face presence estimation region 601 satisfies the third determination condition" may be determined. In the false detection determination unit 107, when the average brightness of the estimation area pixels satisfies the first determination condition and the overexposure area of the face existence estimation area 601 satisfies the third determination condition, the face detection unit 102 sets the driver's face. Is determined to have been erroneously detected.

When the erroneous detection determination unit 107 determines that the information about the image pickup device 20 satisfies the face erroneous detection condition, it is assumed that the face detection unit 102 erroneously detects the driver's face, and an area reduction instruction for reducing the face detection area is given. , Output to the area reduction unit 109. Further, the erroneous detection determination unit 107 notifies the optical setting control unit 106 of the recontrol requirement for executing the control for returning the exposure time, the lighting time, or the gain value to the initial value, the recontrol instruction unit 108. Output to.
On the other hand, when the erroneous detection determination unit 107 determines that the information regarding the image pickup device 20 does not satisfy the face erroneous detection condition, the face detection unit 102 assumes that the driver's face can be correctly detected, and operates the face detection presence information. Output to the person monitoring unit 103.

When the false detection determination unit 107 outputs a recontrol required notification, the recontrol instruction unit 108 causes the optical setting control unit 106 to control the optical setting of the image pickup apparatus 20 to return to an appropriate value. Is output. In the first embodiment, the appropriate value of the optical setting of the imaging device 20 is the initial value of the optical setting of the imaging device 20.
The recontrol instruction unit 108, for example, causes the exposure control unit 1061 to perform exposure control for returning the exposure time to the initial value based on the recontrol required notification, or the image processing unit 1062 initially performs the gain value. It is possible to specify whether or not the gain adjustment to return to the value is performed. The erroneous detection determination unit 107 shall include information on which control of exposure control or gain adjustment is to be performed in the re-control required notification.

When the area reduction instruction is output from the erroneous detection determination unit 107, the area reduction unit 109 reduces the face detection area. At this time, the area reduction unit 109 reduces the face detection area so that the area does not include the driver's face area that was erroneously detected by the face detection unit 102. The area reduction unit 109 acquires the face information output by the face detection unit 102 via the erroneous detection determination unit 107, and identifies the driver's face area erroneously detected by the face detection unit 102 from the acquired face information. Just do it.

7A and 7B are diagrams for explaining an image of an example of the face detection region reduced by the region reduction unit 109 in the first embodiment. The image 200 shown in FIGS. 7A and 7B returns from the state in which the image was captured as shown in FIG. 4 to the state in which the part of the first face 202a, which is the driver's face, is captured again. It is supposed to be.
The area reduction unit 109, for example, moves one side on the right side of the rectangle indicating the face detection area to the left to narrow the face detection area. At that time, the area reduction unit 109 narrows the face detection area until the area does not include the driver's face area detected by the face detection unit 102 (see FIG. 7A).

Further, for example, the area reduction unit 109 narrows the face detection area in the direction of the upper left apex of the lower right vertex of the rectangle indicating the face detection area while maintaining the aspect ratio of the face detection area. At that time, the area reduction unit 109 narrows the face detection area until the area does not include the driver's face area detected by the face detection unit 102 (see FIG. 7B).

The area reduction unit 109 updates, for example, the face detection area defined in the information about the face detection area stored in the storage unit to the reduced face detection area.
Then, the area reduction unit 109 outputs information to the effect that the face detection area has been reduced to the re-detection instruction unit 110.

When the re-detection instruction unit 110 outputs information indicating that the face detection area has been reduced from the area reduction unit 109, the face detection area after the area reduction unit 109 has been reduced with respect to the face detection unit 102. Inside, the driver's face is rediscovered.

The vehicle-mounted image processing device 10 according to the first embodiment has the configuration described above.
It should be noted that the vehicle-mounted image processing device 10 does not include the erroneous detection determination unit 107, and the vehicle-mounted image processing device 10 has a function of determining whether or not the face detection unit 102 erroneously detects the driver's face. It is assumed that it did not have. That is, unlike the conventional technique described above, it is not considered that the face detected from the image may be a face other than the driver.
In that case, as in the state shown in the image of FIG. 4 above, if, for example, some disturbance causes a state in which the driver's face parts cannot be detected temporarily, the face detection unit 102 operates. The face of an occupant other than the person can be detected as the face of the driver. Then, the face detection unit 102 outputs face information regarding the face of the occupant other than the driver, in which the face of the occupant other than the driver is the face of the driver, to the driver monitoring unit 103 and the brightness calculation unit 104. ..

As described above, since it is harder for the light of the lighting device 30, for example, to reach the rear seats than the driver's seat, the occupants of the rear seats are imaged darker in the image. Therefore, when the face detection unit 102 erroneously detects the face of the occupant in the rear seat as the driver's face, the average brightness of the driver's face area pixels calculated by the brightness calculation unit 104 is used as the average brightness of the driver's face area pixels. It will be significantly smaller than the assumed average brightness, and the optical setting control unit 106 may control the optical setting according to the face region of the occupant in the rear seat. As a result, for example, even if the state in which the driver's face parts cannot be detected temporarily returns to the state in which the driver's face parts can be detected (see FIG. 3), the driver's face area is displayed on the image. Since the brightness of the pixels of the above is not optimized, it becomes difficult for the face detection unit 102 to re-detect the driver's face as a face. Specifically, the vicinity of the driver's seat, where light such as the lighting device 30 is more likely to reach than the rear seat, becomes too bright, causing overexposure on the image, for example, and the face detection unit 102 takes an image with the optical settings in the default state. It is more difficult to detect the driver's face as a face than the image. Then, the face detection unit 102 will continue to detect the face of the occupant in the rear seat as the face of the driver.
In addition, the driver monitoring unit 103 will continue to monitor the state of the face of the occupant in the rear seat. For example, if the driver monitoring unit 103 determines that the occupant in the rear seat is in a dozing state, an alarm is output from the output device. This can be a useless alarm, as it is not really an alarm for the driver being asleep.

On the other hand, the vehicle-mounted image processing device 10 according to the first embodiment includes the erroneous detection determination unit 107 as described above. The false detection determination unit 107 controls the optical settings according to the average brightness of the driver's face region pixels detected by the face detection unit 102, and as a result, whether or not the information regarding the image pickup apparatus 20 satisfies the face false detection condition. By determining, it is determined whether or not the driver's face is erroneously detected. Then, when it is determined that the driver's face is erroneously detected, the erroneous detection determination unit 107 causes the recontrol instruction unit 108 to execute a control for returning the exposure time, the lighting time, or the gain value to the default state. The recontrol instruction is output to the optical setting control unit 106. In this way, the vehicle-mounted image processing device 10 makes it possible to prevent control of inappropriate optical settings based on the erroneously detected driver's face. Further, the in-vehicle image processing device 10 keeps the state in which the original driver's face is difficult to be detected as a result of controlling inappropriate optical settings based on the erroneously detected driver's face. It is possible to prevent it.

The operation of the vehicle-mounted image processing device 10 according to the first embodiment will be described.
FIG. 8 is a flowchart for explaining the operation of the vehicle-mounted image processing device 10 according to the first embodiment.
The in-vehicle image processing device 10 repeats the operation shown in the flowchart of FIG. 8 while the vehicle is running.

The image acquisition unit 101 acquires an image from the image pickup apparatus 20 (step ST801).
The image acquisition unit 101 outputs the acquired image to the face detection unit 102 and the brightness calculation unit 104.

The face detection unit 102 detects the driver's face in the face detection area set on the image based on the image acquired by the image acquisition unit 101 in step ST801 (“YES” in step ST802). case).
The face detection unit 102 outputs the detected face information of the driver to the driver monitoring unit 103, the brightness calculation unit 104, and the erroneous detection determination unit 107. Further, the face detection unit 102 stores the detected face information of the driver in the storage unit.

When the face detection unit 102 cannot detect the driver's face in the face detection area (when "NO" in step ST802), the processing by the vehicle-mounted image processing device 10 returns to step ST801.

The brightness calculation unit 104 calculates the average brightness of the driver's face region pixels in the image acquired by the image acquisition unit 101 in step ST801 (step ST803).
The brightness calculation unit 104 outputs the face area brightness information to the brightness determination unit 105.

The brightness determination unit 105 determines whether or not the average brightness of the driver's face region pixels is equal to or greater than the brightness determination threshold based on the face region brightness information output from the brightness calculation unit 104 (step ST804).
In step ST804, when the brightness determination unit 105 determines that the average brightness of the driver's face area pixels is less than the brightness determination threshold value (when “NO” in step ST804), the average brightness of the driver's face area pixels Is output to the optical setting control unit 106 to the effect that is less than the luminance determination threshold value.

When the brightness determination unit 105 determines in step ST804 that the average brightness of the driver's face region pixels is equal to or greater than the brightness determination threshold value (when “YES” in step ST804), the vehicle-mounted image processing device 10 processes. To finish.

The optical setting control unit 106 controls the optical setting of the image pickup apparatus 20 according to the average brightness of the driver's face region pixels calculated by the brightness calculation unit 104 (step ST805).

The false detection determination unit 107 makes a false detection determination based on the result of the control performed by the optical setting control unit 106 in step ST805 (step ST806).

When the face detection unit 102 determines that the driver's face is erroneously detected (in the case of "YES" in step ST807), the erroneous detection determination unit 107 gives an area reduction instruction to reduce the face detection area. Output to 109. Further, the erroneous detection determination unit 107 outputs a re-control required notification to the re-control instruction unit 108.
On the other hand, when the erroneous detection determination unit 107 determines that the face detection unit 102 has not erroneously detected the driver's face (when “NO” in step ST807), the face detection presence information is transmitted to the driver monitoring unit 103. The in-vehicle image processing device 10 ends the processing.

When the erroneous detection determination unit 107 outputs a re-control required notification in step ST807, the re-control instruction unit 108 outputs a re-control instruction to the optical setting control unit 106.
When the recontrol instruction unit 108 outputs the recontrol instruction, the optical setting control unit 106 controls to return the optical setting of the image pickup apparatus 20 to the initial value (step ST808).

When the area reduction instruction is output from the erroneous detection determination unit 107 in step ST807, the area reduction unit 109 reduces the face detection area (step ST809).
Then, the area reduction unit 109 outputs information to the effect that the face detection area has been reduced to the re-detection instruction unit 110.

When the re-detection instruction unit 110 outputs information indicating that the face detection area has been reduced from the area reduction unit 109 in step ST809, the area reduction unit 109 is reduced with respect to the face detection unit 102. The driver's face is rediscovered in the face detection area of the above (step ST810). Then, the in-vehicle image processing device 10 ends the processing.

In the operation of the vehicle-mounted image processing device 10 described with reference to FIG. 8, when the vehicle-mounted image processing device 10 finishes processing, the area reduction unit 109 reduces the face detection area (step of FIG. 8). (See ST809), the face detection area is returned to the initial value in the information about the face detection area stored in the storage unit.

Further, regarding the operation of the in-vehicle image processing device 10 described above with reference to FIG. 8, when the face detection area is reduced, the area reduction unit 109 reduces the face detection area for a certain period of time. If the face detection unit 102 does not detect the driver's face even after the lapse of time (see step ST802 in FIG. 8), the face detection area may be returned to the initial value. In that case, the area reduction unit 109 acquires, for example, information on whether or not the driver's face has been detected from the face detection unit 102.

Here, FIG. 9 is a flowchart for explaining the operation of steps ST804 to ST807 of FIG. 8 in more detail.
In step ST901, the brightness determination unit 105 determines whether or not the average brightness of the driver's face region pixels is equal to or greater than the brightness determination threshold based on the face region brightness information output from the brightness calculation unit 104 in step ST803 of FIG. (Step ST901). The operation of step ST901 is the operation of step ST804 of FIG.
When the brightness determination unit 105 determines in step ST901 that the average brightness of the driver's face region pixels is equal to or greater than the brightness determination threshold value (when “YES” in step ST901), the vehicle-mounted image processing device 10 processes. To finish.

In step ST901, when the brightness determination unit 105 determines that the average brightness of the driver's face area pixels is less than the brightness determination threshold value (when “NO” in step ST901), the average brightness of the driver's face area pixels is Information indicating that the brightness is less than the threshold value for determining brightness is output to the exposure control unit 1061, and the exposure control unit 1061 controls the exposure of the imaging device 20 (step ST902).

When the exposure control unit 1061 performs exposure control in step ST902, the brightness determination unit 105 determines whether or not the average brightness of the driver's face region pixels after the exposure control is equal to or greater than the brightness determination threshold value (step ST903). ).
In step ST903, when the brightness determination unit 105 determines that the average brightness of the driver's face region pixels after exposure control is equal to or greater than the brightness determination threshold value (when “YES” in step ST903), the vehicle-mounted image processing device. The process according to 10 proceeds to step ST906.
In step ST903, when the brightness determination unit 105 determines that the average brightness of the driver face area pixels after exposure control is less than the brightness determination threshold value (when “NO” in step ST903), the driver face area pixels. Information that the average brightness is less than the brightness determination threshold value is output to the image processing unit 1062, and the image processing unit 1062 adjusts the gain (step ST904).

When the image processing unit 1062 adjusts the gain in step ST904, the brightness determination unit 105 determines whether or not the average brightness of the driver's face region pixels after the gain adjustment is equal to or greater than the brightness determination threshold value (step ST905). ).
In step ST905, when the brightness determination unit 105 determines that the average brightness of the driver face region pixels after gain adjustment is less than the brightness determination threshold value (when “NO” in step ST905), the driver face region pixels Information indicating that the average brightness of the above is less than the brightness determination threshold value is output to the exposure control unit 1061, and the processing by the in-vehicle image processing device 10 returns to step ST902.

In step ST905, when the brightness determination unit 105 determines that the average brightness of the driver's face region pixels after gain adjustment is equal to or greater than the brightness determination threshold value (when “YES” in step ST905), the erroneous detection determination unit 107 Makes a false positive determination based on the result of the control performed by the optical setting control unit 106 (step ST906). Specifically, in the false detection determination unit 107, when the exposure control unit 1061 performs exposure control in step ST902, the information regarding the image pickup apparatus 20 obtained as a result of performing the exposure control determines the face false positive detection condition. Determine if it meets or not. Further, in the false detection determination unit 107, when the image processing unit 1062 adjusts the gain in step ST904, whether or not the information about the image pickup apparatus 20 obtained as a result of the gain adjustment satisfies the face false detection condition. Is determined.

In the false detection determination unit 107, when the information about the image pickup device 20 obtained as a result of the exposure control satisfies the face false detection condition, or when the information about the image pickup device 20 obtained as a result of performing the gain adjustment is the face. When the erroneous detection requirement is satisfied (when “YES” in step ST907), the face detection unit 102 determines that the driver's face has been erroneously detected, and issues an area reduction instruction to reduce the face detection area 109. Output to. Further, the erroneous detection determination unit 107 outputs a re-control required notification to the re-control instruction unit 108. Then, the processing by the vehicle-mounted image processing device 10 proceeds to step ST808 in FIG.

In the false detection determination unit 107, the information about the image pickup device 20 obtained as a result of performing exposure control does not satisfy the face false detection condition, and the information about the image pickup device 20 obtained as a result of performing gain adjustment is the face. If the erroneous detection condition is not satisfied (“NO” in step ST907), the face detection unit 102 determines that the driver's face has not been erroneously detected, and outputs the face detection presence information to the driver monitoring unit 103. do. Then, the in-vehicle image processing device 10 ends the processing.

In the first embodiment described above, the in-vehicle image processing device 10 includes an area reduction unit 109, and when the area reduction instruction is output from the erroneous detection determination unit 107, the area reduction unit 109 uses the face detection area. It was decided to reduce it. However, the present invention is not limited to this, and the in-vehicle image processing device 10 may be configured not to include the area reduction unit 109. In that case, when the optical setting control unit 106 controls to return the optical setting of the image pickup apparatus 20 to the initial value based on the recontrol instruction output from the recontrol instruction unit 108, the recontrol instruction unit 108 changes the face detection unit 108. The driver's face is re-detected in the face detection area of the image after the optical setting is returned to the initial value with respect to 102.
However, if the in-vehicle image processing device 10 is provided with the area reduction unit 109, the driver's face is more correctly detected when the driver's face is re-detected after determining that the driver's face is erroneously detected. You will be able to detect faces.
Further, in the above-described first embodiment, it is assumed that the face detection area is set on the image, but the present invention is not limited to this, and it is essential that the face detection area is set on the image. is not it.

Further, in the above-described first embodiment, the vehicle-mounted image processing device 10 controls the optical setting of the image pickup device 20 to be an appropriate value, which is performed when it is determined that the driver's face is erroneously detected in the erroneous detection determination. Is a control that sets the optical setting of the image pickup apparatus 20 to the default state. However, the present invention is not limited to this, and the in-vehicle image processing device 10 performs imaging as a control for setting the optical setting of the imaging device 20 to an appropriate value, which is performed when it is determined that the driver's face is erroneously detected in the erroneous detection determination. The exposure time of the device 20 is set to the exposure time set one before in the time series, the lighting time of the imaging device 20 is set to the lighting time set one before in the time series, or the imaging is performed. Control may be performed so that the gain value of the device 20 is set to the gain value set immediately before in chronological order. Immediately before the image pickup device 20 erroneously detects the driver's face, the image pickup device 20 has taken an image of the driver's face by the optical setting in which these values are set. The set exposure time, the lighting time as the lighting time set one before in chronological order, or the gain value as the gain value set one before in chronological order. Even so, it is assumed that the driver's face can be sufficiently imaged.
The appropriate value of the optical setting referred to in the first embodiment is the above-mentioned initial value of the exposure time, the initial value of the lighting time, the initial value of the gain value, and the exposure time set immediately before. Includes the lighting time that was set immediately before, or the gain value that was set immediately before.

As described above, in the in-vehicle image processing device 10 according to the first embodiment, the range in the vehicle including the range in which the face of the driver sitting in the driver's seat should exist is imaged from the image pickup device 20. The image acquisition unit 101 that acquires an image, the face detection unit 102 that detects the driver's face and the driver's face area on the image based on the image acquired by the image acquisition unit 101, and the face detection unit 102 A brightness calculation unit 104 that calculates the average brightness of the detected pixels in the driver's face region, and a brightness determination unit 105 that determines whether or not the average brightness calculated by the brightness calculation unit 104 is equal to or greater than the brightness determination threshold. When the brightness determination unit 105 determines that the average brightness is less than the brightness determination threshold, the optical setting control unit 106 that controls the optical setting of the image pickup apparatus 20 according to the average brightness, and the optical setting control unit Based on the result of the control performed by 106, the erroneous detection determination unit 107 for determining whether or not the face detection unit 102 erroneously detects the driver's face, the erroneous detection determination unit 107, and the face detection unit 102 for the driver When it is determined that the face is erroneously detected, the optical setting control unit 106 is provided with a recontrol instruction unit 108 that outputs a recontrol instruction for controlling the optical setting of the image pickup device 20 to return to an appropriate value. Configured. Therefore, the in-vehicle image processing device 10 can prevent improper control of optical settings based on the driver's face that has been erroneously detected.

Embodiment 2.
In the second embodiment, an embodiment in which the accuracy of erroneous detection determination can be improved by considering the position of the driver's face detected from the image on the image will be described.

Since the configuration of the vehicle-mounted image processing device 10 according to the second embodiment is the same as the configuration of the vehicle-mounted image processing device 10 according to the first embodiment, duplicate description will be omitted.
The vehicle-mounted image processing device 10 according to the second embodiment is different from the vehicle-mounted image processing device 10 according to the first embodiment in the specific operation of the erroneous detection determination unit 107.
In the second embodiment, the erroneous detection determination unit 107 erroneously detects the driver's face based on the result of the control performed by the optical setting control unit 106 and the position of the driver's face detected by the face detection unit 102. Make a judgment. Specifically, in the false detection determination unit 107, the information about the image pickup device 20 obtained as a result of the control performed by the optical setting control unit 106 satisfies the face false detection condition, and the operation detected by the face detection unit 102. Determine if the position of the person's face is within the non-driver area.

In the second embodiment, the non-driver area refers to an area in the face detection area where it is highly probable that the faces of occupants other than the driver are imaged. The non-driver area is set in advance by the user or the like according to, for example, a face detection area set in consideration of the imaging range of the imaging device 20.
For example, here, it is assumed that the installation position and the imaging range of the imaging device 20 are the installation position and the imaging range as shown in FIG. 2A. The imaging range is basically set to a range centered on, for example, the headrest so that the driver's face is imaged at a substantially center position in the left-right direction of the image. Further, in the image in which the imaging range is captured, the face detection region is, for example, the driver's face even if the driver's face, which is normally captured in the substantially center of the image, moves to various positions. It may be set as wide as it can be detected. If the face of the occupant in the back seat is depressed into the face detection area set in this way, the face of the occupant in the rear seat may be depressed into the edge of the face detection area. Is high. Therefore, in this case, the non-driver area is set at the end of the face detection area. It is assumed that the non-driver area is associated with the face detection area and stored in the storage unit.

As a result of the control by the optical setting control unit 106, it is possible that the light of the lighting device 30 does not easily reach the driver's face as a factor that significantly changes the exposure time, lighting time, or gain value. Is also possible. For example, when the driver is wearing a hat, the hat may block the light from the lighting device 30 and the driver's face may be imaged darkly depending on the angle. In this case, the optical setting control unit 106 controls according to the correctly detected driver's face area, so that, for example, the average brightness of the driver's face area pixels can temporarily become equal to or higher than the driver's seat brightness determination threshold. .. In this way, even when the average brightness of the driver's face area pixels temporarily reaches the threshold value for determining the driver's seat brightness, it cannot be said that the control is performed according to the face areas of the occupants other than the driver. It may not be there.
Therefore, in the second embodiment, the erroneous detection determination unit 107 considers the position of the driver's face detected by the face detection unit 102 in addition to the face erroneous detection condition, and the face detection unit 102 determines the driver's face. Determine if it was erroneously detected.
The erroneous detection determination unit 107 may calculate the position on the image of the driver's face detected by the face detection unit 102 from the face information. For example, the erroneous detection determination unit 107 may use the coordinates of the center of the face region based on the face information as the position of the driver's face. Further, the erroneous detection determination unit 107 may use a frame indicating the driver's face area based on the face information output from the face detection unit 102 as the position of the driver's face.

When the position of the driver's face detected by the face detection unit 102 is within the non-driver area, the driver's face detected by the face detection unit 102 is likely to be the face of an occupant other than the driver. On the contrary, if the position of the driver's face is not within the non-driver's area, the position of the driver's face has not moved significantly, and the optical setting control unit 106 is set to the correctly detected driver's face area. At the same time, it can be said that it is highly possible that the necessary control was performed.
In the false detection determination unit 107, the result of the control performed by the optical setting control unit 106 satisfies the face false detection condition, and the position of the driver's face detected by the face detection unit 102 is within the non-driver area. In a certain case, it is determined that the face detection unit 102 erroneously detects the driver's face.

The operation of the vehicle-mounted image processing device 10 according to the second embodiment will be described.
Since the basic operation of the vehicle-mounted image processing device 10 according to the second embodiment is the same as the operation described with reference to FIG. 8 in the first embodiment, duplicate description will be omitted.
FIG. 10 is a flowchart for explaining in more detail the operations of steps ST804 to ST807 of FIG. 8 in the vehicle-mounted image processing device 10 according to the second embodiment.
The detailed operation of the false positive determination unit 107 in the second embodiment, which is different from the first embodiment, will be described with reference to FIG.
Since the specific operations of steps ST901 to ST907 of FIG. 10 are the same as the specific operations of steps ST901 to ST907 of FIG. 9 described in the first embodiment, duplicate description will be omitted.

The false detection determination unit 107 relates to the image pickup device 20 obtained as a result of performing exposure control in step ST907 when the information regarding the image pickup device 20 satisfies the face error detection condition or as a result of performing gain adjustment. When the information satisfies the face erroneous detection condition (“YES” in step ST907), it is determined whether or not the position of the driver's face detected by the face detection unit 102 is within the non-driver area (when the information satisfies the face error detection condition (“YES” in step ST907)). Step ST1001).

When it is determined in step ST1001 that the position of the driver's face is within the non-driver area (when "YES" in step ST1001), the erroneous detection determination unit 107 causes the face detection unit 102 to detect the driver's face. Judge that it was erroneously detected. Then, the erroneous detection determination unit 107 outputs an area reduction instruction for reducing the face detection area to the area reduction unit 109. Further, the erroneous detection determination unit 107 outputs a re-control required notification to the re-control instruction unit 108. The processing by the in-vehicle image processing device 10 proceeds to step ST808 in FIG.

On the other hand, when it is determined in step ST1001 that the position of the driver's face is not within the non-driver area (when "NO" in step ST1001), the erroneous detection determination unit 107 has the face detection unit 102 of the driver. It is determined that the face is not erroneously detected. Then, the erroneous detection determination unit 107 outputs the face detection presence information to the driver monitoring unit 103. The in-vehicle image processing device 10 ends the processing.

In this way, the erroneous detection determination unit 107 makes an erroneous detection determination of the driver's face based on the result of the control performed by the optical setting control unit 106 and the position of the driver detected by the face detection unit 102. Therefore, the accuracy of erroneous detection determination of the driver's face can be improved.

In the above description, in the erroneous detection determination unit 107, the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition, and the position of the driver's face detected by the face detection unit 102 is determined. , False detection judgment is made by judging whether or not it is in the non-driver area.
However, the determination performed in addition to determining whether or not the result of the control performed by the optical setting control unit 106 satisfies the face error detection condition is not limited to the above-mentioned determination.
The erroneous detection determination unit 107 is, for example, the position of the driver's face based on the current face information output from the face detection unit 102, which triggered the control by the optical setting control unit 106, and the face detection unit 102. Compare with the position of the driver's face based on the previous face information in chronological order with respect to the output face information. Then, as a result of this comparison, the erroneous detection determination unit 107 determines whether or not the difference in the position coordinates indicating the position of the driver's face based on both face information is equal to or greater than a preset threshold value. , May be added to the determination of whether or not the face erroneous detection condition is satisfied. Further, in this case, the erroneous detection determination unit 107 does not, for example, compare the positions of the driver's faces, but compares the feature points of the driver's face or the size of the frame indicating the driver's face area. Whether or not the difference in facial feature points is equal to or greater than the threshold value, or whether or not the difference in the size of the frame indicating the face region is equal to or greater than the threshold value is determined whether or not the face erroneous detection condition is satisfied. It may be performed in addition to the judgment of.

When the erroneous detection determination unit 107 determines whether or not the face erroneous detection condition is satisfied in addition to the determination regarding the position of the driver's face or the determination regarding the feature points of the driver's face, for example, If the installation position and imaging range of the imaging device 20 are the installation position and imaging range as shown in FIG. 2B, the face of the passenger in the passenger seat is erroneously detected as the face of the driver. It is possible to improve the accuracy of false detection determination. If the position of the driver's face or the feature points of the driver's face change significantly, it is highly possible that the face detection unit 102 erroneously detects the face of the passenger in the passenger seat as the driver's face. It can be said that. That is, when the information about the image pickup apparatus 20 satisfies the face erroneous detection condition, the control performed by the optical setting control unit 106 may be the control performed according to the average brightness of the pixels in the face region of the passenger in the passenger seat. Can be said to be high. In addition to the determination of whether or not the face erroneous detection condition is satisfied, the erroneous detection determination unit 107 determines the position of the driver's face or the feature points of the driver's face. In addition, it can be determined that the face detection unit 102 erroneously detects the face of the passenger in the passenger seat as the face of the driver.

Also in the second embodiment, as in the first embodiment, the in-vehicle image processing device 10a may be configured not to include the region reduction unit 109. Further, it is not essential that the face detection area is set on the image.

As described above, the vehicle-mounted image processing device 10 according to the second embodiment has the same configuration as the vehicle-mounted image processing device 10 according to the first embodiment, and the false detection determination unit 107 has an optical setting. The erroneous detection determination is made based on the result of the control performed by the control unit 106 and the position of the driver's face detected by the face detection unit 102. Therefore, the in-vehicle image processing device 10a can prevent control of inappropriate optical settings based on the erroneously detected driver's face, and can prevent the erroneously detected driver's face when performing the erroneous detection determination. By considering the position of, the accuracy of the false positive determination can be improved.

Embodiment 3.
In the third embodiment, the embodiment in which the accuracy of the false positive determination can be improved by considering the pull-out amount of the seat belt will be described.

FIG. 11 is a diagram showing a configuration example of the vehicle-mounted image processing device 10a according to the third embodiment.
In FIG. 11, the same components as those of the vehicle-mounted image processing device 10 described with reference to FIG. 1 in the first embodiment are designated by the same reference numerals, and duplicate description will be omitted.
The vehicle-mounted image processing device 10a according to the third embodiment is different from the vehicle-mounted image processing device 10 according to the first embodiment in that it includes a drawing amount detection unit 111.

The pull-out amount detection unit 111 detects the pull-out amount of the seat belt installed in the driver's seat.
The pull-out amount detection unit 111 detects the pull-out amount of the seat belt from the movement amount of the position of the mark attached to the seat belt on the image based on the image acquired by the image acquisition unit 101, for example. It is assumed that the seat belt is marked in advance. The mark is detectable by the image pickup apparatus 20. For example, if the image pickup apparatus 20 is an infrared camera, the mark can be detected by the infrared camera.
Further, the withdrawal amount detection unit 111 is installed in the vicinity of the seatbelt installed in the driver's seat, for example, from the rotation amount detected by a take-up sensor (not shown) having a portion that rotates according to the movement amount of the seatbelt. , The seatbelt pull-out amount may be detected by calculating the seatbelt pull-out amount.

The pull-out amount of the seatbelt detected by the pull-out amount detection unit 111 is, for example, a pull-out amount based on the seatbelt in the unused state (hereinafter, referred to as “pull-out amount based on the unused state”).
Further, the pull-out amount of the seatbelt detected by the pull-out amount detection unit 111 is, for example, a pull-out amount based on the seatbelt in a state where the driver wears the seatbelt in a normal sitting posture (hereinafter, "usage state standard"). It may be called "withdrawal amount"). When the driver changes his / her posture from the normal sitting posture for some purpose, the amount of pulling out the seat belt may also change. When the withdrawal amount detection unit 111 detects the withdrawal amount based on the usage state, it is more accurate to change the posture of the driver in consideration of the physical disparity of the driver than to detect the withdrawal amount based on the non-use state. It is possible to detect the amount of pulling out of the seat belt according to the situation.
The withdrawal amount detection unit 111 outputs the detected withdrawal amount to the erroneous detection determination unit 107.
The withdrawal amount detection unit 111 repeats the detection operation at least at the same timing as the face detection unit 102 performs the detection operation. Then, the detected withdrawal amount is associated with the face information and stored in the storage unit. For example, when the face detection unit 102 stores the face information in the storage unit, the face detection unit 102 acquires the information regarding the withdrawal amount from the withdrawal amount detection unit 111 and stores it in the storage unit in association with the face information.

The erroneous detection determination unit 107 makes an erroneous detection determination based on the result of the control performed by the optical setting control unit 106 and the withdrawal amount detected by the withdrawal amount detection unit 111.
Specifically, the erroneous detection determination unit 107 includes, for example, the extraction amount corresponding to the current face information output from the face detection unit 102, which triggered the control by the optical setting control unit 106, and the face detection unit. Compare with the withdrawal amount corresponding to the previous face information in chronological order with respect to the face information output from 102. Then, the erroneous detection determination unit 107 determines whether or not the difference in the withdrawal amount is equal to or less than a preset threshold value (hereinafter referred to as "first withdrawal amount determination threshold value"), based on the face erroneous detection condition. It is performed in addition to the judgment of whether or not to satisfy.
The erroneous detection determination unit 107 is a face detection unit when the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition and the difference in the withdrawal amount is equal to or less than the first withdrawal amount determination threshold value. It is determined that 102 erroneously detects the driver's face.
On the other hand, in the false detection determination unit 107, when the result of the control performed by the optical setting control unit 106 satisfies the face false detection condition, but the difference in the withdrawal amount is larger than the first withdrawal amount determination threshold value, It is determined that the face detection unit 102 does not erroneously detect the driver's face.

Specifically, in the erroneous detection determination unit 107, for example, the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition, and the withdrawal amount detected by the withdrawal amount detection unit 111 is a threshold value. (Hereinafter referred to as "second withdrawal amount determination threshold value".) It may be determined whether or not it is equal to or less than or equal to.
When the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition and the withdrawal amount is equal to or less than the second withdrawal amount determination threshold value, the erroneous detection determination unit 102 causes the face detection unit 102 to It is determined that the driver's face is erroneously detected.
On the other hand, the erroneous detection determination unit 107 detects a face when the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition, but the withdrawal amount is larger than the second withdrawal amount determination threshold value. It is determined that the unit 102 does not erroneously detect the driver's face.

As a result of the control by the optical setting control unit 106, one of the factors that significantly change the exposure time, the lighting time, or the gain value is considered to be, for example, that the driver moves his / her body significantly. When the driver moves his body a lot, the position of the driver's face also moves a lot. For example, the driver may move his / her body significantly to visually check the rear of the vehicle and move his / her face to the rear where the light of the lighting device 30 is difficult to reach.
Therefore, even if the average brightness of the driver's face area pixel temporarily exceeds the threshold value for determining the driver's seat brightness, it cannot be said that the control is performed according to the face area of the occupant other than the driver. In some cases.

Therefore, in the third embodiment, the erroneous detection determination unit 107 makes an erroneous detection determination in consideration of the pull-out amount of the driver's seat belt in addition to the face erroneous detection condition. As described above, when the driver moves his / her body significantly, it is considered that the position of the driver's face also moves significantly. Then, when the driver moves his / her body from the normal sitting posture, the seat belt of the driver's seat is pulled out, and the pull-out amount detection unit 111 detects the pull-out amount according to the movement of the seat belt. Become.
When the pull-out amount detection unit 111 detects the pull-out amount based on the unused state, the pull-out amount is more from the position of the face when the driver is in the normal sitting posture than the pull-out amount when the driver is in the normal sitting posture. The amount of drawer is larger when the face is moved a lot. Therefore, it is possible to determine whether or not the driver has moved his / her face significantly based on the difference between the withdrawal amount at a certain time point and the withdrawal amount at another time point. Alternatively, when the withdrawal amount is larger than a certain value, it can be determined whether or not the driver has moved the face significantly from the position of the face in the normal sitting posture.
When the withdrawal amount detection unit 111 detects the withdrawal amount based on the usage state, the withdrawal amount when the driver is in the normal sitting posture is set to, for example, 0 as a reference value, and the driver is normal. The withdrawal amount when the face is largely moved from the position of the face in the sitting posture becomes larger than the reference value. In this case as well, it is possible to determine whether or not the driver has moved his / her face significantly based on the difference between the withdrawal amount at a certain time point and the withdrawal amount at another time point. Alternatively, when the withdrawal amount is larger than a certain value, it can be determined whether or not the driver has moved the face significantly from the position of the face in the normal sitting posture.
If it can be determined that the driver has moved the face significantly, it is considered that the optical setting control unit 106 has performed control to change the optical setting significantly because the face has moved significantly as described above. That is, if it can be determined that the driver has moved his face significantly, it does not necessarily mean that the driver's face has been erroneously detected. On the other hand, if it can be determined that the driver does not move his face significantly, it is probable that the optical setting control unit 106 has performed control to change the optical setting significantly because the driver's face was erroneously detected. ..

Therefore, in the erroneous detection determination unit 107, for example, the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition, and the current withdrawal amount and the previous withdrawal amount in the time series are combined. When the difference is equal to or less than the first withdrawal amount determination threshold value, the face detection unit 102 determines that the driver's face has been erroneously detected. This is because when the difference between the current withdrawal amount and the previous withdrawal amount in the time series is equal to or less than the first withdrawal amount determination threshold value, it is considered that the driver's face is not moved significantly.
On the other hand, in the false detection determination unit 107, when the result of the control performed by the optical setting control unit 106 satisfies the face false detection condition, but the difference in the withdrawal amount is larger than the first withdrawal amount determination threshold value, It is determined that the face detection unit 102 does not erroneously detect the driver's face. This is because when the difference in the withdrawal amount is larger than the threshold value for determining the first withdrawal amount, it is considered that the driver's face has been greatly moved.
Further, the erroneous detection determination unit 107 determines, for example, when the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition and the withdrawal amount is equal to or less than the second withdrawal amount determination threshold value. The detection unit 102 determines that the driver's face has been erroneously detected. This is because when the withdrawal amount is equal to or less than the second withdrawal amount determination threshold value, it is considered that the driver's face is not moved significantly.
On the other hand, the erroneous detection determination unit 107 detects a face when the result of the control performed by the optical setting control unit 106 satisfies the face erroneous detection condition, but the withdrawal amount is larger than the second withdrawal amount determination threshold value. It is determined that the unit 102 does not erroneously detect the driver's face. This is because when the withdrawal amount is larger than the second withdrawal amount determination threshold value, it is considered that the driver's face has been greatly moved.

The operation of the vehicle-mounted image processing device 10a according to the third embodiment will be described.
Since the basic operation of the vehicle-mounted image processing device 10a according to the third embodiment is the same as the operation described with reference to FIG. 8 in the first embodiment, duplicate description will be omitted.
FIG. 12 is a flowchart for explaining in more detail the operations of steps ST804 to ST807 of FIG. 8 in the vehicle-mounted image processing device 10a according to the third embodiment. The detailed operation of the false positive determination unit 107 in the third embodiment, which is different from the first embodiment, will be described with reference to FIG.
Since the specific operations of steps ST901 to ST907 of FIG. 12 are the same as the specific operations of steps ST901 to ST907 of FIG. 9 described in the first embodiment, duplicate description will be omitted.
Further, in the operation shown in FIG. 12, the false detection determination unit 107 is based on the result of the control performed by the optical setting control unit 106 and the difference between the current withdrawal amount and the previous withdrawal amount in chronological order. , False detection judgment is to be performed.

The false detection determination unit 107 relates to the image pickup device 20 obtained when the information about the image pickup device 20 obtained as a result of performing the exposure control in step ST907 satisfies the face false positive detection condition or as a result of performing the gain adjustment. When the information satisfies the face erroneous detection condition (“YES” in step ST907), it is determined whether or not the difference in the withdrawal amount is equal to or less than the first withdrawal amount determination threshold value (step ST1201).

In step ST1201, when the erroneous detection determination unit 107 determines that the difference in the withdrawal amount is equal to or less than the first withdrawal amount determination threshold value (when “YES” in step ST1201), the face detection unit 102 tells the driver's face. Is determined to have been erroneously detected. Then, the erroneous detection determination unit 107 outputs an area reduction instruction for reducing the face detection area to the area reduction unit 109. Further, the erroneous detection determination unit 107 outputs a re-control required notification to the re-control instruction unit 108. The processing by the in-vehicle image processing device 10 proceeds to step ST808 in FIG.

On the other hand, when it is determined in step ST1201 that the difference in the withdrawal amount is larger than the threshold value for determining the first withdrawal amount (when "NO" in step ST1201), the erroneous detection determination unit 107 has the face detection unit 102 of the driver. It is determined that the face is not erroneously detected. Then, the erroneous detection determination unit 107 outputs the face detection presence information to the driver monitoring unit 103. The in-vehicle image processing device 10 ends the processing.

As described above, the erroneous detection determination unit 107 makes an erroneous detection determination based on the result of the control performed by the optical setting control unit 106 and the pull-out amount of the seat belt detected by the pull-out amount detection unit 111. Therefore, the accuracy of erroneous detection determination can be improved.

In the third embodiment as well, the in-vehicle image processing device 10a may not be provided with the area reduction unit 109, as in the first and second embodiments. Further, it is not essential that the face detection area is set on the image.

As described above, the vehicle-mounted image processing device 10a according to the third embodiment detects the amount of pulling out of the seat belt installed in the driver's seat in addition to the configuration of the vehicle-mounted image processing device 10 according to the first embodiment. The erroneous detection determination unit 107 includes the withdrawal amount detection unit 111, so that the erroneous detection determination unit 107 makes an erroneous detection determination based on the result of the control performed by the optical setting control unit 106 and the withdrawal amount detected by the withdrawal amount detection unit 111. Configured. Therefore, the in-vehicle image processing device 10a can prevent control of inappropriate optical settings based on the erroneously detected driver's face, and pull out the driver's seat belt when making an erroneous detection determination. By considering the amount, the accuracy of the false positive determination can be improved.

13A and 13B are diagrams showing an example of the hardware configuration of the vehicle-mounted image processing devices 10 and 10a according to the first to third embodiments.
In the first to third embodiments, the image acquisition unit 101, the face detection unit 102, the driver monitoring unit 103, the brightness calculation unit 104, the brightness determination unit 105, and the optical setting control unit 106 are erroneously used. The functions of the detection determination unit 107, the area reduction unit 109, the re-detection instruction unit 110, and the withdrawal amount detection unit 111 are realized by the processing circuit 1301. That is, the in-vehicle image processing devices 10 and 10a perform erroneous detection determination as to whether or not the driver's face is erroneously detected, and control the optical setting of the image pickup apparatus 20 based on the result of the erroneous detection determination. A processing circuit 1301 is provided.
The processing circuit 1301 may be dedicated hardware as shown in FIG. 10A, or may be a CPU (Central Processing Unit) 1305 that executes a program stored in the memory 1306 as shown in FIG. 10B.

When the processing circuit 1301 is dedicated hardware, the processing circuit 1301 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array) or a combination of these is applicable.

When the processing circuit 1301 is the CPU 1305, the image acquisition unit 101, the face detection unit 102, the driver monitoring unit 103, the brightness calculation unit 104, the brightness determination unit 105, the optical setting control unit 106, and the erroneous detection determination unit The functions of the 107, the area reduction unit 109, the re-detection instruction unit 110, and the extraction amount detection unit 111 are realized by software, firmware, or a combination of software and firmware. That is, the image acquisition unit 101, the face detection unit 102, the driver monitoring unit 103, the brightness calculation unit 104, the brightness determination unit 105, the optical setting control unit 106, the false detection determination unit 107, and the area reduction unit. The 109, the rediscovery instruction unit 110, and the withdrawal amount detection unit 111 are processing circuits such as an HDD (Hard Disk Drive) 1302, a CPU 1305 that executes a program stored in a memory 1306, and a system LSI (Large-Scale Integration). Is realized by. The programs stored in the HDD 1302, the memory 1306, etc. are the image acquisition unit 101, the face detection unit 102, the driver monitoring unit 103, the brightness calculation unit 104, the brightness determination unit 105, and the optical setting control unit 106. It can also be said that the computer is made to execute the procedure or method of the erroneous detection determination unit 107, the area reduction unit 109, the re-detection instruction unit 110, and the withdrawal amount detection unit 111. Here, the memory 1306 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Online Memory), an EPROM (Electrically Memory), etc. This includes sexual or volatile semiconductor memories, magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versailles Disc), and the like.

The image acquisition unit 101, the face detection unit 102, the driver monitoring unit 103, the brightness calculation unit 104, the brightness determination unit 105, the optical setting control unit 106, the false detection determination unit 107, and the area reduction unit The functions of the 109, the re-detection instruction unit 110, and the withdrawal amount detection unit 111 may be partially realized by dedicated hardware and partly realized by software or firmware. For example, the image acquisition unit 101 and the extraction amount detection unit 111 are realized by the processing circuit 1301 as dedicated hardware, and the face detection unit 102, the driver monitoring unit 103, the brightness calculation unit 104, and the brightness Regarding the determination unit 105, the optical setting control unit 106, the false detection determination unit 107, the area reduction unit 109, and the re-detection instruction unit 110, the processing circuit reads and executes the program stored in the memory 1306. It is possible to realize the function.
Further, the in-vehicle image processing devices 10 and 10a include a device such as an image pickup device 20 or a lighting device 30, an input interface device 1303 and an output interface device 1304 that perform wired communication or wireless communication.

In the above-described first to second embodiments, the in-vehicle image processing devices 10 and 10a are mounted on the vehicle, but this is only an example.
For example, a part or all of the components of the in-vehicle image processing devices 10 and 10a as described with reference to FIG. 1 or 11 may be provided in the server.

Further, in the above-described first to third embodiments, in the in-vehicle image processing devices 10 and 10a, the optical setting control unit 106 can perform both exposure control and gain adjustment as control of the optical setting. However, this is just an example. The optical setting control unit 106 may perform only one of exposure control and gain adjustment as the control of the optical setting.

Further, in the present invention, within the scope of the invention, any combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment is possible. ..

Since the in-vehicle image processing device according to the present invention is configured to prevent improper control of optical settings based on a falsely detected driver's face, the vehicle is driven based on an image of an occupant in the vehicle. It can be applied to an in-vehicle image processing device that detects a person's face.

10,10a In-vehicle image processing device, 101 image acquisition unit, 102 face detection unit, 103 driver monitoring unit, 104 brightness calculation unit, 105 brightness determination unit, 106 optical setting control unit, 1061 exposure control unit, 1062 image processing unit , 107 False positive determination unit, 108 re-control instruction unit, 109 area reduction unit, 110 re-detection instruction unit, 111 withdrawal amount detection unit, 20 imaging device, 30 lighting device, 1301 processing circuit, 1302 HDD, 1303 input interface device, 1304 output interface device, 1305 CPU, 1306 memory.

Claims (11)

An image acquisition unit that acquires an image of the area inside the vehicle including the area where the driver's face sitting in the driver's seat should exist from the image pickup device.
Based on the image acquired by the image acquisition unit, the face detection unit that detects the driver's face and the driver's face region on the image, and the face detection unit.
A brightness calculation unit that calculates the average brightness of pixels in the driver's face region detected by the face detection unit, and a brightness calculation unit.
A brightness determination unit that determines whether or not the average brightness calculated by the brightness calculation unit is equal to or greater than the brightness determination threshold value.
When the brightness determination unit determines that the average brightness is less than the brightness determination threshold value, the optical setting control unit controls the optical settings of the image pickup apparatus according to the average brightness.
Based on the result of the control performed by the optical setting control unit, the erroneous detection determination unit that determines whether or not the face detection unit erroneously detects the driver's face,
When the erroneous detection determination unit determines that the face detection unit erroneously detects the driver's face, the optical setting control unit controls the optical setting control unit to return the optical setting of the image pickup apparatus to an appropriate value. An in-vehicle image processing device characterized by being provided with a recontrol instruction unit that outputs recontrol instructions. The optical setting control unit
The vehicle-mounted image processing device according to claim 1, wherein exposure control is performed on the image pickup device. The optical setting control unit
The vehicle-mounted image processing device according to claim 1 or 2, wherein the gain of the image pickup device is adjusted. The face detection unit detects the driver's face and the driver's face area in the face detection area set on the image.
The erroneous detection determination unit is in a region in which it is more likely that the driver's face is imaged in the face detection region as a result of the exposure control performed by the optical setting control unit. It is determined whether or not the average brightness of the pixels satisfies the first determination condition, and when the average brightness satisfies the first determination condition, the face detection unit determines that the driver's face is erroneously detected. The vehicle-mounted image processing device according to claim 2, wherein the image processing device is characterized by this. The false detection determination unit
As a result of the gain adjustment by the optical setting control unit, it is determined whether or not the gain value of the image pickup apparatus satisfies the second determination condition, and when the gain value of the image pickup apparatus satisfies the second determination condition. The vehicle-mounted image processing device according to claim 3, wherein the face detection unit determines that the driver's face has been erroneously detected. The face detection unit detects the driver's face and the driver's face area within the face detection area set on the image.
The false detection determination unit is white in an area where it is more likely that the driver's face is imaged in the face detection area as a result of the gain adjustment by the optical setting control unit. It is determined whether or not the skip area satisfies the third determination condition, and when the whiteout area satisfies the third determination condition, it is determined that the face detection unit has erroneously detected the driver's face. 3. The vehicle-mounted image processing device according to claim 3. The face detection unit detects the driver's face and the driver's face area within the face detection area set on the image.
When the erroneous detection determination unit determines that the face detection unit erroneously detects the driver's face, the area reduction unit that reduces the face detection area and the area reduction unit.
A request for the face detection unit to be provided with a re-detection instruction unit that rediscovers the driver's face and the driver's face area in the face detection area after the area reduction unit is reduced. Item 1. The in-vehicle image processing apparatus according to Item 1. The area reduction section
The vehicle-mounted image processing apparatus according to claim 7, wherein the face detection area is reduced so that the face detection unit does not include the driver's face that has been erroneously detected. The false detection determination unit
The vehicle-mounted image according to claim 1, wherein the false detection determination is performed based on the result of the control performed by the optical setting control unit and the position of the driver's face detected by the face detection unit. Processing equipment. A drawer amount detection unit for detecting the withdrawal amount of the seatbelt installed in the driver's seat is provided.
The false detection determination unit
The vehicle-mounted image processing apparatus according to claim 1, wherein the false detection determination is performed based on the result of control performed by the optical setting control unit and the withdrawal amount detected by the withdrawal amount detection unit. A step in which the image acquisition unit acquires an image obtained by capturing an image of a range in the vehicle including a range in which the driver's face seated in the driver's seat should exist from the image pickup device.
A step in which the face detection unit detects the driver's face and the driver's face area on the image based on the image acquired by the image acquisition unit.
A step in which the brightness calculation unit calculates the average brightness of the pixels in the driver's face region detected by the face detection unit.
A step in which the brightness determination unit determines whether or not the average brightness calculated by the brightness calculation unit is equal to or greater than the brightness determination threshold value.
When the optical setting control unit determines that the average brightness is less than the brightness determination threshold value, the brightness determination unit controls the optical setting of the image pickup apparatus according to the average brightness.
A step in which the erroneous detection determination unit determines whether or not the face detection unit erroneously detects the driver's face based on the result of control performed by the optical setting control unit.
When the recontrol instruction unit determines that the erroneous detection determination unit erroneously detects the driver's face by the face detection unit, the optical setting of the image pickup apparatus is set to an appropriate value for the optical setting control unit. An in-vehicle image processing method characterized by having a step of outputting a recontrol instruction for performing control to return to.

Images