JP5353734B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  2. Description of the Related Art Conventionally, as a display device for a vehicle, there is a head-up display for notifying a driver of various information by projecting light emitted from the display device in front of the driver and displaying a display image. In this head-up display, it is necessary to adjust the display position of the display image so that the driver can visually recognize the display image, and the display position of the display image is adjusted according to the position of the eyes of the driver. Since the position of the driver's eyes changes according to the physique and posture of the driver, the correct eye position of the driver is detected, and the display position of the display image is adjusted according to the detected eye position. .

  As a head-up display capable of adjusting the display position of a display image, there is a technique described in Japanese Patent Laid-Open No. 2008-155720 (Patent Document 1). The head-up display described in Patent Literature 1 includes an infrared irradiation unit that irradiates infrared rays toward the user (driver), and an infrared camera that senses the infrared rays reflected from the user and images the user. . In the head-up display described in Patent Document 1, the user is imaged by a plurality of infrared cameras, and the eyeball position is calculated by calculating the three-dimensional coordinates of the eyeball position based on the plurality of captured face images. Yes.

JP 2008-155720 A Japanese Patent Laid-Open No. 11-95156

  However, the technique described in Patent Document 1 has a problem that when strong sunlight from the outside enters the vehicle interior, the driver's image becomes unclear and it becomes difficult to recognize the driver's image. .

  The present invention has been made to solve such a problem, and is capable of reducing the influence of incident waves incident from the outside and improving the sharpness of the image of the imaged subject. An object is to provide an apparatus.

  An imaging apparatus according to the present invention is provided on an optical path between an imaging unit that captures an image of a subject projected on a projection surface and the projection surface and the imaging unit, and from the subject side of the projection surface. A first polarization means for realizing a first polarization state in which a reflected wave due to the incident wave and an incident wave from a side opposite to the subject side of the projection surface are transmitted or reflected; and a projection surface A second polarization unit that is provided on an optical path between the first imaging unit and the imaging unit and that realizes a second polarization state in which an incident wave from the opposite side of the projection surface is transmitted or reflected; It is characterized by comprising difference image acquisition means for acquiring a difference image between the first captured image captured in the polarization state and the second captured image captured in the second polarization state.

  In such an imaging apparatus, an imaging unit that captures an image of the subject projected on the projection surface, and an optical path between the projection surface and the imaging unit are provided from the subject side of the projection surface. A first polarization means for realizing a first polarization state that is a state in which a reflected wave by the incident wave and an incident wave from a side opposite to the subject side of the projection surface are transmitted or reflected; and a projection surface; And a second polarization unit that is provided on an optical path between the imaging unit and that realizes a second polarization state in which an incident wave from the opposite side of the projection surface is transmitted or reflected. Here, in the present invention, a difference image between the first captured image captured in the first polarization state and the second captured image captured in the second polarization state can be acquired. Of the subject image reflected in the first captured image and the image due to the incident wave from the opposite side of the projection surface, the image due to the incident wave from the opposite side of the projection surface reflected in the second captured image is removed. It becomes possible. Thereby, the incident wave from the opposite side of the projection surface can be taken into account, and the influence of the incident wave from the opposite side of the projection surface can be reduced, and the sharpness of the subject image can be improved.

  Further, as an imaging device for a vehicle, provided on an optical path between the windshield and the imaging means, an imaging means for capturing an image of a driver who is a subject projected on a windshield having a projection surface, First polarized light that realizes a first polarization state in which a reflected wave due to an incident wave from the windshield driver side and an incident wave from the opposite side of the windshield driver side are transmitted or reflected And a second polarization unit that is provided on an optical path between the windshield and the imaging unit and that realizes a second polarization state in which an incident wave from the opposite side of the windshield is transmitted or reflected A configuration including these is preferable. Thereby, it is possible to remove the image of the incident light from the outside of the vehicle reflected in the second captured image from the driver image reflected in the first captured image and the image of the incident light from the outside of the vehicle. Become. As a result, incident light from the outside of the vehicle can be taken into consideration, and even when strong sunlight is incident as incident light from the outside of the vehicle, it is possible to capture a driver image while reducing the influence of sunlight. Therefore, it is possible to prevent the driver image that is captured from becoming unclear.

  According to the imaging apparatus of the present invention, since the incident light incident from the outside is taken into consideration, it is possible to improve the clarity of the subject image to be captured.

It is a block block diagram which shows the display apparatus for vehicles provided with the imaging device which concerns on embodiment of this invention. It is a typical sectional view of a display for vehicles provided with an imaging device concerning an embodiment of the present invention. It is a top view which shows the camera part which concerns on embodiment of this invention from the upper direction of a windshield. It is a graph which shows the relationship between an incident angle and a reflectance. It is a figure which shows the optical path of the S polarization wave and virtual image optical path which are incident on the 1st camera which image | photographs an S polarization image. It is a schematic diagram which shows an S polarization image, a P polarization image, and a difference image. It is a figure which shows the optical path of the P polarization wave and virtual image optical path which are incident on the 2nd camera which image | photographs a P polarization image. It is a flowchart which shows the procedure of the automatic adjustment process of the virtual image display position performed by HUD-ECU. It is a flowchart which shows the procedure of the image process performed by an image process part. It is a flowchart which shows the procedure of the process performed in a tracking calculating part. It is a schematic perspective view for demonstrating the relationship between the drive control of a mirror, and a display optical path. It is a flowchart which shows the procedure of the driving | running | working attitude | position discrimination | determination process performed with an image process part.

  Hereinafter, a vehicle display device including an imaging device according to a preferred embodiment of the present invention will be described in detail with reference to the drawings. In the description of the present embodiment, the reference of “position” is the position of the vehicle. In addition, the eyeball of the driver in the present embodiment is the eyeball of both eyes of the driver.

  First, the configuration of a vehicle display device including the imaging device according to the present embodiment will be described with reference to FIGS. The vehicle display device M shown in FIGS. 1 to 3 is a so-called head-up display (hereinafter referred to as “HUD”), which is mounted on a vehicle and displays a display image in the field of view of the driver D of the vehicle. It is. As shown in FIG. 1, the vehicle display device M includes a camera unit (imaging unit) 2, a display unit 3, a mirror 4, a mirror driving unit 5, and a HUD-ECU 10. The camera unit 2, the display unit 3, and the mirror driving unit 5 are each electrically connected to the HUD-ECU 10. An ignition button 9 is electrically connected to the HUD-ECU 10.

  FIG. 2 shows a situation where the driver D of the vehicle drives the vehicle while looking at the front of the vehicle. As shown in FIG. 2, the vehicle display device M includes the camera unit 2, the display unit 3, the mirror 4, the mirror driving unit 5, and the HUD-ECU 10 in the HUD main body 1 that is a main part of the device. Contained. The HUD main body 1 is disposed in a dashboard 7 provided below the windshield 6. The top plate 7a of the dashboard 7 is provided with an opening 7b, and the HUD main body 1 is disposed below the opening 7b.

The display unit 3 is a liquid crystal display, for example, and receives a signal from the HUD-ECU 10 to display a display image. The mirror 4 is disposed on the front side of the display unit 3 and reflects the display image projected from the display unit 3 toward the windshield glass 6. The mirror 4 is a concave mirror, for example, and has a reflecting surface that is curved so as to be recessed toward the side opposite to the display unit 3. The mirror drive unit 5 controls the drive of the mirror 4 and includes, for example, a motor, a rack and pinion, a guide rail, and the like, and adjusts the angle and position of the mirror 4. Mirror driver 5 controls the driving of the mirror 4 so as to form a display optical path _{L 1} calculated by the HUD-ECU 10.

  The HUD-ECU 10 has a display optical path in which light emitted from the display unit 3 is incident on the windshield 6 at an angle deviating from the Brewster angle, and light reflected by the windshield 6 is incident on the eye position of the driver D The mirror driving unit 5 is controlled so as to form

  The camera unit 2 captures the face image of the driver D reflected by the windshield 6. The camera unit 2 photographs the driver D using the virtual image of the driver D reflected on the windshield 6. The camera unit 2 includes a first camera 21A, a second camera 21B, a first polarizing plate 22A, and a second polarizing plate 22B. The first camera 21 </ b> A, the second camera 21 </ b> B, the first polarizing plate 22 </ b> A, and the second polarizing plate 22 </ b> B are disposed so as to correspond to the opening 7 b of the dashboard 7.

The first camera 21A and the second camera 21B have a reflected light path (hereinafter referred to as “virtual image light path”) L of the virtual image of the driver D reflected on the windshield 6 at a Brewster angle (or an angle near the Brewster angle). ₂ is installed. FIG. 4 is a graph showing the relationship between the incident angle and the reflectance. As shown in FIG. 4, when the light incident at the Brewster angle is reflected, the P-polarized component of the reflected light becomes zero. The camera unit 2 takes a virtual image of the driver D using this characteristic.

The first polarizer 22A is a polarizer which transmits only S-polarized light component, is provided on the virtual image optical path L ₂ incident on the first camera 21A. For example, the first polarizing plate 22A is disposed immediately before the first camera 21A. The first polarizer 22A is equivalent to the first polarizing means provided on the virtual image optical path L ₂ between the windshield 6 and the first camera 21A, the reflected waves and the vehicle exterior from the vehicle interior It has the function to make it the 1st polarization state which permeate | transmits or reflects the incident wave from.

The second polarizer 22B is a polarizer which transmits only P-polarized light component, is provided on the virtual image optical path L ₂ entering the second camera 22B. For example, the second polarizing plate 22B is disposed immediately before the second camera 21B. The second polarizing plate 22B is equivalent to a second polarization means, provided on the virtual image optical path L ₂ between the windshield 6 and the second camera 21B, the incident wave L ₂ from the vehicle exterior It has a function of setting the second polarization state to be transmitted or reflected.

  Next, the HUD-ECU 10 will be described. The HUD-ECU 10 includes a CPU that performs arithmetic processing, a ROM and a RAM that are storage units, an input signal circuit, an output signal circuit, a power supply circuit, and the like. In the HUD-ECU 10, the image processing unit 30, the tracking calculation unit 40, and the display control unit 50 are constructed by executing the program stored in the storage unit.

  The image processing unit 31 receives the image signal input from the camera unit 2 and performs image processing. The image processing unit 31 acquires a difference image using the S-polarized image and the P-polarized image, and the difference image. A feature point detection unit 32 that detects a feature point of the head of the driver D, a face direction detection unit 33 that detects the face direction of the driver D, and a pupil position that detects the positions of the left and right pupils of the driver D A detection unit 34 is included. The tracking calculation unit 40 includes a display optical path calculation unit 41 that calculates a display optical path, and a drive control unit 42 that controls the mirror driving unit 5.

Next, a method for capturing an S-polarized image will be described. As shown in FIG. 5, the S-polarized wave L ₃ from the passenger compartment enters the windshield 6, and a virtual image of the driver D is projected on the windshield 6. Angle at which the S polarized light L ₃ from the passenger compartment and enters the windshield 6 is a Brewster angle (or angle of Brewster angle near). Then, the S-polarized wave L ₃ from the passenger compartment is reflected by the windshield 6 and travels toward the camera unit 2. At this time, passes S polarized light wave L ₃ reflected by the windshield 6, overlap and S-polarized light waves L ₄ passing through the windshield 6 incident from the vehicle exterior, an opening 7b proceeds along a virtual image optical path L ₂ To do. S-polarized waves (L ₃ , L ₄ ) that have passed through the opening 7b pass through the first polarizing plate 22A and are photographed by the first camera 21A. At that time, the first polarizing plate 22A is in the state shown in FIG. 6A, and allows only the light of the S polarization component to pass therethrough. As shown in FIG. 6B, the first camera 21A captures an S-polarized image that is an image in which the driver D and the scenery S outside the vehicle overlap.

Next, a method for capturing a P-polarized image will be described. As shown in FIG. 7, the P polarized light L ₅ from the vehicle exterior is incident on the windshield 6, and transmitted through the windshield 6, passes through the opening 7b. The P-polarized wave that has passed through the opening 7b passes through the second polarizing plate 22B and is photographed by the second camera 21B. At that time, the second polarizing plate 22B is in the state shown in FIG. 6C, and allows only the P-polarized component light to pass therethrough. As shown in FIG. 6D, the second camera 22B captures a P-polarized image that is an image of only the scenery S.

  Next, a method for automatically adjusting the driver's state (physique) virtual image display position will be described. FIG. 8 is a flowchart showing a procedure in the automatic adjustment processing of the virtual image display position. The step may be abbreviated as S. First, when the ignition button is turned on (powered on) by the driver D, the vehicle display device M is also powered on (S1).

  When the power is turned on, the display control unit 50 of the HUD-ECU 10 projects the boot screen on the display unit 3 in order to display the boot screen as the virtual image V of the HUD (S2).

  The drive control unit 42 controls the mirror drive unit 5 to drive the mirror 4, sets the mirror 4 to the initial state position (S3), and when the position of the mirror 4 is set to the initial state, the display control unit 50 erases the boot screen (S4).

  Next, the display control unit 50 projects the normal display on the display unit 3 in order to display the normal screen of the HUD as the virtual image V of the HUD (S5). Examples of the normal screen of the HUD include speed display and navigation information display.

  Here, the HUD-ECU 10 performs image processing (S6) and tracking calculation processing (in order to allow the driver D to always visually recognize the virtual image V of the HUD in accordance with the state (physique) of the driver D. S7) is performed. “Image processing” and “tracking calculation processing” will be described later.

  In subsequent step 8, the HUD-ECU 10 determines whether or not the ignition button 9 has been turned off. The HUD-ECU 10 ends the process when detecting the OFF state of the ignition button 9, and repeats the processes of Step 5 to Step 8 when the OFF state of the ignition button 9 is not detected.

  Next, image processing in the image processing unit will be described. FIG. 9 is a flowchart illustrating a processing procedure in the image processing unit. When the image processing unit 30 is turned on, the image processing unit 30 inputs the S-polarized image output from the first camera 21A (see FIG. 6B) and the P output from the second camera 21B. A polarization image (see FIG. 6D) is input.

  Next, the image processing unit 30 corrects the S-polarized image and the P-polarized image (S11). The image processing unit 30 performs image correction in consideration of the curvature of the windshield 6 with reference to the position of either the first camera 21A or the second camera 21B. Specifically, when the position of the first camera 21A is used as a reference, the P-polarized image captured by the second camera 21B is corrected by image processing as if it was captured at the position of the first camera 21A. Note that the reference position in the image correction may be another position.

  Subsequently, the differential image acquisition unit 31 of the image processing unit 30 acquires a differential image between the S-polarized image and the P-polarized image (S12). Thereby, the difference image acquisition part 31 can acquire the virtual image image of only the driver | operator D as a difference image, as shown to FIG.6 (E).

  Next, the feature point detection unit 32 of the image processing unit 30 detects the position of the feature point on the head of the driver D by performing image processing using the difference image acquired in Step 12 (S13). Here, examples of the feature points include a nose, eyes, ears, mouth, eyebrows, chin, and forehead.

  Subsequently, the face direction detection unit 33 of the image processing unit 30 detects the face direction of the driver D based on the position of the feature point on the driver D's head (S14). Next, the pupil position detection unit 34 of the image processing unit 30 specifies (estimates) the positions of the left and right pupils of the driver D based on the positions of the feature points on the driver D's head (S15).

  Then, the image processing unit 30 calculates the central part based on the positions of the left and right pupils estimated in step 15, and determines the central position of the eye box. Next, the image processing unit 30 determines whether or not the center position of the eye box has been determined (S16). When the center position of the eye box is determined, the process is terminated and the tracking calculation process of step 7 is executed. When the center position of the eye box is not detected, the steps 11 to 16 are performed. The process is repeated sequentially.

Next, the tracking calculation process in the tracking calculation unit will be described. FIG. 10 is a flowchart illustrating a processing procedure in the tracking calculation unit. When the center position of the eye box is detected by the image processing in step 7, the display optical path calculation unit 41 of the tracking calculation unit 40 calculates the optical path from the HUD light source (display unit 3) to the eye box center position (S21). The “optical path” here is the display optical path L ₁ shown in FIG.

Subsequently, the drive control unit 42 of the tracking calculation unit 40 controls the mirror drive unit 5 to adjust the angle of the mirror 4 so as to match the display optical path calculated in step 22 (S22). FIG. 11 is a schematic perspective view for explaining the relationship between mirror drive control and the display optical path. For example, the position of the eye box in the display optical path L _1b, when high adjust the position of the eye boxes, tilt the display optical path L _1b to the display unit 3 side to display the optical path L _1a to. On the other hand, the position of the eye box in the display optical path L _1b, when adjusting lowered position of the eye boxes, is inclined to the side opposite to the display light path L _1c in the display section 3 to display the optical path L _1b.

In the vehicle display device to the present embodiment, since the display optical path L ₁ is incident on the windshield 6 at an angle off the Brewster angle, whether crab driver D is wearing polarized sunglasses Regardless, the driver D can visually recognize the flat virtual image V. In addition, even when intense light such as sunlight enters the camera unit 2 from the windshield 6, an image in which the influence of sunlight is suppressed by generating a differential image between the S-polarized image and the P-polarized image. Can be acquired. Therefore, it is possible to capture the driver image from the front while suppressing the influence of sunlight.

  In this way, by capturing the virtual image of the driver D reflected by the windshield 6 using the polarizing plates 22A and 22B and the cameras 21A and 21B, the driver D is captured from the front without giving a sense of tension. The HUD capable of detecting the position of the driver's eyes, grasping the position of the reliable eye point, and automatically adjusting the position can be realized.

  Next, a first modification of the imaging device according to the present invention will be described. For example, a neutral density filter can be superimposed on the polarizing plates 22A and 22B. When sunlight incident from the outside is strong, it is possible to further reduce the influence of sunlight by using a neutral density filter superimposed on the polarizing plates 22A and 22B. When the above-described image processing unit 31 obtains a difference image based on the S-polarized image and the P-polarized image, the difference is greater when the neutral density filter is used than when the neutral density filter is not used. Since the contrast in the image increases, there is an advantage that it is easy to detect the feature point position of the head of the driver D.

  Next, a second modification of the imaging device according to the present invention will be described. In the above image processing, for example, the center position of the eye box is calculated based on the position of the feature point on the head such as the nose, eyes, ears, mouth, eyebrows, chin, and forehead. The position of the center of the eye box may be calculated based on the shape and position of the body. For example, the position of the center point of the eye box may be detected by detecting the shoulder shape of the driver D using the difference image and determining the physique of the driver.

  Further, in the above-described image processing, processing for determining the driving posture of the driver may be executed. FIG. 12 is a flowchart illustrating a procedure of driving posture determination processing executed by the image processing unit. The driving posture determination process shown in FIG. 12 is executed after the pupil position detection in step 15 is executed and before the eyebox center position determining process in step 16 is executed.

  In the driving posture determination process, first, based on the face direction of the driver D detected in step 14 and the positions of the left and right pupils detected in step 15, the driver's D line-of-sight position (the driver D sees). Direction) is detected (S31).

  Next, the image processing unit 30 performs the look-aside determination of the driver D based on the line-of-sight position detected in step 31 (S32). Aside look determination is performed by determining whether or not the line-of-sight position of the driver D is within a preset range. For example, the image processing unit 30 determines whether or not the angle between the reference line-of-sight position (for example, ahead of the traveling direction of the host vehicle) and the detected line-of-sight position of the driver D is equal to or greater than a determination threshold. . If the angle between the reference line-of-sight position and the driver's D line-of-sight position is greater than or equal to the determination threshold, it is determined that the driver is looking aside and the process proceeds to step 33, and the angle between the reference line-of-sight position and the driver D's line-of-sight Is not determined to be equal to or greater than the determination threshold value, the process proceeds to step 34 without determining that the driver is looking aside.

  In step 33, the image processing unit 30 outputs a warning signal to the display control unit 50 and causes the display unit 3 to display a warning image. By displaying the warning image, the driver D is alerted to stop driving aside.

  In step 34, the image processing unit 30 awakens based on the face direction of the driver D detected in step 14, the positions of the left and right pupils detected in step 15, and the line-of-sight direction detected in step 31. A determination is made (S34). When it is not less than the determination threshold that is a preset determination condition, it is determined that the arousal level is low, and the process proceeds to step 35, and when it is not determined that it is equal to or greater than the determination threshold, it is determined that the arousal level is high. The driving posture determination process is terminated without executing the process of step 35.

  In step 35, the image processing unit 30 outputs a warning signal to the display control unit 50 and causes the display unit 3 to display a warning image. The driver D is alerted by displaying the warning image. In addition to the warning image display, the driver D may be alerted by emitting a warning sound. After the driving posture discrimination process is completed, the process proceeds to step 16 to determine the center position of the eye box. By performing such a driving posture determination process, a warning can be issued according to the state of the driver D.

  Next, a third modification of the imaging device according to the present invention will be described. The imaging apparatus includes a first camera 21A that acquires an S-polarized image using the first polarizing plate 22A and a second camera 21B that acquires a P-polarized image using the second polarizing plate 22B. However, it is good also as an imaging device provided with one polarizing plate and one camera. In such an imaging apparatus, a driving device that changes the position of the polarizing plate is provided, and the rotation position of the polarizing plate is rotated 90 degrees to change the state shown in FIG. 6A and the state shown in FIG. 6C. Can be switched. As another method, a configuration in which the first polarization state and the second polarization state are electrically generated may be employed. In the camera unit, a strobe signal is output at predetermined intervals, and an image is captured by the camera that has received the strobe signal. In the polarization unit, the first polarization state and the second state are alternately generated according to the strobe signal. By setting it as such a structure, the image of the driver | operator D can be image | photographed using one polarizing plate and one camera.

  As mentioned above, although this invention was concretely demonstrated based on the embodiment, this invention is not limited to the said embodiment. The first polarizing means includes a reflected wave due to an incident wave from the subject (driver D) side of the projection surface (windshield 6) and an incident wave (sunlight from the side opposite to the subject side of the projection surface). ) May be realized in a first polarization state which is a reflected state. The second polarization unit may realize a second polarization state in which an incident wave from the opposite side of the projection surface is reflected.

  Further, the image pickup apparatus may be mounted on a vehicle, and may be mounted on another vehicle such as an airplane and image a driver who controls the vehicle. Further, for example, as an imaging device mounted on a display device used outdoors, an imaging device that captures a virtual image of a subject reflected by a combiner may be used. Moreover, you may use the imaging device of this invention with the display apparatus used indoors.

  M ... Vehicle display device (head-up display unit), 1 ... HUD main body, 2 ... Camera unit, 21A ... First camera, 21B ... Second camera, 22A ... First polarizing plate, 22A ... Second polarizing plate, 3 Display unit, 4 Mirror, 5 Mirror drive unit, 6 Windshield, 10 HUD-ECU, 30 Image processing unit, 31 Difference image acquisition unit, 32 Feature point detection unit, 33 Face detection 34, a pupil position detection unit, 40 a tracking calculation unit, 41 a display optical path calculation unit, 42 a drive control unit, and 50 a display control unit.

Claims (2)

First and second imaging means for capturing an image of the subject projected on the projection surface of the windshield ;
Provided on the optical path between the projection surface and the first imaging means, the reflected wave due to the incident wave from the subject side of the projection surface and the opposite side of the projection surface to the subject side First polarization means for realizing a first polarization state that is a state in which an incident wave from is transmitted or reflected;
Provided on the optical path between the projection surface and the second imaging means to realize a second polarization state that is a state in which an incident wave from the opposite side of the projection surface is transmitted or reflected A second polarizing means;
Differential image acquisition means for acquiring a differential image between a first captured image captured in the first polarization state and a second captured image captured in the second polarization state ;
An image processing unit that corrects the image according to the curvature of the projection surface with respect to the position of the first or second imaging unit ;
When the image processing unit is based on the position of the first imaging unit, the image processing unit corrects the image captured by the second imaging unit as if it was captured at the position of the first imaging unit, If the basis of the position of the second imaging means, the imaging apparatus characterized that you correcting the image captured by the first imaging means as imaged by the position of the second imaging means. A display device comprising the imaging device according to claim 1.
  A display unit for displaying a display image;
  A mirror that reflects the display image projected from the display unit toward the projection surface;
  A feature point detection unit that performs image processing on the difference image acquired by the difference image acquisition unit and detects a position of a feature point on the subject's head;
  A display optical path calculation unit that calculates a display optical path from the display unit to the subject;
  A drive control unit that drives and controls the mirror so as to form the display optical path,
  The first and second imaging means are installed on a reflected light path of a virtual image of the subject reflected on the windshield at a Brewster angle or an angle near the Brewster angle,
  The drive control unit controls the mirror so that the light emitted from the display unit forms the display optical path that is incident on the projection surface at an angle excluding the Brewster angle. .

Images