JP7679764B2 - Image display device, imaging device, and image processing device - Google Patents

Description

The present invention relates to an image display device, an imaging device, and an image processing device.

The imaging system of Patent Document 1 is provided with an adjustment mechanism that reduces changes in brightness of the captured image output from the imaging element of the camera device. The imaging system also provides a zoom function in the camera device, and when the zoom position is on the optical side of the threshold value, the adjustment mechanism is stopped while the wiper is operating, thereby preventing the image captured by the camera device from appearing unnatural.

The image display device of Patent Document 2 displays an image of the rear of the vehicle captured by a camera on a display. This image display device detects the illuminance in front of and behind the vehicle, and sets the display brightness to a first luminance when the front illuminance is less than a set value and the rear illuminance is equal to or greater than the set value, and sets the display brightness to a second luminance that is smaller than the first luminance when the front illuminance is equal to or greater than the set value and the rear illuminance is less than the set value.

Patent No. 5404250 Patent No. 6763324

However, raindrops and water films on the windshield scatter light. For this reason, when an image captured through the windshield is displayed on a display, if the light from the headlights of another vehicle shines on the windshield, the raindrops and water films will become bright and cause glare to the occupants viewing the display. Some cameras and other devices are equipped with an automatic dimming function, which adjusts the brightness to reduce glare.

However, for example, when the wiper device is operating, the wiper (wiper blade) repeatedly crosses the shooting area, causing repeated changes in brightness, such as flickering, in the image displayed on the display. Such changes in brightness (flickering) in the display can cause discomfort, such as restlessness, to the occupants viewing the display.

In addition, brightness adjustment using an automatic dimming function generally requires a certain amount of response time, and cannot respond to sudden blinking such as that which occurs when a wiper passes over the shooting area. Patent Document 1 was devised in consideration of such issues, but the causes of blinking when the wiper device is operating include the wiper itself blocking the shooting area, as well as the scattering of light caused by raindrops and water film on the windshield. In particular, when the water film spreads over a wide area of the glass surface due to the wiper wiping the glass surface, it has a significant effect on scattering of light, and the spreading of the water film caused by wiping is immediately eliminated after the wiper passes. For this reason, scattering of light occurs only after the wiper passes, and the immediate elimination of this scattering exacerbates the blinking.

The present invention was made in consideration of the above-mentioned facts, and aims to provide an image display device, an imaging device, and an image processing device that can obtain captured images that do not cause discomfort to passengers when displayed on a display.

The image display device of claim 1 for achieving the above object includes a display means for displaying a display image corresponding to a captured image of one side of the vehicle outside through the windshield by an imaging means, an environmental detection means for detecting an environmental state that can estimate whether water droplets are attached to the windshield and whether water droplets attached to the windshield affect the brightness of the image captured by the imaging means, and an adjustment means for adjusting the brightness of the display image to be lower when the environmental state detected by the environmental detection means estimates that water droplets are attached to the windshield and that the water droplets attached to the windshield affect the brightness of the display image, compared to when the environmental state detected by the environmental detection means does not estimate that water droplets are attached to the windshield or does not estimate that the water droplets attached to the windshield affect the brightness of the display image.

In the image display device of claim 1, a display image corresponding to a captured image of one side of the vehicle outside through the windshield by the imaging means is displayed on the display means. The environmental detection means detects an environmental state that can estimate whether water droplets are attached to the windshield, and whether water droplets attached to the windshield affect the brightness of the captured image by the imaging means. The adjustment means adjusts the brightness of the display image when it is displayed on the display means.

Here, when the environmental condition is estimated to be such that water droplets are attached to the windshield and the water droplets attached to the windshield are estimated to affect the brightness of the displayed image, the adjustment means suppresses the brightness of the displayed image more than when the environmental condition is estimated to be such that water droplets are not attached to the windshield or the water droplets attached to the windshield are estimated to affect the brightness of the displayed image.

As a result, even if the light from the headlights of another vehicle shines on a windshield with raindrops on it in a dark environment, the brightness of the image displayed on the display means is suppressed, so the image displayed on the display means will not be too bright and cause dazzling sensation.

The image display device of claim 2 is the same as claim 1, in which the environment detection means includes a light/dark detection means for detecting the brightness outside the vehicle, and a wiper device that wipes water droplets from the windshield when activated, and the adjustment means reduces the brightness of the displayed image when the brightness outside the vehicle is below a predetermined level and the wiper device is activated.

The image display device of claim 2 includes a light/dark detection means for detecting the brightness outside the vehicle as the environmental detection means, and a wiper device that is activated when it rains to wipe off water droplets adhering to the windshield.

The adjustment means reduces the brightness of the displayed image when the brightness outside the vehicle detected by the brightness detection means is equal to or lower than a predetermined brightness and the wiper device is operating.

This makes it possible to estimate how water droplets on the windshield affect the brightness of the image captured by the imaging means, so that even if the headlights of another vehicle shine on the windshield, the image displayed on the display means will not be too bright and cause dazzling sensations.

The image display device of claim 3 is the same as claim 1 or claim 2, in which the adjustment means suppresses the luminance of the display image when the display image is displayed on the display means, thereby suppressing the brightness of the display image.

In the image display device of claim 3, when the display image is displayed on the display means, the adjustment means suppresses the luminance of the display image to suppress the brightness of the display image. For example, color correction (tone correction) or gamma correction can be applied to suppress the luminance. This prevents the image displayed on the display means from being perceived as dazzling even if the captured image is too bright.

The image display device of claim 4 is any one of claims 1 to 3, in which the adjustment means reduces the brightness of the display means to reduce the brightness of the displayed image.

In the image display device of claim 4, the adjustment means reduces the brightness of the display means, thereby reducing the brightness of the displayed image. To reduce the brightness of the display means, for example, reduction in the brightness of the backlight of an LCD or the like can be applied. This prevents the image displayed on the display means from being too dazzling even if the captured image becomes too bright.

The image display device of claim 5 is the same as claim 1 or claim 2, in which the adjustment means reduces the brightness of the image displayed on the display means to suppress the brightness of the displayed image.

In the image display device of claim 5, the adjustment means reduces the brightness of the image displayed on the display means to suppress the brightness of the displayed image. This suppresses the brightness of the image displayed on the display means (darkens it), so that the image displayed on the display means does not cause a sense of glare even if the captured image becomes too bright.

The imaging device of claim 6 includes an imaging means for imaging one side of the vehicle outside through a windshield, an environmental detection means for detecting an environmental state that can estimate whether water droplets are attached to the windshield and whether water droplets attached to the windshield affect the brightness of an image captured by the imaging means, and an adjustment means for adjusting the brightness of the captured image when the environmental state detected by the environmental detection means estimates that water droplets are attached to the windshield and that the water droplets attached to the windshield affect the brightness of the image captured by the imaging means, compared to when the environmental state detected by the environmental detection means does not estimate that water droplets are attached to the windshield or does not estimate that the water droplets attached to the windshield affect the brightness of the captured image.

In the imaging device of claim 6, when the environmental condition is estimated to be that water droplets are attached to the windshield and that the water droplets attached to the windshield affect the brightness of the captured image, the brightness of the captured image is suppressed compared to when the environmental condition is estimated to be that water droplets are not attached to the windshield or that the water droplets attached to the windshield are not estimated to affect the brightness of the captured image.

As a result, when capturing an image through raindrop-covered windshield in a dark environment, even if the light from the vehicle's headlights is shone on the captured image, the captured image is prevented from becoming too bright, and even when the captured image is displayed on a display device, the user does not feel any glare.

The image processing device of claim 7 includes an environmental detection means for detecting an environmental condition that can estimate whether water droplets are attached to the windshield and whether water droplets attached to the windshield affect the brightness of an image captured by an imaging means, and an adjustment means for adjusting the brightness of an image captured by an imaging means when the environmental condition detected by the environmental detection means estimates that water droplets are attached to the windshield and that the water droplets attached to the windshield affect the brightness of the image, compared to when the environmental condition detected by the environmental detection means does not estimate that water droplets are attached to the windshield or when the water droplets attached to the windshield are not estimated to affect the brightness of the image.

In the image processing device of claim 7, even if the headlights of a vehicle are irradiated onto windshield that is assumed to have water droplets and that the adhering water droplets are assumed to affect the brightness of the captured image, the captured image captured through the windshield is prevented from becoming dazzling, so that even if the captured image is displayed on the display means, the user does not feel any glare.

As described above, according to the present invention, the image displayed on the display means does not cause glare, and therefore the image displayed on the display means does not cause discomfort.

1 is a schematic configuration diagram of a digital mirroring system according to a first embodiment. 1 is a plan view showing a schematic configuration of a vehicle according to a first embodiment. FIG. 2 is a front view showing an example of an inner mirror. 5 is a flowchart showing an outline of a dimming process setting according to the first embodiment. (A) to (C) are front views of the inner mirror displaying an image at night, where (A) shows an image that has been dimmed when it is raining, (B) shows an image that is displayed when it is not raining, and (C) shows an image that has not been dimmed when it is raining. 4 is a diagram showing an outline of conversion characteristics of the luminance values of an output image relative to the luminance values of an input image, which are applied to luminance adjustment in the first embodiment; FIG. FIG. 7 is a diagram showing an example of color tone characteristics applied to luminance adjustment in place of FIG. 6 . 10A is a schematic diagram of an electronic mirror system according to a second embodiment, and FIG. 10B is a diagram showing an example of gamma characteristics for a display according to the second embodiment. FIG. 11 is a schematic configuration diagram of an electronic mirroring system according to a third embodiment. 13 is a flowchart showing an outline of a main part of a dimming process setting according to the third embodiment. FIG. 13 is a schematic configuration diagram of an electronic mirroring system according to a fourth embodiment. 13 is a flowchart showing an outline of a dimming process setting according to the fourth embodiment. FIG. 13 is a schematic configuration diagram of an electronic mirroring system according to a fifth embodiment. 13 is a flowchart showing an outline of a dimming process setting according to the fifth embodiment. FIG. 13 is a schematic configuration diagram of an electronic mirroring system according to a sixth embodiment. FIG. 13 is a schematic configuration diagram of an imaging device according to a seventh embodiment. FIG. 23 is a plan view showing a schematic configuration of a vehicle according to a seventh embodiment. 23 is a flowchart showing an outline of a dimming process setting according to the seventh embodiment. FIG. 13 is a schematic configuration diagram of an imaging device according to an eighth embodiment. 13 is a flowchart showing an outline of a dimming process setting according to the eighth embodiment. 10 is a diagram showing an outline of a change in brightness of a captured image when a wiper device is not in operation and when it is in operation;

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
First Embodiment
Fig. 1 shows a schematic configuration diagram of an electronic mirror system 10 according to the first embodiment, and Fig. 2 shows a schematic configuration of a vehicle 12 according to the first embodiment in a plan view. In the drawings, the front side of the vehicle is indicated by an arrow FR, the right side in the vehicle width direction is indicated by an arrow HR, and the upward direction is indicated by an arrow UP.

As shown in FIG. 2, the vehicle 12 has a front windshield glass (windshield) 16F installed at the front of the vehicle interior 14, and a rear windshield glass (rear glass) 16R installed as a windshield at the rear of the vehicle interior 14. In addition, the vehicle interior 14 is equipped with a driver's seat 14D, a passenger seat 14P on the left side of the driver's seat 14D in the vehicle width direction, and a row of rear seats 14R on the rear side of the driver's seat 14D and the passenger seat 14P. One-box cars and two-box cars such as station wagons, minivans, and SUVs (Sport Utility Vehicles) can be used as such vehicles 12. In addition, various types of automobiles such as three-box cars (so-called sedan types) can be used as the vehicle 12.

The vehicle 12 is equipped with a wiper device 18 and a rear wiper device 20. The wiper device 18 is installed on the windshield 16F, and when a wiper switch (not shown) is turned on, the wiper 18A (wiper blade) is operated to wipe the outer surface (front surface of the vehicle) of the windshield 16F. This removes raindrops and other debris adhering to the windshield 16F.

The rear wiper device 20 is installed on the rear windshield 16R, and when a rear wiper switch (not shown) is turned on, the wiper 20A (wiper blade) is operated to wipe the outer surface (front surface of the vehicle) of the rear windshield 16R and remove raindrops, etc.

The vehicle 12 may be equipped with an automatic light device 24 that turns on and off the headlights 22. The automatic light device 24 detects the brightness (darkness) of the surroundings of the vehicle 12 using an illuminance sensor (not shown) or the like, and turns on the headlights 22, etc. (sidelights and taillights (not shown), etc.) when the surroundings of the vehicle 12 reach a predetermined brightness (darkness set when the headlights 22 are turned on).

As shown in Figures 1 and 2, the electronic mirror system 10 includes an imaging device 26, an inner mirror 28 as a display means, and a display controller (display ECU) 30 for displaying an image captured by the imaging device 26 on the inner mirror 28. In the first embodiment, the inner mirror 28 and the display controller 30 function as an image display device, and the display controller 30 functions as an image processing device.

The imaging device 26 includes a camera (camera device) 32 as an imaging means, and the camera 32 is an imaging element (image sensor) such as a CCD image sensor or a CMOS image sensor. The camera 32 is installed in the vehicle interior 14 on the rear side of the vehicle (the vehicle interior 14 side of the rear window 16R) and at the upper part of the center in the vehicle width direction (the ceiling surface side), and the camera 32 captures an image of the rear of the vehicle as one side of the outside of the vehicle through the rear window 16R.

As shown in FIG. 2, the inner mirror 28 is installed in the vehicle interior 14 at the front of the vehicle (the inside of the vehicle interior 14 of the windshield 16F) and at the upper part of the center in the vehicle width direction, and the display surface of the inner mirror 28 faces the rear of the vehicle. As a result, the inner mirror 28 does not obstruct the driver's view of the area in front of the vehicle seated in the driver's seat 14D, allowing the driver to view the displayed image. FIG. 3 shows an outline of the inner mirror 28 in a front view looking rearward from the vehicle.

As shown in FIG. 3, the inner mirror 28 is provided with a display (monitor) 34 constituting a display means, and the display 34 uses an LCD (Liquid Crystal Display) and LEDs, etc. (both not shown). An electrical signal corresponding to image data is input to the display 34, and an image (video) corresponding to the image data is displayed on the inner mirror 28. The inner mirror 28 is provided with a semi-transparent half mirror on the front side (rear side of the vehicle) of the display 34, which can function as a mirror, and the half mirror functions as an optical mirror when the display 34 is in an inoperative state (no image is displayed or the backlight is turned off).

As shown in FIG. 1, the display controller 30 is formed with an image processing unit 36 and an environment detection unit 38, and the image processing unit 36 is formed with a brightness adjustment unit 40 as an adjustment means. The image capture device 26 and the inner mirror 28 (display 34) are connected to the display controller 30, and the display controller 30 applies a predetermined image processing to the captured image (image data) captured by the image capture device 26 (camera 32) in the image processing unit 36 and outputs the processed image to the inner mirror 28 (display 34).

Such a display controller 30 includes a microcomputer in which a CPU, ROM, RAM, storage as non-volatile memory, and an input/output interface are connected to each other via a bus for data communication, and required functional circuits are connected to the input/output interface (all not shown). In the display controller 30, the CPU reads out programs for image processing and environment detection stored in the ROM and storage, and executes them while expanding them into the RAM, thereby realizing the functions of the image processing unit 36, environment detection unit 38, and brightness adjustment unit 40.

The image processing unit 36 performs the necessary image processing on the image data (video data) of the captured image captured by the camera 32 in order to display an image corresponding to the captured image on the inner mirror 28 (display 34). The necessary image processing in the image processing unit 36 includes viewpoint conversion to match the captured image to the driver's viewpoint, and processing to invert the captured image from left to right.

The brightness adjustment unit 40 adjusts the brightness of the captured image so that the brightness (luminance) of the image displayed on the display 34 falls within a predetermined range. The brightness adjustment unit 40 can switch between normal processing and dimming processing in which the brightness (brightness) is suppressed (dimmed) more than in the normal processing. In the electronic mirror system 10, normal processing is performed in the brightness adjustment unit 40, so that when an image captured by the camera 32 is displayed on the inner mirror 28 (display 34), the brightness is suppressed if the area behind the vehicle is too bright, and the darkness is suppressed (brightened) if the area behind the vehicle is too dark.

The brightness adjustment unit 40 also switches between normal processing and dimming processing depending on the detection result (determination result) of the environment detection unit 38.

An environmental detection means is connected to the environmental detection unit 38, and the environmental detection means detects the environmental condition to estimate whether water droplets are attached to the rear window 16R and whether the water droplets attached to the rear window 16R affect the brightness of the image captured by the camera 32. The environmental detection means includes a light/dark detection means for estimating the brightness (light/dark or day/night) around the vehicle 12, and a weather detection means (rainfall detection means) for estimating rainfall as a weather condition around the vehicle 12.

The environment detection unit 38 determines (estimates) whether the brightness of the surroundings of the vehicle 12 is equal to or lower than a preset brightness based on the detection result of the light/darkness detection means. As the light/darkness detection means, an illuminance sensor (or a sunshine sensor) for detecting the brightness outside the vehicle can be used, but in the first embodiment, the automatic light device 24 is used, and the automatic light device 24 is connected to the environment detection unit 38.

The brightness of the image of the rear of the vehicle captured by the camera 32 of the imaging device 26 changes depending on the brightness of the surroundings, and the captured image becomes darker as the surroundings become darker. For this reason, in the first embodiment, in addition to the auto light device 24, the imaging device 26 (captured image) is used as a light/dark detection means, and the imaging device 26 is connected to the environment detection unit 38.

The brightness (light and dark) around the vehicle 12 can be obtained from calendar information indicating sunrise and sunset at the driving location, and therefore a GPS device capable of acquiring the position information of the vehicle 12 and calendar information corresponding to the position information can be applied to the light and dark detection means.

The weather detection means detects whether the surrounding weather is rain (which may include snow or fog). A raindrop sensor can be used as the weather detection means, but in the first embodiment, the wiper device 18 that the driver operates in rainy weather can be used as the weather detection means. In the first embodiment, the wiper device 18 and the rear wiper device 20 are connected to the environment detection unit 38. Note that if the rear wiper device 20 is not provided, only the wiper device 18 may be provided.

The environment detection unit 38 is connected to the brightness adjustment unit 40, and the brightness adjustment unit 40 switches from normal processing to dimming processing when the environment detection unit 38 determines that the brightness around the vehicle 12 is below a predetermined level and that rainfall has been detected. In the first embodiment, the environment detection unit 38 sets normal processing or dimming processing based on the detection result of the environment detection means, and the brightness adjustment unit 40 performs normal processing or dimming processing based on this setting.

Next, the operation of the first embodiment will be described.
The electronic mirror system 10 is activated and starts operation when an ignition switch (not shown) of the vehicle 12 is turned on, and stops operation when the ignition switch is turned off. When the electronic mirror system 10 is operated, the imaging device 26 captures an image of the rear of the vehicle through the rear window 16R using the camera 32, and displays an image corresponding to the captured image on the inner mirror 28 disposed in front of the driver's seat 14D. This allows the driver to see the rear of the vehicle in the electronic mirror system 10.

On the other hand, if raindrops or a water film created by wiping off raindrops are attached to the rear windshield 16R, the water film attached to the rear windshield 16R will be reflected (displayed) in the inner mirror 28. In addition, the water film attached to the rear windshield 16R tends to diffusely reflect light that is irradiated onto it. For this reason, for example, when it is dark and raining, such as at night, the rear windshield 16R will receive light from the headlights of the vehicle behind, causing the image displayed on the inner mirror 28 to become too bright. This causes the driver looking at the inner mirror 28 to feel dazzled, reducing the driver's visibility behind the vehicle (causing dazzle).

From this point, the electronic mirror system 10 executes a brightness adjustment process to prevent glare (suppress glare). Figure 4 shows a flow chart outlining the brightness adjustment process (settings for brightness adjustment) to prevent glare.

The flowchart in FIG. 4 is executed by the display controller 30 at a predetermined time interval when the electronic mirror system 10 is started, and ends when the electronic mirror system 10 is stopped. When the electronic mirror system 10 is started, the display controller 30 is set so that the brightness adjustment unit 40 performs normal processing (normal brightness processing).

As shown in FIG. 4, in the first step 200, the display controller 30 performs brightness detection using the image captured by the camera 32, and in step 202, it checks whether the brightness of the captured image is equal to or lower than a predetermined threshold value. In step 204, the display controller 30 performs brightness detection using the operation of the automatic light device 24, and in step 206, it checks whether the automatic light device 24 is operating.

A threshold value for determining the brightness (luminance value) detected from the image captured by the camera 32 is set in the display controller 30, and the display controller 30 detects the brightness of the surroundings of the vehicle 12 from the image captured by the camera 32. The brightness (luminance value) of the captured image for determining the brightness of the surroundings of the vehicle 12 can be the brightness of the peripheral part of the captured image (average brightness value), etc. Also, the threshold value for the captured image can be the brightness (luminance value) at which vehicles traveling nearby may turn on their headlights.

If the display controller 30 judges negative in step 202 because the brightness (luminance value) detected from the image captured by the camera 32 exceeds the threshold value, and if the display controller 30 judges negative in step 206 because the automatic light device 24 is not operating, the display controller 30 proceeds to step 208. In other words, if the display controller 30 judges that the brightness of the surroundings of the vehicle 12 exceeds the threshold value in both the image captured by the camera 32 and the automatic light device 24, the display controller 30 proceeds to step 208.

The display controller 30 sets the display to perform normal processing as the processing for adjusting the brightness of the captured image in step 208. As a result, the image of the rear of the vehicle captured by the camera 32 is displayed on the inner mirror 28 after being normally processed for brightness.

When the display controller 30 detects that the brightness around the vehicle 12 is below a predetermined level in at least one of the images captured by the camera 32 and the automatic light device 24, it makes a positive determination in step 202 or step 206 and proceeds to step 210.

In step 210, the display controller 30 detects rainfall using the wiper device 18. In step 210, the display controller 30 acquires the operation of the wiper device 18, and in step 212, checks whether the wiper device 18 is operating. At this time, if the wiper device 18 is turned off (not operating), the display controller 30 makes a negative determination in step 212 and proceeds to step 208. As a result, even if the display controller 30 determines that the surroundings of the vehicle 12 are dark, if rainfall is not detected, brightness adjustment is performed by normal processing.

In contrast, when it is determined that the surroundings are dark and the driver operates the wiper switch (not shown) to operate the wiper device 18 to remove raindrops adhering to the windshield 16F, the display controller 30 makes a positive determination in step 212 and proceeds to step 214.

In step 214, the display controller 30 sets the display to perform dimming processing as a process for adjusting the brightness of the captured image. That is, the display controller 30 sets the display to perform dimming processing as a brightness adjustment when the surrounding brightness is equal to or lower than a threshold value, or when it is estimated that the surroundings have become dark due to the automatic light device 24 being activated, and it can be determined (estimated) that it is raining. As a result, an image corresponding to the captured image of the rear of the vehicle captured by the camera 32 is dimmed and displayed on the inner mirror 28.

When the display controller 30 is set to perform dimming processing for brightness adjustment, in step 216, it performs brightness detection using the image captured by the camera 32, and in step 218, it checks whether the brightness of the captured image exceeds a threshold value. In addition, in step 220, the display controller 30 performs brightness detection using the automatic light device 24, and in step 222, it checks whether the surroundings are bright (whether the operation of the automatic light device 24 has been stopped). In addition, in step 224, the display controller 30 performs rainfall detection using the wiper device 18, and in step 226, it checks whether the rain has stopped (whether the wiper device 18 has been stopped).

As a result, the display controller 30 continues the dimming process if the brightness (luminance value) of the captured image is below the threshold value (negative determination in step 218), if the automatic light device 24 is operating (negative determination in step 222), or if the wiper device 18 is operating (negative determination in step 226).

Therefore, even if the vehicle behind has its headlights on, the image displayed on the inner mirror 28 does not cause the occupants to feel dazzled. In addition, while the surroundings are dark and the wiper device 18 is operating, the brightness of the image displayed on the inner mirror 28 is kept suppressed, so the occupants do not feel the image flickering.

In addition, if the brightness (luminance value) of the captured image exceeds the threshold value, the automatic light device 24 is stopped, and the wiper device 18 is stopped, the display controller 30 makes a positive determination in each of steps 218, 222, and 226 and proceeds to step 208. As a result, the luminance adjustment in the luminance adjustment unit 40 is returned from the dimming process to the normal process.

5(A) to 5(C) show a schematic front view of the inner mirror 28 on which an image corresponding to an image captured by the camera 32 at night is displayed, and Fig. 6 shows a schematic diagram of the conversion characteristics used for brightness adjustment, and Fig. 5(A) shows the inner mirror 28 on which an image dimmed during rain is displayed, Fig. 5(B) shows the inner mirror 28 on which an image (image with normal brightness processing) is displayed when it is not raining, and Fig. 5(C) shows the inner mirror 28 on which an image not dimmed during rain is displayed. In Fig. 6, the horizontal axis is the brightness (brightness value) of the input image, and the vertical axis is the output (output brightness level), and the conversion characteristics applied to normal processing are shown by dashed lines, and the conversion characteristics applied to suppression processing are shown by solid lines.

As shown in FIG. 6, the conversion characteristics applied to the dimming process are shifted to lower the luminance value compared to the conversion characteristics applied to the normal process. As a result, the image displayed on the inner mirror 28 is less bright overall when the dimming process is applied (see FIG. 5(A)) compared to when the normal process is applied (see FIG. 5(B)).

On the other hand, when it is raining, raindrops adhere to the rear windshield 16R, and when the rear wiper device 20 is operated, a water film or the like is formed. As shown in FIG. 5(C), when the headlights of a vehicle behind are shone on the rear windshield 16R with water droplets or a water film attached thereto at night, the water droplets or the like diffuse the light of the headlights. In particular, when the rear wiper device 20 is operated and a water film is formed on the rear windshield 16R, not only does the water film area become brighter, causing the driver to feel dazzled, but the visibility of the displayed image (the driver's visibility behind the vehicle) is also reduced.

Here, the electronic mirror system 10 is set to perform dimming processing when it is determined that it is dark around the vehicle 12 and raining, and adjusts the brightness so that the brightness value of the displayed image is lowered. Therefore, as shown in FIG. 5(A), the brightness of the image displayed on the inner mirror 28 is suppressed. This prevents the driver from feeling dazzled and does not reduce visibility behind the vehicle.

In this way, the electronic mirror system 10 uses the image captured by the camera 32 and the automatic light device 24 for light/dark detection, and the wiper device 18 for rain detection. This makes it possible to reliably estimate whether the image displayed on the inner mirror 28 is likely to be dazzling. Furthermore, because the function of the vehicle 12 is used to determine (estimate) whether the image displayed on the inner mirror 28 is likely to be perceived as dazzling, there is no need to add a functional component to prevent the image displayed on the inner mirror 28 from being perceived as dazzling.

In the electronic mirror system 10, when detecting light and dark using the image captured by the camera 32, the brightness (luminance value) of the captured image is applied, but this is not limited to this, and the contrast of the captured image may also be applied.

In addition, the display controller 30 switches from dimming processing to normal processing when the brightness (luminance value) of the captured image exceeds the threshold value, the operation of the auto light device 24 is stopped, and the wiper device 18 is stopped (positive judgments in each of steps 218, 222, and 226). However, the switch from dimming processing to normal processing may be made only when the light from the headlights of the rear vehicle or the like is no longer dazzling. For example, regardless of the brightness of the surroundings, the switch may be made to normal processing after a predetermined time (a time when it can be determined that the influence of water droplets on the rear windshield 16R has disappeared, for example, a preset time of several minutes to several dozen minutes) has elapsed since it can be determined that the rain has stopped, such as by stopping the operation of the wiper device 18. In addition, when the surroundings become bright enough to turn off the headlights, the switch may be made to normal processing regardless of whether it is raining or not (regardless of whether the wiper device 18 is operating/stopped).

In addition, the luminance adjustment unit 40 uses the conversion characteristics shown in FIG. 6 for the luminance adjustment. However, tone characteristics may be applied to the luminance adjustment, or gamma characteristics (gamma coefficients) may be applied instead of tone characteristics. Furthermore, lightness adjustment may be applied to the luminance adjustment. FIG. 7 shows an outline of an example of tone characteristics (tone curve) applied to the luminance adjustment in a line diagram. In FIG. 7, the horizontal axis represents the brightness (luminance value) of the input image, and the vertical axis represents the brightness (output luminance level) that is output.

As shown in Figure 7, when applying color tone characteristics to brightness adjustment, the color tone characteristics applied to the dimming process (shown by the solid line in Figure 7) are set to be darker than the color tone characteristics applied to normal processing (shown by the dashed line in Figure 7) in areas where the brightness of the input image is relatively dark. Also, when applying gamma (gamma correction) to brightness adjustment, the gamma coefficient γ is made smaller for the dimming process compared to normal processing. Furthermore, when applying brightness adjustment, the brightness of the captured image is adjusted to be lower in the dimming process than in normal processing.

As a result, the dimming process emphasizes black in the image of the rear of the vehicle displayed on the inner mirror 28, reducing the overall brightness (effectively dimming). As a result, even if the light from the headlights of the vehicle behind is irradiated onto water droplets or a water film adhering to the rear windshield 16R, the water droplets or water film areas are prevented from becoming too bright, and the driver does not feel dazzled when viewing the image displayed on the inner mirror 28.

Second Embodiment
Next, a second embodiment of the present invention will be described. In the second embodiment, the same functional parts as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and detailed description thereof will be omitted.

Figure 8 (A) shows a schematic diagram of an electronic mirror system 50 according to the second embodiment. In the second embodiment, brightness adjustment is performed by varying gamma correction for the image displayed on the display 34, and Figure 8 (B) shows a diagram of an example of gamma characteristics applied to the gamma correction for the display 34. In Figure 8 (B), the horizontal axis represents the brightness level of the input image, and the vertical axis represents the brightness level of the output image. In Figure 8 (B), the dashed line shows the gamma characteristics applied to normal processing, and the solid line shows the gamma characteristics applied to dimming processing.

As shown in FIG. 8A, in the electronic mirror system 50, a brightness adjustment unit 54 is formed as an adjustment means in the display controller 52. The electronic mirror system 50, the display controller 52, and the image processing unit 56 of the second embodiment differ from the electronic mirror system 10, the display controller 30, and the image processing unit 36 of the first embodiment in that the brightness adjustment unit 54 is formed instead of the brightness adjustment unit 40.

The image processing unit 56 performs the necessary image processing on the image data of the captured image captured by the camera 32 in order to display the captured image on the inner mirror 28 (display 34). The processing performed by the image processing unit 56 includes viewpoint conversion to match the captured image to the driver's viewpoint, and processing such as flipping the captured image left to right.

The brightness adjustment unit 54 adjusts the brightness of the image displayed on the display 34 by performing gamma correction as a brightness adjustment (brightness adjustment) for the image (image data) output from the image processing unit 56 to the display 34 using gamma characteristics that have been preset for the display 34.

As shown in FIG. 8 (B), the brightness adjustment unit 54 uses different gamma characteristics (gamma coefficients) between normal processing and dimming processing. In dimming processing, the gamma coefficient is set so that the output brightness is lower relative to the input brightness compared to normal processing, and the gamma coefficient applied to dimming processing is smaller than the gamma coefficient applied to normal processing. As a result, the brightness adjustment unit 54 applies different gamma coefficients between normal processing and dimming processing.

As shown in FIG. 8 (A), the detection result of the environment detection unit 38 (the brightness processing setting according to the judgment result) is input to the brightness adjustment unit 54. The brightness adjustment unit 54 switches between normal processing and dimming processing based on the detection result (the brightness processing setting) input from the environment detection unit 38.

In the display controller 52 of the electronic mirror system 50 configured in this manner, normal processing is set based on the detection result of the environment detection unit 38, and the brightness adjustment unit 54 adjusts the brightness of the image output from the image processing unit 56 to the display 34 based on the gamma characteristic set for normal processing. In addition, when the image captured by the camera 32 and the operating state of the automatic light device 24 indicate that the brightness around the vehicle 12 is below a predetermined level, and when the operating state of the wiper device 18 indicates that it is raining (rainfall), the display controller 52 sets the dimming process (see FIG. 4 of the first embodiment).

When dimming processing is set, the brightness adjustment unit 54 adjusts the brightness of the image (image data) output from the image processing unit 56 to the display 34 based on the gamma characteristic set for the dimming processing. As a result, the brightness is suppressed and the brightness is reduced in the image of the rear of the vehicle displayed on the inner mirror 28. This prevents the water droplets or water film from becoming too bright even if the headlights of the vehicle behind are turned on at night or other times when there are water droplets or a water film on the rear windshield 16R.

Therefore, in the electronic mirror system 50, like the electronic mirror system 10, the rear image displayed on the inner mirror 28 during rainy hours, such as at night, does not cause the driver to feel dazzled, and does not reduce visibility behind the vehicle.

On the other hand, in the electronic mirror system 50, the brightness of the image displayed on the inner mirror 28 is adjusted by performing a different specific gamma correction in the brightness adjustment unit 54. The adjustment of the brightness of the image displayed on the inner mirror 28 is not limited to this, and the brightness (luminance) of a backlight (not shown) provided on the display 34 may also be adjusted. In this case, in the dimming process, the brightness of the backlight may be suppressed (the amount of light may be reduced) compared to normal processing. This prevents the water droplets or water film from becoming too bright even if the light from the headlights of the vehicle behind is irradiated onto water droplets or water film adhering to the rear windshield 16R.

Third Embodiment
Next, a third embodiment of the present invention will be described. In the third embodiment, functional parts similar to those in the first or second embodiment are denoted by the same reference numerals as the corresponding functional parts in the first or second embodiment, and detailed description thereof will be omitted.

FIG. 9 is a schematic diagram showing the configuration of an electronic mirror system 60 according to the third embodiment.
As shown in FIG. 9, the electronic mirror system 60 includes an imaging device 26, an inner mirror 28, and a display controller 62. The display controller 62 includes an image processing unit 66 in which a luminance adjustment unit 64 is formed, an adjustment switching unit 68 as a switching means, and an environment detection unit 70. In the electronic mirror system 60 according to the third embodiment, the environment detection unit 70 of the display controller 62 has a function similar to that of the environment detection unit 38 of the first embodiment. The luminance adjustment unit 64 formed in the image processing unit 66 of the display controller 62 performs normal processing and dimming processing, but differs from the luminance adjustment unit 40 of the first embodiment in that a first dimming processing and a second dimming processing are set in the dimming processing. The display controller 62 also includes an adjustment switching unit 68, and differs from the display controller 30 of the first embodiment in that the luminance adjustment unit 64 is formed instead of the luminance adjustment unit 40 in the display controller 62, and the adjustment switching unit 68 is formed.

The brightness adjustment unit 64 adjusts the brightness in the same manner as the brightness adjustment unit 40 of the first embodiment. In addition, the brightness adjustment unit 64 has set a normal process, a first dimming process for suppressing (dimming) the brightness (brightness) more than the normal process, and a second dimming process for further suppressing (dimming) the brightness (brightness) more than the first dimming process in the brightness adjustment process. The brightness adjustment unit 64 is switched between the normal process, the first dimming process, and the second dimming process according to the detection result of the environment detection unit 70.

The adjustment switch 68 is connected to a rear wiper device 20 for wiping raindrops off the rear windshield 16R. The rear wiper device 20 applied to the third embodiment can select a continuous mode in which the wiper 20A operates continuously, and an intermittent mode in which the wiper 20A operates intermittently at a predetermined time interval. The continuous mode and the intermittent mode are selected by operating a rear wiper switch (not shown).

When the environment detection unit 70 detects that the brightness around the vehicle 12 is below a predetermined level and it is estimated that rain is falling, it switches from normal processing to dimming processing. When the environment detection unit 70 is set to perform dimming processing, the adjustment switching unit 68 sets the adjustment switching unit 68 to perform dimming processing in response to the operation of the rear wiper device 20.

The average wiping speed (movement speed) of the wiper 20A (movement time per unit time) differs between the continuous mode and the intermittent mode of the rear wiper device 20, and the average wiping speed is faster in the continuous mode than in the intermittent mode. Water droplets adhering to the rear windshield 16R are spread on the rear windshield 16R by being wiped away by the wiper 20A, forming a water film, etc., which gradually narrows. In this case, when there is a lot of rain, the continuous mode is often selected, and even if the wiper 20A wipes away frequently, the water film is likely to remain, and the remaining water film causes glare in the image displayed on the inner mirror 28.

From here, the adjustment switching unit 68 sets the brightness adjustment unit 64 to perform the second dimming process when the rear wiper device 20 is operated in the continuous mode, and sets the brightness adjustment unit 64 to perform the first dimming process when the rear wiper device 20 is operated in the intermittent mode.

Next, the dimming process executed by the display controller 62 as the operation of the third embodiment will be described.
4 is executed in the electronic mirror system 60 in the same manner as the display controller 30 of the first embodiment. As a result, the display controller 62 switches from normal processing to dimming processing when it is determined that the brightness of the surroundings of the vehicle 12 is equal to or lower than a predetermined value and rainfall is detected, and switches from dimming processing to normal processing when the brightness of the surroundings exceeds the predetermined value and rainfall is no longer detected.

Here, when the dimming process is set in the display controller 62, the dimming process is executed in response to the operation of the rear wiper device 20, and the strength of the dimming process is changed. Figure 10 shows a flow chart outlining the dimming process setting process executed by the display controller 62 in the electronic mirror system 60 by transitioning to step 214 in Figure 4.

The flowchart in FIG. 10 is executed at a predetermined time interval when the display controller 62 sets the environment detection unit 70 to perform dimming processing (transition to step 214 in FIG. 4), and the operation of the rear wiper device 20 is detected in the first step 230, and it is confirmed in step 232 whether the rear wiper device 20 is operating or not. At this time, if the rear wiper device 20 is not operating, a negative judgment is made in step 232, and the process proceeds to step 216 in FIG. 4. As a result, the brightness adjustment unit 64 performs normal processing on the captured image. Note that when the flowchart in FIG. 4 is applied in the third embodiment, if a negative judgment is made in any of steps 218, 222, and 226, the process returns to step 214.

In contrast, when the rear wiper device 20 is operated, the display controller 62 makes a positive determination in step 232 and proceeds to step 234 to check whether the rear wiper device 20 is operating in the intermittent mode. At this time, if the rear wiper device 20 is operating in the intermittent mode, the display controller 62 makes a positive determination in step 234 and proceeds to step 236 to set the first dimming process as the dimming process. As a result, the brightness adjustment unit 64 performs the first dimming process on the captured image, and an image with reduced brightness compared to normal processing is displayed on the inner mirror 28.

If the rear wiper device 20 is operating in the continuous mode, the display controller 62 makes a negative determination in step 234 and proceeds to step 238, where it sets the second dimming process as the dimming process. As a result, the brightness adjustment unit 64 performs the second dimming process on the captured image, which reduces the brightness more than the first dimming process, and an image with even less brightness than the normal process is displayed on the inner mirror 28.

Furthermore, the display controller 62 checks in step 240 whether the rear wiper device 20 has been stopped, and if the rear wiper device 20 has not been stopped, a negative determination is made in step 240, and the process proceeds to step 234. As a result, the first dimming process or the second dimming process continues.

On the other hand, when the rear wiper device 20 is stopped, the display controller 62 makes a positive determination in step 240 and proceeds to step 242. In step 242, the display controller 62 sets the normal processing and ends the setting processing of the dimming processing. Note that, when setting the normal processing in step 242, the display controller 62 sets the normal processing after a predetermined time (for example, several tens of seconds to several minutes, a time during which the change in the display image of the inner mirror 28 does not bother the driver) has elapsed. As a result, in the electronic mirror system 60, even if the rear wiper device 20 is repeatedly turned on and off or the operation time is short, the brightness of the display image of the inner mirror 28 is suppressed from flickering, preventing the driver from feeling uncomfortable.

In the third embodiment, the dimming process is set to two stages. However, the dimming process may be set to three stages. In this case, a third dimming process that reduces the light more weakly than the first dimming process is provided between the normal process and the first dimming process, and the third dimming process is set when the rear wiper device 20 is not operating (when a negative determination is made in step 232 in FIG. 10).

In addition, in the third embodiment, the intensity of dimming of the captured image (image displayed on the inner mirror 28) is varied depending on whether the rear wiper device 20 is in intermittent mode or continuous mode. However, the wiping speed (average wiping speed, wiping frequency) of the wiper 20A may be detected from the captured image of the camera 32, and the higher the wiping speed, the stronger the dimming may be.

In this case, the wiper (wiper blade) 20A can be detected from the captured image by extracting the shadow of the wiper 20A reflected in the captured image. In addition, the wiping speed of the wiper 20A can be calculated as an average movement amount per unit time, etc., from the change in the extracted position of the wiper 20A by extracting the position of the wiper 20A from each frame of the captured image or from frames at a predetermined time interval.

The dimming process may also be performed with a strength corresponding to the set dimming ratio, which is set as an index showing the strength of dimming relative to the wiping speed of the wiper 20A and increases as the wiping speed increases.

Fourth Embodiment
Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, functional parts similar to those in any of the first to third embodiments are given the same reference numerals as the corresponding functional parts in any of the first to third embodiments, and detailed descriptions thereof will be omitted.

FIG. 11 is a schematic diagram showing the configuration of an electronic mirror system 80 according to the fourth embodiment.
11 , the electronic mirror system 80 includes an imaging device 26, an inner mirror 28, and a display controller 82. The display controller 82 is provided with an image processing unit 66, an adjustment switching unit 84 as a switching means, and an environment detection unit 38. The electronic mirror system 80 according to the fourth embodiment differs from the electronic mirror system 60 according to the third embodiment in that the adjustment switching unit 84 and the environment detection unit 38 are formed in the display controller 82 instead of the adjustment switching unit 68 and the environment detection unit 70 in the display controller 62 according to the third embodiment.

The image captured by the imaging device 26 is input to the adjustment switching unit 84. The adjustment switching unit 84 detects the brightness from the captured image, and determines whether or not to perform dimming processing according to the detected brightness, and sets the strength of the dimming processing if dimming processing is performed. The brightness adjustment unit 64 switches between normal processing and dimming processing based on the setting of the adjustment switching unit 84, and also switches the strength of the dimming processing when dimming processing is performed.

Next, a dimming process executed by the display controller 82 will be described as a fourth embodiment.
FIG. 12 is a flow chart showing an outline of the brightness adjustment process that is executed at predetermined time intervals in the display controller 82 when the digital mirror system 80 is started up.

As shown in FIG. 12, when the brightness of the surroundings of the vehicle 12 is below a predetermined brightness in either the image captured by the camera 32 or the automatic light device 24, and the wiper device 18 is operated to detect rainfall, the display controller 82 proceeds to step 250.

In step 250, the display controller 82 acquires an image captured by the camera 32, and in the next step 252, it is confirmed whether or not the brightness of the acquired captured image is equal to or greater than a preset brightness. The preset brightness may be the brightness of an image captured when it is not raining at night (when there are no raindrops on the rear window 16R) and the rear vehicle has its headlights turned on. Alternatively, the preset brightness may be a brightness that is not perceived as dazzling by the driver viewing the image displayed on the inner mirror 28, even when it is raining at night.

If the brightness of the captured image does not exceed the preset brightness, it can be determined that the driver does not find the image displayed on the inner mirror 28 too dazzling, so the display controller 82 makes a negative judgment in step 252 and proceeds to step 208.

Here, if the headlights of the vehicle behind are turned on during rainy conditions such as at night, water droplets adhering to the rear window 16R will brighten the captured image. If the brightness of the captured image exceeds a preset brightness, the display controller 82 makes a positive determination in step 252 and proceeds to step 214, where it sets the dimming process.

Then, the display controller 82 detects the brightness of the captured image in step 250. When the brightness of the captured image becomes equal to or lower than a preset brightness, the display controller 82 makes a positive determination in step 252 and proceeds to step 208, where the brightness processing is set to normal processing.

In this way, the electronic mirror system 80 (display controller 82) performs dimming processing when the image captured by the camera 32 becomes too bright for the driver to perceive as dazzling, even during rainfall at night, etc. This prevents dimming processing from being performed when there is no vehicle behind with its headlights on that would cause glare, for example, and prevents the image displayed on the inner mirror 28 from appearing too dark due to dimming processing.

In addition, the electronic mirror system 80 (display controller 82) can prevent the image displayed on the inner mirror 28 from being dazzling to the driver due to lighting behind the vehicle 12, not limited to that of the vehicle behind, thereby improving the driver's visibility behind the vehicle.
The light reduction process set in step 242 may be set in accordance with the brightness detected from the captured image, with the strength of the light reduction process being set to be stronger as the detected brightness becomes brighter.

Fifth Embodiment
Next, a fifth embodiment of the present invention will be described. In the fifth embodiment, functional parts similar to those in any of the first to fourth embodiments are denoted by the same reference numerals as the corresponding functional parts in any of the first to fourth embodiments, and detailed descriptions thereof will be omitted.

In the fourth embodiment, the normal processing is switched to the dimming processing when the brightness detected from the image captured by the camera 32 becomes equal to or higher than a preset brightness. In contrast, in the fifth embodiment, a rear vehicle is detected, and when a rear vehicle is detected, the dimming processing is performed on the captured image (the image displayed on the inner mirror 28).

FIG. 13 is a schematic diagram showing the configuration of an electronic mirror system 90 according to the fifth embodiment.
13, the electronic mirror system 90 according to the fifth embodiment includes the imaging device 26, the inner mirror 28, and a display controller 92. The display controller 92 is formed with the image processing unit 66, an adjustment switching unit 94 as a switching means, and an environment detection unit 38. The electronic mirror system 90 according to the fifth embodiment differs from the electronic mirror system 80 according to the fourth embodiment in that the adjustment switching unit 94 is formed in the display controller 92 instead of the adjustment switching unit 84 of the fourth embodiment.

A vehicle detection device 96 that detects a rear vehicle is connected to the adjustment switching unit 94, and the adjustment switching unit 94 switches the brightness adjustment process of the brightness adjustment unit 64 depending on whether a rear vehicle is detected.

Various configurations can be used for the vehicle detection means applied to the vehicle detection device 96 as long as they are capable of detecting a rear vehicle. For example, the vehicle detection means can be a millimeter wave radar method that uses radio waves with frequencies of 30 GHz to 300 GHz called millimeter waves and can measure the position and distance of an object. In addition, the vehicle detection means can be a known method such as a LiDAR (Light Detection Ranging) method that uses light such as infrared light (laser light) to measure the direction of light irradiation and the time it takes for the light reflected by an object to be received.

In addition, since the rear vehicle is captured in the image captured by camera 32, camera 32 (the image captured by camera 32) can be applied to the vehicle detection means. In this case, image recognition processing can be performed, such as extracting an image of the rear vehicle from the image captured by camera 32.

Furthermore, if the vehicle behind has its headlights on at night when it is dark, the brightness differs between the front and rear of the vehicle 12, so the vehicle detection means can use the camera 32 and an imaging means (front imaging means) for capturing images of the front of the vehicle 12, etc. In this case, the brightness (average brightness, or brightness between a preset area such as the center of the captured image, etc.) of the image captured by the camera 32 and the image captured by the front imaging means can be compared to determine whether there is a difference in brightness that would allow the presence of a vehicle behind to be estimated.

The adjustment switching unit 94 detects a rear vehicle using the vehicle detection device 96, and when a rear vehicle is detected, sets (switches) the brightness adjustment unit 64 to perform dimming processing. At this time, the adjustment switching unit 94 may change the strength of the dimming processing depending on whether or not there is a rear vehicle in a rainy condition where the surroundings are dark.

Next, the dimming process executed by the display controller 92 as the operation of the fifth embodiment will be described. Figure 14 shows a flow chart outlining the process for setting the dimming process executed by the display controller 92 at a predetermined time interval when the electronic mirror system 90 is started.

As shown in FIG. 14, when the brightness of the surroundings of the vehicle 12 is equal to or lower than a predetermined brightness in both the image captured by the camera 32 and the automatic light device 24, and the wiper device 18 is operated and it is determined that it is raining (positive determination in step 212), the display controller 92 proceeds to step 260.

In step 260, the display controller 92 checks whether the vehicle detection device 96 has detected a rear vehicle. If the vehicle detection device 96 has not detected a rear vehicle, the display controller 92 makes a negative determination in step 262 and proceeds to step 208.

In contrast, when the vehicle detection device 96 detects a rear vehicle, the display controller 92 makes a positive determination in step 262 and proceeds to step 264. In step 264, the display controller 92 sets the processing in the brightness adjustment unit 64 to dimming processing. This allows the driver to see behind the vehicle without feeling that the image displayed on the inner mirror 28 is too dazzling, even if the headlights of the rear vehicle are irradiated onto the rear windshield 16R.

In addition, when the brightness adjustment is set to dimming processing and the vehicle detection device 96 no longer detects a rear vehicle, the display controller 92 makes a negative determination in step 262 and proceeds to step 208. This sets the brightness adjustment to normal processing. Note that even if the vehicle detection device 96 detects a rear vehicle (a positive determination is made in step 252), the display controller 92 proceeds to step 208 (returns to normal processing) when positive determinations are made in each of steps 218, 222, and 226.

In this way, the electronic mirror system 90 (display controller 92) performs normal processing if no vehicle is detected behind the vehicle, even at night or during rain. This prevents dimming processing from being performed when there is no vehicle behind with its headlights on, which can cause glare, and prevents the image displayed on the inner mirror 28 from appearing dark due to dimming processing.

Sixth Embodiment
Next, a sixth embodiment of the present invention will be described. In the sixth embodiment, functional parts similar to those in any of the first to fifth embodiments are denoted by the same reference numerals as the corresponding functional parts in any of the first to fifth embodiments, and detailed descriptions thereof will be omitted.

FIG. 15 is a schematic diagram showing the configuration of a digital mirror system 100 according to the fourth embodiment.
15, the electronic mirror system 100 includes an inner mirror 28, a display controller 102 for displaying an image on the inner mirror 28, and an imaging device 104 for imaging the rear of the vehicle through the rear glass 16R. In the sixth embodiment, the imaging device 104 functions as the imaging device of the present invention. The electronic mirror system 100 of the sixth embodiment differs from the electronic mirror systems 10, 50, 60, 80, and 90 of the first to fifth embodiments in that the imaging device 104 is provided with a function for adjusting the brightness of the display image on the inner mirror 28 (display 34).

The display controller 102 of the electronic mirror system 100 is formed with an image processing unit 106, which performs the necessary image processing on the captured image (image data) input from the imaging device 104 in order to display the captured image on the inner mirror 28 (display 34).

The imaging device 104 includes a camera 32 and an imaging controller (imaging ECU) 108 as an image processing device for controlling imaging using the camera 32 and outputting a captured image. The imaging controller 108 is formed with a brightness adjustment unit 40 as an environment detection means and adjustment means.

The imaging controller 108 includes a microcomputer in which a CPU, ROM, RAM, storage as non-volatile memory, and an input/output interface are connected by a bus to enable data communication, and the required functional circuits are connected to the input/output interface (all not shown). In the imaging controller 108, the CPU reads out required programs stored in the ROM and storage and executes them while expanding them into the RAM, thereby realizing functions for imaging using the camera 32, a function for adjusting brightness, and a function for outputting a captured image. As a result, the imaging controller 108 is formed with an environment detection unit 38 and a brightness adjustment unit 40.

In the electronic mirror system 100 configured in this manner, the imaging controller 108 executes the same dimming process as the display controller 30 in the first embodiment. When the brightness (luminance value) detected from the image captured by the camera 32 is equal to or less than a threshold value, the auto light device 24 is in operation, and the wiper device 18 is operated, the imaging controller 108 executes dimming process on the captured image.

In addition, when the brightness detected from the image captured by the camera 32 exceeds the threshold value, the operation of the automatic light device 24 is stopped (the headlights 22 are turned off), and the wiper device 18 is stopped, the imaging controller 108 returns the brightness adjustment for the captured image to normal processing.

This prevents the headlights of the vehicle behind from making the image displayed on the inner mirror 28 too bright for the driver to perceive as dazzling at night when it is raining, improving the driver's visibility behind the vehicle even when the vehicle behind is moving.

In addition, in the imaging device 104, when the imaging controller 108 estimates the environmental condition to be nighttime and raining, a dimming process is performed on the image captured by the camera 32. This allows the imaging device 104 to output a captured image that is not dazzling to the driver even when displayed on the inner mirror 28.

Such an imaging device 104 can be used not only in the electronic mirror system 100 but also in a drive recorder that records captured images in a recording medium such as a non-volatile storage . As a result, when the captured images recorded in the recording medium of the drive recorder are played back, no halation or the like occurs even in images captured at night when it is raining, and high-quality captured images can be obtained.

In the sixth embodiment, the imaging device 104 is provided with an adjustment function for the display controller 30. However, the imaging device may be provided with an adjustment function for the display controller 52 according to the second embodiment, an adjustment function for the display controller 62 according to the third embodiment, an adjustment function for the display controller 82 according to the fourth embodiment, or a display controller 92 according to the fifth embodiment.

In addition, in cameras such as 32, the brightness of the captured image can be changed by adjusting the aperture (adjusting the light receiving time of the image sensor). Therefore, in the imaging device, the aperture can be used for light reduction processing, and when light reduction processing is performed, the aperture can be adjusted so that the brightness of the captured image is lower than in the case of normal processing.

Seventh Embodiment
Next, a seventh embodiment of the present invention will be described. In the seventh embodiment, functional parts similar to those in any of the first to sixth embodiments will be denoted by the same reference numerals as those corresponding to the functional parts in any of the first to sixth embodiments, and detailed descriptions thereof will be omitted.

Figure 16 shows a schematic diagram of the imaging device 110 according to the seventh embodiment, and Figure 17 shows a plan view of the schematic configuration of the vehicle 12A according to the seventh embodiment.

As shown in FIG. 16, the imaging device 110 includes a camera 32 as an imaging means, and an imaging controller 112 that controls imaging using the camera 32 and outputs a captured image. As shown in FIG. 17, in the seventh embodiment, a vehicle 12A is applied instead of the vehicle 12 in the first embodiment, and the basic configuration of the vehicle 12A is the same as that of the vehicle 12.

A camera 32 is disposed in the vehicle interior 14 at the front of the vehicle (on the inside of the vehicle interior 14 of the windshield 16F) and at the top of the center in the vehicle width direction. The camera 32 is also directed toward the front of the vehicle, and captures an image of the area in front of the vehicle through the windshield 16F.

Such an imaging device 110 can be used, for example, as a recording device such as a drive recorder that records captured images of the area in front of the vehicle, or for capturing images to be displayed on a monitor (display) provided on an instrument panel.

The imaging controller 112 is formed with a brightness adjustment unit 114 as a brightness adjustment means, an environment detection unit 38, and an adjustment switching unit 116. The imaging controller 112 is equipped with a microcomputer in which a CPU, a ROM, a RAM, a storage as a non-volatile memory, an input/output interface, etc. are connected to each other via a bus so that data communication is possible, and a required functional circuit is connected to the input/output interface (all not shown). In the imaging controller 112, the CPU reads out the required programs stored in the ROM and the storage and executes them while expanding them in the RAM, thereby realizing the functions of the environment detection unit 38, the brightness adjustment unit 114, and the adjustment switching unit 116 as well as the function of imaging using the camera 32 and the function of outputting the captured image. At this time, in the imaging controller 112, the function of the brightness adjustment unit 114 is realized as the same function as the brightness adjustment unit 64 in which the dimming process is switched between multiple stages, for example.

The environment detection unit 38 estimates whether the environmental conditions can be considered to be nighttime based on the image captured by the camera 32 and the operation of the automatic light device 24, and estimates whether the environmental conditions can be considered to be raining based on the operation of the wiper device 18. The environment detection unit 38 also sets the brightness adjustment to normal processing or dimming processing depending on the estimated environmental conditions.

The wiper device 18 can be operated in either intermittent or continuous mode by operating a wiper switch (not shown), and the operation interval (intermittent time, interval) in intermittent mode and the wiping speed in continuous mode can be selected.

The adjustment switching unit 116 is connected to the wiper device 18, and the adjustment switching unit 116 applies the wiping speed as the operating state of the wiper device 18 and switches the strength of the dimming process according to the wiping speed.

The switching adjustment unit 116 is set with a dimming ratio for dimming processing according to the wiping speed of the wiper device 18 (the reciprocating period of the wiper 18A), and the switching adjustment unit 116 is set with a dimming ratio as the strength of the dimming processing according to the wiping speed. The wiping speed is calculated according to the movement amount of the wiper 18A per predetermined time (unit time), and the wiping speed in the intermittent mode is set lower (slower) as the intermittent time is longer compared to when the intermittent time is short. In addition, the dimming ratio is set to be higher as the wiping speed is higher compared to when the wiping ratio is low, and the strength of the dimming processing is stronger (brightness is greatly reduced) as the dimming ratio is higher compared to when the dimming ratio is small. Note that the switching adjustment unit 116 is set with the strength of the dimming processing when the wiper device 18 is stopped (wiping speed is zero), and the dimming processing at this time is set to the weakest (it may be the same as the normal processing).

Next, the dimming process in the imaging device 110 will be described with reference to FIG. 18. FIG. 18 shows a flow chart outlining the dimming process settings executed by the imaging controller 112.

As shown in FIG. 18, the imaging controller 112 executes steps 200 to 206 and steps 210 and 212, and when the brightness of the surroundings of the vehicle 12 in both the image captured by the camera 32 and the automatic light device 24 is equal to or lower than a predetermined brightness, and the wiper device 18 is operated to determine that it is raining (a positive determination in step 212), the imaging controller 112 proceeds to step 270.

In step 270, the imaging controller 112 acquires the speed of the wiper device 18 being operated. That is, it acquires whether the wiper device 18 is being operated in intermittent mode or continuous mode, as well as the intermittent time in intermittent mode and the wiping speed in continuous mode (e.g., high speed or low speed), and sets the dimming ratio based on the acquired wiping speed. Next, in step 272, the imaging controller 112 sets the dimming process so that the process is performed with the strength set according to the dimming ratio.

When the dimming process and the strength of the dimming process for brightness adjustment are set in this manner, the imaging controller 112 executes steps 216 to 226, and if a positive determination is made in each of steps 218, 222, and 226, the imaging controller 112 proceeds to step 208 (returning to normal processing). Note that if a negative determination is made in step 218 or step 222 and the wiper device 18 is stopped, the imaging controller 112 sets the wiping speed to the minimum value (e.g., 0) and continues the dimming process.

Here, when the wiper device 18 operates, if the wiper 18A cycle is short, the interval between changes in brightness caused by light scattering and the shadow of the wiper 18A becomes short. For this reason, when the wiping speed of the wiper device 18 is high, compared to when the wiping speed is low, the flickering that occurs in the captured image tends to be greater and the changes in brightness tend to be greater, which can be annoying when viewing the captured image.

In this way, when the imaging device 110 estimates that the environmental conditions will cause the captured image to be dazzling when viewed, it performs a dimming process on the captured image. This means that even if an oncoming vehicle has its headlights on, the captured image will not be dazzling when viewed.

The imaging device 110 also performs dimming processing with a strength based on the wiping speed of the wiper device 18. At this time, when the wiping speed is high, the strength of the dimming processing is stronger than when the wiping speed is low. This makes it possible to reduce the change in brightness that occurs in the captured image, and does not cause annoyance when viewing the captured image.

Eighth embodiment
Next, an eighth embodiment of the present invention will be described. In the eighth embodiment, functional parts similar to those in any of the first to seventh embodiments are denoted by the same reference numerals as the corresponding functional parts in any of the first to seventh embodiments, and detailed descriptions thereof will be omitted.

FIG. 18 is a schematic diagram showing the configuration of an image pickup device 120 according to the eighth embodiment.
18, the imaging device 120 is provided with an imaging controller 122, and an adjustment switching unit 124 is formed in the imaging controller 122. The imaging device 120 according to the eighth embodiment differs from the imaging device 110 according to the seventh embodiment in that the imaging controller 122 is formed with an adjustment switching unit 124 instead of the adjustment switching unit 116 of the imaging controller 112 according to the seventh embodiment.

The image captured by the camera 32 is input to the adjustment switching unit 124 of the imaging controller 122. In addition, in the imaging device 120, the wiper device 18 is connected to the adjustment switching unit 124. Furthermore, in the imaging device 120, a defogger device 126 that operates as an anti-fogging device for the windshield 16F may be connected.

The adjustment switching unit 124 acquires (monitors) images captured by the camera 32 that captures images of the area ahead of the vehicle through the windshield 16F, and calculates the average value (non-operating average value) of the brightness (luminance) of the captured images over a predetermined time interval when the wiper device 18 is not in operation. When the wiper device 18 is in operation, the adjustment switching unit 124 also calculates the average value (operating average value) of the brightness of the captured images over a predetermined time interval when the wiper device 18 is in operation. At this time, the predetermined time interval is set to a time shorter than one period of the wiping operation of the wiper 18A of the wiper device 18.

When setting dimming processing for a captured image, the adjustment switching unit 124 compares the average non-operating value of the captured image with the average operating value (contrasts brightness), and sets the dimming processing to be performed when the peak (maximum value ) of the difference obtained by the comparison exceeds a predetermined threshold value and the interval between the peaks is synchronized with the wiping cycle of the wiper device 18.

Next, the dimming process in the imaging device 120 will be described with reference to Figures 20 and 21. Note that Figure 20 shows an outline of the dimming process settings executed in the imaging controller 122 in the form of a flow chart, and Figure 21 shows an outline of the change in brightness obtained from the captured image in an environment where the image captured by the camera 32 is dazzling. In Figure 21, the horizontal axis represents time, and the vertical axis represents the brightness (average luminance) of the captured image.

As shown in FIG. 20, the imaging controller 122 executes steps 200 to 206 and steps 210 and 212, and when the brightness of the surroundings of the vehicle 12 is equal to or lower than a predetermined brightness in both the image captured by the camera 32 and the automatic light device 24, and the wiper device 18 is operated and it is determined that it is raining (a positive determination in step 212), the imaging controller 122 proceeds to step 280.

In step 280, the imaging controller 122 calculates the average operating value of the brightness when the wiper device 18 is in operation from the image captured by the camera 32, and compares the calculated average operating value with the average non-operating value calculated before the wiper device 18 is operated. In step 282, the imaging controller 122 obtains a period (anti-glare period) in which the difference between the average non-operating value and the average operating value is equal to or greater than a preset threshold value. At this time, the threshold value can be set based on the average brightness (lower limit value) of the captured image at which dazzling is felt relative to the average non-operating value.

When the imaging controller 122 acquires the anti-glare cycle, it determines in step 284 whether the anti-glare cycle is synchronized with the wiping cycle of the wiper 18A. At this time, if the anti-glare cycle and the wiping cycle are not synchronized, the imaging controller 122 makes a negative determination in step 284 and proceeds to step 208.

In response to this, if the anti-glare cycle and the wiping cycle are synchronized, the imaging controller 122 makes a positive judgment in step 284 and proceeds to step 286 to set the dimming process to be performed (set dimming process).

Then, the imaging controller 122 proceeds to step 208 when a positive determination is made in each of steps 218 and 222, or a negative determination is made in step 284 because the wiper device 18 has been stopped, etc.

As shown in Fig. 21, when raindrops or the like are attached to the windshield 16F at night, the wiper device 18 operates, and the average brightness (operating average brightness) changes in synchronization with the wiping action of the wiper 18A, and the difference with the non-operating average brightness exceeds the threshold value. At this time, the wiper 18A's remaining wiping residue forms a water film or the like, causing the operating average brightness to increase. In addition, the wiper 18A is reflected in the captured image, casting a shadow of the wiper 18A, causing the operating average brightness to decrease.

Therefore, it is possible to determine whether or not a change in brightness of a captured image is caused by the operation of the wiper device 18 from the change in the average brightness of the captured image. Furthermore, if the change in the average brightness of a captured image affects the operation of the wiper device 18, the period (anti-glare period) will be close to the wiping period of the wiper 18A.

Then, if the imaging controller 122 determines that the anti-glare cycle and the wiping cycle are synchronized, it performs dimming processing on the captured image. This makes it possible to prevent glare from occurring when the captured image is displayed, even if the wiper device 18 is operating.

In this way, even if the imaging device 120 operates the wiper device 18 when it is estimated that the environmental conditions will cause the driver to feel dazzled when viewing the captured image, it is possible to prevent the driver from feeling dazzled when viewing the captured image.

On the other hand, the vehicle 12A is equipped with a defogger device 126, and by operating this defogger device 126, the defogger device 126 can be activated to promote the defogging of the windshield 16F. Therefore, the imaging device 120 activates the defogger device 126 when performing the dimming process. This effectively prevents glare from appearing in the captured image.

In addition, a rear defogger is often provided on the rear window 16R. When capturing an image of the rear of the vehicle through the rear window 16R, activating the rear defogger in conjunction with the dimming process can effectively reduce glare in the captured image.

In the above-described first to eighth embodiments, automobiles such as vehicles 12 and 12A have been used as examples of vehicles. However, the vehicle may be any vehicle that has a wiper device installed on the windshield, and may also be a train such as an electric locomotive.

In the first to eighth embodiments described above, the programs executed by the display controllers 30, 52, 62, 82, 92, 102 and the imaging controllers 108, 112, 122 may be pre-installed in a ROM or storage. These programs may also be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory, or may be downloaded from an external device via a network.

In addition, the present invention is not limited to the first to eighth embodiments, and various modifications can be made without departing from the spirit of the invention. For example, the spirit of the present invention is not lost even if all or part of the components of the display controllers 30, 52, 62, 82, 92, 102 and the imaging controllers 108, 112, 122 are integrated into the inner mirror 28 or the camera 32.

10, 50, 60, 80, 90, 100, Electronic mirror system 16F Front glass (windshield)
16R Rear glass (window glass)
18 Wiper device (environment detection means)
20 Rear wiper device 24 Auto light device (environment detection means)
26 Imaging device 28 Inner mirror (display means, image display device)
30, 52, 62, 82, 92, 102 Display controller (image display device, image processing device)
32 Camera (imaging means, environment detection means)
38, 70 Environment detection unit (environment detection means)
40, 54, 64, 114 Brightness adjustment unit (adjustment means)
68, 84, 116, 124 Adjustment switching units 104, 110, 120 Imaging devices 108, 112, 122 Imaging controllers (image processing devices)

Claims (7)

a display means for displaying a display image corresponding to an image captured by the imaging means through the windshield of one side of the vehicle exterior;
an environmental detection means for detecting an environmental condition capable of estimating whether water droplets are attached to the window glass and whether water droplets attached to the window glass affect the brightness of an image captured by the imaging means;
an adjustment means for performing an adjustment so as to reduce the brightness of the display image when the environmental condition detected by the environmental detection means is estimated to include water droplets on the window glass and the water droplets on the window glass are estimated to affect the brightness of the display image, compared to when the environmental condition detected by the environmental detection means is estimated not to include water droplets on the window glass or the water droplets on the window glass are estimated not to affect the brightness of the display image;
An image display device comprising: The environment detection means includes a light/dark detection means for detecting brightness outside the vehicle, and a wiper device for wiping off water droplets adhering to the windshield when the wiper device is activated.
2. The image display device according to claim 1, wherein the adjustment means reduces the brightness of the displayed image when the brightness outside the vehicle is equal to or lower than a predetermined value and the wiper device is in operation. The image display device according to claim 1 or 2, wherein the adjustment means suppresses the luminance of the display image to suppress the brightness of the display image when the display image is displayed on the display means. The image display device according to any one of claims 1 to 3, wherein the adjustment means reduces the brightness of the display means to reduce the brightness of the displayed image. The image display device according to claim 1 or 2, wherein the adjustment means suppresses the brightness of the displayed image by lowering the brightness of the image displayed on the display means. An imaging means for imaging one side of the outside of the vehicle through a windshield;
an environmental detection means for detecting an environmental condition capable of estimating whether water droplets are attached to the window glass and whether water droplets attached to the window glass affect the brightness of an image captured by the imaging means;
an adjustment means for making an adjustment when the environmental condition detected by the environmental detection means is estimated to include water droplets on the window glass and when the water droplets on the window glass are estimated to affect the brightness of the image captured by the imaging means, so as to reduce the brightness of the captured image compared to when the environmental condition detected by the environmental detection means is estimated to include no water droplets on the window glass or when the water droplets on the window glass are estimated to affect the brightness of the captured image;
An imaging device comprising: an environmental detection means for detecting an environmental condition capable of estimating whether water droplets are attached to a window glass and whether water droplets attached to the window glass affect the brightness of an image captured by an imaging means;
A photographed image of one side of the outside of the vehicle taken through the windshield by the imaging means,
an adjustment means for making an adjustment when the environmental condition detected by the environmental detection means is estimated to include water droplets on the window glass and when the water droplets on the window glass are estimated to affect the brightness of the captured image, so as to reduce the brightness of the captured image compared to when the environmental condition detected by the environmental detection means is estimated not to include water droplets on the window glass or not to affect the brightness of the captured image;
2. An image processing device comprising: