JP4961734B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus such as an electronic camera and an imaging method.

A conventional image processing technique is described in, for example, Patent Document 1 below.
In this prior art, first, an area including a characteristic portion of a digital image is extracted, and a first luminance average value of pixels in the extracted area is calculated. Then, when the calculated first luminance average value is equal to or smaller than the threshold value, the luminance value of the target pixel is corrected based on the luminance values of the pixels within the predetermined range including the target pixel of the digital image. .
For this correction, for example, the second luminance average value of surrounding 3 × 3 pixels including the target pixel itself is calculated, and the RGB value of the target pixel is added to the second luminance average value multiplied by, for example, 0.6. To do. By performing this process for all pixels, only the luminance value of each pixel is increased as a whole, the noise component is relatively reduced, and the SN ratio is improved.

JP 2001-45298 A

In the above conventional technique, since the noise level is reduced by smoothing the luminance with the peripheral pixels by image processing, not only the noise but also the original signal level is smoothed. There is a problem that deterioration of the image occurs (that is, the image becomes blurred or not sharp).
An object of the present invention is to provide an imaging apparatus and an imaging method capable of improving the SN ratio and improving the image quality.

  In order to solve the above problems, the present invention includes an image sensor that performs photoelectric conversion of input image light acquired by an image acquisition unit, and offset light output unit that outputs offset light to be superimposed on the input image light. It is configured to do.

  According to the present invention, it is possible to provide an imaging apparatus and an imaging method capable of improving the SN ratio and improving the image quality.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings described below, components having the same function are denoted by the same reference numerals, and repeated description thereof is omitted.
Embodiment 1
<Configuration of imaging device>
FIG. 1 is a block diagram showing the configuration of the imaging apparatus according to Embodiment 1 of the present invention.
In FIG. 1, 1 is an image acquisition device, 2 is an image sensor that performs photoelectric conversion of input image light (described later) acquired by the image acquisition device 1, and 3 is an input image from the image acquisition device 1 to the image sensor 2. An offset light output device 4 that outputs offset light to be superimposed on light is a control device that controls the output of the offset light output device 3 based on the output of the image sensor 2. The image acquisition device 1 is a device that forms an image of light ahead of the imaging device on the image sensor 2 that performs photoelectric conversion, and is specifically an optical system such as a lens. The configuration of the imaging apparatus shown in FIG. 1 is common to all of the following first to third embodiments.

FIG. 2 is a diagram illustrating a specific configuration of the imaging apparatus of FIG.
In FIG. 2, 10 is a housing of the imaging device, 11 is a lens (or a lens group) that is a component of the image acquisition device 1 (see FIG. 1), 31 is a diffusion plate, 32 is a half mirror, and 100 is an image. Input image light input from the acquisition device 1 to the image sensor 2, 300 is offset light output from the offset light output device 3 to the image sensor 2, and 130 is input image light from the image acquisition device 1 to the image sensor 2. The input light is formed by superimposing the offset light 300 from the offset light output device 3 on 100. Thus, in this specification, the input light input from the image acquisition device 1 to the image sensor 2 is referred to as “input image light” (100), and the input image light 100 from the image acquisition device 1 to the image sensor 2 is The input light formed by superimposing the offset light 300 output from the offset light output device 3 is simply referred to as “input light” (130) (note that the input light 130 includes the case where the offset light 300 is zero). ).

As an element of the image acquisition device 1 (FIG. 1), the input image light 100 of the housing 10 of the imaging device has an input unit that forms light on the image sensor 2 in front of the imaging device. A lens (lens group) 11 is provided. A pinhole or a mirror can be used instead of the lens (lens group) 11.
The image sensor 2 is a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Metal Oxide Semiconductor) sensor, an amorphous sensor, or the like. Has been.

The offset light output device 3 is a device that generates light, and for example, various light sources such as an incandescent bulb, a halogen bulb, a krypton bulb, an LED (light Emitting Diode), and an LD (Laser Diode) can be used. .
A diffusion plate 31 is provided in the optical path of the offset light 300 between the offset light output device 3 and the half mirror 32. Here, the diffusion plate 31 is disposed in the vicinity of the output unit of the offset light output device 3 between the output unit of the offset light output device 3 and the half mirror 32. The diffusing plate 31 is provided so that the offset light 300 output from the offset light output device 3 is irradiated to the image sensor 2 at a uniform rate.

In addition, the image sensor 2 is irradiated with the input image light 100 input from the image acquisition device 1 (FIG. 1) via the lens 11 which is one component thereof, and the offset light 300 output from the offset light output device 3. A half mirror 32 is disposed to irradiate the image sensor 2 with the image. In other words, the half mirror 32 irradiates the input light 130 formed by superimposing the offset light 300 from the offset light output device 3 on the input image light 100 from the image acquisition device 1 to the image sensor 2. Yes.
The input image light 100 imaged on the image sensor 2 through the lens 11 is photoelectrically converted by the image sensor 2 and output as a digital image (FIG. 1).

<principle>
The principle that the image pickup apparatus having such a configuration can improve the SN ratio and improve the image quality will be described.
FIG. 3 is a diagram showing output characteristics (sensitivity characteristics) of a general image sensor. The horizontal axis represents light input, and the vertical axis represents output (voltage).

As shown in FIG. 3, the sensitivity characteristics of an image sensor are generally not linear with respect to light input, but have a good sensitivity (exactly differential sensitivity, which will be described later) (that is, a region with a large slope). ) On the left and right sides of 400, there are insensitive regions 500 (regions having a small inclination). The region 400 with good sensitivity is a region where a sufficient output difference can be obtained with respect to the difference in brightness. The region 500 with poor sensitivity has a difference in output even though there is a difference in brightness. It is an area that cannot be obtained sufficiently. In the sensitive region 400, the light intensity range in which the sensitivity is not less than the first threshold Th _{1 and not} more than the second threshold Th ₂ is referred to as a peak section of the sensitivity characteristic of the image sensor (FIG. In the case of 3, the sensitive area 400 and the peak section are the same). Sensitivity is differential sensitivity, and differential sensitivity is an increase in output with respect to a constant increment of light input, and is proportional to the slope of the sensitivity curve.
600 does not overlap the offset light 300 (FIG. 2), that is, the input light 130 when the offset light 300 is 0, that is, the intensity range of the input image light 100, and 700 is the input light where the offset light 300 is superimposed. A range of 130 intensities.

  Conventionally, in the low sensitivity region 500, there is a problem that a sufficient output difference cannot be obtained even though there is a difference in brightness. In contrast, in the present invention, the above-described sensitivity characteristics of the image sensor 2 are taken into consideration, and the offset light output device 3 offsets the input image light 100 (FIG. 2) from the image acquisition device 1 to the image sensor 2. By superimposing the light 300, as shown in FIG. 3, the intensity range 600 of the input image light 100 with respect to the image sensor 2 is in the region 400 where the image sensor 2 has good sensitivity (the peak section of the sensitivity characteristic of the image sensor). Can be moved to a range 700 of the intensity of the input light 130. As a result, the output signal level of the image sensor 2 is increased, the SN ratio is improved, the image quality is improved, and a sharp screen without blur is obtained. That is, even for an imaging target that is dark and has a small difference in luminance and a small difference from noise, the sensitivity can be increased without changing the noise level, and the imaging target can be seen clearly.

4A is a diagram showing the relationship between the light intensity of the input light 130 of FIG. 2 and the position of the image sensor 2 on, for example, a horizontal scanning line, and FIG. 4B is the output of the image sensor 2 and the image sensor 2. For example, it is a figure which shows the relationship with the position on a horizontal scanning line.
The dotted line in (a) indicates the input light 130 that does not overlap the offset light 300 (that is, the input image light 100), and the solid line indicates the input light 130 that overlaps the offset light 300. Further, the dotted line in (b) indicates the output of the image sensor 2 in which the offset light 300 is not superimposed, and the solid line indicates the output of the image sensor 2 in which the offset light 300 is superimposed. The dotted line and the solid line in (a) correspond to the dotted line and the solid line in (b), respectively.
In (b), 800 is the input light 130 on which the offset light 300 is not superimposed, that is, the output range of the image sensor 2 by the input image light 100, and 900 is the image sensor 2 by the input light 130 on which the offset light 300 is superimposed. Indicates the output range.
As shown in FIGS. 4A and 4B, by superimposing the offset light 300 on the input image light 100, the output difference of the image sensor 2 can be increased even when the light intensity difference is small. Can be improved.

<Processing flow>
Hereinafter, the flow of operation in the present embodiment will be described with reference to the flowchart of FIG.
First, in S1 of FIG. 5, the input image light 100 (input light 130. FIG. 2) input from the image acquisition device 1 (FIG. 1) is input to the image sensor 2 to obtain a digital image.
Next, in S <b> 2, the control device 4 obtains the intensity range 600 (FIG. 3) of the input light 130 based on the output of the image sensor 2.
Next, in S <b> 3, the control device 4 determines whether or not the intensity range 600 of the input light 130 to the image sensor 2 is in the region 400 with good sensitivity of the image sensor 2. That is, the control device 4 obtains the intensity range of the input light 130, and if the intensity range of the input light 130 is within a range equal to or less than the _first threshold Th ₁ (FIG. 2), the offset light output device 3 If the output of the offset light 300 is increased and the output is within the range of the second threshold Th ₂ or more, the output of the offset light 300 of the offset light output device 3 is decreased. In this case, the control device 4, the range of the intensity of the input light 130 is to have a portion that exceeds at least a first threshold value Th _1, increasing the output of the offset light 300, and at least a second smaller than the threshold Th ₂ The output of the offset light 300 is lowered so as to have a portion. In this case, the control device 4 controls the output of the offset light 300 so that the intensity range of the input light 130 falls within the peak section of the sensitivity characteristic of the image sensor 2.

Next, if it is not in the sensitive range 400 in S3, the control device 4 controls the output of the offset light 300 of the offset light output device 3 in S4. That is, if the intensity range 600 of the input image light 100 is low with respect to the sensitive area 400 of the image sensor 2, the output of the offset light 300 of the offset light output device is increased, and the intensity range 600 of the input image light 100 is increased. If it is higher, the output of the offset light 300 is lowered and the process returns to S1. If the output of the offset light 300 is lowered to 0 and the image sensor 2 does not enter the sensitive area 400, no further treatment is performed. Although not performed here, in such a case, it is also possible to control using a diaphragm, a filter, or the like.
If the intensity range 600 of the input image light 100 is in the sensitive range 400 in S3, the image obtained in S1 is output in S5, and the process returns to S1.

  As described above, the imaging device according to the present embodiment includes the image acquisition device 1 (FIG. 1), the image sensor 2 that performs photoelectric conversion of the input image light 100 (FIG. 2) acquired by the image acquisition device 1, and the image. And an offset light output device 3 that outputs an offset light 300 to be superimposed on the input image light 100 from the acquisition device 1 to the image sensor 2. Further, in the imaging method of the present embodiment, the offset light 300 is added to the input image light 100 from the image acquisition device 1 to the image sensor 2 that performs photoelectric conversion of the input image light 100 acquired by the image acquisition device 1. It is something that overlaps.

  As described above, in the present embodiment, by providing the offset light output device 3, the offset light 300 can be superimposed on the input image light 100 (FIG. 2) to the image sensor 2. Output signal level can be increased. Further, it is possible to improve the S / N ratio of an image shot at a brightness with low sensitivity of the image sensor 2 and to improve the image quality. In addition, the optical system such as the image sensor 2 and the lens 11 is improved so that the sensitivity characteristic is improved in a dark luminance region to widen the dynamic range, or the imaging target is directly irradiated with light in order to brighten the imaging target. There is no need to use a lighting device. In this embodiment, since the image sensor 2 is irradiated with the offset light 300, the sensitivity characteristic can be improved by using a small lighting device with low power consumption.

In addition, a half mirror 32 that superimposes the input image light 100 and the offset light 300 is provided. Thus, the input image light 100 and the offset light 300 can be superimposed with a simple configuration.
A diffusion plate 31 is provided in the optical path of the offset light 300 between the offset light output device 3 and the half mirror 32. As a result, the offset light 300 from the offset light output device 3 can be diffused, so that the uniform offset light 300 can be given to the image sensor 2.

  In addition, the controller 4 controls the output of the offset light 300 of the offset light output device 3 based on the output of the image sensor 2 (for example, the output voltage output in time series from the image sensor 2). Since the output of the offset light output device 3 can be adjusted based on the output of the image sensor 2 in this way, the SN of the photographed image is adjusted according to the brightness when photographing and the sensitivity characteristics of the image sensor 2. The image quality can be improved while improving the ratio.

Further, the control device 4 obtains an intensity range of the input light 130 formed by superimposing the offset light 300 on the input image light 100 based on the output of the image sensor 2, and the intensity range of the input light 130 is the first range. If it is within the range below the threshold Th ₁ (FIG. 2), the output of the offset light 300 of the offset light output device 3 is increased, and if it is within the second threshold Th ₂ or more, the offset light output device 3 The output of the offset light 300 is lowered. Thus, the range of the intensity of the input light 130 is obtained based on the output of the image sensor 2, and the output of the offset light output device 3 is adjusted so that the intensity of the input light 130 enters the region 400 with high sensitivity of the image sensor 2. Therefore, the SN ratio of the photographed image can be improved and the image quality can be improved in accordance with the brightness at the time of photographing and the sensitivity characteristics of the image sensor 2.

Further, the control device 4, the range of the intensity of the input light 130 is to have a portion that exceeds at least a first threshold value Th _1, increasing the output of the offset light 300, and at least a second smaller than the threshold Th ₂ The output of the offset light 300 is lowered so as to have a portion. As described above, the range of the intensity of the input light 130 is obtained from the output of the image sensor 2, and the output of the offset light output device 3 is adjusted so that the intensity of the input light 130 enters the region 400 with good sensitivity of the image sensor 2. Therefore, the SN ratio of the captured image can be improved and the image quality can be improved in accordance with the brightness at the time of shooting and the sensitivity characteristics of the image sensor 2.

  In addition, the control device 4 controls the output of the offset light 300 so that the intensity range of the input light 130 falls within the peak section of the sensitivity characteristic of the image sensor 2. Thus, when adjusting the output of the offset light output device 3, the output of the offset light output device 3 can be adjusted so that the intensity of the input light 130 falls within the peak section of the sensitivity characteristic of the image sensor 2. According to the brightness at the time of shooting and the sensitivity characteristics of the image sensor 2, the SN ratio of the shot image can be improved and the image quality can be improved.

<< Embodiment 2 >>
FIG. 6 is a diagram illustrating the configuration of the imaging apparatus according to the second embodiment of the present invention.
In the present embodiment, when the input image light 130 includes light having a specific wavelength (hereinafter referred to as specific wavelength light), the output signal level corresponding to the specific wavelength light is increased. It is a form.
In FIG. 6, reference numeral 30 denotes an offset light output device composed of a normal light source that emits light of a normal wavelength and a specific wavelength light source that emits light of a specific wavelength. That is, a normal light source and a specific wavelength light source are prepared, and the light output from the two light sources can be switched. As the specific wavelength light source in this case, for example, a laser or a light source including an optical filter that transmits only specific wavelength light can be used. As described above, the offset light output device 30 according to the present embodiment can selectively output offset light having a normal wavelength or offset light having a specific wavelength to the image sensor 2. As a light source of the offset light output device 30, a light source that serves as both a normal light source and a specific wavelength light source can be used. As such a light source, for example, an LED array or the like in which LEDs that emit light of a normal wavelength and specific wavelength light are two-dimensionally arranged can be used.

  When it is desired to improve the output of the image sensor 2 with respect to light of all wavelengths of the input image light 100, a normal light source is used, and the output of the image sensor 2 corresponding to specific wavelength light in the input image light 100. When it is desired to improve, the specific wavelength light source is switched and used.

  The specific wavelength light means light output from a light source that outputs only light having a single wavelength, such as laser light. The laser beam is a light that is convenient for irradiating only a specific portion because it has a feature that it goes straight, that is, it travels straight without spreading. For example, by installing the imaging device in a vehicle such as an automobile, irradiating a line (line) -shaped laser beam forward in the traveling direction of the vehicle with an irradiation device, and observing the bent state of the line-shaped laser beam, The presence or absence of obstacles in front of the vehicle and the three-dimensional configuration can be observed. The present embodiment is applicable to such a case, and the offset light 300 of the specific wavelength light having the same wavelength is superimposed on the input input image light 100 by the specific wavelength light source of the offset light output device 30. Thus, the observation accuracy corresponding to the specific wavelength light in the input image light 100 can be improved. That is, the output signal level corresponding to the specific wavelength light can be improved, the SN ratio can be improved, and the image quality can be improved. Therefore, it is not necessary to set an optical filter or prepare a specific image sensor for the input image light 100.

<< Embodiment 3 >>
FIG. 7 is a plan view showing the configuration of the half mirror 33 in the imaging apparatus according to Embodiment 3 of the present invention. Reference numeral 34 denotes an area formed in the half mirror 33 and having a partially high reflectance.
FIG. 8 is a diagram illustrating a configuration of the imaging apparatus according to the present embodiment. The half mirror 33 having the region 34 having a partially high reflectance is movable two-dimensionally (the moving mechanism is not shown).

  In the present embodiment, the output signal level is raised only for a specific region of the image sensor 2 to improve the SN ratio and improve the image quality. That is, in the present embodiment, a region where the intensity of the input light 130 is weaker than other portions is obtained based on the output of the image sensor 2, and the offset light 300 is changed to a region where the intensity of the input light 130 is weak. It has the control apparatus 4 (refer FIG. 1) which irradiates the area | region of the corresponding image sensor 2. FIG.

It is considered that the intensity of the input image light 100 input from the image acquisition device 1 with respect to the image sensor 2 varies within a certain range. If this variation is large, a state occurs in which the intensity of the input image light 100 with respect to the image sensor 2 is small in a certain area and the intensity of the input image light 100 is sufficiently large in a certain area. In such a state, when uniform offset light 300 is applied to the entire image sensor 2, even if it enters the sensitive region 400 (FIG. 3) in one region, it enters the insensitive region 500 in other regions. It can be considered.
For such input image light 100, the image sensor 2 is moved into a region 500 having low sensitivity by moving a two-dimensionally moving half mirror 33 having a region 34 with a high reflectance. By partially superimposing the offset light 300 on the corresponding area of the sensor 2, the output signal level can be increased over the entire area of the image sensor 2, the SN ratio can be improved, and the image quality can be improved. In this way, by giving the offset light 300 only to a dark part on the screen, not the entire screen, the intensity of the input light 130 is changed in a region where sufficient intensity of the input light 130 is obtained. In other words, the intensity of the input light 130 can be increased only for a portion where the intensity of the input light 130 is weak.

  In the present embodiment, as means for superimposing the offset light 300 partially on the image sensor 2, the region 34 having a partially high reflectance as shown in FIG. 7 is provided and moved two-dimensionally. A half mirror 33 having a mechanism is used to extract a weak region of the input image light 100 as compared with other portions based on the output of the image sensor 2 and to give the offset light 300 around that region. Then, the offset light 300 is given by controlling the position of the half mirror 33.

The embodiment described above is described for facilitating understanding of the technical idea of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment includes all design changes and equivalents belonging to the technical scope of the present invention.
For example, in the first to third embodiments, the offset light output device 3 is arranged in the housing 10 of the imaging device as shown in FIGS. 2, 6, and 8, respectively. However, it is also possible to arrange the offset light output device 3 outside the housing 10 and superimpose the offset light 300 on the input image light 100 from the image sensor 2 from the outside of the housing 10.

In the first to third embodiments, the control device 4 obtains the intensity range of the input light 130 based on the output of the image sensor 2, and the intensity range of the input light 130 is a predetermined predetermined number of times. The output of the offset light 300 is increased if it is within the range of ₁ threshold Th ₁ or less, and the output of the offset light 300 is decreased if it is within the range of the predetermined second threshold Th ₂ or greater. However, instead of the range of the intensity of the input light 130, an average value or maximum value of the intensity of the input light 130 is obtained, and if this average value or maximum value is equal to or less than a predetermined threshold value, the offset light 300 The output of the offset light 300 may be decreased if the output of the offset light 300 is higher than a predetermined threshold.

  In the first to third embodiments, when the control device 4 determines the intensity range (or the average value or the maximum value as described above) of the input light 130 based on the output of the image sensor 2, The output is obtained based on the output from the entire image, but may be obtained based on the output of the image sensor 2 in a predetermined specific area such as the center of the screen.

  The correspondence between each component in the claims and each component in the embodiment of the invention will be described. That is, the image acquisition device 1 according to the embodiment is an image acquisition unit of the claims, the offset light output device 3 is an offset light output unit, the half mirror 32 is an overlay unit, and the diffusion plate 23 is a light diffusion unit. Further, the control device 4 corresponds to the control means.

It is a block diagram which shows the structure of the imaging device of Embodiment 1 of this invention. It is a figure which shows the specific structure of the imaging device of Embodiment 1 of this invention. It is a figure which shows the output characteristic of a general image sensor. (A) is a figure which shows the relationship between the light intensity of the input light to an image sensor, and the position on the horizontal scanning line in an image sensor, (b) is the relationship between the output of an image sensor and the same position in an image sensor. FIG. It is a flowchart which shows the flow of operation | movement in Embodiment 1 of this invention. It is a figure which shows the specific structure of the imaging device of Embodiment 2 of this invention. It is a top view which shows the structure of the half mirror in the imaging device of Embodiment 3 of this invention. It is a figure which shows the specific structure of the imaging device of Embodiment 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image acquisition device 2 ... Image sensor 3, 30 ... Offset light output device 4 ... Control apparatus 10 ... Case 11 ... Lens (lens group)
31 ... Diffuser 32, 33 ... Half mirror 34 ... Partially high reflectance region 100 ... Input image light 130 ... Input light 300 ... Offset light 400 ... High sensitivity region 500 ... Low sensitivity region 600 ... Input image light Intensity range 700 ... Input light intensity range 800 superimposed with offset light ... Image sensor output range 900 with input image light ... Image sensor output range with input light superimposed with offset light

Claims (8)

An imaging device for mounting on a vehicle,
Image acquisition means;
An image sensor that performs photoelectric conversion of input image light acquired by the image acquisition means;
Offset light output means for outputting offset light to be superimposed on the input image light from the image acquisition means to the image sensor;
A moving mechanism for moving a region irradiated with the offset light;
Control means for controlling the output of the offset light of the offset light output means based on the output of the image sensor;
The control means includes
Based on the output of the image sensor, a region where the intensity of the input image light is weaker than other parts is obtained,
An image pickup apparatus that controls the moving mechanism to irradiate the area of the image sensor corresponding to the area where the intensity of the input image light is weak . The imaging apparatus according to claim 1, further comprising a superimposing unit that superimposes the input image light and the offset light. 3. The imaging apparatus according to claim 2, wherein a light diffusing unit is provided in an optical path of the offset light between the offset light output unit and the superimposing unit. The control means includes
Based on the output of the image sensor, obtain a range of the intensity of the input light formed by superimposing the offset light on the input image light,
If the intensity range of the input light is within the first threshold value or less, increase the output of the offset light of the offset light output means,
Within the range of more than the second threshold value, the imaging apparatus according to claim 1, wherein the lower the output of the offset light of the offset light output means. The control means includes
Increasing the output of the offset light so that the range of the intensity of the input light has at least a portion exceeding the first threshold;
The imaging apparatus according to claim 4 , wherein the output of the offset light is lowered so as to have at least a portion that is smaller than the second threshold value. The control means includes
The imaging apparatus according to claim 4 , wherein the output of the offset light is controlled so that a range of the intensity of the input light falls within a peak section of sensitivity characteristics of the image sensor. The offset light output means includes
Certain imaging apparatus according to claim 1, wherein the offset light output relative to the image sensor having a wavelength. An imaging method using an imaging device mounted on a vehicle,
A step of performing photoelectric conversion of input image light acquired by the image acquisition means by an image sensor;
Outputting offset light to be superimposed on the input image light from the image acquisition means to the image sensor;
Moving the region irradiated with the offset light by a moving mechanism;
A control step of controlling the output of the offset light based on the output of the image sensor;
The control step includes
Based on the output of the image sensor, a region where the intensity of the input image light is weaker than other parts is obtained,
An imaging method, wherein the moving mechanism is controlled to irradiate the area of the image sensor corresponding to the area where the intensity of the input image light is weak with the offset light .

Images