JP5074564B2 - Imaging apparatus, noise removal method, and noise removal program - Google Patents

Description

  The present invention relates to an image pickup apparatus such as a digital camera or a video camera having a noise removal function, a noise removal method and a noise removal program using the image pickup apparatus, and in particular, noise that can shoot a higher quality image. The present invention relates to an image pickup apparatus having a removal function, a noise removal method using the image pickup apparatus, and a noise removal program.

  The present invention relates to an imaging apparatus such as a digital camera and a video camera, and more particularly to an imaging apparatus having a fixed pattern noise removal function.

In recent years, electronic still cameras have been developed mainly for capturing and recording still images.
An image sensor used in this type of camera has fixed pattern noise caused by variation in output characteristics of each pixel, abnormal pixels of dark current, which are called white scratches, crystal quality of raw materials, and the like. For this reason, if the output signal of the image sensor is used as it is, the noise is superimposed on an effective signal component, which causes deterioration of the image quality of the captured image.

Further, the pixel defect includes a subsequent pixel defect that occurs transiently due to the influence of the deterioration of the crystal itself over time or the environmental change during imaging.
In order to remove these fixed pattern noises, there is an electronic still camera equipped with a noise reduction function.

  According to this noise reduction function, at the time of image capturing, the image sensor is first exposed with the shutter closed, and the dark image obtained thereby is recorded in the memory. Subsequently, the image sensor is exposed with the shutter opened, and the dark image stored in the memory is subtracted from the bright image signal obtained by this exposure.

According to this method, since the noise component generated at the time of dark output is canceled by the difference, it becomes possible to efficiently remove the defect.
Further, since a dark image is acquired every time an image is acquired, it is possible to remove most fixed pattern noise including a subsequent pixel defect that occurs transiently.

However, there are defects that cannot be removed by the above method alone.
There is an upper limit to the pixel value that can be detected by each pixel in the image sensor. Therefore, if the dark image signal is subtracted from the bright image signal where the defective pixel is saturated at the maximum value at which the pixel value can be detected, it is correct. Difference value cannot be obtained.

  For this reason, for defective pixels having a large pixel value that may be saturated, it is necessary to detect the defect position by a separate means and correct the value to remove the defect. For example, for a difference image obtained by subtracting a dark image from a bright image, a method of determining an upper and lower threshold value of a specific pixel and detecting a pixel having a value exceeding the threshold value as a defective pixel is disclosed (for example, (See Patent Document 1).

  However, since this method detects a difference image as a target, the target image has various shapes depending on the subject. Therefore, pixels having various luminance values are generated, and accurate detection using a certain reference cannot be performed.

  On the other hand, the position of a pixel having a pixel value equal to or greater than a set threshold is detected and stored in the dark image. By using this defect detection data, a technique is disclosed that can correct a defective pixel that occurs on a constant basis regardless of the subject (see, for example, Patent Document 2).

  In addition, each pixel in the image sensor has a fixed capacitance, and there is a limit to the amount of charge that can be accumulated. Therefore, if the dark image signal is subtracted from the bright image signal in which the charge amount is saturated at the maximum value, a correct difference value cannot be obtained. With white scratches, the dark current is abnormally large and the charge is likely to be saturated, and there are many dark current fluctuation components called shot noise, and image correction cannot be performed by dark image subtraction alone. Therefore, for defective pixels having a large dark current output value, it is necessary to detect the defective pixel position and separately perform pixel correction by another method to remove the defect.

  On the other hand, a method of detecting a pixel that exceeds a certain threshold at the time of dark image acquisition as a defective pixel, subtracting the dark image from the bright image, and correcting the defective pixel is shown (for example, (See Patent Document 3). According to this method, even for defective pixels that cannot be corrected by subtraction such as white scratches described above, the pixel positions can be specified and corrected separately, so efficiency is high. Further, more suitable pixel defect detection can be performed by varying the defect detection threshold according to parameters such as exposure time, imaging sensitivity, and temperature.

JP 2002-354340 A JP 2003-169258 A JP 2002-94884 A

  However, when a dark image is acquired during long exposure or high-sensitivity imaging that exceeds several tens of seconds to several minutes, only a part of the pixel area close to the heat source is caused by the heat of the internal circuit of the image sensor. Dark current increases. As a result, a large luminance unevenness occurs in the pixel area on the dark image. For this reason, there has been a problem that accurate defect detection cannot be performed with a uniform defect detection threshold.

FIG. 23 is a diagram illustrating a state in which the dark output value increases.
FIG. 23 shows that the signal value increases in the arrow direction in the figure as the exposure time becomes longer.

FIG. 24 is a diagram illustrating an example of a region where dark current unevenness occurs.
In FIG. 24, the shaded portion indicates a pixel region where dark current unevenness occurs. The tendency of dark current unevenness is not limited to the parameter of time, but also changes depending on the parameter of sensitivity setting and temperature.

  In such a pixel area (a pixel area where dark current unevenness occurs), the amount of charge that can be accumulated is inevitably smaller than in other pixel areas, and therefore normal exposure is performed in other pixel areas. There is a high possibility of charge saturation. If noise reduction is performed in this situation, a normal image cannot be obtained because output value inversion occurs due to subtraction.

FIG. 25 is a diagram showing that output value inversion occurs due to subtraction.
In order to prevent such a reversal phenomenon, when the sensitivity is reduced or the exposure time is shortened, an appropriate light amount and signal value cannot be obtained as a whole image, and the output image becomes very dark overall. . For this reason, there is a problem that any method can provide only an unnatural image having a sense of incongruity to the observer.

  In addition, since there is a limit to the amount of charge that can be stored in the image sensor, the amount of charge that can be stored in pixels in that region decreases as the dark current noise generated at this time increases. That is, the dynamic range (hereinafter referred to as the D range) that enables imaging is reduced. In order to avoid the above state, it is necessary to perform imaging with low sensitivity with less dark current noise and amplify each value of the acquired image by image processing. However, this method has a problem that the S / N ratio of the acquired image is greatly deteriorated, so that a high-quality image cannot be acquired.

  The present invention has been made in view of the above-described drawbacks of the prior art. When noise reduction is performed, accurate defect detection is performed even if dark current noise occurs, and the imaging dynamic range is increased by increasing dark current noise. An object of the present invention is to provide an image pickup apparatus that can prevent a decrease in noise as much as possible, a noise removal method and a noise removal program using the image pickup apparatus.

  In addition, the present invention provides an imaging device capable of normally removing noise and obtaining a natural image maintaining imaging sensitivity even when high sensitivity or a sufficient dynamic range cannot be ensured during long exposure. It is an object of the present invention to provide a noise removal method and a noise removal program by the imaging apparatus.

The present invention employs the following configuration in order to solve the above problems.
That is, according to one aspect of the present invention, an imaging apparatus of the present invention obtains a bright signal obtained in an unshielded state by an imaging unit having a plurality of pixels for imaging a subject and the imaging unit. A time signal acquisition means, a dark signal acquisition means for acquiring a dark signal obtained in a light-shielded state by the imaging means, and a first amplification means for amplifying the bright signal or dark signal obtained by the imaging means; Subtracting means for subtracting the dark signal acquired by the dark signal acquiring means from the bright signal acquired by the bright signal acquiring means and outputting a subtracted signal; and the subtracted signal subtracted by the subtracting means a second amplifier means for amplifying the imaging condition acquisition means for acquiring the exposure time and the imaging sensitivity of the imaging means is an imaging condition for obtaining a bright-field signal by the bright-field signal acquisition means Based on the exposure time and the imaging sensitivity of the imaging means is an image pickup condition acquired by the imaging-condition obtaining unit, a gain correction means for changing the gain of the gain and second amplifying means of the first amplifying means, the It is characterized by providing.

Further, in the imaging apparatus according to the present invention, the gain correction unit may determine the gain of the first amplification unit and the gain based on the exposure time and imaging sensitivity of the imaging unit, which are imaging conditions acquired by the imaging condition acquisition unit. It is desirable to change the gain of the first amplifying means and the gain of the second amplifying means so that the product of the gain of the second amplifying means becomes equal to the imaging sensitivity.
In the imaging apparatus of the present invention, it is desirable that the imaging condition further includes the temperature of the imaging means.
Further, the imaging apparatus of the present invention includes a defective pixel position detecting unit that detects a pixel position exceeding a predetermined pixel defect detection threshold among the dark signal acquired by the dark signal acquiring unit, and the defective pixel position. Image signal correction means for correcting the subtraction signal output by the subtraction means in a pixel corresponding to the pixel position detected by the detection means, and the imaging condition acquisition means includes the second imaging condition acquisition means. It is desirable to change the pixel defect detection threshold while changing the gain of the amplification means.

Also, according to one aspect of the present invention, a method of denoising invention, in the noise removing method performed by the image pickup device having an imaging unit having a plurality of pixels for imaging a subject, by the image pickup means A light signal obtained in a non-light-shielded state is obtained, a dark signal obtained in a light-shielded state is obtained by the imaging means, a light signal or a dark signal obtained by the imaging means is amplified, and the obtained signal is obtained. The exposure time of the imaging means, which is an imaging condition when the bright signal is acquired by subtracting the acquired dark signal from the bright signal and outputting a subtracted signal, amplifying the subtracted signal and acquires the imaging sensitivity, based on the exposure time and the imaging sensitivity of the imaging means is the acquired imaging conditions, especially to change the gain of the amplifier gain and the subtraction signal of the amplifier of the dark signal To.

In the noise removal method of the present invention, the dark signal amplification gain and the subtraction signal amplification gain are determined based on the exposure time and imaging sensitivity of the imaging means, which are the imaging conditions. It is desirable to change so that the product of the gain of amplification and the gain of amplification of the subtraction signal is equal to the imaging sensitivity.
In the noise removal method of the present invention, it is desirable that the imaging condition further includes the temperature of the imaging means.
Further, the noise removal method of the present invention further detects a pixel position exceeding a predetermined pixel defect detection threshold from the acquired dark signal, and the pixel corresponding to the detected pixel position It is desirable to correct the output subtraction signal to change the gain of amplification of the dark signal and to change the pixel defect detection threshold.

Further, according to one aspect of the present invention, the noise removal program of the present invention is a noise removal program for causing an imaging device including an imaging unit having a plurality of pixels for imaging a subject to execute the program. A procedure for obtaining a light signal obtained in a non-light-shielded state by an image pickup means, a procedure for obtaining a dark signal obtained in a light-shielded state by the image pickup means, and a light signal or dark signal obtained by the image pickup means. Amplifying procedure, subtracting the acquired dark signal from the acquired bright signal and outputting a subtracted signal, a procedure for amplifying the subtracted subtracted signal, and acquiring the bright signal a step of obtaining the exposure time and the imaging sensitivity of the imaging means is an image pickup conditions for, based on the exposure time and the imaging sensitivity of the imaging means is the acquired imaging conditions, the dark A step of changing the gain of the amplifier gain and the subtraction signal of the amplifier of the signal, a computer-executable denoising program for execution.
Further, in the noise removal program of the present invention, the procedure for changing the amplification gain of the dark signal and the amplification gain of the subtraction signal is based on the exposure time and imaging sensitivity of the imaging means, which are the imaging conditions. It is desirable to change the product of the amplification gain of the dark signal and the amplification gain of the subtraction signal to be equal to the imaging sensitivity.

  According to the present invention, even when dark current noise increases during long-time exposure or high-sensitivity imaging, defect detection can be performed with a constant accuracy in all pixel regions of the imaging device. That is, it is possible to acquire a high-quality image in the entire pixel area regardless of the imaging conditions.

  In addition, according to the present invention, it is possible to appropriately switch the threshold setting according to the imaging condition and to minimize the decrease in the D range due to the increase in dark current noise. As a result, the imaging conditions under which noise reduction can be normally performed are widened, and the imaging condition range in which an image can be acquired without deteriorating the S / N ratio is increased. That is, a wide range of imaging performance can be provided.

  Further, according to the present invention, it is possible to change the defect detection threshold in a region where dark current noise is greatly generated and another region. As a result, the increase in the number of detected defects that occurs when the threshold value is lowered can be minimized, so that the processing time is shortened and the cost is reduced by saving the memory usage.

  Further, according to the present invention, even when high sensitivity or a sufficient dynamic range cannot be ensured during long exposure, it is possible to normally remove noise and obtain a natural image with imaging sensitivity maintained.

  In addition, according to the present invention, it is possible to prevent the development of pixel defects that occur in the image processing step and to obtain an excellent image with fewer noise components.

It is a block diagram which shows the basic composition of the digital camera concerning the 1st Embodiment of this invention. It is a flowchart which performs the 1st Embodiment of this invention. It is a figure which shows the adjacent pixel with respect to the specific pixel x on an image pick-up element. It is the figure which showed the graph of the pixel position at the time of using the defect detection method used in the 1st Embodiment of this invention, and an output value. It is the figure which showed the defect detection method by the conventional method. FIG. 6 is a diagram (part 1) illustrating a graph representing a relationship between a pixel position and a luminance value according to the first embodiment of the present invention. It is FIG. (2) which shows the graph showing the relationship between a pixel position and a luminance value about the 1st Embodiment of this invention. FIG. 6 is a third diagram illustrating a graph representing a relationship between a pixel position and a luminance value according to the first embodiment of the present invention. It is FIG. (4) which shows the graph showing the relationship between a pixel position and a luminance value about the 1st Embodiment of this invention. It is a block diagram which shows the basic composition of the digital camera concerning the 2nd Embodiment of this invention. It is a figure for demonstrating the dark current noise area | region which generate | occur | produces in an image pick-up element. It is a flowchart which performs the 2nd Embodiment of this invention. It is a figure which shows the table for determining an addition threshold value with an imaging state. It is FIG. (1) which shows the graph showing the relationship between a pixel position and a luminance value about the 2nd Embodiment of this invention. It is FIG. (2) which shows the graph showing the relationship between a pixel position and a luminance value about the 2nd Embodiment of this invention. It is a block diagram which shows the basic composition of the digital camera concerning the 3rd Embodiment of this invention. It is a figure which shows the look-up table (LUT) used for 3rd Embodiment. It is an auxiliary explanatory view for explaining LUT. It is a flowchart which performs the 3rd Embodiment of this invention. It is a block diagram which shows the basic composition of the digital camera concerning the 4th Embodiment of this invention. It is a figure which shows the look-up table (LUT) used for 4th Embodiment. It is a flowchart which performs the 4th Embodiment of this invention. It is a figure which shows a mode that the output value at the time of darkness is rising. It is a figure which shows an example of the area | region where the dark current nonuniformity has generate | occur | produced. It is a figure which shows that output value inversion will arise by subtraction.

Embodiments of the present invention will be described below with reference to the drawings.
First, a first embodiment according to the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing a basic configuration of a digital camera according to the first embodiment of the present invention.

  In FIG. 1, a digital camera 1 includes an optical system 11 composed of various lenses, a shutter 12 for selectively blocking light from a subject, an image sensor 13, an analog amplifier (A-AMP) 14, and an A / D converter. 15, bright image storage unit 16, dark image storage unit 17, addition circuit 18, threshold setting unit 20, defect detection unit 21, defect storage unit 22, defect correction unit 23, output image storage unit 24, CPU 25, timing generator (TG) 26 is provided.

  The image of the subject is formed on the light receiving surface of the image sensor 13 via the optical system 11 and the shutter 12. At this time, the exposure time can be controlled by controlling the drive timing of the shutter 12 and the image sensor 13 by the timing generator 26. Further, the CPU 25 controls each unit including the timing generator 26.

  The analog amplifier 14 applies a gain corresponding to the imaging sensitivity set by the CPU 25 to the analog imaging signal obtained by the imaging device 13. The A / D converter 15 converts an analog signal into a digital value. The bright-time image storage unit 16 is a memory for storing a bright-time signal (a signal obtained by performing normal imaging when the imaging device is in a non-light-shielded state), and the dark-time image storage unit 17 is a dark-time image storage unit 17 It is a memory for storing a signal (a signal obtained by performing imaging with the imaging element in a light-shielded state). The adder circuit 18 subtracts the dark signal stored in the dark image storage unit 17 from the bright signal stored in the bright image storage unit 16 and outputs the result.

  The threshold setting unit 20 calculates a defect detection threshold based on the dark image signal obtained from the dark image storage unit 17. The defect detection unit 21 compares the signal from the threshold setting unit 20 with the dark signal stored in the dark image storage unit 17 to detect a pixel defect. The address of the detected defective pixel is stored in the defect storage unit 22. The defect correction unit 23 corrects the output image data obtained from the addition circuit 18 based on the defect information stored in the defect storage unit 22 and stores the correction result in the output image storage unit 24.

Next, the operation in the first embodiment will be described with reference to FIGS. 2 and 3.
FIG. 2 is a flowchart for executing the first embodiment of the present invention, and FIG. 3 is a diagram showing adjacent pixels to the specific pixel x on the image sensor.

The CPU 25 performs all control described below in an integrated manner.
First, in step S1, when the user starts an imaging operation, in step S2, the shutter 12 is opened for the exposure time set by the timing generator 26 to expose the image sensor 13, and a bright image is obtained. The acquired bright-time image is stored in the bright-time image storage unit 16 through the A / D converter 15 when the gain is increased by the analog amplifier 14 by the imaging sensitivity.

Next, in step S <b> 3, the dark image is acquired with the shutter 12 closed and shielded from light, and stored in the dark image storage unit 17.
Thereafter, in step S4, the threshold value setting unit 20 sets the defect detection threshold value TH (x) from the dark image stored in the dark image storage unit 17. At this time, the defect detection threshold value TH (x) is obtained for the pixel x shown in FIG. 3 and the eight peripheral pixels r _{1 to} r ₈ adjacent to the pixel x using the following equation (1). Note that V (x) is the pixel value of the pixel x, and the addition threshold value AddTH is a constant determined in advance in design.

TH (x) = (V (x) + V (r ₁ ) + V (r ₂ ) + V (r ₃ ) + V (r ₄ ) + V (r ₅ ) + V (r ₆ ) + V (r ₇ ) + V (r ₈ ) ) / 9 + AddTH Formula (1)
Thereafter, in step S5, the defect detection formula TH (x) and the dark image stored in the dark image storage unit 17 are compared, and defect detection is performed for each pixel. A pixel having a pixel value exceeding the threshold value TH (x) in each pixel is detected as a defect. The defect detected at this time is stored in the defect storage unit 22 with the pixel position as defect data.

  When the defect data is acquired, defect correction is performed in step S6. At this time, first, the light image is read out from the light image storage unit 16, the dark image is read out from the dark image storage unit 17, and the addition circuit 18 calculates the difference to reduce noise. Next, using this difference image, the defect correction unit 23 performs image correction based on the defect data stored in the defect storage unit 22. At this time, assuming that the defective pixel is x and the corrected output value is Mod (x), the pixel is corrected using the following equation (2).

Mod (x) = (V (x) + V (r ₁ ) + V (r ₂ ) + V (r ₃ ) + V (r ₄ ) + V (r ₅ ) + V (r ₆ ) + V (r ₇ ) + V (r ₈ ) ) / 8 ... Formula (2)
Finally, in step S7, when all defective pixels have been corrected, the image data is stored in the output image storage unit 24.

  FIG. 4 is a diagram showing a graph of pixel positions and output values when using the defect detection method used in the first embodiment of the present invention, and FIG. 5 shows a defect detection method according to a conventional method. It is a figure.

By adopting the method as described above with reference to FIGS. 1 to 3, the detection threshold changes following the peak of dark current noise as shown in FIG. 4, and pixel defects can be detected.
According to the conventional method, since the detection threshold is fixed, there are defects (x3, x4) that cannot be detected due to dark current noise as shown in FIG. 5, and many residual defects that are not corrected can occur. There was sex. However, according to the method of the first embodiment, even if dark current noise occurs, defects can be accurately detected in all regions, and as a result, a high-quality image can be acquired. It becomes possible.

Next, a second embodiment according to the present invention will be described with reference to FIGS.
First, the range of the D range in the noise reduction algorithm according to the second embodiment will be described with reference to FIGS. 6 to 9 which are graphs showing the relationship between the pixel position and the luminance value.

FIG. 6 is a graph showing output values of a light image and a dark image, and FIG. 7 is a graph showing a difference value between the light image and the dark image.
The light output is V (x), the dark output is D (x), and the maximum detection level is MaxLevel. At this time, the pixel x satisfying TH (x) <D (x) is detected as a pixel defect. In FIG. 6, pixels x2, x3, and x5 are detected as defective pixels.

  As shown in FIG. 6, when the light output value Vmax at the position where the peripheral pixel average is maximum satisfies the following formula (3), the difference between the light image and the dark image is shown in FIG. As described above, small defects such as defective pixels x1 and x4 disappear at the point of the difference output V ′.

Vmax ≦ MaxLevel-AddTH Formula (3)
In addition, if the value is clipped by MaxLevel as in the defective pixels x2 and x5, the defect remains, but defect detection is performed so that TH (x2) <D (x2) is satisfied. I can do it. In addition, a large defect such as a defective pixel x3 usually disappears due to a difference, but if the defect value becomes large, the linearity of the defect is easily lost between the dark image and the light image, and the defect remains after the difference. There is a high possibility that However, since the defect pixel x3 is detected by the threshold TH (x), it can be corrected in a later process.

  However, when the amount of received light increases and Vmax satisfies the following formula (4) as shown in FIG. 8, the defective pixel x1, in addition to the defective pixels x2, x3, and x5, is determined as a defect at the point of the differential output V ′. It will remain.

Vmax> MaxLevel-AddTH (4)
Since the defective pixel x1 is TH (x1)> D (x1), no defect is detected, and unlike other defective pixels, it remains as a residual defect indefinitely. That is, there is a possibility that an undetectable pixel such as the defective pixel x1 exists when the expression (4) is satisfied. For this reason, the D range for normally outputting the bright image V is expressed by the following formula (5) using the maximum value Dmax of the dark image.

Dmax ≦ Vmax ≦ MaxLevel-AddTH (5)
Next, a second embodiment according to the present invention will be specifically described with reference to FIGS.

FIG. 10 is a block diagram showing a basic configuration of a digital camera according to the second embodiment of the present invention.
In FIG. 10, a digital camera 10 includes an optical system 11 composed of various lenses, a shutter 12 for selectively blocking light from a subject, an image sensor 13, an analog amplifier (A-AMP) 14, and an A / D converter. 15, bright image storage unit 16, dark image storage unit 17, addition circuit 18, imaging state acquisition unit 19, threshold setting unit 20, defect detection unit 21, defect storage unit 22, defect correction unit 23, output image storage unit 24, a CPU 25, a timing generator (TG) 26, and a dark current noise generation area storage unit 27.

  10, the digital camera 10 is characterized in that it further includes an imaging state acquisition unit 19 and a dark current noise generation region storage unit 27, compared to the digital camera 1 described with reference to FIG. The description of the same part as 1 is omitted.

  The imaging state acquisition unit 19 has an addition threshold value determination table for determining an addition threshold value based on the exposure time and light reception sensitivity at the time of imaging. When the CPU 25 acquires exposure time and imaging sensitivity information, the imaging state acquisition unit 19 is based on those values. The addition threshold value AddTH is determined by referring to the addition threshold value determination table and sent to the threshold value setting unit 20.

Further, the dark current noise generation area storage unit 27 stores position information of a dark current noise generation area described later.
FIG. 11 is a diagram for explaining a dark current noise region generated in the image sensor.

In FIG. 11, it is shown that the entire pixel region of the image sensor is composed of a region R1 and a region R2.
The imaging element has a region R1 where dark current noise occurs during long exposure and high sensitivity imaging. Since the region R1 is unique to each element, the area R1 is set in advance for each image sensor in design such as when the image sensor is adjusted. As a setting method of the region R1, it is assumed that a dark image is acquired under a predetermined imaging condition with the shutter closed, and a continuous region in which dark current noise is a certain value or more is selected. The region R2 indicates a region obtained by removing R1 from the entire CCD pixel region. The set position information of the regions R1 and R2 is stored in the dark current noise generation region storage unit 27 in advance.

FIG. 12 is a flowchart for executing the second embodiment of the present invention, and FIG. 13 is a diagram showing a table for determining the addition threshold value according to the imaging state.
Steps S1 to S3 in FIG. 12 are the same as steps S1 to S3 in the flowchart for executing the first embodiment described with reference to FIG.

After the process of acquiring the dark image in step S3, in step S11, the imaging state acquisition unit 19 acquires the exposure time and light reception sensitivity values from the CPU 25.
In step S12, the addition threshold value (AddTH2) of the region R1 is determined based on the exposure time and light reception sensitivity values acquired in step S11, in accordance with the addition threshold value determination table shown in FIG. The addition threshold value determination table is set such that the longer the exposure time, the smaller the addition threshold value (the same applies to columns A4 ≦ A3 ≦ A2 ≦ A1, B, C, and D), and the addition threshold value increases as the sensitivity increases. It is set to be a large value (A4 ≦ B4 ≦ C4 ≦ D4, 1, 2, and 3 rows). Further, the addition threshold AddTH of the region R2 is unchanged.

Thereafter, in step S13, the threshold value setting unit 20 calculates the defect detection threshold value TH for the region R1 from the dark image stored in the dark image storage unit 17 and the added threshold value AddTH acquired from the imaging state acquisition unit 19. (X) A defect detection threshold TH (x) _R2 is set for _R1 and region R2. In this case, the calculation formula of the defect detection threshold TH (x) _R1 and TH (x) _R2 is the same as in the first embodiment both regions R1, R2. That is, the detection threshold value in the region R1 is obtained by the following equation (6) using AddTH2 determined by the imaging state acquisition unit 19, and the detection threshold value in the region R2 is calculated using the normal addition threshold value AddTH. It calculates | requires by the following formula | equation (7).

TH (x) = (V (r ₁ ) + V (r ₂ ) + V (r ₃ ) + V (r ₄ ) + V (r ₅ ) + V (r ₆ ) + V (r ₇ ) + V (r ₈ )) / 9 + AddTH 2 ..Formula (6)
TH (x) = (V (r ₁ ) + V (r ₂ ) + V (r ₃ ) + V (r ₄ ) + V (r ₅ ) + V (r ₆ ) + V (r ₇ ) + V (r ₈ )) / 9 + AddTH • ..Formula (7)
Subsequent steps S5 to S7 are the same as steps S5 to S7 in the flowchart for executing the first embodiment described with reference to FIG.

14 and 15 are graphs showing the relationship between the pixel position and the luminance value for the second embodiment of the present invention.
According to the method described above, as shown in FIG. 14, the threshold value is changed only in the region R1 by lowering the addition threshold AddTH2 under the imaging condition with a long exposure time (FIG. 13: A4 ≦ A3 ≦ A2 ≦ A1). It becomes possible to make it. As the exposure time becomes longer, dark current noise increases and the D range becomes narrower. At this time, the addition threshold value is lowered, and the D range can be secured wider by (AddTH−AddTH2). At this time, since the threshold value is lowered more than necessary, variation pixels in the dark image, small defects that can be corrected by the image difference, etc. are applied to the threshold value, and false detection of defects increases (x1 in FIG. 15). . However, by limiting the range in which the addition threshold value is lowered to the region R1, it is possible to minimize the amount of erroneous detection of a defect and to prevent an increase in memory consumption and processing time. .

  Furthermore, since the output value tends to vary when the sensitivity is improved, if the defect detection threshold value is too low, the variation value is applied to the threshold value, and there is a high risk of erroneous detection of the defect. Therefore, the above situation can be prevented by increasing the addition threshold value in the high sensitivity imaging condition (FIG. 13: A4 ≦ C4 ≦ B4 ≦ D4).

In the first embodiment and the second embodiment, the addition threshold AddTH is a constant. However, the addition threshold AddTH may be set by a function α (V) having the dark output level V as a variable.
In the second embodiment, the setting method of the region R1 is determined at the time of image sensor adjustment. However, even if the region R1 is determined each time based on the pixel value of the dark image acquired at the time of noise reduction. good.

Next, a third embodiment according to the present invention will be described with reference to FIGS.
FIG. 16 is a block diagram showing a basic configuration of a digital camera according to the third embodiment of the present invention.

  In FIG. 16, the digital camera 30 amplifies signals according to an optical system 51 composed of various lenses, a shutter 52 for selectively blocking light from a subject, an image sensor 53 such as a CCD, and a set imaging gain. Amplifier (AMP1) 54, A / D converter 55 for converting an analog signal into a digital value, bright image memory 56, dark image memory 57, pixel defect detector 58, and the address of the detected defective pixel are stored. Defect address memory 59, signal subtraction unit 60, defect correction unit 61, adjustment amplifier (AMP2) 62 for adjusting signal output, acquired image memory 63 for storing acquired images, look-up table (LUT) 64, imaging A condition determination unit 65, a timing generator (TG) 66, and a CPU 67 are provided.

  In such a digital camera 30, the image of the subject is formed on the light receiving surface of the image sensor 53 via the optical system 51 and the shutter 52. At this time, the exposure time can be controlled by controlling the drive timing of the shutter 52 and the image sensor 53 by the TG 66.

  Here, the bright-time image memory 56 is a memory for storing a bright-time image signal (image signal obtained by performing normal imaging when the imaging device is in a non-light-shielded state), and the dark-time image memory 57 is This is a memory for storing a dark image signal (image signal obtained by imaging with the image sensor in a light-shielded state).

  The pixel defect detection unit 58 compares a dark signal value stored in the dark image memory 57 with a pixel defect detection threshold set in advance in advance, detects a signal exceeding the pixel defect detection threshold as a pixel defect, The address (pixel position) of the pixel determined to be defective is stored in the defect address memory 59.

  The signal subtracting unit 60 subtracts the signal stored in the dark image memory 57 from the signal stored in the bright image memory 56 and outputs the result, which is transmitted to the defect correcting unit 61. The defect correction unit 61 corrects the output data of the signal subtraction unit 60 based on the defect information stored in the defect address memory 59. This defect correction is performed by replacing the value with the signal average value of the peripheral pixels when the current pixel address matches the pixel address stored in the defect address memory 59.

  The adjustment amplifier (AMP2) 62 performs signal amplification so that the signal reduced by the dark current component by the dark image subtraction performed by the signal subtraction unit 60 can be utilized up to the full range. The signal value obtained as a result is stored in the acquired image memory 63.

  The imaging condition determination unit 65 reads a setting value suitable for the imaging condition obtained from the CPU 67 from the LUT 64, and performs gain setting of the amplifier (AMP1) 54 and the adjustment amplifier (AMP2) 62. Note that the LUT 64 stores determination values and various setting values used for imaging condition determination.

  Then, by controlling the drive timing of the shutter 52 and the image sensor 63 by the TG 66, it is possible to control the exposure time and the light shielding or non-light shielding state. The CPU 67 controls each unit including the timing generator 66.

Next, the setting contents of the LUT used in the third embodiment will be described.
FIG. 17 is a diagram showing an input / output relationship of a lookup table (LUT) used in the third embodiment, and FIG. 18 is an auxiliary explanatory diagram for explaining the LUT.

  In FIG. 17, the input value of the LUT 64 has exposure time and imaging sensitivity, and the output value has gain setting values of the amplifier (AMP1) 54 and the adjustment amplifier (AMP2) 62. For example, the portion corresponding to “A × 2” shown in FIG. 18 is imaged and noise-reduced under the same conditions as the portion corresponding to “A”, and is doubled by the adjustment amplifier (AMP2) 62 (× 2) performing the gain correction. It is assumed that the adjustment amplification described above is separately performed by the adjustment amplifier (AMP2) 62.

  According to this LUT 64, when paying attention to a specific exposure time, for the sensitivity setting above a certain level, the amount of amplification performed by the amplifier (AMP1) 54 is limited, and the gain correction by the adjustment amplifier (AMP2) 62 is used. The total gain is increased. With these gain settings, amplification by the amplifier (AMP1) 54 is suppressed, and the peak value of the dark current at the time of dark image subtraction performed by the signal subtracting unit 60 is adjusted so as not to exceed a predetermined value.

Next, the operation in the third embodiment will be described with reference to FIG.
FIG. 19 is a flowchart for executing the third embodiment of the present invention.
The CPU 67 performs all control described below in an integrated manner.

  First, when the user starts an imaging operation in step S191, in step S192, the imaging condition determination unit 65 obtains the set exposure time and imaging sensitivity values from the CPU 67, and the LUT 64 described with reference to FIG. Against. In step S193, when the imaging condition determination unit 65 obtains a corresponding setting value from the LUT 64, the imaging condition determination unit 65 sets the gain values of the amplifier (AMP1) 54 and the adjustment amplifier (AMP2) 62.

  Next, in step S194, a bright image and a dark image are acquired. Specifically, the shutter 52 is opened for a predetermined exposure time using the TG 66 to expose the image sensor 53, and a bright image is acquired. The acquired image is subjected to signal amplification in the amplifier (AMP1) 54 and stored in the bright image memory 56. Similarly, a dark image is acquired with the shutter 52 closed and shielded from light, and is stored in the dark image memory 57 through the same processing.

  In step S195, noise reduction is executed. Specifically, first, a pixel defect is extracted by the pixel defect detection unit 58, and the address (pixel position) of the pixel determined to be defective is stored in the defect address memory 59. On the other hand, the signal subtracting unit 60 subtracts the image signal stored in the dark image memory 57 from the image signal stored in the bright image memory 56 and outputs the result. The defect correction unit 61 corrects the pixel corresponding to the position stored in the defect address memory 59 with respect to the subtraction result output from the signal subtraction unit 60. Noise reduction is performed by these series of operations.

  Thereafter, in step S196, a correction gain is applied to the output image using the gain value of the adjustment amplifier (AMP2) 62 set in step S193. Finally, in step S197, the obtained image is stored in the acquired image memory 63.

  According to the above method, by setting the LUT 64 in advance, it is possible to limit the gain with respect to the amount of dark current generated and to prevent inversion during signal subtraction. Thereby, it is possible to provide a noise reduction function having a wider usage environment. That is, even when dark current unevenness occurs during high sensitivity or long-time exposure, it is possible to normally remove noise and obtain a natural image with apparent sensitivity maintained.

  A configuration in which several types of LUT 64 are created and the LUT 64 to be used is changed can be easily realized. In this way, when performing image acquisition, various processing methods can be selected depending on the purpose or application.

  Furthermore, although the exposure time and the imaging sensitivity are given as input parameters of the LUT 64, it can also be realized by adding the temperature of the imaging element 53 as a parameter. As a result, the gain setting can be defined more finely, and a suitable image can be obtained for each imaging condition.

Next, a fourth embodiment according to the present invention will be described with reference to FIGS. Note that description of the same parts as those in the third embodiment is omitted.
The fourth embodiment is characterized in that the imaging condition determination unit 65 in the third embodiment has a function of changing the defect detection threshold.

First, the principle of the fourth embodiment will be described.
In a series of noise reduction processes, pixel defects that are equal to or higher than the defect detection threshold (TH) are corrected by the defect correction unit 61, and pixel defects that are equal to or lower than the defect detection threshold (TH) are corrected by subtraction by the signal subtracting unit 60. . At this time, strictly speaking, there are some pixel defects that remain below the defect detection threshold (TH) but cannot be subtracted and corrected. However, since the level of the pixel defect is sufficiently small, it is not regarded as a defect.

  However, by performing gain correction in the adjustment amplifier (AMP2) 62 after noise reduction as shown in the third embodiment, even if the pixel defect is not noticeable at the time of dark image subtraction, it is a conspicuous level after gain correction. There is a possibility of development. Therefore, when the adjustment amplifier (AMP2) 62 performs gain correction of twice or more, the defect detection thresholds TH1, TH2, TH3,... (TH ≧ TH1 ≧ TH2) lower than the defect detection threshold (TH) used during normal operation. By using ≧ TH3 ≧..., It is possible to suppress developing pixel defects.

  At this time, if pixel defects are excessively detected and corrected, the original information of the image may be lost, so care must be taken not to reduce the defect detection threshold excessively. Therefore, the defect detection threshold values (TH1, TH2, TH3,...) Are determined in advance so as to maintain a balance between prevention of pixel defect development and excessive defect detection.

  FIG. 20 is a block diagram showing a basic configuration of a digital camera according to the fourth embodiment of the present invention, and FIG. 21 is a diagram showing a look-up table (LUT) used in the fourth embodiment. .

  20, a digital camera 40 includes an optical system 51 including various lenses, a shutter 52, an image sensor 53, an amplifier (AMP1) 54, an A / D converter 55, a bright image memory 56, a dark image memory 57, and pixels. Defect detection unit 71, defect address memory 59, signal subtraction unit 60, defect correction unit 61, adjustment amplifier (AMP2) 62, acquired image memory 63, lookup table (LUT) 72, imaging condition determination unit 73, timing generator (TG) ) 66 and a CPU 67.

  In FIG. 20, the digital camera 40 includes a pixel defect detection unit 71 instead of the pixel defect detection unit 58 as compared with the digital camera 30 described with reference to FIG. 16, and performs a lookup instead of the lookup table (LUT) 64. Since the table (LUT) 72 is provided and the imaging condition determination unit 73 is provided instead of the imaging condition determination unit 65, description of the same parts as those of the digital camera 30 is omitted.

  The LUT 72 has defect detection threshold values (TH1, TH2, TH3,...) In addition to the gain values of the amplifier (AMP1) 54 and the adjustment amplifier (AMP2) 62 as output values. In the configuration diagram of the LUT 72 shown in FIG. 21, since the defect detection threshold TH is determined by the exposure time t and the imaging sensitivity g, it is shown as TH (t, g) in the drawing.

  The imaging condition determination unit 73 reads out setting values that match the imaging conditions obtained from the CPU 27 from the LUT 72, sets the gains of the amplifier (AMP1) 54 and the adjustment amplifier (AMP2) 62, and uses the defects used in the pixel defect detection unit 71. Set the detection threshold.

  Then, the pixel defect detection unit 71 detects a defective pixel based on the dark image signal stored in the dark image memory 57 and the threshold value sent by the imaging condition determination unit 73, and the address of the pixel determined to be defective. Is stored in the defective address memory 59.

Next, the operation in the fourth embodiment will be described with reference to FIG.
FIG. 22 is a flowchart for executing the fourth embodiment of the present invention.
The CPU 67 performs all control described below in an integrated manner.

  The start of the imaging operation in step S191 and the determination of the imaging condition in step S192 are the same as those in step S191 and step S192 in the third embodiment of the present invention described with reference to FIG.

  In step 221, when the imaging condition determination unit 73 obtains a corresponding setting value from the LUT 72, the imaging condition determination unit 73 sets the gain values of the amplifier (AMP1) 54 and the adjustment amplifier (AMP2) 62. In step S222, a defect detection threshold value used in the pixel defect detection unit 71 is set.

The acquisition of the bright image and the dark image in step S194 is the same as that in step S194 in the third embodiment.
Thereafter, noise reduction is executed in step S223. Specifically, first, the pixel defect detection unit 71 extracts a pixel defect using the defect detection threshold set by the imaging state determination unit 73 and stores it in the defect address memory 59. On the other hand, the signal subtracting unit 60 subtracts the image signal stored in the dark image memory 57 from the image signal stored in the bright image memory 56 and outputs the result. The defect correction unit 61 corrects the pixel corresponding to the position stored in the defect address memory 59 with respect to the subtraction result output from the signal subtraction unit 60. Noise reduction is performed by these series of operations.

The correction gain in step S196 and the image storage in step S197 are the same as those in step S196 and step S197 in the third embodiment.
According to the above method, when the gain of the adjustment amplifier (AMP2) 62 is set, the pixel defect detection threshold can be lowered from the normal value. As a result, even pixel defects having a level lower than normal are extracted, so that pixel defects that develop when amplification is performed by the adjustment amplifier (AMP2) 62 can be suppressed. Further, by adjusting the LUT 72 in advance, it becomes possible to optimize the output image more, so that an excellent image with fewer noise components can be provided.

  As described above, each embodiment of the present invention has been described with reference to the drawings. However, an imaging apparatus to which the present invention is applied is limited to the above-described embodiments and the like as long as the function is executed. Needless to say, a single device, a system composed of a plurality of devices, an integrated device, or a system that performs processing via a network such as a LAN or WAN may be used. .

  It can also be realized by a system comprising a CPU, ROM or RAM memory connected to the bus, input device, output device, external recording device, medium driving device, portable recording medium, and network connection device. That is, a ROM or RAM memory, an external recording device, and a portable recording medium that record the program code of the software that realizes the system of each of the above-described embodiments is supplied to the imaging device, and the computer of the imaging device performs the program. Needless to say, this can also be achieved by reading and executing the code.

  In this case, the program code itself read from the portable recording medium or the like realizes the novel function of the present invention, and the portable recording medium or the like on which the program code is recorded constitutes the present invention. .

  Examples of portable recording media for supplying program codes include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, DVD-RAMs, magnetic tapes, and non-volatile memories. Various recording media recorded through a network connection device (in other words, a communication line) such as a card, a ROM card, electronic mail or personal computer communication can be used.

  In addition, the functions of the above-described embodiments are realized by executing the program code read out on the memory by the computer, and the OS running on the computer is actually executed based on the instruction of the program code. The functions of the above-described embodiments are also realized by performing part or all of the process.

  Furthermore, a program code read from a portable recording medium or a program (data) provided by a program (data) provider is provided in a function expansion board inserted into a computer or a function expansion unit connected to a computer. The CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are also performed by the processing. Can be realized.

  That is, the present invention is not limited to the above-described embodiments and the like, and can take various configurations or shapes without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 Digital camera 10 Digital camera 11 Optical system 12 Shutter 13 Image pick-up element 14 Analog amplifier (A-AMP)
DESCRIPTION OF SYMBOLS 15 A / D converter 16 Light image storage part 17 Dark image storage part 18 Addition circuit 19 Imaging state acquisition part 20 Threshold setting part 21 Defect detection part 22 Defect storage part 23 Defect correction part 24 Output image storage part 25 CPU
26 Timing generator (TG)
27 Dark Current Noise Generation Area Storage Unit 30 Digital Camera 40 Digital Camera 51 Optical System 52 Shutter 53 Image Sensor 54 Amplifier (AMP1)
55 A / D converter 56 Bright image memory 57 Dark image memory 58 Pixel defect detection unit 59 Defect address memory 60 Signal subtraction unit 61 Defect correction unit 62 Adjustment amplifier (AMP2)
63 Acquired image memory 64 Look-up table (LUT)
65 Imaging Condition Determination Unit 66 Timing Generator (TG)
67 CPU
71 Pixel Defect Detection Unit 72 Look-up Table (LUT)
73 Imaging condition determination unit

Claims (10)

Imaging means having a plurality of pixels for imaging a subject;
A light signal acquisition means for acquiring a light signal obtained in a non-shielded state by the imaging means;
Dark signal acquisition means for acquiring a dark signal obtained in a light-shielded state by the imaging means;
First amplification means for amplifying a light signal or dark signal obtained by the imaging means;
Subtracting means for subtracting the dark signal acquired by the dark signal acquiring means from the bright signal acquired by the bright signal acquiring means and outputting a subtracted signal;
Second amplification means for amplifying the subtraction signal subtracted by the subtraction means;
An imaging condition acquisition unit that acquires an exposure time and an imaging sensitivity of the imaging unit, which is an imaging condition when the bright signal is acquired by the bright signal acquisition unit;
A gain correction unit that changes the gain of the first amplification unit and the gain of the second amplification unit based on the exposure time and imaging sensitivity of the imaging unit, which are imaging conditions acquired by the imaging condition acquisition unit;
An imaging apparatus comprising: The gain correction unit is configured to obtain a gain of the first amplifying unit and a gain of the second amplifying unit based on an exposure time and an imaging sensitivity of the imaging unit which are imaging conditions acquired by the imaging condition acquiring unit. The imaging apparatus according to claim 1, wherein the gain of the first amplifying unit and the gain of the second amplifying unit are changed so that a product becomes equal to the imaging sensitivity. The imaging conditions, further, the imaging apparatus according to claim 1 or 2, characterized in that it comprises a temperature of the imaging means. Of the dark signal acquired by the dark signal acquisition means, a defective pixel position detection means for detecting a pixel position exceeding a predetermined pixel defect detection threshold;
Image signal correction means for correcting the subtraction signal output by the subtraction means in the pixel corresponding to the pixel position detected by the defective pixel position detection means;
Further comprising
The imaging-condition obtaining unit, the image pickup apparatus according to any one of claims 1 to 3, characterized in that to change the pixel defect detection threshold with changing the gain of the second amplifying means. In a noise removal method executed by an imaging apparatus including an imaging unit having a plurality of pixels for imaging a subject,
Obtaining a bright signal obtained in a non-shielded state by the imaging means;
Obtaining a dark signal obtained in a light-shielded state by the imaging means;
Amplifying the light signal or dark signal obtained by the imaging means;
Subtracting the acquired dark signal from the acquired light signal to output a subtracted signal;
Amplifying the subtracted subtracted signal;
Obtaining an exposure time and imaging sensitivity of the imaging means, which is an imaging condition when acquiring the light signal,
A noise removal method, wherein the gain of amplification of the dark signal and the gain of amplification of the subtraction signal are changed based on an exposure time and imaging sensitivity of the imaging means which are the acquired imaging conditions. The amplification gain of the dark signal and the amplification gain of the subtraction signal are determined based on the exposure time and imaging sensitivity of the imaging means, which are the imaging conditions, and the amplification of the subtraction signal. The noise removal method according to claim 5, wherein a product of the gain and the gain is changed to be equal to the imaging sensitivity. The imaging condition, further noise removing method according to claim 5 or 6, characterized in that it comprises a temperature of the imaging means. further,
Among the acquired dark signal, detect a pixel position that exceeds a predetermined pixel defect detection threshold,
Performing correction on the output subtraction signal in the pixel corresponding to the detected pixel position;
Noise removing method according to any one of claims 5 to 7, characterized in that to change the pixel defect detection threshold with changing the gain of amplification of the dark signal. A noise removal program for causing an image pickup apparatus including an image pickup unit having a plurality of pixels for picking up an image of an object,
A procedure for obtaining a bright signal obtained in a non-shielded state by the imaging means;
A procedure for obtaining a dark signal obtained in a light-shielded state by the imaging means;
Amplifying the light signal or dark signal obtained by the imaging means;
Subtracting the acquired dark signal from the acquired light signal to output a subtracted signal;
Amplifying the subtracted subtracted signal;
A procedure for acquiring an exposure time and imaging sensitivity of the imaging means, which is an imaging condition for acquiring the light signal;
A procedure for changing the amplification gain of the dark signal and the amplification gain of the subtraction signal based on an exposure time and imaging sensitivity of the imaging means that is the acquired imaging condition;
A computer-executable noise removal program for running. The procedure for changing the amplification gain of the dark signal and the amplification gain of the subtraction signal is based on the exposure time and imaging sensitivity of the imaging means, which are the imaging conditions, and the amplification gain of the dark signal and the The noise removal program according to claim 9, wherein a product of a gain of the subtraction signal and an amplification gain is changed to be equal to the imaging sensitivity.