JP6705252B2 - Imaging device, pixel defect correction device, and pixel defect correction method - Google Patents

Description

The present invention relates to an image pickup apparatus including an image pickup device in which a plurality of photoelectric conversion layers are stacked, and more particularly, to pixel defect correction of an image pickup device in which a plurality of photoelectric conversion layers are stacked.

An imaging device used in a digital camera or the like uses a plurality of photodiode layers in which a plurality of photoelectric conversion units forming pixels are two-dimensionally arranged by utilizing that the light absorption characteristic of a semiconductor substrate varies depending on the wavelength of light. An image pickup device stacked in a substrate is known. For example, in a p-type silicon substrate, an n-type layer, a p-type layer, and an n-type layer are sequentially formed from the substrate surface along the depth direction (optical path), and three photoelectric conversion units have the same depth in the same pixel. Stacked along the direction.

In such an image pickup device, noise such as dark current in which the generation location is not constant in the depth direction is generated. That is, even in the same pixel, there are a photoelectric conversion unit in which a defect occurs and a photoelectric conversion unit in which no defect occurs. However, the pixel signal generated by the photoelectric conversion unit having no defect is also affected by the pixel signal deterioration due to the defective photoelectric conversion unit.

In order to prevent this, when it is detected that noise is included in the pixel signal generated by one of the photoelectric conversion units in the same pixel, that is, it is detected that there is a defect, another photoelectric conversion without a defect in the same pixel is performed. The correction process is also performed on the pixel signal generated by the unit (see Patent Document 1).

JP, 2008-42944, A

Since all the pixel signals generated by the photoelectric conversion unit of the same pixel are corrected, the pixel signal information of that pixel is completely lost. As a result, the pixel defect correction may significantly deteriorate the image quality.

Therefore, it is required to perform pixel defect correction that suppresses deterioration of image quality in an image pickup device in which a plurality of photoelectric conversion layers are stacked.

The imaging device of the present invention provides an imaging device in which a plurality of photoelectric conversion layers in which a plurality of photoelectric conversion units are two-dimensionally arranged are stacked, and a pixel signal output from a defective photoelectric conversion unit in the plurality of photoelectric conversion layers. A correction unit that corrects is provided, and the correction unit performs the correction based on the pixel signal output from the photoelectric conversion unit of the photoelectric conversion layer having no defective photoelectric conversion unit.

The correction unit may perform correction based on a correlation between a pixel signal output from a photoelectric conversion unit of another photoelectric conversion layer included in the same pixel and a pixel signal output from a photoelectric conversion unit around the photoelectric conversion unit. it can. For example, the correction unit may perform the correction based on the ratio of the pixel signal output from the photoelectric conversion unit of the other photoelectric conversion layer included in the same pixel and the pixel signal output from the photoelectric conversion unit around it. You can Alternatively, the correction can be performed based on the difference between the pixel signal output from another photoelectric conversion unit included in the same pixel and the pixel signal output from the peripheral photoelectric conversion unit.

In addition, the correction unit can perform the correction based on the pixel signal output from the photoelectric conversion unit having a value that is closest to the pixel value of another photoelectric conversion unit included in the same pixel.

On the other hand, the correction unit, when there is a peripheral photoelectric conversion unit having a defect in the adjacent photoelectric conversion unit in the other photoelectric conversion layer, except for the pixel signal output from the peripheral photoelectric conversion unit, You can make a correction.

A pixel defect correction apparatus according to another aspect of the present invention outputs from a photoelectric conversion unit having a defect in a plurality of photoelectric conversion layers of an image pickup device in which a plurality of photoelectric conversion layers in which a plurality of photoelectric conversion units are two-dimensionally arranged are stacked. The pixel defect correction device corrects the pixel signal to be corrected, and performs the correction based on the pixel signal output from the photoelectric conversion unit of the photoelectric conversion layer having no defective photoelectric conversion unit.

A pixel defect correction method according to another aspect of the present invention is directed to output from a photoelectric conversion unit having a defect in a plurality of photoelectric conversion layers of an imaging device in which a plurality of photoelectric conversion layers in which a plurality of photoelectric conversion units are two-dimensionally arranged are stacked. Pixel defect correction method for correcting a pixel signal that is generated, the correction is performed based on the pixel signal output from the photoelectric conversion unit of the photoelectric conversion layer having no defective photoelectric conversion unit.

According to the present invention, it is possible to perform pixel defect correction processing while suppressing deterioration of image quality in an image pickup device in which photoelectric conversion units are stacked.

It is a block diagram of the digital camera which is a 1st embodiment. It is the figure which showed the internal structure of the single-plate type|mold multilayer image sensor. It is a figure showing the internal structure of the predetermined pixel of an image sensor. It is the figure which showed the arrangement|sequence in case one defective photoelectric conversion part exists in one photoelectric conversion layer. It is the figure which showed the block when one defective photoelectric conversion part exists in two photoelectric conversion layers. It is the figure which showed the block in which the center photoelectric conversion part of the block of three photoelectric conversion layers has a defect in all. It is the figure which showed the block in which the defect has arisen in the peripheral photoelectric conversion part in one photoelectric conversion layer. It is the figure which showed the block in which the defect has arisen in the peripheral photoelectric conversion part in two photoelectric conversion layers. It is a figure showing an example of an image sensor which laminated a photodiode in a silicon substrate.

The digital camera of this embodiment will be described below with reference to the drawings. FIG. 1 is a block diagram of a digital camera according to the first embodiment.

The digital camera 10 includes a camera body 30 and an interchangeable lens 20 that is detachable from the camera body 30. The system control circuit 40 including the camera CPU outputs a control signal to the lens control circuit 56, the image processing circuit 34, etc. in accordance with an input operation on a power button, a release button, a mode selection dial (none of which is shown), and the focus A series of camera operation controls such as adjustment, shooting control, image processing, recording processing, reproduction display processing are performed.

The system control circuit 40 has an arithmetic unit, a ROM, a RAM, an interface circuit and the like (all not shown), and a program for controlling camera operation is stored in a recording medium (not shown) such as a ROM. The operation switch group 38 includes a power switch, a shooting mode selection switch, a release switch, and the like. A timing generator (not shown) outputs a clock pulse signal having a predetermined frequency to each circuit. The operation timing of each circuit follows the clock pulse signal sent.

In the shooting mode, the light from the subject that has passed through the shooting optical system 22 forms an image on the light receiving surface of the image sensor 32, and a subject image is formed on the image sensor 32. The image sensor 32 is a single-plate CMOS image sensor in which (M×N) pixels are arrayed here, and is configured as a multilayer image sensor in which a plurality of photoelectric conversion layers are stacked and arranged, as described later. ing. The image sensor drive circuit 36 drives the image sensor 32 and reads pixel signals for one field or one frame from the image sensor 32 at predetermined time intervals. An XY address type image pickup device other than CMOS may be used.

The pixel signal for one field/frame read from the image sensor 32 is converted into a digital signal by the AD converter 33, and then temporarily stored in the image memory 35. The image processing circuit 34 performs color interpolation processing, gamma correction processing, white balance adjustment, and the like on the pixel signal for one field/frame to generate a color image signal. The system control circuit 40 drives the display device 37 such as an LCD based on the color image signal, whereby a through image is displayed on the display device 37.

When the release button is pressed halfway, focus adjustment is performed. The system control circuit 40 executes AF processing based on the pixel signal read from the image sensor 32. That is, the defocus amount is calculated and the image plane is shifted to the in-focus position. Further, AF processing by multi-point distance measurement can be executed. The lens CPU 28 can communicate with the lens control circuit 56 via the camera side mount contact and the lens side mount, and controls the lens drive mechanism 26 based on a command from the lens control circuit 56. The lens driving mechanism 26 moves the focusing lens of the photographing optical system 22 along the optical axis direction in accordance with a control signal from the lens CPU 28. When the release button is pressed halfway down, the system control circuit 40 detects the brightness of the subject based on the pixel signals read from the image sensor 32 and calculates the exposure value (shutter speed, aperture value, etc.).

When the release button is fully pressed, the system control circuit 40 controls the exposure. Here, the electronic shutter function of the image sensor 32 adjusts the shutter speed, that is, the exposure time. The lens CPU 28 adjusts the opening degree of the diaphragm 23 according to the transmitted aperture value data, and adjusts the amount of light incident on the image sensor 32.

When the pixel signal for one frame is read from the image sensor 32, the image processing circuit 34 performs white balance processing, γ correction processing, and the like on the pixel signal for one frame to generate still image data. The still image data is recorded in a removable recording medium 54 such as a memory card after being compressed or temporarily stored in the image memory 35 in an uncompressed state.

As will be described later, the system control circuit 40 performs pixel defect correction in order to prevent deterioration of the R, G, and B pixel signals sent from the image sensor 32 due to defective pixels (photoelectric conversion unit). To do. Pixel defect data indicating the position of the defective pixel is stored in advance in a pixel defect data memory (not shown) by prior measurement or the like, and the pixel value of the defective pixel is corrected based on this information.

FIG. 2 is a diagram schematically showing a laminated structure of the image sensor. FIG. 3 is a diagram showing an internal structure of a predetermined pixel of the image sensor.

As shown in FIG. 2, in the image sensor 32, three photoelectric conversion layers 62B, 62G, and 62R are laminated on the silicon substrate 75 in this order from the incident side, and the insulating films 60, 61, and 62 are interposed between them. Intervene in. The photoelectric conversion layer 62B selectively absorbs and photoelectrically converts light according to the wavelength range of B, and transmits light according to light (including G and R) in other wavelength ranges. The photoelectric conversion layer 62G absorbs and photoelectrically converts light in a wavelength range corresponding to G, and transmits light in other wavelength ranges. The photoelectric conversion layer 62R absorbs and photoelectrically converts light in the wavelength range corresponding to R, and transmits light in the other wavelength ranges.

The photoelectric conversion layers 62B, 62G, and 62R are each configured by a thin photoelectric conversion film that selectively absorbs light in a wavelength range corresponding to B, G, and R and photoelectrically converts the light. It consists of a conversion film. The photoelectric conversion layer 62B can be made of, for example, a perylene derivative, the photoelectric conversion layer 62G can be made of, for example, quinacridone, and the photoelectric conversion layer 62R can be made of a phthalocyanine derivative. is there. It should be noted that the molding may be made of an inorganic material or an organic-inorganic mixed material instead of the organic material.

As shown in FIG. 3, the photoelectric conversion layer 62B is sandwiched between the pixel electrode 92B and the counter electrode 91B, and when light corresponding to B is incident, charges are generated. The generated charges are stored in the storage diode 66B formed on the silicon substrate 75 via the plug 63. A pixel signal read circuit 74 is formed on the silicon substrate 75, and a pixel signal is read from the storage diode 66B by a drive pulse.

The photoelectric conversion layers 62G and 62R are also sandwiched between the pixel electrodes 82G and 72R and the counter electrodes 81G and 71R, respectively, and charges are stored in the storage diodes 66G and 66R via the plugs 65 and 67. It The pixel electrodes 72R, 82G, 92B and the counter electrodes 71R, 81G, 91B are formed of a transparent electrode material, and the pixel electrodes 92B, 82G, 72R and the counter electrodes 71R, 81G, 91B are two-dimensionally arranged in a matrix. This defines the pixel area of the image sensor 32.

The pixel regions of the photoelectric conversion layers 62B, 62G, and 62R face each other, and each pixel of the image sensor 32 has the pixel regions of the three photoelectric conversion layers 62B, 62G, and 62R (hereinafter referred to as photoelectric conversion units). A microlens array is formed on the photoelectric conversion layer 62B in accordance with the pixel position, and the light incident on each microlens is at the same position along the depth direction of the photoelectric conversion layers 62B, 62G, and 62R. It is incident on the photoelectric conversion unit. By stacking the photoelectric conversion layers 62B, 62G, and 62R that function as R, G, and B color filters on each pixel of the image sensor 32, the pixel signals of R, G, and B are output from each pixel of the image sensor 32. To be done.

FIG. 4 is a diagram showing an arrangement in which one defective photoelectric conversion unit is present in one photoelectric conversion layer. Here, 3×3 blocks defined in the partial regions 32R, 32G, and 32B of the image pickup region IM of the image pickup device 32 shown in FIG. 2 are shown. Similar correction processing is performed on the other blocks.

In FIG. 4, in the 3×3 block BR of the photoelectric conversion layer 62R, the case where the central photoelectric conversion portion R ₀ has a defect is shown. In this case, the correction process is performed using the pixel signals output from the photoelectric conversion units of the photoelectric conversion layers 62G and 62B that do not include the defective photoelectric conversion unit.

More specifically, _{initially, R n / (R n +} G n + B n) (n = 1,2, ··, 8) , and calculates an average value or median value as P. Then, R ₀ =(G ₀ +B ₀ )/(1/P-1) is obtained from the equation of P=R ₀ /(R ₀ +G ₀ +B ₀ ).

That is, the photoelectric conversion layer 62B, 62G, at _{62R (0 B, G 0,} R 0) other than the output from the photoelectric conversion section _{(B n, G n, R} n) pixel values of R for the pixel signal sum of The value of the pixel signal of the defective photoelectric conversion unit R ₀ is corrected by assuming that the ratio average of the defect photoelectric conversion unit R ₀ is equal to the ratio average of (B ₀ , G ₀ , R ₀ ).

Alternatively, the pixel defect correction processing may be performed based on the ratio. First, B _n /G _n (n=1, 2,..., 8) is calculated, and B _m /G _m closest to the value of B ₀ /G ₀ is calculated, and then Calculate P _m =R _m /(R _m +G _m +B _m ) according to the position m. Then, R ₀ is calculated from the equation P=R ₀ /(R ₀ +G ₀ +B ₀ )=P _m .

The ratio between the photoelectric conversion units G ₀ and B _{0 at the} center position and the adjacent photoelectric conversion units is calculated, and the position where the value is the closest to the value of B ₀ /G ₀ , that is, the position where the correlation is strong. Is detected, and pixel defect correction processing is performed based on the pixel signal of the photoelectric conversion unit at that position.

It is also possible to perform pixel defect correction processing based on the difference. _{First, (B n -G n) (} n = 1,2, ····, 8) to calculate the closest to the value of in the _{(B 0 -G 0) (B} m -G m) And the photoelectric conversion part R _m of the photoelectric conversion layer 62R corresponding to the position m is determined as R ₀ . However, it may be obtained by R ₀ =R _m ×(B ₀ −G ₀ )/(B _m −G _m ). By performing such a difference process, the calculation time and the power consumption can be reduced.

As described above, according to the present embodiment, the defect photoelectric conversion units among the plurality of photoelectric conversion units arranged two-dimensionally in the photoelectric conversion layers 62B, 62G, and 62R corresponding to B, G, and R are centered on the defective photoelectric conversion units. Pixel defect correction processing is performed using a pixel signal output from a photoelectric conversion unit having no defect in another photoelectric conversion layer in the pixel block. Since the correction processing is not performed on the other photoelectric conversion units included in the same pixel, it is possible to suppress the image quality deterioration due to the pixel defect correction. In addition, since the correction is performed based on the pixel signal output from the photoelectric conversion unit having a high correlation in the vicinity, the pixel value can be properly corrected.

Next, a second embodiment will be described with reference to FIG. In the second embodiment, when the two photoelectric conversion layers have defective photoelectric conversion units, pixel defect correction is performed based on the pixel signals output from the photoelectric conversion units of the remaining photoelectric conversion layers without defective photoelectric conversion units. ..

FIG. 5 is a diagram showing a block when one defective photoelectric conversion unit exists in two photoelectric conversion layers. FIG. 5 shows a case where the photoelectric conversion section R _{0 of the} photoelectric conversion layer 62R and the photoelectric conversion section B ₀ of the photoelectric conversion layer 62B have defects.

First, in G _n (n=1, 2,..., 8), the photoelectric conversion unit G _m closest to the value of G ₀ is obtained, and the photoelectric conversion units R _m and B _{m at the} position m are obtained. Are defined as R ₀ and B ₀ , respectively. However, it may be obtained by the formulas R ₀ =R _m ×G ₀ /G _m and B ₀ =B _m ×G ₀ /B _m . As described above, by substituting the value of the photoelectric conversion part having a strong correlation with the defective photoelectric conversion part, even if there are two defective photoelectric conversion parts in the photoelectric conversion layer, deterioration of image quality can be reduced. it can.

FIG. 6 is a diagram showing a block in which each of the central photoelectric conversion units B ₀ , G ₀ , and R ₀ of the blocks BB, BG, and BR of the three photoelectric conversion layers has a defect. When all of the central photoelectric conversion units B ₀ , G ₀ , and R ₀ are defective, the correction process may be performed according to the conventional correction process along the strong correlation. For example, when the correlation is strong in the vertical direction, it may be corrected by R ₂ , R ₆ , G ₂ , G ₆ , B ₂ , and B ₆ .

Next, a third embodiment will be described with reference to FIG. In the third embodiment, when a defective photoelectric conversion part is formed around another photoelectric conversion layer that does not have a defective photoelectric conversion part in the center, the pixel defect correction process is performed excluding it. The other configuration is the same as that of the first embodiment.

FIG. 7 is a diagram showing a block in which a peripheral photoelectric conversion part has a defect in one photoelectric conversion layer. In FIG. 7, defects occur in the photoelectric conversion unit R _{0 at} the center of the photoelectric conversion layer 62R and the peripheral photoelectric conversion unit B _{3 of the} photoelectric conversion unit B ₀ which corresponds to the same pixel as the photoelectric conversion unit R ₀ in the photoelectric conversion layer 62B. There is. For example, in the case of correcting R ₀ , the correction process excluding the defective photoelectric conversion unit B ₃ and R ₃ and G ₃ according to the position (n=1, 2,..., 8 (n≠3)) Apply. The other calculation method is the same as that of the first embodiment. As described above, even if the peripheral photoelectric conversion unit has a defective photoelectric conversion unit, it can be appropriately corrected.

FIG. 8 is a diagram showing a block in which a peripheral photoelectric conversion section has a defect in two photoelectric conversion layers. In FIG. 8, the photoelectric conversion part R _{0 at} the center of the photoelectric conversion layer 62R, the peripheral photoelectric conversion part G _{5 of the} photoelectric conversion part G ₀ which corresponds to the same pixel as the photoelectric conversion part R ₀ in the photoelectric conversion layer 62G, and the photoelectric conversion layer 62B. In, there is a defect in the photoelectric conversion unit R ₀ and the peripheral photoelectric conversion unit B _{3 of the} photoelectric conversion unit B ₀ which corresponds to the same pixel. In this case, the same correction processing as in the first embodiment is performed except for B ₅ , G ₅ , R ₅ and B ₃ , G ₃ , R ₃ .

It is also possible to form a three-layered photodiode structure in the silicon substrate to form the R, G, B photoelectric conversion films. FIG. 9 is a diagram showing an example of an image pickup device in which photodiodes are stacked in a silicon substrate. In the image sensor 32', an n-type layer, a p-type layer, and an n-type layer are sequentially formed along the depth direction. The R, G, and B pixel signals are output by utilizing the property that the light incident from the substrate surface side penetrates deeper as the wavelength increases. In this case, the depth of the pn junction is designed so that the n-type layer, the p-type layer, and the n-type layer selectively absorb light in the B, G, and R wavelength ranges from above.

In the first to third embodiments, based on the defective photoelectric conversion unit data existing in the photoelectric conversion layer 62R, a block including a peripheral photoelectric conversion unit around the photoelectric conversion unit of the pixel is defined for each photoelectric conversion layer, Pixel defect correction processing is executed based on the pixel signals output from the defect-free photoelectric conversion unit in the block, but correction processing is executed based on the defect photoelectric conversion unit of the other (predetermined) photoelectric conversion layer. You may.

Further, the detection unit of the defective photoelectric conversion unit is not limited to the configuration in which the pixel defect information is stored in advance as data, and considering that a pixel that becomes unsuitable as the pixel data due to the usage condition of the camera during manufacturing is also corrected. The determination unit may be provided inside the camera.

It is also possible to form a photoelectric conversion layer other than R, G, and B, and a plurality of photoelectric conversion layers that selectively absorb light having different wavelength regions and perform photoelectric conversion may be formed.

The pixel defect correction processing may be performed by a method other than the above-described calculation method. In this case, when a certain pixel area is targeted, the correction process may be performed by using at least the photoelectric conversion unit of the photoelectric conversion layer having no defective photoelectric conversion unit.

10 digital camera 32 image sensor 62R photoelectric conversion layer 62G photoelectric conversion layer 62B photoelectric conversion layer R _n photoelectric conversion unit G _n photoelectric conversion unit B _n photoelectric conversion unit

Claims (8)

An image pickup device in which a plurality of photoelectric conversion layers in which a plurality of photoelectric conversion units are two-dimensionally arranged are stacked,
A pixel signal output from a defective photoelectric conversion unit in the plurality of photoelectric conversion layers is a pixel signal output from a defect-free photoelectric conversion unit of another photoelectric conversion layer included in the same pixel, and a photoelectric signal around the pixel signal. A correction unit that corrects based on the correlation with the pixel signal output from the conversion unit,
The correction unit performs correction based on a pixel signal output from the photoelectric conversion unit in the position having the strongest correlation with the defect-free photoelectric conversion unit among the peripheral photoelectric conversion units. Image pickup device. The correction unit, for each of the plurality of photoelectric conversion layers, based on a pixel signal output from the defect-free photoelectric conversion unit and a pixel signal output from the peripheral photoelectric conversion unit, the defect-free photoelectric conversion unit. The image pickup apparatus according to claim 1, wherein the photoelectric conversion unit located at a position having the strongest correlation with the conversion unit is detected . The correction unit detects a photoelectric conversion unit in a position having the strongest correlation, based on a ratio of pixel values of corresponding photoelectric conversion units among the plurality of photoelectric conversion layers. The imaging device according to. The correction unit detects a photoelectric conversion unit in a position having the strongest correlation, based on a difference between pixel values of corresponding photoelectric conversion units among the plurality of photoelectric conversion layers. The imaging device according to. The correction unit may perform correction based on a pixel signal output from a photoelectric conversion unit having a pixel value closest to a pixel value of a photoelectric conversion unit having no defect in another photoelectric conversion layer included in the same pixel. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is provided. When the correction unit has a peripheral photoelectric conversion unit having a defect in a positional relationship adjacent to the defective photoelectric conversion unit in another photoelectric conversion layer, the correction is performed except for the pixel signal output from the peripheral photoelectric conversion unit. The image pickup apparatus according to claim 1, wherein Pixel signals output from defective photoelectric conversion units in the plurality of photoelectric conversion layers of the image pickup device in which the plurality of photoelectric conversion layers in which the plurality of photoelectric conversion units are two-dimensionally arranged are stacked are included in the same pixel. A pixel defect correction device for correcting , based on a correlation between a pixel signal output from a photoelectric conversion unit having no defect in the photoelectric conversion layer and a pixel signal output from a photoelectric conversion unit around the photoelectric conversion unit ,
Pixel defect correction characterized by performing a correction based on a pixel signal output from the photoelectric conversion unit in the position having the strongest correlation with the defect-free photoelectric conversion unit in the surrounding photoelectric conversion units. apparatus. Pixel signals output from defective photoelectric conversion units in the plurality of photoelectric conversion layers of the image pickup device in which the plurality of photoelectric conversion layers in which the plurality of photoelectric conversion units are two-dimensionally arranged are stacked are included in the same pixel. A pixel defect correction method for correcting , based on a correlation between a pixel signal output from a photoelectric conversion unit having no defect of the photoelectric conversion layer and a pixel signal output from a photoelectric conversion unit around the photoelectric conversion unit ,
Pixel defect correction characterized by performing a correction based on a pixel signal output from the photoelectric conversion unit in the position having the strongest correlation with the defect-free photoelectric conversion unit in the surrounding photoelectric conversion units. Method.

Images