JP6789722B2 - Imaging device and its driving method - Google Patents

Description

The present invention relates to an image pickup apparatus in which a plurality of photoelectric conversion units are provided in one pixel and a driving method thereof.

In recent digital cameras, an image sensor having various functions is mounted by providing a plurality of photoelectric conversion units (photoelectric conversion elements) in one pixel. For example, by having a structure in which a plurality of photoelectric conversion units share one microlens and divide the pupil of the photographing lens to receive light, not only the imaging signal is obtained as in the conventional case, but also the signal required for focus detection. Image sensors that can also be obtained have appeared.

In the image sensor in which a plurality of photoelectric conversion units are provided in one pixel in this way, there is a problem that the readout time increases as the photoelectric conversion units are doubled. In order to deal with this problem, Patent Document 1 proposes a method of suppressing an increase in the readout time in an image pickup apparatus in which two photoelectric conversion units are provided for one pixel.

Specifically, in Patent Document 1, after each of the reset signal and the signal of the first photoelectric conversion unit is read, the signal of the second photoelectric conversion unit is added and read without resetting, and the second is read. A technique for obtaining the signal of the photoelectric conversion unit by subtracting the signal of the first photoelectric conversion unit from the addition signal is described. Then, according to such a technique, since the reset signal can be commonly used in the signals of the first and second photoelectric conversion units, the reading of the reset signal can be shortened by one time.

Japanese Unexamined Patent Publication No. 2013-106194

However, in the technique described in Patent Document 1, when the signals of a plurality of photoelectric conversion units are added and read, the dynamic range of the processing circuit after the addition is exceeded to cause circuit saturation, and proper signal processing can be performed. There was a problem that there was no risk.

The present invention has been made in view of such a problem, and provides a mechanism capable of performing appropriate signal processing in the processing circuit after the addition when the signals of a plurality of photoelectric conversion units are added and read. The purpose is to do.

The image pickup apparatus of the present invention includes a plurality of photoelectric conversion units, and charges generated by a first photoelectric conversion unit, which is one of the plurality of photoelectric conversion units, are transferred to the plurality of photoelectric conversion units. A pixel array in which a plurality of pixels configured to be able to be stored in a second photoelectric conversion unit other than the first photoelectric conversion unit are provided in an array, and a plurality of pixels provided in the pixel array. For each pixel in the pixel, a first signal based on the charge accumulated in the first photoelectric conversion unit and a second signal based on the charge accumulated in the plurality of photoelectric conversion units including the first photoelectric conversion unit. When the value of the second signal output from the reading means for each pixel reaches a predetermined threshold value, the reading means for reading and outputting the signal of is a pixel related to the second signal. Correction to correct the value of the second signal in the pixel by using the value of the first signal output from the reading means and the saturation characteristic value based on the saturation characteristic of the second photoelectric conversion unit. have a means, the correction means adds the said saturation characteristic value and the value of the first signal to calculate a sum value is corrected to replace the value of said second signal to said additional value ..
The present invention also includes the above-described method for driving the imaging device.

According to the present invention, when the signals of a plurality of photoelectric conversion units are added and read, it is possible to perform appropriate signal processing in the processing circuit after the addition.

It is a figure which shows an example of the schematic structure of the image pickup apparatus which concerns on embodiment of this invention. It is a figure which shows the embodiment of this invention and shows an example of the internal structure of one pixel and one column signal processing circuit shown in FIG. It is a figure which shows the embodiment of this invention and shows an example of the timing chart of various signals supplied to the pixel of the nth row from the pixel control circuit shown in FIG. It is a figure which shows the embodiment of this invention, and schematically shows an example of each potential barrier of PD, transfer switch, and FD shown in FIG. FIG. 5 is a characteristic diagram showing an embodiment of the present invention and showing an example of the relationship between the amount of incident light incident on the pixels shown in FIG. 2 and the output value of the corresponding column signal processing circuit (ADC circuit). It is a figure which shows the embodiment of this invention and shows an example of the functional structure of the correction part shown in FIG. 1 and FIG. It is a flowchart which shows the driving method of the image pickup apparatus which concerns on embodiment of this invention, and shows an example of the processing procedure performed in the correction determination processing part shown in FIG.

Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of a schematic configuration of an image pickup apparatus 100 according to an embodiment of the present invention.
As shown in FIG. 1, the image pickup apparatus 100 includes a pixel array 101, a column signal processing circuit 20, a pixel control circuit 30, and a correction unit 40.

The pixel array 101 is formed by providing a plurality of pixels 10 including photodiodes (hereinafter, referred to as “PD”) 12a and 12b, which are a plurality of photoelectric conversion units (photoelectric conversion elements), in an array. Here, in the present embodiment, the pixel 10 transfers the electric charge generated by one PD (first photoelectric conversion unit: for example, PD12a) of the plurality of photoelectric conversion units PD12a and 12b to a plurality of photoelectrics. It is configured so that it can be stored in a PD (second photoelectric conversion unit: for example, PD12b) other than the PD corresponding to the first photoelectric conversion unit among the conversion units PD12a and 12b. Further, each pixel 10 shown in FIG. 1 is configured with one microlens 11 arranged so as to cover PD12a and 12b. In the pixel array 101 shown in FIG. 1, an example in which pixels 10 having 6 rows × 4 columns are provided is shown. Further, in the pixel array 101 shown in FIG. 1, the uppermost row shown in FIG. 1 is the nth row as a notation of the number of rows of the pixel 10.

A plurality of column signal processing circuits 20 are provided corresponding to a plurality of vertical output lines 21 that connect the pixels 10 in the pixel array 101 in column units, and process signals output from the pixels 10 in the corresponding columns. .. The output of the signal from the pixel 10 to the column signal processing circuit 20 is controlled by the selection pulse signal φSEL for each pixel 10 by the pixel control circuit 30.

The pixel control circuit 30 controls the pixel 10 in the pixel array 101 on a line-by-line basis. Specifically, the pixel control circuit 30 controls the signal output of the pixel 10 by supplying the signals φRES, φTXa, φTXb, and φSEL. In FIG. 1, "signal φRES on the nth line" is described as "signal φRES (n)" with a subscript n. The same applies to the signals φTXa, φTXb, and φSEL.

The correction unit 40 corrects the signals output from the plurality of column signal processing circuits 20 as necessary, and outputs the signals to the outside of the image pickup apparatus 100.

FIG. 2 is a diagram showing an embodiment of the present invention and showing an example of an internal configuration of one pixel 10 and one column signal processing circuit 20 shown in FIG. In FIG. 2, the same reference numerals are given to configurations similar to those shown in FIG.

The pixel 10 has a microlens 11, PD12a and 12b, transfer switches 13a and 13b, a floating diffusion (hereinafter referred to as “FD”) 14, an amplification MOS amplifier 15, a selection switch 16, and a reset switch 17. It is configured. Further, the signals φTXa, φTXb, φSEL and φRES input to the pixel 10 are supplied from the pixel control circuit 30 shown in FIG.

PD12a and 12b are photoelectric conversion units that generate and store electric charges according to the incident light that sequentially passes through the photographing lens (not shown) of the imaging device 100 and the microlens 11 from the outside of the imaging device 100 or the like. is there. The PDs 12a and 12b are provided via the microlens 11 corresponding to different pupil regions of the above-mentioned photographing lens (not shown), respectively. Further, in PD12a and 12b, the potential barrier related to the boundary is set lower than that of the other boundary so that when one PD is charged more than the saturation amount, the potential barrier related to the boundary is preferentially leaked to the other PD12. Has been done. Thereby, as described above, the pixel 10 in the present embodiment is configured to be able to accumulate the electric charge generated by one PD in the other PD.

The transfer switches 13a and 13b are driven by the transfer pulse signals φTXa and φTXb input to their gate terminals, respectively, and transfer the charges stored in the PD12a and 12b to the common FD14, respectively.

The FD 14 functions as a charge-voltage conversion unit that temporarily stores charges and converts the accumulated charges into voltage signals.

The amplification MOS amplifier 15 functions as a source follower, and a signal whose charge and voltage are converted by the FD 14 is input to the gate. Further, the drain of the amplification MOS amplifier 15 is connected to the power supply line VDD, and the source of the amplification MOS amplifier 15 is connected to the selection switch 16.

The selection switch 16 is driven by the selection pulse signal φSEL input to its gate, its drain is connected to the amplification MOS amplifier 15, and its source is connected to the vertical output line 21. Specifically, the selection switch 16 becomes conductive when the selection pulse signal φSEL reaches the active level (high level), and the source of the corresponding amplification MOS amplifier 15 is electrically connected to the vertical output line 21. ..

The drain of the reset switch 17 is connected to the power line VDD, and the source of the reset switch 17 is connected to the FD 14. The reset switch 17 is driven by the reset pulse signal φRES input to the gate and removes the electric charge accumulated in the FD 14.

Further, as shown in FIG. 2, the column signal processing circuit 20 includes a vertical output line 21, a constant current source 22, a CDS circuit 23, and an ADC circuit (AD conversion circuit) 24.

The vertical output line 21 is a wiring for connecting the pixels 10 in the pixel array 101 in column units.

The constant current source 22 is connected to the vertical output line 21 and supplies a constant current to the amplification MOS amplifier 15 selected by the selection switch 16 through the vertical output line 21 to cause the amplification MOS amplifier to function as a source follower. At this time, the amplification MOS amplifier 15 outputs the voltage signal of the FD 14 to the column signal processing circuit 20 through the vertical output line 21. Here, the amplification MOS amplifier 15 outputs a reset signal when the FD 14 is reset by the reset pulse signal φRES. Further, the amplification MOS amplifier 15 outputs a transfer signal A including a photoelectric conversion signal of the PD 12a when the charge of the PD 12a is input to the FD 14 by the transfer pulse signal φTXa. After that, when the charge of PD12b is superimposed on the FD14 by the transfer pulse signal φTXb, the amplification MOS amplifier 15 includes a transfer signal including a signal to which the photoelectric conversion signal of PD12a and the photoelectric conversion signal of PD12b are added. Output A + B.

The CDS circuit 23 clamps the reset signal output from the pixel 10 to the reference voltage Vclmp, and the image pickup signal A which is the difference between the transfer signal A and the reset signal and the image pickup which is the difference between the transfer signal A + B and the reset signal. The signal A + B is output to the ADC circuit 24.

The ADC circuit 24 converts various analog signals input from the CDS circuit 23 into various digital signals, and outputs these to the correction unit 40. Here, the ADC circuit 24 defines a predetermined output maximum value Dmax, and when there is an input exceeding the input value Amax corresponding to the maximum value, the output is clipped to the output maximum value Dmax. Isuzu.

If necessary, an analog amplifier may be provided in the CDS circuit 23 to amplify the input signal and output it to the ADC circuit 24. Further, in the present embodiment, as shown in FIGS. 1 and 2, the ADC circuit 24 is provided for each row of pixels 10 in the pixel array 101, but in reality, the signals input from each row are sequentially AD-converted. Only one or more possible circuits are required.

Next, the operation of the pixel control circuit 30 when the reset signal, the transfer signal A, and the transfer signal A + B are output from the pixel 10 will be described with reference to FIG.
FIG. 3 is a diagram showing an embodiment of the present invention and showing an example of a timing chart of various signals supplied from the pixel control circuit 30 shown in FIG. 1 to the pixel 10 on the nth row.

At time t1 in FIG. 3, the selection pulse signal φSEL (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “H”, and the pixel 10 on the nth row has a corresponding vertical output line. A signal is output to 21. At the same time, at time t1, the reset pulse signal φRES (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “H”, and the pixel 10 on the nth row discharges unnecessary charges of the FD14. Reset the potential of FD14.

At time t2, the reset pulse signal φRES (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “L”, and the pixel 10 on the nth row receives the reset signal of the FD 14 by the amplification MOS amplifier 15. Output to the vertical output line 21. Then, the reset signal output to the vertical output line 21 is input to the column signal processing circuit 20, and is clamped to the reference voltage Vclmp by the CDS circuit 23.

At time t3, the transfer pulse signal φTXa (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “H”. In this case, the pixel 10 on the nth row transfers the signal charge accumulated in the PD 12a to the FD 14, and the FD 14 inputs the signal potential corresponding to the signal charge of the PD 12a to the gate of the amplification MOS amplifier 15.

At time t4, the transfer pulse signal φTXa (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “L”. In this case, the amplification MOS amplifier 15 of the pixel 10 in the nth row outputs the signal potential of the FD 14 after transferring the charge of the PD 12a to the vertical output line 21 as the transfer signal A. Then, the transfer signal A output to the vertical output line 21 is input to the column signal processing circuit 20, passes through a clamping process and an AD conversion process, and is output to the correction unit 40.

At time t5, the transfer pulse signal φTXb (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “H”. In this case, the pixel 10 on the nth row transfers the signal charge accumulated in the PD 12b to the FD 14, and the FD 14 inputs the signal potential to which the signal charge of the PD 12b is added to the gate of the amplification MOS amplifier 15.

At time t6, the transfer pulse signal φTXb (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth row becomes “L”. In this case, the amplification MOS amplifier 15 of the pixel 10 in the nth row outputs the signal potential of the FD 14 after transferring the charge of the PD 12b to the vertical output line 21 as the transfer signal A + B. Then, the transfer signal A + B output to the vertical output line 21 is input to the column signal processing circuit 20, passes through a clamping process and an AD conversion process, and is output to the correction unit 40.

At time t7, the selection pulse signal φSEL (n) supplied from the pixel control circuit 30 to the pixel 10 on the nth line becomes “L”, and the pixel 10 on the nth line is electrically connected to the corresponding vertical output line 21. Is cut off.

With the above control, the pixel control circuit 30 performs a signal output operation for continuously outputting the reset signal, the transfer signal A, and the transfer signal A + B to each pixel 10. In the present embodiment, the operation of supplying the transfer pulse signal φTXa (n) and the transfer pulse signal φTXb (n) at different timings is shown, but the transfer pulse signal φTXa (n) may be set to “H” until time t6. However, it may be set to "H" at the same timing. Here, when "H" is set at the same timing, PD12a and PD12b can be treated as one PD. Further, the reset signal may be read out by setting the reset pulse signal φRES (n) to “H” again between the time t4 and the time t5.

Next, the function of the potential barrier provided at the boundary between PD12a and PD12b will be described with reference to FIG.
FIG. 4 shows an embodiment of the present invention, and is a diagram schematically showing an example of each potential barrier of PD12a and 12b, transfer switches 13a and 13b, and FD14 shown in FIG.

In FIG. 4, the potential barrier Vab is provided between PD12a and PD12b, and is lower than the potential barrier Vtx of the transfer switches 13a and 13b when the transfer pulse signals φTXa and φTXb are “L”. Further, although not shown in FIG. 4, the potential barrier Vab is also provided so as to be lower than the outer potential barrier surrounding the PD 12a and 12b, and is generally provided so that the potential barrier Vab is the lowest. It is assumed that it is. Therefore, in the following description, the outer potential barrier surrounding PD12a and 12b will be treated as equivalent to the potential barrier Vtx. Further, in the FD 14 shown in FIG. 4, for the sake of explanation, a set of PD12a and 12b and one FD14 corresponding to the transfer switches 13a and 13b are described twice on the left and right.

As shown in FIG. 4A, for example, when the amount of light incident on the PD12a is larger than that of the PD12b and a charge exceeding the potential barrier Vab is generated in the PD12a, the electric charge of the PD12a exceeding the potential barrier Vab flows into the PD12b. In the following description, a state in which a charge exceeding the potential barrier Fab is generated in the PD will be referred to as "subpixel saturation".

When more light is incident than in the case shown in FIG. 4 (a1), as shown in FIG. 4 (b1), when PD12b also reaches subpixel saturation, the inflow of charges from PD12a stops.

When more light is incident than in the case shown in FIG. 4 (b1), the generated charges are accumulated in the PDs 12a and 12b, respectively, until the potential barrier Vtx is reached, as shown in FIG. 4 (c1).

The FD14 of FIGS. 4 (a1), 4 (b1), and 4 (c1) is in a state of being reset by the reset switch 17, and the FD 14 at this time is via the amplification MOS amplifier 15 and the selection switch 16. Then, the reset signal is being output.

Next, from the states of FIGS. 4 (a1), 4 (b1), and 4 (c1), when the transfer pulse signal φTXa is turned on, a part of the electric charge is transferred to the FD14, which is shown. These are FIGS. 4 (a2), 4 (b2), and 4 (c2), respectively.

Here, in FIGS. 4 (a2) and 4 (b2), the electric charge accumulated in the PD 12a is transferred to the FD 14. On the other hand, in FIG. 4C2, in addition to the electric charge accumulated in the PD12a, among the electric charges accumulated in the PD12b by being read out earlier, the electric charge exceeding the potential barrier Vab is transferred to the FD14. In other words, all the other charges are transferred to the FD 14, leaving the charge amount for the subpixel saturation of PD12b. In FIGS. 4 (a2), 4 (b2), and 4 (c2), the FD 14 is in a state of outputting the transfer signal A via the amplification MOS amplifier 15 and the selection switch 16.

Then, from the states of FIGS. 4 (a2), 4 (b2), and 4 (c2), the transfer pulse signal φTXa was turned off and the transfer pulse signal φTXb was turned on, so that the remaining charge was transferred to the FD14. The states are FIG. 4 (a3), FIG. 4 (b3), and FIG. 4 (c3), respectively.

Here, FIGS. 4 (a3), 4 (b3), and 4 (c3) are all in a state where all the charges are transferred to the FD 14 and added, and the FD 14 at this time is the amplification MOS amplifier 15. And, the transfer signal A + B is output via the selection switch 16.

FIG. 5 shows an embodiment of the present invention, and shows an example of the relationship between the amount of incident light on the pixel 10 shown in FIG. 2 and the output value of the corresponding column signal processing circuit 20 (ADC circuit 24). It is a figure.

The reset signal output from the pixel 10 is clamped to the reference voltage Vclmp by the CDS circuit 23. After that, the CDS circuit 23 outputs the image pickup signal A, which is the difference between the transfer signal A and the reset signal, and the image pickup signal A + B, which is the difference between the transfer signal A + B and the reset signal, to the ADC circuit 24. In FIG. 5, for the sake of simplicity, the output value of the AD conversion result of the reference voltage Vclmp is treated as 0.

Then, the ADC circuit 24 converts various analog signals input from the CDS circuit 23 into various digital signals. As described above, the ADC circuit 24 is provided with an output maximum value Dmax capable of output, and when there is an input exceeding the incident light amount corresponding to this output maximum value Dmax, the output value is , Clipped to the maximum output value Dmax. Here, the maximum output value Dmax is, for example, 16383 is the maximum output value Dmax in the case of 14-bit digital conversion.

As for the input / output characteristics of the imaging signal A + B, the electric charge proportional to the amount of incident light is transferred to the FD 14, and is input to the ADC circuit 24 via the CDS circuit 23 while maintaining the proportional relationship. Therefore, the maximum output is as shown in FIG. It exhibits linear properties until it reaches the amount of light clipped to the value Dmax.

On the other hand, the input / output characteristics of the imaging signal A are different from those of the imaging signal A + B because the charge transferred to the FD 14 is not proportional to the amount of incident light, as described above with reference to FIG. Here, when the incident light amount shown in FIG. 5 is 0 to L (a), when it is L (a) to L (b), and when it is L (b) to, the incident light amount L (a) is divided. ) And L (b) are defined as follows.

Specifically, as shown in FIG. 4 (a1), the incident light amount L (a) is such that the amount of light incident on the PD12a is larger than that of the PD12b, the PD12a reaches the subpixel saturation first, and the charge exceeds the potential barrier Vab. Is the amount of light that begins to flow into the PD12b. Further, the incident light amount L (b) is defined as the amount of light at which the PD12b subsequently reaches subpixel saturation and the inflow of electric charges from the PD12a is stopped, as shown in FIG. 4 (b1).

Here, when the input light amount is 0 to L (a), as shown in FIG. 4A2, the charge amount corresponding to the light amount incident on the PD12a is transferred to the FD14, so that the output is proportional to the incident light amount of the PD12a. Is obtained.

Further, when the input light amount is L (a) to L (b), as shown in FIG. 4 (b2), the charge amount corresponding to the subpixel saturation of PD12a is transferred to the FD14, so that it is constant with respect to the incident light amount. The output is obtained.

Further, when the input light amount is L (b) to L (b), as shown in FIG. 4 (c2), the total amount of the electric charge exceeding the potential barrier Vab of PD12a and 12b and the electric charge of the subpixel saturation of PD12a is Transferred to FD14. At this time, the amount of charge for the subpixel saturation of PD12a is constant, and the amount of charge for exceeding the potential barrier Vab depends on the amount of light incident on both PD12a and 12b, so that the obtained outputs are PD12a and 12b. Is proportional to the amount of incident light.

Here, assuming that the amount of incident light that the output of the ADC circuit 24 reaches the maximum output value Dmax is M (a) for the image pickup signal A and M (a + b) for the image pickup signal A + B, FIG. 4 (c1) shows. The incident light amount L (c) in the state can be treated as M (a + b) <L (c) <M (a).

Therefore, the imaging signal A outputs a desired value without being clipped to the maximum output value Dmax, but on the other hand, the imaging signal A + B exceeds the dynamic range of the ADC circuit 24, and the output is the maximum output value Dmax. The desired value cannot be obtained because it is clipped to. However, the difference between the imaging signals A and A + B at this time is based on the amount of charge for the subpixel saturation of PD12b. Therefore, in the present embodiment, using this characteristic, an output value obtained by AD-converting only the charge amount for the subpixel saturation of PD12b is stored in advance, and this is stored as the output value of the AD conversion result of the imaging signal A. By adding to, the output value of the AD conversion result of the imaging signal A + B is calculated. Here, the output value obtained by AD-converting only the charge amount for the subpixel saturation of PD12b in advance corresponds to, for example, S (b) in FIG.

Based on the above points, in the present embodiment, processing will be described when the output value of the AD conversion result of the imaging signal A + B reaches the maximum output value Dmax and a desired value cannot be obtained.

FIG. 6 is a diagram showing an embodiment of the present invention and showing an example of the functional configuration of the correction unit 40 shown in FIGS. 1 and 2.

The image pickup signal A + B output from the ADC circuit 24 and the image pickup signal A output from the ADC circuit 24 are input to the correction unit 40 shown in FIG. Here, in the present embodiment, the image pickup signal A output from the ADC circuit 24 corresponds to the "first signal" based on the electric charge accumulated in the PD12a (first photoelectric conversion unit). Further, in the present embodiment, the image pickup signal A + B output from the ADC circuit 24 is a “second signal” based on the charges accumulated in the plurality of photoelectric conversion units PD12a and PD12b formed in one pixel 10. Corresponds to. Further, in the present embodiment, for example, the processing circuit in the subsequent stage after FD14 in the pixel 10 of FIG. 2 and the column signal processing circuit 20 shown in FIG. 2 have the above-mentioned imaging signal A (first) for each pixel 10. Signal) and the above-mentioned imaging signal A + B (second signal) are read and output, which corresponds to a “reading means”.

As shown in FIG. 6, the correction unit 40 includes a signal branching unit 41, a memory 42, an addition unit 43, and a correction determination processing unit 44.

The signal branching unit 41 branches the signal input from the ADC circuit 24 into an imaging signal A (first signal 601) and an imaging signal A + B (second signal 602).

The memory 42 is the subpixel saturation of PD12b (second photoelectric conversion unit) excluding PD12a, which is the first photoelectric conversion unit, among the PD12a and PD12b which are a plurality of photoelectric conversion units formed in one pixel 10. It is a storage means that stores in advance the output value obtained by AD-converting the amount of electric charge of. Here, the output value stored in the memory 42 corresponds to the "saturation characteristic value based on the saturation characteristic of the second photoelectric conversion unit".

The addition unit 43 AD-converts the value of the image pickup signal A (first signal 601) and the charge amount of the subpixel saturation of PD12b stored in the memory 42 (saturation of the second photoelectric conversion unit). The saturation characteristic value based on the characteristic) is added to calculate the added value. In this embodiment, this added value corresponds to the imaging signal A + S (b) shown in FIG.

The correction determination processing unit 44 determines whether or not the value of the image pickup signal A + B (second signal 602) has reached the output maximum value Dmax which is a predetermined threshold value. Then, when the correction determination processing unit 44 determines that the value of the image pickup signal A + B (second signal 602) has reached the maximum output value Dmax, the correction determination processing unit 44 adds the value of the image pickup signal A + B (second signal 602). The signal is output after making corrections to replace the added value obtained as a result calculated in 43. Further, when the correction determination processing unit 44 determines that the value of the imaging signal A + B (second signal 602) has not reached the maximum output value Dmax (that is, the value of the imaging signal A + B is less than the maximum output value Dmax). The image pickup signal A + B is output as it is without correcting the signal value.

By performing the above-mentioned correction by the correction determination processing unit 44, as shown in FIG. 5, the maximum output value of the imaging signal A + B is expanded from the maximum output value Dmax to the maximum output value Dmax + S (b), which corresponds to the maximum output value. The amount of incident light is also improved from M (a + b) to M (a). When the ADC circuit 24 is a 14-bit digital conversion with a maximum output value of Dmax = 16383, the correction unit 40 extends the bit to 15 bits and outputs the output. In this case, the correction unit 40 needs to have a function capable of processing 15 bits of data at least internally.

FIG. 7 is a flowchart showing a driving method of the image pickup apparatus 100 according to the embodiment of the present invention, and showing an example of a processing procedure performed by the correction determination processing unit 44 shown in FIG. When the processing of the flowchart shown in FIG. 7 is started, the imaging signal A (first signal 601) and the imaging signal A + B (second signal) shown in FIG. 6 are used in the above-described reading means including the ADC circuit 24. It is assumed that the read and output steps of 602) have already been performed.

Further, FIG. 7 shows processing for a set of image pickup signals A (first signal 601) and image pickup signals A + B (second signal 602) from one pixel 10, but in the present embodiment, the processing is shown. It shall be performed for the signals of all the pixels 10 formed in the pixel array 101.

First, in step S701, the correction determination processing unit 44 determines whether or not the value of the second signal 602, which is the output value of the AD conversion result of the imaging signal A + B, has reached the output maximum value Dmax, which is a predetermined threshold value. judge.

As a result of the determination in step S701, when the value of the second signal 602, which is the output value of the AD conversion result of the imaging signal A + B, reaches the output maximum value Dmax, which is a predetermined threshold value (S701 / YES), the step. Proceed to S702.
Proceeding to step S702, the correction determination processing unit 44 adds the value of the second signal 602 that has reached the maximum output value Dmax to the first signal 601 that is the output value of the AD conversion result of the image pickup signal A in the addition unit 43. The addition obtained as a result of adding the value of and the output value (saturation characteristic value based on the saturation characteristic of the second photoelectric conversion unit) obtained by AD-converting the charge amount of the subpixel saturation of PD12b stored in the memory 42. The value is corrected to replace it, and the signal is output. When this step S702 is completed, the processing for the signal of the one pixel 10 is completed, and the processing of the flowchart of FIG. 7 is completed.

On the other hand, as a result of the determination in step S701, when the value of the second signal 602, which is the output value of the AD conversion result of the imaging signal A + B, does not reach the output maximum value Dmax, which is a predetermined threshold value (S701 / NO). ), Proceed to step S703.
Proceeding to step S703, the correction determination processing unit 44 outputs the value of the second signal 602, which is the output value of the AD conversion result of the imaging signal A + B, as it is, without correcting the signal value. When this step S703 is completed, the processing for the signal of the one pixel 10 is completed, and the processing of the flowchart of FIG. 7 is completed.

Then, in the present embodiment, the flow chart shown in FIG. 7 is processed for the signals of all the pixels 10 formed in the pixel array 101.

The output value (saturation characteristic value based on the saturation characteristic of the second photoelectric conversion unit) S (b) obtained by AD-converting the charge amount of the subpixel saturation of PD12b stored in the memory 42 of FIG. 6 in advance is It can be a value measured by another means. Here, as the means for measuring, for example, a subject having a uniform luminance surface such that the amount of incident light shown in FIG. 5 is between L (b) and M (a + b) is photographed, and the imaging signal A + B and the imaging signal A are used. The above-mentioned S (b) can be obtained by calculating the difference between.

Further, for example, in the timing chart shown in FIG. 3, the transfer pulse signal φTXa (n) is also set to “H” even during the period from time t1 to time t2, and a subject in which the entire surface of the pixel 10 is sufficiently saturated is photographed. The above-mentioned S (b) can also be obtained. At this time, even in the period of time t1 to time t2, by setting the transfer pulse signal φTXa (n) to "H", the electric charge exceeding the potential barrier Vab accumulated in the PD12a and 12b and the electric charge accumulated in the PD12a Is discharged through the transfer switch 13a and the reset switch 17 between the time t1 and the time t2. In addition, although the period of time t1 to time t2 shown in FIG. 3 is used here, PD12a is also set with the reset pulse signal φRES (n) and the transfer pulse signal φTXa (n) as “H” during the storage period. And the electric charge exceeding the potential barrier Vab accumulated in 12b and the electric charge accumulated in PD12a may be constantly discharged.

By these operations, the charge amount of only the subpixel saturation of PD12b can be AD-converted and the above-mentioned S (b) can be obtained in the subsequent read operation. Then, by storing the obtained S (b) described above in the frame memory, it is possible to use this as the memory 42.

Here, the output value obtained by AD-converting the charge amount of the sub-pixel saturation of PD12b for each pixel in the plurality of pixels 10 provided in the pixel array 101 in the memory 42 (in the saturation characteristic of the second photoelectric conversion unit). Consider the case where the saturation characteristic value) S (b) is stored. In this case, the correction unit 40 outputs the above-mentioned S (b) of the corresponding pixel that outputs the image pickup signal A + B (second signal 602) that was positively determined (S701 / YES) in step S701 of FIG. 7 from the memory 42. The mode of acquiring the image and performing the correction in step S702 of FIG. 7 is adopted.

In this embodiment, the present embodiment is not limited to this embodiment, and the above-mentioned S (b) stored in the memory 42 is stored in the memory 42 in the following modification 1 and modification from the viewpoint of reducing the memory capacity of the memory 42. It may be 2.

Modification 1 of S (b) stored in the memory 42 is based on the assumption that the value of S (b) is caused by, for example, a pattern such as the layout of pixels 10. In this case, the memory 42 stores one S (b) representing the predetermined pixel group for each predetermined pixel group corresponding to the pattern. At this time, as one S (b) representing the predetermined pixel group, for example, the average value of the S (b) of the pixels 10 constituting the predetermined pixel group can be used. In this case, the correction unit 40 describes the predetermined pixel group to which the corresponding pixel that outputs the image pickup signal A + B (second signal 602) that was positively determined (S701 / YES) in step S701 of FIG. 7 from the memory 42 belongs. S (b) is acquired and the correction in step S702 of FIG. 7 is performed.

In the second modification of S (b) stored in the memory 42, there is little variation in the output value S (b) obtained by AD-converting the charge amount of the subpixel saturation of PD12b in all the pixels 10 provided in the pixel array 101. This is effective when it can be ignored as compared with the optical shot noise generated in the high-luminance portion to be corrected. In this case, the memory 42 stores one S (b) representing all the pixels 10 provided in the pixel array 101. At this time, as one S (b) representing all the pixels 10, for example, the average value of the S (b) of all the pixels 10 can be used. In this case, the correction unit 40 takes the mode of acquiring the above-mentioned S (b) from the memory 42 and performing the correction in step S702 of FIG.

In the above-described example, an example in which the maximum output value Dmax is applied as a predetermined threshold value used in the correction determination processing by the correction determination processing unit 44 has been described, but the present embodiment is not limited to this. Absent. Here, in the present embodiment, if both PD12a and 12b have reached subpixel saturation, the imaging signal A + B can be obtained by correction. Therefore, in the present embodiment, the predetermined threshold value is not limited to the maximum output value Dmax, and can be arbitrarily set as long as the AD conversion output value shown in FIG. 5 is between S (a + b) and Dmax.

Further, in the above-described example, an example assuming a case where the ADC circuit 24 becomes circuit saturated when the signals of a plurality of photoelectric conversion units PD12a and 12b are continuously transferred and added to the FD 14 has been described. The present embodiment is not limited to this. For example, the same effect can be expected when the CDS circuit 23 becomes circuit saturated, and in this case as well, it can be applied to the present embodiment. Also. The dynamic range of the CDS circuit 23 and the ADC circuit 24 has a margin, but it is conceivable that the FD 14 will be saturated instead. Here, the saturation of FD14 refers to a situation in which complete charge transfer becomes impossible when the signals of PD12a and PD12b, which are photoelectric conversion units, are transferred. In particular, the transfer signals A + B are added and read by FD14. Complete charge transfer is not possible, increasing the possibility that it cannot be read correctly. At this time, if the voltage of the FD 14 is Vfd and the AD conversion value of the voltage Vfd is Sfd when complete charge transfer is impossible, the predetermined threshold value that is the correction determination criterion of the correction unit 40 is the AD conversion value Sfd. .. As a result, the correction unit 40 can obtain the image pickup signal A + B from the image pickup signal A in the same manner as in the above-described example when the image pickup signal A + B exceeds the AD conversion value Sfd. This example is effective for an imaging device 100 designed to have a small capacity of the FD 14 for improving SN during high-sensitivity shooting, and by performing the correction processing of the present embodiment during low-sensitivity shooting, high-sensitivity shooting is performed. It can contribute to both improving the SN of the time and securing the dynamic range during low-sensitivity shooting.

In the present embodiment, when a charge equal to or greater than the saturation amount is generated in any of the PDs 12a and 12b provided in one pixel 10, the PDs are preferentially leaked to the other PD12s. The potential barrier is set lower than other potential barriers. With such a structure, when the PD12 signal in all of the pixels 10 is saturated, the signal of any one PD12 (first photoelectric conversion unit) is read out and the first photoelectric conversion is performed. By storing in advance the saturation characteristic values based on the saturation characteristics of other PDs (second photoelectric conversion units) other than the PD corresponding to the unit, it is possible to calculate the total value of the PD12 signals. it can.

Therefore, in the above-mentioned example, an example in which two photoelectric conversion units PD12a and 12b are provided in one pixel 10 has been described, but in the present embodiment, three or more photoelectric conversion units are provided in one pixel 10. It is also applicable to the image pickup apparatus 100 provided with the above. Here, an example in which three photoelectric conversion units PD12a, PD12b and PD12c are provided in one pixel 10 will be described. In this case, the correction unit 40 inputs the image pickup signal A as the first signal described above, and inputs the image pickup signal A + B + C as the second signal described above, similarly to the above example. Then, in this case, the memory 42 has PD12b and PD12c (second) among the three photoelectric conversion units PD12a, PD12b and PD12c formed in one pixel 10, excluding PD12a which is the first photoelectric conversion unit. The output value obtained by AD-converting the charge amount of the subpixel saturation of the photoelectric conversion unit) is stored in advance. Then, the correction determination processing unit 44 determines whether or not the value of the imaging signal A + B + C (second signal) has reached a predetermined threshold value. Then, when the correction determination processing unit 44 determines that the value of the image pickup signal A + B + C (second signal) has reached a predetermined threshold value, the addition unit 43 calculates the value of the image pickup signal A + B + C (second signal). A form is adopted in which a signal is output by making a correction to replace the added value obtained as a result of the calculation. At this time, the addition unit 43 AD-converts the value of the first signal 601 which is the output value of the AD conversion result of the image pickup signal A and the charge amount of the subpixel saturation of PD12b and PD12c stored in the memory 42. The output value (saturation characteristic value based on the saturation characteristic of the second photoelectric conversion unit) is added.
Even when four or more photoelectric conversion units are provided on one pixel 10, the signal value of the second signal is corrected by the same method as when three photoelectric conversion units are provided on one pixel 10 described above. be able to.

In the image pickup apparatus 100 according to the present embodiment, when the value of the image pickup signal A + B (second signal) reaches a predetermined threshold value, the correction unit 40 arranges a line in the corresponding pixel which is a pixel related to the second signal. Using the value of the image pickup signal A (first signal 601) output from the signal processing circuit 20 and the saturation characteristic value based on the saturation characteristic of PD12b (second photoelectric conversion unit) stored in the memory 42 ( (Adding these values), the value of the imaging signal A + B (second signal) is corrected.
According to such a configuration, when the signals of PD12a and PD12b, which are a plurality of photoelectric conversion units, are added and read, it is possible to perform appropriate signal processing in the processing circuit after the addition.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment (for example, the functional configuration of the correction unit 40 shown in FIG. 6) to a system or an apparatus via a network or a storage medium, and supplies the system or the device. It can also be realized by the process of reading and executing a program by one or more processors in the computer of the device. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
This program and a computer-readable storage medium that stores the program are included in the present invention.

It should be noted that the above-described embodiments of the present invention merely show examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. Is. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

10: Pixel, 11: Microlens, 12a, 12b: Photodiode, 20: Column signal processing circuit, 21: Vertical output line, 30: Pixel control circuit, 40: Correction unit, 100: Imaging device 100, 101: Pixel array

Claims (7)

The first photoelectric conversion unit among the plurality of photoelectric conversion units is provided with a plurality of photoelectric conversion units, and the electric charge generated by the first photoelectric conversion unit, which is one of the plurality of photoelectric conversion units. Pixel arrays in which a plurality of pixels configured to be able to be stored in a second photoelectric conversion unit other than the conversion unit are provided in an array, and
For each pixel in the plurality of pixels provided in the pixel array, the first signal based on the electric charge accumulated in the first photoelectric conversion unit and the plurality of photoelectric conversion units including the first photoelectric conversion unit. A reading means for reading and outputting a second signal based on the electric charge accumulated in the
When the value of the second signal output from the reading means for each pixel reaches a predetermined threshold value, the second signal output from the reading means in the corresponding pixel which is the pixel related to the second signal. using the value of the first signal, and a saturation characteristic value based on the saturation characteristics of the second photoelectric conversion unit, it has a correction means for correcting the value of the second signal,
The correction means is characterized in that the value of the first signal and the saturation characteristic value are added to calculate an added value, and the value of the second signal is replaced with the added value. apparatus. The correction means is configured to be able to acquire the saturation characteristic value for each pixel in a plurality of pixels provided in the pixel array, and obtains the saturation characteristic value in the corresponding pixel to perform the correction. The imaging apparatus according to claim 1 , wherein the imaging apparatus is performed. The correction means is configured to be able to acquire one saturation characteristic value representing the predetermined pixel group for each predetermined pixel group in a plurality of pixels provided in the pixel array. The imaging apparatus according to claim 1 , wherein the saturation characteristic value in the predetermined pixel group to which a pixel belongs is acquired and the correction is performed. The correction means is configured to be capable of acquiring one saturation characteristic value representing all pixels in a plurality of pixels provided in the pixel array, and acquires the saturation characteristic value to perform the correction. The imaging apparatus according to claim 1 , wherein the imaging apparatus is performed. Further having a storage means for storing the saturation characteristic value,
The imaging device according to any one of claims 1 to 4 , wherein the correction means acquires the saturation characteristic value from the storage means and performs the correction. The pixel further comprises one microlens.
The imaging device according to any one of claims 1 to 5 , wherein each of the plurality of photoelectric conversion units is provided so as to correspond to a different pupil region of a photographing lens via the microlens. .. The first photoelectric conversion unit among the plurality of photoelectric conversion units is provided with a plurality of photoelectric conversion units, and the electric charge generated by the first photoelectric conversion unit, which is one of the plurality of photoelectric conversion units. Pixels configured to be able to be stored in a second photoelectric conversion unit other than the conversion unit are a driving method of an imaging device including a plurality of pixel arrays provided in an array.
For each pixel in the plurality of pixels provided in the pixel array, the first signal based on the electric charge accumulated in the first photoelectric conversion unit and the plurality of photoelectric conversion units including the first photoelectric conversion unit. A reading process of reading and outputting a second signal based on the electric charge accumulated in the
When the value of the second signal output from the reading means for each pixel reaches a predetermined threshold value, the second signal output from the reading means in the corresponding pixel which is the pixel related to the second signal. using the value of the first signal, and a saturation characteristic value based on the saturation characteristics of the second photoelectric conversion unit, it has a correction step of correcting the value of the second signal,
The correction step is characterized in that the value of the first signal and the saturation characteristic value are added to calculate an added value, and the value of the second signal is replaced with the added value. How to drive the device.