JP7084801B2 - Image sensor and image sensor - Google Patents

Description

The present invention relates to an image pickup device and an image pickup apparatus.

For example, a photon counting type image pickup device that can acquire an incident light amount as a digital value by using an avalanche photodiode (APD) as a light receiving element and counting the number of voltage pulses generated by the incident of photons is known (patented). Document 1).

The photon counting type image sensor directly acquires a digital value corresponding to the amount of light incident on the light receiving element, and does not require transmission or amplification of an analog signal. Therefore, random noise and fixed noise associated with analog signal transmission and amplification do not occur. Since the noise components included in the digital value are only the optical shot noise generated in the pixel and the dark current, it is possible to obtain an image having a high SN ratio especially in a low illuminance environment.

However, in the photon counting type image pickup element, it is necessary to provide a counter circuit (counter, memory, comparator, etc.) for counting the number of voltage pulses for each pixel, and the circuit scale per pixel increases. Therefore, it has been used only in fields where the number of pixels and the demand for miniaturization of the device are not strict, such as industrial cameras.

Japanese Unexamined Patent Publication No. 7-67043

In recent years, by applying a three-dimensional mounting technology that stacks multiple IC chips, a multi-layer type or a multi-layer type in which the chip of the image sensor and the chip on which peripheral circuits such as an A / D conversion circuit and memory are mounted are stacked and housed in the same package. A stacked image sensor has been realized. Therefore, by forming the chip of the image sensor and the chip of the counter circuit separately and mounting them three-dimensionally, it is conceivable to realize miniaturization and high pixel count of the photon counting type image sensor.

However, for example, in the case of realizing a photon counting type image sensor with 20 million pixels, it is necessary to form 20 million counter circuits on one chip, so it is difficult to manufacture all the counter circuits as designed. .. In a general logic LSI, a failure is detected in the inspection process at the time of manufacturing, and a chip in which a failure is detected even in one place is discarded as a defective product. However, if the chip forming the counter circuit of the photon counting type image sensor is discarded if there is even one counter circuit that does not operate normally, it is expected that the manufacturing yield of the chip will be significantly reduced.

An object of the present invention is to alleviate such problems that are expected to occur in the future. Specifically, it is an object of the present invention to provide an image pickup device and an image pickup apparatus capable of correcting a pixel value corresponding to a failed counter circuit among the counter circuits provided for each pixel.

The above-mentioned purpose is a pixel array in which a plurality of pixels are arranged, and each of the plurality of pixels has a photoelectric conversion unit that can detect the incident of a single photon and a counter that counts the pulses output by the photoelectric conversion unit. It is characterized by having a pixel array having a pixel array, a failed counter among a plurality of counters included in the pixel array, and a storage means for storing counter failure information which is information about the content of the failure. Achieved by the image sensor.

With such a configuration, according to the present invention, it is possible to provide an image pickup device and an image pickup device capable of correcting a pixel value corresponding to a failed counter circuit among the counter circuits provided for each pixel.

The figure which shows the structural example of the unit pixel which concerns on embodiment The figure for demonstrating the failure detection method of the counter which concerns on embodiment. The figure which shows the structural example of the image pickup device which concerns on embodiment Block diagram showing a functional configuration example of the image pickup apparatus according to the embodiment The figure which shows the data structure example of the counter failure information which concerns on embodiment The figure which shows the structural example of the correction processing part in 1st Embodiment Timing chart for explaining the operation of the decoder according to the first embodiment The figure which shows the example of the weight control which concerns on 1st Embodiment A block diagram showing a functional configuration example of the image pickup apparatus according to the second embodiment. The figure which shows the example of the weight control which concerns on 2nd Embodiment The figure which shows the structural example of the flag addition part which concerns on 2nd Embodiment The figure which shows the data structure example of the flagged image data of 2nd Embodiment The figure which shows the structural example of the correction processing part of 2nd Embodiment

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings, based on its exemplary embodiments. It should be noted that the embodiments described are merely examples and do not limit the scope of the present invention. The scope of the present invention is defined by the description of the scope of claims. For example, an embodiment relating to an image pickup device using a photon counting type image pickup device having a photoelectric conversion element capable of detecting the incident of a single photon will be described below. However, the present invention can be implemented in any electronic device in which a photon counting type image sensor can be used, that is, it may have a photographing function. Such electronic devices include, but are not limited to, image pickup devices such as digital cameras and digital video cameras, as well as personal computers, tablet terminals, mobile phones, game consoles, drive recorders, robots, drones, and the like.

● (First embodiment)
FIG. 1 is a circuit diagram showing a configuration example of pixels included in a photon counting type image pickup device (hereinafter, simply referred to as an image pickup device) according to the present embodiment. The image pickup device has a pixel array in which a plurality of pixels (unit pixels) having the configuration shown in FIG. 1 are arranged.

The pixel 100 is composed of an avalanche photodiode (APD) 101, a quench resistor 102, a waveform shaping circuit 103, an adder 104, and a flip-flop (FF) circuit 105 with a reset (hereinafter, simply referred to as FF105). In the present embodiment, the FF 105 is configured to hold and output a 12-bit value.

A bias voltage V _APD equal to or higher than the breakdown voltage is applied to the APD 101 via the quench resistor 102. When a photon is incident on the APD 101 in this state, a large photocurrent flows due to the avalanche multiplication, and a voltage drop occurs in the quench resistor 102. As a result, the bias voltage V _APD applied to the APD 101 decreases, and when the bias voltage V _APD drops to the breakdown voltage, the avalanche multiplication stops. As a result, the photocurrent stops flowing, and the bias voltage VAPD is applied to the _APD 101 again. The quench resistance 102 is a resistance element for stopping the avalanche multiplication of the APD 101. The quench resistance 102 may be realized by utilizing the resistance component of the transistor.

The waveform shaping circuit 103 generates a voltage pulse from the voltage change caused by the incident of photons by amplifying the voltage at the connection point between the APD 101 and the quench resistor 102 and then detecting the edge. As described above, the APD 101, the quench resistor 102, and the waveform shaping circuit 103 form a photoelectric conversion unit capable of detecting the incident of a single photon. Alternatively, it can be said that the APD 101, the quench resistor 102, and the waveform shaping circuit 103 form a 1-bit type AD converter by converting the presence or absence of photon incident into a voltage pulse.

The adder 104 and the FF 105 constitute a counter that counts the voltage pulses generated by the waveform shaping circuit 103. Then, the count value of 12 bits is output from the FF 105 as the count result output 106. Therefore, a 12-bit digital value corresponding to the amount of light incident on the light receiving element (APD101) during the exposure period is output. The count value held by the FF 105 can be reset by the reset signal from the reset input 109.

The test input 107 and the test output 108 of the FF 105 are input and output of test data for detecting a failure of the counter. In the test step after manufacturing the chip, serial test data having a length equal to the number of storage bits of the FF 105 is sequentially input to each pixel through the test input 107. Then, after executing one cycle of counting operation, serial data is read from FF105 through the test output 108 and compared with the data to be output during normal operation. By this test step, it is possible to identify whether or not the operation of each counter is normal, and if it is abnormal, which bit is abnormal.

A method of detecting a failure of the counters of the vertically adjacent pixels 100-1 and 100-3 will be described by using an example of the case where the pixel 100 has a 4-bit counter shown in FIG. 2. In this embodiment, as shown in FIG. 2, the test is performed through a failure detection path in which a plurality of counters are connected in series. Here, it is assumed that the wiring of the bit 2 of the counter (FF105) of the pixel 100-1 is short-circuited to the power supply and is out of order. When performing a test, Scan_Test_Mode of the target pixel is set to a high level (logic 1) through TG201.

Then, the serial input "0000 0000" is given from the test input 107, and each bit of the FF 105 is set to 0. After that, Scan_Test_Mode is returned to the low level (logic 0) and operated for one cycle (that is, each counter value is increased by +1). Next, the Scan_Test_Mode is set to a high level again, and the counter value is read out as serial data from the test output 108.

If the read serial data is "1000 1000", the counter can be judged to be normal. However, in the example shown in FIG. 2, since the bit 2 of the counter of the pixel 100-1 is out of order, the serial data to be read is “1000 1010”. From the serial data read in this way, it can be detected that bit 2 of pixel 100-1 is out of order. Since the test process exemplified here is the same as the scanning method known as a test process after manufacturing the logic LSI, the details thereof will be omitted.

When forming a stacked image sensor using a three-dimensional mounting, the APD 101, the quench resistor 102, and the waveform shaping circuit 103 are mounted on the same chip among the circuits constituting the unit pixel 100, and the adder 104 and the FF105 are mounted. Can be mounted on the same chip. In forming the stacked image sensor, there is no particular limitation on which circuit is mounted on each chip, but if the circuit provided for each pixel is distributed and mounted on a plurality of chips, an increase in the mounting area is suppressed. can do.

FIG. 3 is a diagram showing a schematic circuit configuration example of an image pickup device having a pixel array in which the unit pixels shown in FIG. 1 are two-dimensionally arranged. Here, for convenience, the pixel array is indicated by four unit pixels 100-1 to 100-4, but in reality, more than 10 million unit pixels are arranged. In addition to the pixel array, the image pickup element 200 includes a timing signal generation circuit (TG) 201, a vertical scanning circuit 202, an output switch 203 for a vertical transmission line, a horizontal scanning circuit 204, an output switch 205 for an output transmission line, and a control unit. Has 206.

In this embodiment, the pixel array is provided with a color filter having a predetermined pattern. Here, it is assumed that the color filter of the primary color bayer array is used. Therefore, each pixel is provided with a red (R), blue (B), or green (G) color filter according to the bayer arrangement. Hereinafter, the pixel provided with the red filter is referred to as a red pixel or an R pixel. Pixels provided with filters of other colors are also referred to in the same manner.

The TG 201 generates a timing signal for controlling the start and end of an exposure period, a transfer period, and the like, and supplies the timing signal to the vertical scanning circuit 202, the horizontal scanning circuit 204, and the control unit 206.
The vertical scanning circuit 202 supplies the read signal READ_EN to the output switch 203 connected to the pixel in the row where the imaging period has ended. The output switch 203 turns on when READ_EN is received, and outputs the counting result from the corresponding unit pixel 100 to the vertical transmission line.

The horizontal scanning circuit 204 controls the output switch 205 to the output transmission line to sequentially select the vertical transmission line, and the counting result (pixel data) obtained from the pixel 100 of the row to which the vertical scanning circuit 202 supplies the read signal READ_EN. ) Is output to the outside via the output transmission line.

The control unit 206 resets the count value of the pixel 100 before the exposure period starts, based on the timing signal supplied from the TG 201.

The test input 207 and the test output 208 are similar to the test input 107 and the test output 108 in FIG. In the example of FIG. 3, the test data is supplied from the test input 107 of the unit pixel 100-1, and is output from the test output 108 of the unit pixel 100-1. This test output 108 is input to the unit pixel 100-3 as a test input. Similarly, the test data is input from the unit pixel 100-3 to the unit pixel 100-4, and finally is output to the outside as a test output 208 through the test output 108 of the unit pixel 100-4. Here, an example of performing a test using a path in which counters of four pixels are connected in series is shown, but a test may be performed by connecting more pixels.

In the example of FIG. 3, in order to detect a physical failure in a circuit in which four counters of unit pixels 100 are connected in series, serial test data having a total length of 48 bits of 12 bits × 4 pixels is used. The principle of failure detection does not depend on the length of the test data. By collating the 48-bit test data with the data to be output during normal operation, it is possible to detect which bit of which pixel has a physical failure. Then, based on the test result, information (counter failure information) regarding the counter (pixel) in which the physical failure has occurred and the failure content (here, the bit position in failure) can be referred to from the outside of the image pickup element 200. It can be saved in a non-volatile memory. This non-volatile memory may be a built-in memory of the image pickup device 200, or may be a memory of an electronic device using the image pickup element 200. The test for detecting the failure and the storage of the counter failure information can be performed, for example, at the time of manufacturing the image pickup device 200 or the image pickup device 300.

The counter failure information is generated for each counter in which a failure is detected. Further, as the final counter failure information, specific data (referred to as EOD data) indicating that there is no counter in which a failure has been detected can be stored thereafter. As described above, by storing the counter failure information at the time of manufacturing the image pickup element and the electronic device using the image pickup device, it is possible to correct the count value of the counter in which the failure has occurred. The details of the correction process will be described later.

FIG. 4 is a block diagram showing a functional configuration example of the image pickup apparatus 300 using the image pickup device according to the embodiment of the present invention.
The lens unit 301 is a photographing optical system, and has an optical lens group including a focusing lens for adjusting the focusing distance, a lens, a lens control unit for controlling the optical system such as a diaphragm, a forceing lens, and driving of the diaphragm. .. In the present embodiment, the lens unit 301 is fixed to the image pickup apparatus 300, but may be detachable. Further, the aperture may also serve as a shutter. The lens unit 301 forms an optical image on the image pickup surface of the image pickup element 200.

The system control unit 302 has a programmable processor such as a CPU, and controls each part of the image pickup apparatus 300 by reading the program stored in the non-volatile memory 306 into the system memory 308 and executing the program, and controls the image pickup apparatus 300. Realize the function. Even the block that is not connected to the system control unit 302 in the figure actually operates according to the control of the system control unit 302. The system control unit 302 executes automatic focus adjustment (AF) and automatic exposure control (AE) processing based on the evaluation value obtained from the image processing unit 304. For example, the system control unit 302 drives the focusing lens of the lens unit 301 through the optical system drive unit 303 based on the evaluation value indicating the defocus amount of the lens unit 301. Further, the system control unit 302 determines shooting conditions such as aperture, shutter speed, and shooting sensitivity based on the evaluation value regarding brightness. The system control unit 302 may execute the AF process of the contrast detection method based on the contrast evaluation value. Further, when the image processing unit 304 detects a predetermined subject area such as a face area, AE processing is performed so that the subject area becomes an appropriate exposure, or AF processing is performed so that the subject area is in focus. It can be carried out.

The system control unit 302 sets the drive information of the image sensor 200, such as the exposure period and the readout interval of the image sensor 200, according to the shooting conditions and the shooting mode (for example, still image shooting mode or moving image shooting mode) determined by the AE processing. To control the operation of the TG201. When the system control unit 302 also detects an operation of the operation unit 309, which is a general term for a group of input devices such as keys and buttons, the system control unit 302 executes a process according to the operation. For example, when the system control unit 302 detects that the shutter button is half-pressed in the still image shooting mode, it recognizes it as a shooting preparation instruction and executes AF and AE processing. When the shutter button is fully pressed, it is recognized as a shooting start instruction, and still image shooting processing and recording processing are executed.

The correction processing unit 307 corrects the image data output by the image pickup element 200 based on the counter failure information stored in the non-volatile memory 306, and outputs the image data to the image processing unit 304. The details of the operation of the correction processing unit 307 will be described later.

The image processing unit 304 uses images such as demosaic (color interpolation) processing, white balance adjustment processing, subject detection processing, AF evaluation value calculation processing, and AE evaluation value calculation processing for the image data supplied from the correction processing unit 307. The processing is applied according to the shooting mode and the like. Further, the image processing unit 304 applies scaling processing, coding processing, decoding processing, D / A conversion processing, and the like, depending on the use of the image data. For example, the image processing unit 304 applies coding processing to the image data for recording, generates additional information according to the recording format, generates an image data file, and records the image data in the recording device 305. Can be done. Further, the image data for display can be D / A converted and output to the display device 310 after applying a scaling process according to the display resolution. Further, when the encoded image data is read from the recording device 305, the decoding process can be applied. The processes listed here are merely examples, and the image processing unit 304 may execute other processes. The image processing unit 304 is often realized by dedicated hardware such as an ASIC, but even if at least a part of the processing is realized by the microprocessor of the image processing unit 304 or the system control unit 302 executing a program. good.

The recording device 305 records data on a recording medium and reads data from the recording medium. The recording medium may be a memory card, a magnetic disk drive, or the like. Further, the recording medium may or may not be removable.

The non-volatile memory 306 stores programs that can be executed by a microprocessor such as the system control unit 302, various setting values, GUI data, counter failure information, and the like. The non-volatile memory 306 may be rewritable.

The operation unit 309 is a general term for a group of input devices, and is used by a user to input an instruction to the image pickup apparatus 300. A shutter button, a moving image shooting / stop button, a shooting mode switching dial, a direction key, an enter button, a menu button, and the like are typical examples of input devices included in the operation unit 309. When the display device 310 is a touch display, the touch panel portion is included in the operation unit 309.

The system memory 308 reads a program executed by the system control unit 302 and stores information necessary for executing the program. The system memory 308 is also used as a buffer memory in which the image processing unit 304 temporarily stores image data.

The display device 310 is, for example, an LCD or an organic EL display, and is used for displaying an image taken by the image pickup device 300, information of the image pickup device 300, a menu screen, an image reproduced from a recording medium, and the like. During standby for still image shooting or during video recording, the moving image being shot is displayed on the display device 310 in substantially real time, and the display device 310 functions as an electronic viewfinder.
Although not shown for convenience, the image pickup apparatus 300 has a configuration included in a general digital camera in addition to the above-mentioned functional block.

FIG. 5 is a diagram showing an example of a data format of counter failure information stored in the non-volatile memory 306. In this embodiment, each counter failure information is composed of 21 bits, and 16 bits from bit 0 to bit 15 are used for distance information between pixels in which the counter is failed. Further, 4 bits from bit 16 to bit 19 are used for counter bit information indicating the failure bit position of the counter. The bit 20 is used as a dummy flag. The dummy flag is set to 1 to indicate that the counter of the pixel represented by the distance information (65535 pixels) has not failed when the distance information cannot be represented by 16 bits (more than 65535 pixels). Further, the EOD data stored as the last counter failure information may be data in which the dummy flag is set to 1 and the distance information and the counter bit information do not exist. Further, the number of bits of the counter bit information can be determined according to the number of bits of the counter included in the image pickup device 200.

Next, a functional configuration example and an operation example of the correction processing unit 307 of the present embodiment will be described with reference to FIGS. 6 to 8.
FIG. 6 is a block diagram showing a functional configuration example of the correction processing unit 307. Further, FIG. 7 is a timing chart showing the operation timing of the correction processing unit 307 from the time when the image pickup apparatus 300 is started (Init).

The SRAM 501 reads and stores the counter failure information stored in the non-volatile memory 306 via the data input 509 at the time of starting the image pickup apparatus 300 (the Data-Load period from the time Init to Vstart1 in FIG. 7). It is a buffer memory for. Further, among the counter failure information read into the SRAM 501, the distance information (bits 0 to 15) is input to the comparison circuit 503, and the counter bit information and the dummy flags (bits 16 to 20) are input to the decoder 504.

The reset input 510 is input from the system control unit 302 at the timing at the beginning of the image frame (Vstart 1 in FIG. 7), and resets the counter 502. The counter 502 counts one pixel as one cycle for the image data input from the image sensor 200 as the pixel input 507. That is, the count value of the counter 502 indicates the pixel number from the beginning of the frame that the input pixel data is. In the present embodiment, the counter 502 is a 16-bit counter, and the count value is input to the comparison circuit 503.

The comparison circuit 503 compares the distance information Dn (n is an integer of 1 or more) of the nth counter failure information with the count value of the counter 502, and when both match, notify the decoder 504 and the counter. Reset 502. The comparison circuit 503 does not notify the decoder 504 when the distance information and the count value do not match.

Then, when the decoder 504 (weight control means) receives the notification, the decoder 504 determines the weight based on the counter bit information of the nth counter failure information and gives it to the weighting addition circuit 506. Then, the decoder 504 reads the next (n + 1st) counter failure information from the SRAM 501. As a result, the counter failure information of the decoder 504 and the comparison circuit 503 is updated (each of P1 to P4 in FIG. 7).

When the dummy flag is not set (not 1) for the cycle in which the notification is input from the counter 502 (pixel data which is the output of the failed counter circuit), the decoder 504 of the weighting addition circuit 506 is based on the counter bit information. Control the weighting. In FIG. 7, the value of the dummy flag is shown as Dummy. For example, when the decoder 504 is notified at the timing of P1, the decoder 504 confirms the value of the dummy flag Dummy read from the SRAM 501 at the timing of Vstart1. Similarly, when the decoder 504 is notified at the timing of P2, the decoder 504 confirms the value of the dummy flag Dummy read from the SRAM 501 at the timing of P1. FIG. 7 shows the operation timing when there are four counter failure information excluding EOD data.

The decoder 504 does not give a weight to the weighting addition circuit 506 or gives a weight of 0 for the cycle in which the notification is not input from the counter 502. In either case, the correction by the weighting addition circuit 506 is not performed. If the counter failure information read from the SRAM 501 is EOD data, the decoder 504 does not read the counter failure information from the SRAM 501 until the next frame is started, and does not give weight to the weighting addition circuit 506? , Gives a weight of 0.

The weighting addition circuit 506 weights and adds the pixel data (input pixel data) from the pixel input 507 and the correction data from the correction data generation circuit 505 based on the following equation to correct the pixel data.
Corrected pixel data = (1-w / 100) × input pixel data + w / 100 × correction data Here, w [%] (0 ≦ w ≦ 100) is the weight (or degree of correction) given by the decoder 504. be. The weighting addition circuit 506 outputs the input pixel data as it is for the cycle in which the weight is not given by the decoder 504 and the cycle in which the weight is given 0. When the next frame is started (time Vstart2 in FIG. 7), the decoder 504 reads the first counter failure information from the SRAM 501.

The correction data generation circuit 505 generates correction data (correction value) using pixel data in a predetermined range around the pixel data from the pixel input 507. The method of generating the correction data from the peripheral pixel data is not particularly limited, and for example, the average value of the peripheral pixel data may be used as the correction data. Further, the pixel data to be corrected may also be used to generate the correction data. Although not shown in FIG. 6 for simplification of the drawings and description, for example, when pixel data input after the pixel data to be corrected is used for generating correction data, the input pixel data is used. Use a buffer that stores a certain amount of data. Further, the processing timing of the comparison circuit 503 and the decoder 504 is also adjusted.

Here, the weighting control by the decoder 504 will be described with reference to FIG. FIG. 8 is a diagram showing an example of weights set by the decoder 504 with respect to the weighting addition circuit 506. In this embodiment, the decoder 504 changes the weight depending on which bit of the counter is failing. Here, since the image sensor 200 uses a 12-bit counter, the failure bit position on the horizontal axis in FIG. 8 is from bit-0 to bit-11.

Here, FIG. 8A shows an example of a weight for a G pixel, and FIG. 8B shows an example of a weight for an R pixel and a B pixel. In each case, basically, the weight (correction degree) for the failure bit position n is equal to or greater than the weight for the failure bit position (n-1). This is because the failure of the high-order bit increases the difference from the correct count value. On the other hand, due to the pattern of the color filter and the visual characteristics of human beings, the deterioration of the image quality due to the correction is not noticeable for the R pixel and the B pixel as compared with the G pixel. Therefore, the weight for the same failure bit position is larger (stronger correction) for the R pixel and the B pixel than for the G pixel. For example, for the G pixel, when the failure bit position is near the least significant bit (here, bit 0 or 1), the correction degree is set to 0% and no correction is performed.

The color of the color filter provided for each pixel (hereinafter referred to as the pixel color) can be stored in the non-volatile memory 306 in advance and notified from the system control unit 302 to the decoder, but it is not essential. do not have. Since the correspondence between the pixel position and the color is fixed and regular, if the color of the pixel to be read first is known, the color of the pixel to be read after that can be specified. Therefore, for example, if the position of the first pixel to be read and the reading order are fixed, the color of each pixel can be specified without storing the information. Alternatively, the system control unit 302 may notify the decoder 504 of the color information of the pixel to be read first, and then the decoder 504 may specify the color of each pixel according to the regularity of the color filter.

It is not essential to make the weights different depending on the color of the pixels, and a common weight according to the failure bit position may be applied to all the pixels. Also in this case, basically, the weight for the failure bit position n is equal to or greater than the weight for the failure bit position (n-1).

In FIG. 7, Bit indicates the fault bit position. Therefore, the decoder 504, for example, at time P1, gives the weighting addition circuit 506 a weight corresponding to the failure bit position 4 (and, if necessary, the color of the pixel). Similarly, the decoder 504 gives a weight corresponding to the failure bit position 7 at the time P3 and the failure bit position 11 at the time P4 to the weighting addition circuit 506. On the other hand, since the dummy flag is 1 for the time P2, the target pixel should not be corrected. Therefore, the decoder 504 gives a weight corresponding to a correction degree of 0% (that is, a weight of 0 for the correction data) to the weighting addition circuit 506.

When the EOD data is read at time P4, the decoder 504 stops reading the counter failure information and providing the weight to the weighting addition circuit 506 until the reading of the pixel data for the current frame is completed (continues to provide the weight 0). May be).

As described above, according to the present embodiment, among the counter circuits provided for each pixel in the photon counting type image pickup device, the counter failure relating to the counter circuit in which the failure is detected and the failed bit position Changed to retain information. This makes it possible to correct the pixel data corresponding to the failed counter among the read pixel data. Therefore, it is possible to increase the yield of chips on which a huge number of counter circuits are mounted. Further, by making corrections to a degree (strength) according to the bit position where the failure has occurred, appropriate corrections can be made according to the magnitude of the influence of the failure. Further, by making the degree (intensity) different according to the color of the pixel to be corrected, the image deterioration due to the correction can be made inconspicuous. Further, by implementing this embodiment as a stacked image sensor, it can be suitably implemented for an image sensor having tens of millions of pixels or more, typically a large image sensor of APS-C size or full size or more.

● (Second embodiment)
Next, a second embodiment of the present invention will be described. FIG. 9 is a block diagram showing a configuration example of the image pickup apparatus 800 according to the present embodiment. In FIG. 9, the same components as those of the image pickup apparatus 300 described in the first embodiment are designated by the same reference numerals as those in FIGS. 4 and 6, and the description thereof will be omitted.

The image pickup device 800 mounts a part of the configuration that the image pickup device 300 has outside the image pickup device 200 in the first embodiment on a chip enclosed in the same package as the image pickup device 200 (image pickup device package 801). is doing. Further, it has a configuration for imparting information on weights for correcting pixel data read from the image sensor package 801 to the pixel data in the image sensor package 801. There are no particular restrictions on the number of chips to be mounted three-dimensionally in the image sensor package 801 and which circuit is mounted on each chip.

In the present embodiment, the flag addition unit 802 adds correction information (correction flag) according to the instruction of the decoder 804 to each of the pixel data output by the image sensor 200 and outputs the correction information (correction flag) to the data transmission unit 803. The decoder 804 is the same as the decoder 504 of the first embodiment except that a correction flag is given to the flag addition unit 802 instead of the weight.

That is, the decoder 804 determines the correction degree (weight) according to the failure bit position for the pixel data corresponding to the failed counter, and gives the flag addition unit 802 a correction flag indicating the determined correction degree. .. The relationship between the fault bit position and the correction degree may be the same as in the first embodiment, but here, in order to reduce the number of bits to be added, the types of correction degree are reduced. Specifically, as shown in FIG. 10, the degree of correction is set to 4 types of 0%, 25%, 50%, and 100% so that it can be expressed by a 2-bit correction flag. Moreover, the color of the pixel is not considered. It should be noted that when the dummy flag is 1, the correction degree is set to 0%, which is common to the first embodiment.

FIG. 11 is a diagram showing a circuit configuration example of the flag addition unit 802. The flag addition unit 802 adds a 2-bit correction flag output by the selector 904 to the pixel data 901 of m bits (here, m = 12) output by the image pickup element 200, and a pixel with a flag of (m + 2) bits. It is output to the data transmission unit 803 as data 903. FIG. 12 shows a data configuration example of flagged pixel data. Here, the correction flag is added as the most significant 2 bits, but it may be added as the least significant 2 bits. Further, if the pixel data can be associated with the correction flag, the correction flag may be output separately from the pixel data.

The selector 904 outputs one of the four types of correction flags according to the correction degree 902 given by the decoder 804. Here, the selector 904 is "00" when the correction degree is 0 [%], "01" when the correction degree is 25 [%], and "10" when the correction degree is 50 [%]. When the degree is 100 [%], the correction flag of "11" is output.

The data transmission unit 803 transmits the flagged pixel data 903 to the data reception unit 805, which is an external circuit of the image sensor package 801. The data receiving unit 805 outputs the flagged image data 903 to the correction processing unit 806.

FIG. 13 is a block diagram showing a functional configuration example of the correction processing unit 806 of the present embodiment. The correction processing unit 806 receives the 14-bit flagged image data 1101 received from the data reception unit 805. Of the 14 bits, the most significant 2 bits are input to the flag detection unit 1103, and the remaining 12 bits are input to the correction data generation circuit 505.

The flag detection unit 1103 performs the reverse operation of the selector 904 of the flag addition unit 802, converts a 2-bit flag into a correction degree, and gives it to the weighting addition circuit 506. That is, the flag detection unit 1103 is 0 [%] if the correction flag is "00", 25 [%] if it is "01", 50 [%] if it is "10", and "11". A correction degree of 100 [%] is given to the weighting addition circuit 506. Since the operation of the correction data generation circuit 505 and the weighting addition circuit 506 is the same as that of the first embodiment, the description thereof will be omitted.

Also in this embodiment, the same effect as that of the first embodiment can be realized. Further, in the present embodiment, the image pickup element package holds the counter failure information, and if necessary, outputs the pixel data to which the information regarding the correction amount (correction flag) is added, so that the user of the image pickup element needs to detect the failure of the counter. There is no. Therefore, there is an advantage that it is suitable for distribution as a single image sensor. It should be noted that finer correction control may be performed by increasing the number of bits of the correction flag or considering the color of the pixel.

(Other embodiments)
The present invention supplies a program that realizes at least the function of the correction processing unit in the above-described embodiment to the system or device via a network or storage medium, and one or more processors in the computer of the system or device provide the program. It can also be realized by the process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 ... unit pixel, 104 ... adder, 105 ... FF, 200 ... image pickup element, 302 ... system control unit, 306 ... non-volatile memory, 307, 806 ... correction processing unit, 504, 804 ... decoder, 505 ... correction data generation Circuit, 506 ... Weighting adder circuit

Claims (16)

A pixel array in which a plurality of pixels are arranged, each of the plurality of pixels has a photoelectric conversion unit capable of detecting the incident of a single photon, and a counter for counting pulses output by the photoelectric conversion unit. With a pixel array,
Among the plurality of counters included in the pixel array, a failed counter, a storage means for storing counter failure information which is information about the content of the failure, and a storage means.
An image pickup device characterized by having. The image pickup device according to claim 1, wherein the content of the failure is a failed bit position. The image pickup device according to claim 1 or 2, further comprising an output means for generating and outputting correction information for correcting the count value of the failed counter based on the counter failure information. .. The content of the failure is the failed bit position, and the output means provides correction information such that the correction strength for the bit position (n + 1) is equal to or greater than the correction strength for the bit position n. The image pickup device according to claim 3, wherein the image pickup device is generated. The output means generates correction information so that when the color of the color filter provided in the pixel having the failed counter is red or blue, the correction is stronger than when the color is green. The image pickup device according to claim 4, which is characterized. The image pickup device according to any one of claims 1 to 5, wherein the counter failure information is stored based on a test result at the time of manufacturing the image pickup device. The image pickup device according to any one of claims 1 to 6, further comprising a path for detecting a failure, in which a plurality of counters are connected in series. The image pickup device according to any one of claims 1 to 7, wherein the photoelectric conversion unit and the counter are laminated image pickup devices mounted on separate chips. The image pickup device according to any one of claims 3 to 5.
A correction means for correcting the count value read as pixel data from the image sensor based on the correction information,
An imaging device characterized by having. The imaging device according to claim 9, wherein the correction means corrects the count value by weighting and adding the count value and the correction data using weights based on the correction information. An image pickup device in which a plurality of pixels are arranged, each of the plurality of pixels has a photoelectric conversion unit capable of detecting the incident of a single photon, and a counter for counting pulses output by the photoelectric conversion unit. Image sensor and
Among the plurality of counters included in the image pickup element, a failed counter, a storage means for storing counter failure information which is information about the content of the failure, and a storage means.
A correction means for correcting pixel data read from a pixel having the failed counter based on the counter failure information, and a correction means.
An imaging device characterized by having. The imaging device according to claim 11, wherein the content of the failure is a bit position where the failure has occurred. The imaging according to claim 12, wherein the correction means corrects the pixel data so that the correction strength for the bit position (n + 1) is equal to or higher than the correction strength for the bit position n. Device. 13. The correction means according to claim 13, wherein when the color of the color filter provided in the pixel having the failed counter is red or blue, the correction means makes a stronger correction than when the color is green. Imaging device. One of claims 11 to 14, wherein the correction means corrects the pixel data by weighting and adding the pixel data and the correction data using weights based on the counter failure information. The imaging device according to. A program for causing a computer included in an image pickup apparatus to function as a correction means for the image pickup apparatus according to any one of claims 10 to 15.

Images