JP7624372B2 - Pixel array including evenly arranged phase detection pixels and image sensor including the same - Google Patents

Description

The present invention relates to a pixel array, and more particularly to a pixel array including phase detection pixels evenly spaced for each color pixel, and an image sensor including the same.

The image sensor also includes an array of pixels that senses received light signals.
According to the image sensor providing an autofocusing function, the pixel array includes phase detection pixels, which are non-contiguous and irregularly arranged within the pixel array.
Due to the increasing demand for high quality images, the pixels of the image sensor are required to be highly integrated.
As pixel sizes in image sensors decrease and pixel array patterns become non-uniform, an increasing problem of inter-pixel cross-talk arises.

Crosstalk refers to the influence of signals generated in adjacent pixels on a target pixel, which changes the spectral characteristics of the signal generated in the target pixel and reduces color reproducibility.
Crosstalk causes degradation of image quality.
Therefore, the effects of crosstalk need to be eliminated or minimized.

JP 2017-118477 A

The present invention has been made in consideration of the problems with the conventional image sensors described above, and the object of the present invention is to provide a pixel array that minimizes noise for each color channel of a pixel by evenly arranging phase detection pixels, and an image sensor including the same.

In order to achieve the above object, a pixel array according to the present invention includes a plurality of pixel groups, each of the plurality of pixel groups includes a plurality of color pixels arranged in a 2×2 matrix , each of the plurality of color pixels includes a plurality of sub-pixels arranged in an (m)×(n) matrix (m and n are natural numbers equal to or greater than 2) sensing light of the same wavelength band, the plurality of sub-pixels including a phase detection pixel that detects a phase and a sensing pixel that senses an image, the plurality of color pixels include a first color pixel and a second color pixel arranged adjacent to each other in a first diagonal direction, and a position of a first phase detection pixel included in the first color pixel within the first color pixel is the same as a position of a second phase detection pixel included in the second color pixel within the second color pixel .

In order to achieve the above object, an image sensor according to the present invention includes: a pixel array including a plurality of color pixels arranged in a matrix , each of which includes a plurality of sub-pixels sensing light of the same wavelength band ; a readout circuit configured to convert a sensing signal received from the pixel array via a plurality of column lines into binary data; a row decoder configured to generate a row selection signal for controlling the pixel array to output the sensing signal for each row via a plurality of row lines; and control logic configured to control the row decoder and the readout circuit, wherein the sub-pixels included in each of the plurality of color pixels include a phase detection pixel that detects a phase and a sensing pixel that senses an image, and the plurality of color pixels include a first color pixel and a second color pixel that are adjacently arranged in a first diagonal direction, and a position of a first phase detection pixel included in the first color pixel within the first color pixel is the same as a position of a second phase detection pixel included in the second color pixel within the second color pixel .

In order to achieve the above object, an image sensor according to the present invention includes a plurality of color pixels arranged in a matrix , each of the color pixels including a plurality of sub-pixels sensing light of the same wavelength band , and each of the color pixels includes a phase detection pixel arranged at a predetermined position, a readout circuit configured to convert a sensing signal received from the pixel array via a plurality of column lines into binary data, a row decoder configured to generate a row selection signal for controlling the pixel array to output the sensing signal for each row via a plurality of row lines, and a control logic configured to control the row decoder and the readout circuit and to change an output method of the sensing signal based on a mode signal , wherein the plurality of color pixels include a first color pixel and a second color pixel arranged in a first diagonal direction, and a position of a first phase detection pixel included in the first color pixel within the first color pixel is the same as a position of a second phase detection pixel included in the second color pixel within the second color pixel .

According to the pixel array and image sensor of the present invention, a predetermined number of phase detection pixels are arranged at specific positions for each color channel, so that the amount of crosstalk induction is substantially the same for each color channel, and constant crosstalk is easily removed without separate correction. In addition, the phase detection pixels P are uniformly arranged at specific positions for each color channel, so that the number of phase detection pixels included in the color channels is minimized, and the signal-to-noise ratio (SNR) of the sensing signal can be improved.

1 is a block diagram showing a schematic configuration of an image sensor according to an embodiment of the present invention. 1 is a diagram illustrating a pixel array according to an embodiment of the present invention; 1 is a diagram illustrating a pixel group according to an embodiment of the present invention; 1 is a diagram illustrating a pixel group according to an embodiment of the present invention; 11A and 11B are diagrams for explaining data values of color pixels including sub-pixels. 2 is a vertical cross-sectional view illustrating a schematic diagram of a phase detection pixel pair according to an embodiment of the present invention; 2 is a vertical cross-sectional view illustrating a schematic diagram of a phase detection pixel pair according to an embodiment of the present invention; 2 is a vertical cross-sectional view illustrating a schematic diagram of a phase detection pixel pair according to an embodiment of the present invention; FIG. 5C is a plan view of the dual photodiode of FIG. 5B. FIG. 5C is a plan view of the dual photodiode of FIG. 5B. FIG. 5C is a plan view of the dual photodiode of FIG. 5B. This is a vertical cross-sectional view taken along line A-A' in Figure 6A. FIG. 13 is a diagram for explaining phase detection pixels that are not uniformly arranged for each color pixel. 10A and 10B are diagrams illustrating phase detection pixels uniformly arranged for each color pixel according to an embodiment of the present invention. 10A and 10B are diagrams illustrating phase detection pixels uniformly arranged for each color pixel according to an embodiment of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 1A-1C show various examples of uniformly spaced phase detection pixels according to embodiments of the present invention. 4A-4C are diagrams illustrating various examples of uniformly arranged phase detection pixels according to embodiments of the present invention. 4A-4C are diagrams illustrating various examples of uniformly arranged phase detection pixels according to embodiments of the present invention. 4A-4C are diagrams illustrating various examples of uniformly arranged phase detection pixels according to embodiments of the present invention. 11A to 11C are diagrams illustrating different data outputs for different modes according to an embodiment of the present invention. FIG. 2 is an equivalent circuit diagram of a pixel. FIG. 2 is a circuit diagram of a pixel that performs summation of sensing signals according to an embodiment of the present invention. FIG. 2 is a circuit diagram of a pixel that performs summation of sensing signals according to an embodiment of the present invention. 1 is a block diagram showing a schematic configuration of an electronic device including a multi-camera module to which an image sensor according to an embodiment of the present invention is applied; FIG. 14 is a block diagram showing a detailed configuration of the multi-camera module of FIG. 13. 1 is a block diagram showing a schematic configuration of an electronic device according to an embodiment of the present invention. 1 is a block diagram showing a schematic configuration of an electronic device according to an embodiment of the present invention.

Next, specific examples of embodiments for implementing the pixel array and image sensor according to the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of an image sensor 100 according to an embodiment of the present invention.
The image sensor 100 is also installed in an electronic device having an image sensing function or a light sensing function.

For example, the image sensor 100 may be installed in electronic devices such as cameras, smartphones, wearable devices, Internet of Things (IoT) devices, home appliances, tablet PCs (personal computers), PDAs (personal digital assistants), PMPs (portable multimedia players), navigation devices, drones, and advanced driver-assistance systems (ADAS).
The image sensor 100 is also mounted on electronic devices provided as components in vehicles, furniture, manufacturing equipment, doors, various measuring instruments, and the like.

The image sensor 100 includes a lens LS, a pixel array 110, a row decoder 120, control logic 130, a ramp generator 140, and a readout circuit 150.
Although not shown in FIG. 1, it should be understood that the image sensor 100 may further include a clock signal generator, a signal processor, a column decoder, and/or a memory.

The image sensor 100 converts an optical signal of an object incident through an optical device into an electrical signal, and generates image data (IDAT) based on the electrical signal.
The optical system is an optical focusing system including mirrors and lenses LS.
For example, the image sensor 100 may use an optical device that can collect various paths of light reflected by an object or change the path of travel of the light by using optical properties such as dispersion and refraction of light.
In the present invention, for convenience of explanation, it is assumed that a lens LS is used, but as described above, it will be understood that the technical concept of the present invention can be embodied using various optical devices.

The pixel array 110 is a complementary metal oxide semiconductor image sensor (CIS) that converts optical signals into electrical signals.
The optical signal transmitted through the lens LS reaches the light receiving surface of the pixel array 110 and forms an image of the subject.
Such a pixel array 110 can adjust the sensitivity of optical signals, etc., under the control of the control logic 130.
The pixel array 110 is connected to a plurality of row lines RLs and a plurality of column lines CLs that transmit signals to a plurality of pixels arranged in a matrix.

For example, each of the row lines RLs transmits a control signal output from the row decoder 120 to each of the transistors included in the pixel, and each of the column lines CLs transmits pixel signals of the pixels to the readout circuit 150 on a column basis in the pixel array 110.
Each of the column lines CLs extends in the column direction and connects pixels arranged in the same column to the readout circuit 150 .
Each of the plurality of pixels in the pixel array 110 includes at least one photoelectric conversion element and a transistor.

For example, the pixel array 110 may be implemented as a photoelectric conversion element (or a light sensing element) such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), but may also be implemented as various other types of photoelectric conversion elements.
According to one embodiment, the photoelectric conversion element senses light and converts the sensed light into photo-charges.
For example, the photoelectric conversion element may be a light sensing element made of an organic or inorganic material, such as an inorganic photodiode, an organic photodiode, a perovskite photodiode, a phototransistor, a photogate, or a pinned photodiode.
In one embodiment, the transistor transfers the charge stored in the photoelectric conversion element, resets it to a power supply voltage, or converts the charge into an electrical signal.

A microlens and a color filter are laminated on top of each of the pixels, and the color filters of the pixels form a color filter array.
The color filter transmits light of a specific hue, in other words, wavelengths in a specific hue range, among the light incident via the microlens.
The color filter attached to a pixel determines the hues that the pixel can sense.
However, without being limited thereto, in one embodiment, the photoelectric conversion element provided in the pixel can convert light corresponding to a wavelength in the color range into an electrical signal depending on the level of the electrical signal applied thereto, e.g., a voltage level, and thus the color that the pixel can sense can be determined depending on the level of the electrical signal applied to the photoelectric conversion element.

In one embodiment, each of the pixels of the pixel array 110 includes a microlens and at least one photoelectric conversion element disposed side-by-side under the microlens.
For example, each of the plurality of pixels includes at least one first photoelectric conversion element and at least one second photoelectric conversion element arranged side by side.
The pixel outputs a first image signal generated from the first photoelectric conversion element or a second image signal generated from the second photoelectric conversion element.
The pixel also outputs a sum image signal generated from the first photoelectric conversion element and the second photoelectric conversion element.

The plurality of pixels includes color pixels, for example, a red pixel, a green pixel, and a blue pixel.
The color pixels generate image signals containing color information based on light passing through different color filters.
In the present invention, each color filter that produces different color information, or each color pixel or collection of color pixels that produces an image signal containing different color information, is referred to as a color channel.
For example, the red channel refers to the red filter itself or a red pixel which is a pixel that processes light that has passed through a red filter, the blue channel refers to the blue filter itself or a blue pixel which is a pixel that processes light that has passed through a blue filter, and the green channel refers to the green filter itself or a green pixel which is a pixel that processes light that has passed through a green filter.
Although the present invention has been exemplified by red, green, and blue colors, the plurality of pixels may include pixels combined in other color configurations, for example, yellow pixels, cyan pixels, and white pixels.

The red pixel responds to wavelengths in the red region of the visible light range and generates an image signal (or charge) corresponding to a red color signal.
The green pixel responds to wavelengths in the green region of the visible light region and generates an image signal (or charge) corresponding to a green color signal.
The blue pixel responds to wavelengths in the blue region of the visible light range and generates an image signal (or charge) corresponding to a blue color signal.
However, without being limited thereto, the plurality of pixels may further include a white pixel.
As another example, the plurality of pixels may include cyan pixels, yellow pixels, magenta pixels, or white pixels.

The row decoder 120, under the control of the control logic 130 (e.g., a row control signal (CTR_X)), generates control signals for driving the pixel array 110, and drives multiple pixels of the pixel array 110 row by row via multiple row lines RLs.
Each of the row lines RLs extends in the row direction and is connected to pixels arranged in the same row.

In one embodiment, the row decoder 120 controls the pixels in the pixel array 110 so that multiple pixels sense incident light simultaneously or row by row.
In addition, the row decoder 120 selects a plurality of pixels on a row basis, provides a reset signal to the selected pixels (e.g., pixels in one row) to reset the pixels, and controls the sensing voltages generated by the selected pixels to be output via a plurality of column lines CLs.

Control logic 130 provides control signals that control the timing of row decoder 120 , ramp generator 140 , and readout circuit 150 .
For example, the control logic 130 provides a row control signal (CTR_X) to the row decoder 120, and the row decoder 120 senses the pixel array 110 row by row via the row lines RLs based on the row control signal (CTR_X).
For example, the control logic 130 provides a ramp control signal (CTR_R) to the ramp generator 140 to control the ramp signal, and the ramp generator 140 generates a ramp signal RMP for operation of the readout circuit 150 based on the ramp control signal (CTR_R).
For example, the control logic 130 provides a column control signal (CTR_Y) to the readout circuit 150, which receives and processes pixel signals from the pixel array 110 via the column lines CLs based on the column control signal (CTR_Y).

According to an embodiment of the present invention, the control logic 130 generally controls the image sensor 100 based on the mode signal MD.
For example, the control logic 130 receives a mode signal MD instructing high-resolution sensing or low-resolution sensing from the application processor, and controls the row control signal (CTR_X), column control signal (CTR_Y), and ramp control signal (CTR_R) so that each of the multiple pixels of the pixel array 110 outputs an independent pixel signal, the pixel array 110 outputs each of the multiple pixels based on the row control signal (CTR_X) and column control signal (CTR_Y), and the readout circuit 150 samples and processes the pixel signal based on the ramp signal RMP.
For example, the application processor determines the imaging mode of the image sensor 100 based on various scenarios such as the illuminance of the imaging environment, the user's resolution setting, and the sensing or learning state, and provides the result as a mode signal MD.

The control logic 130 may be embodied as a processing circuit, such as hardware including logic circuits, or as a combination of hardware and software, such as a processor executing software that performs the compression operations.
In particular, the control logic 130 may be implemented as a central processing unit (CPU) included in the image sensor 100, an arithmetic logic unit (ALU) performing arithmetic and logical operations, bit shifting, etc., a digital signal processor (DSP), a microprocessor, an application specific integrated circuit (ASIC), control logic, etc., but is not limited thereto. For example, the control logic 130 may further include an accelerator that supports an artificial neural network, etc., or uses the artificial neural network itself, such as an NPU (neural processing unit).

The ramp generator 140 generates a ramp signal RMP that gradually increases or decreases with a predetermined gradient, and provides the ramp signal RMP to a readout circuit 150 .
The readout circuit 150 receives pixel signals output from the pixel array 110 via column lines CLs, processes the pixel signals, and outputs them as image data IDAT.
The readout circuitry 150 may include a correlated double sampling (CDS) circuitry 151 , an analog-to-digital converting (ADC) circuitry 153 , and a buffer 155 .

The correlated double sampling circuit 151 includes a number of comparators and compares pixel signals received from the pixel array 110 via a number of column lines CLs with the ramp signal RMP from the ramp generator 140 .
The comparator compares the received pixel signal with the buffered ramp signal RMP and outputs the comparison result as a logic low or logic high.
For example, when the level of the ramp signal RMP is the same as the level of the pixel signal, the comparator outputs a comparison signal that transitions from a first level (e.g., logic high) to a second level (e.g., logic low), and the point in time at which the level of the comparison signal transitions is determined by the level of the pixel signal.

The multiple pixel signals output from the multiple pixels have deviations due to characteristics unique to each pixel (e.g., fixed pattern noise (FPN) and the like) and/or deviations due to differences in characteristics of logic for outputting pixel signals from the pixels (e.g., a transistor for outputting charges stored in a photoelectric conversion element within a pixel).
In this way, in order to compensate for the deviation between multiple pixel signals output through multiple column lines CLs, a reset voltage (or reset component) and a sensing voltage (or sensing component) are obtained for each pixel signal, and the difference (e.g., voltage difference) is extracted as a valid signal component. This is called correlated double sampling (CDS).
The comparator outputs a comparison result (eg, a comparison output) to which a correlated double sampling (CDS) technique is applied.
As a result, the correlated double sampling circuit 151 generates a comparison result to which the correlated double sampling (CDS) technique is applied.

The analog-to-digital converting circuit 153 converts the comparison result of the correlated double sampling circuit 151 into digital data, thereby generating and outputting pixel values corresponding to a plurality of pixels on a row-by-row basis.
The analog-to-digital converting circuit 153 includes a plurality of counters.
The plurality of counters are connected to the outputs of the plurality of comparators, respectively, and count the comparison results output from the comparators.
The counter counts the comparison results of logic high or logic low output from the comparator based on a counting clock during a reset conversion period in which a reset signal is sensed and during an image conversion period in which a sensing signal is sensed, and outputs digital data (e.g., a pixel value) based on the counting result.

The counter includes a latch circuit and an arithmetic circuit.
The latch circuit latches the code value received as the counting clock signal at the point in time when the level of the comparison signal received from the comparator transitions.
The latch circuitry latches a code value corresponding to the reset signal, for example a reset value, and a code value corresponding to the image signal, for example an image signal value, respectively.
The arithmetic circuit performs an operation on the reset value and the image signal value to generate an image signal value from which the reset level of the pixel has been removed.
The counter outputs the image signal value with the reset level removed as the pixel value.
However, the present invention is not limited thereto, and the counter may be realized as an up counter and an arithmetic circuit in which the count value is sequentially increased based on a counting clock signal, an up/down counter, or a bit-wise inversion counter.

The buffer 155 stores each pixel value output from the analog-to-digital converting circuit 153 .
A buffer 155 stores the digital data (eg, pixel values) for each row.
In one embodiment, the buffer 155 temporarily stores a plurality of digital data output from the counter, amplifies the data, and outputs the data.
That is, the buffer 155 is an output buffer.
The buffer 155 temporarily stores the digital data output from each of the counters, and then sequentially or selectively outputs the digital data to a sense amplifier (not shown), which amplifies and outputs the received digital data.
The buffer 155 outputs the amplified image data IDAT to the outside based on a column control signal (CTR_Y) of a column decoder (not shown) that selects a column under the control of the control logic 130 .
The buffer 155 may be implemented by, but is not limited to, a static random access memory (SRAM), a latch, a flip-flop, or a combination thereof.
In one embodiment, buffer 155 may be included within analog-to-digital converting circuitry 153 as a memory.

In one embodiment, the image sensor 100 supports an auto focusing (AF) function, and a phase difference AF technique is used for the auto focusing.
Phase difference AF is a method of adjusting focus by detecting disparity in the phase of an image formed on the image sensor 100 .
Phase difference AF has the characteristic that the phase difference becomes large in both the front focus and rear focus cases.
Phase-difference AF has an advantage in that focusing is possible by moving the focus lens only once since it is possible to know not only the phase difference value but also the pin direction information through the detected phase difference.
For example, the image sensor 100 calculates in advance a lens movement value based on the phase difference and the pin direction information, and drives the focus lens once with reference to the calculated value to perform focusing.
Therefore, the image sensor 100 using the phase difference AF allows for quick focusing without shaking of the electronic viewfinder.

In order to utilize phase-difference AF, the question arises as to how to arrange phase detection pixels in the pixel array 110 .
If too many phase detection pixels are arranged in the pixel array 110, the number of sensing pixels is reduced, causing degradation of resolution, and if too few phase detection pixels are arranged, accurate phase difference detection is impossible.
In addition, the image sensor 100 supports various imaging modes such as a live view mode, a still image mode, a video mode, a preview mode, and a high-resolution capture mode. In such various imaging modes, the positions of the sensing pixels for image generation may differ, and the imaging performance in each mode may differ depending on the arrangement of the phase detection pixels.
Therefore, there is a need for a phase detection pixel placement technique that can maximize the imaging performance of the image sensor 100.

FIG. 2 is a diagram illustrating a pixel array 110 according to an embodiment of the present invention.
Referring to FIG. 2, the pixel array 110 includes a number of pixel groups PG arranged in a repetitive manner.
The pixel group PG includes a red channel, a green channel, and a blue channel, and is a color expression unit that can reproduce an object OBJECT (FIG. 1) by a combination of red pixels, green pixels, and/or blue pixels.

In one embodiment, one pixel group PG also includes Bayer pattern color pixels including red, green and blue colors.
Each color pixel includes nine sub-pixels arranged in a 3×3 matrix, and each sub-pixel receives light that has passed through the same color filter.
A pixel group PG consisting of nine red sub-pixels R, nine green sub-pixels Gr, nine blue sub-pixels B, and nine green sub-pixels Gb (hereinafter, the green pixels Gr are referred to as the first green pixels, and the green pixels Gb are referred to as the second green pixels), each of which is arranged in a 3×3 matrix, is also referred to as a nona cell.
In the present invention, for convenience of explanation, nine sub-pixels (i.e., nona cells) arranged in a 3 x 3 matrix are illustrated as an example, but the technical idea of the present invention is not limited thereto, and it should be understood that the pixel group PG can be configured with a combination of various numbers (e.g., M x N, where M and N are natural numbers).

The pixel group PG includes a phase detection pixel P as one of a plurality of sub-pixels included in a color pixel.
According to an embodiment of the present invention, the phase detection pixels P may be evenly arranged (or arrayed) for each color channel (or color pixel).
The image sensor 100 (FIG. 1) calculates the distance to an object OBJECT based on the pixel signal difference (eg, disparity) of a pair of phase detection pixels P, thereby automatically detecting focus.
Therefore, the phase detection pixels P may be arranged in the pixel array 110 in left-right pairs or top-bottom pairs.
Although the present invention illustrates a phase detection pixel pair PHD arranged horizontally, the technical concept of the present invention may include phase detection pixel pairs of various geometrical structures, such as up-down or diagonal.

According to the technical idea of the present invention, each color channel of the pixel group PG includes the same number of phase detection pixels P.
For example, one pixel of the phase detection pixel pair PHD is located at the lower right edge of the first green pixel, and the other pixel of the phase detection pixel pair PHD is located at the lower left edge of the red pixel.
Similarly, one pixel of the other phase detection pixel pair is located at the bottom left edge of the blue pixel, and the other pixel of the phase detection pixel pair is located at the bottom right edge of the second green pixel.
The pixel groups PG are arranged in a repetitive manner, so that the phase detection pixel P located at the bottom left end of the blue pixel is included in another pixel group PG and abuts the phase detection pixel P located at the bottom right end of the second green pixel, forming a pair.

According to an embodiment of the present invention, a phase detection pixel P located at the first green pixel and a phase detection pixel P located at the red pixel constitute a phase detection pixel pair PHD.
For example, the phase detection pixel P arranged at the lower right end of the first green pixel senses a left side image of the object OBJECT, and the phase detection pixel P arranged at the lower left end of the red pixel senses a right side image of the object OBJECT.
Thereby, disparity according to the phase difference between the left image and the right image for the same object OBJECT is calculated.

The pixel groups PG are repeatedly arranged throughout the pixel array 110, so that the phase detection pixel P arranged at the bottom left end of the blue pixel forms a phase detection pixel pair PHD with the phase detection pixel of a pixel group (e.g., the second green pixel) located to the left of the pixel group PG, and the phase detection pixel P arranged at the bottom right end of the second green pixel forms a phase detection pixel pair PHD with the phase detection pixel of a pixel group (e.g., the blue pixel) located to the right of the pixel group PG.
For the sake of convenience of explanation, the present invention illustrates an embodiment of a Bayer pattern in which a red pixel is disposed to the right of a first green pixel, a blue pixel is disposed below the first green pixel, and a second green pixel is disposed diagonally from the first green pixel. However, the technical idea of the present invention is not limited thereto, and it will be understood that the positions of the red pixel, green pixel, and blue pixel may be interchanged, a white pixel may be disposed in place of one of the two green pixels, or the red pixel and the blue pixel may be combined with other color configurations such as a yellow pixel and a cyan pixel.

According to an embodiment of the present invention, pixel array 110 includes a phase detection pixel P evenly for each color pixel, so that cross-talk between the phase detection pixel P and adjacent pixels may be induced relatively consistently.
Since some crosstalk is eliminated without additional correction, phase detection performance and image quality can be improved.
The crosstalk induced between the phase detection pixel P and adjacent pixels is described in further detail with reference to Figures 7A-7C.

In addition, according to an embodiment of the present invention, the image sensor 100 may include a phase detection pixel P evenly in each color pixel of the pixel array 110, thereby minimizing the number of phase detection pixels P included in a color pixel.
In the color pixel, the number of sub-pixels sensing the object OBJECT excluding the phase detection pixel P is increased, so that the signal-to-noise ratio (SNR) of the image sensor 100 can be improved.

3A and 3B are diagrams illustrating a pixel group PG1 or PG2 according to an embodiment of the present invention.
3A and 3B, the terms pixel group (PG1, PG2), color pixel (CP1, CP2), and sub-pixel (SP1, SP2) used in the present invention will be defined in more detail.

Referring to FIG. 3A, a first pixel group PG1 includes color pixels of a Bayer pattern including red, green, and blue colors, and each color pixel includes four sub-pixels arranged in a 2×2 matrix.
A pixel group PG1 composed of four red sub-pixels (R1 to R4), four green sub-pixels (Gr1 to Gr4), four blue sub-pixels (B1 to B4), and four green sub-pixels (Gb1 to Gb4), each of which is arranged in a 2×2 matrix, is called a tetra cell.

The first pixel group PG1 includes two green pixels, one red pixel, and one blue pixel as color pixels.
For example, among the color pixels, the green pixel located to the left of the red pixel is the first color pixel CP1.
The first color pixel CP1 includes a plurality of sub-pixels having the same color information.
For example, the green pixel as the first color pixel CP1 includes four sub-pixels (Gr1, Gr2, Gr3, Gr4) arranged in a 2×2 matrix.
Among the sub-pixels, the sub-pixel Gr1 located at the upper left corner of the first color pixel CP1 is the first sub-pixel SP1.

Referring to FIG. 3B, the second pixel group PG2 includes color pixels of a Bayer pattern including red, green, and blue colors, and each color pixel includes nine sub-pixels arranged in a 3×3 matrix.
As described above, pixel group PG2, which is composed of nine red subpixels (R1 to R9), nine green subpixels (Gr1 to Gr9), nine blue subpixels (B1 to B9), and nine green subpixels (Gb1 to Gb9), each arranged in a 3 × 3 matrix, is a nona cell.

The second pixel group PG2 includes two green pixels, one red pixel, and one blue pixel as color pixels.
For example, the green pixel located to the left of the red pixel among the color pixels is the second color pixel CP2.
The second color pixel CP2 includes a plurality of sub-pixels having the same color information.
For example, the green pixel as the second color pixel CP2 includes nine sub-pixels Gr1 to Gr9 arranged in a 3×3 matrix.
Among the sub-pixels, the sub-pixel Gr1 located at the upper left corner of the second color pixel CP2 is the second sub-pixel SP2.

In Figures 3A and 3B, a tetracell or nonacell is illustrated as an example, and the definitions of pixel groups, color pixels, and subpixels (e.g., pixel groups (PG1, PG2), color pixels (CP1, CP2), and subpixels (SP1, SP2)) are described. However, it will be understood that the technical concept of the present invention can be equally applied to combinations of various numbers of subpixels contained in each color pixel (e.g., subpixels arranged in an M x N matrix).

FIG. 4 is a diagram for explaining data values of color pixels including sub-pixels.
In FIG. 4, a nonacell, which is a set of nine subpixels arranged in a 3×3 matrix, is shown as an example to explain the technical concept of the present invention, but as described above, the present invention can also be expanded to subpixels arranged in an M×N matrix.

Referring to FIG. 4, a pixel group PG includes color pixels composed of sub-pixels arranged in an M×N (=MN) matrix.
For example, the pixel group PG includes a first green pixel and a second green pixel, a red pixel, and a blue pixel, where the first green pixel includes (MN) green sub-pixels Gr, the second green pixel includes (MN) green sub-pixels Gb, the red pixel includes (MN) red sub-pixels R, and the blue pixel includes (MN) blue sub-pixels B.

According to an embodiment of the present invention, each color pixel (ie, the first and second green pixels, the red pixel, and the green pixel) includes the same number of phase detection pixels P.
By disposing the phase detection pixels P evenly in each color pixel, there is a possibility that crosstalk is induced relatively uniformly between each phase detection pixel P and its adjacent pixels.
For example, one phase detection pixel P is located at the bottom right corner of the first green pixel, the bottom left corner of the red pixel, the bottom left corner of the blue pixel, and the bottom right corner of the second green pixel, and each color channel includes the same number of phase detection pixels P.

In scenarios where a high resolution image is not required, where the illumination of the imaging environment is low and sufficient light cannot be secured, or where rapid image processing is required (e.g., preview images, etc.), the sensing signals or sensing data of multiple sub-pixels included in a color pixel are summed.
When analog signals generated by a plurality of light sensing elements are summed, this is also called analog summation, and when digital conversion results of the sensing signals are summed, this is also called digital summation.
Analog summing is described in further detail with reference to FIGS. 11 and 12, while digital summing is described in further detail in FIG.

According to one embodiment, signals or data generated in one color channel are summed into one color pixel information.
For example, each color pixel includes M×N (=MN) sub-pixels, one of which is a phase detection pixel P for disparity calculation, and the rest of which are sensing pixels for sensing an image.
Referring to the first green pixel according to an embodiment, in the (MN) sub-pixels, sensing pixels corresponding to the number of remaining sub-pixels (M×N−#P) excluding the number of phase detection pixels (#P) are summed.
The signal or data of the summed sensing pixel contains more information (eg, resolution, contrast, sensitivity, etc.) than the signals or data of each of the multiple sub-pixels.
The red pixel, blue pixel, and second green pixel are summed in a similar manner to produce more information than is contained in the individual sub-pixels.

5A, 5B, and 5C are vertical cross-sectional views that diagrammatically illustrate phase detection pixel pairs (PHDa, PHDb, PHDc) according to embodiments of the present invention.
5A to 5C, overlapping descriptions will be omitted.
Referring to FIG. 5A, the phase detection pixel pair PHDa includes a microlens, a photo-sensing element, and a color filter.
For example, each of the pixels (PX1, PX2) is an example of a light sensing element, and includes a photodiode (PD1, PD2) and a color filter (CF1, CF2), and a microlens (ML1, ML2) is provided on the upper part of the color filter (CF1, CF2).
The vertical cross-sectional shape of the microlens ML according to the embodiment can be an arc having a circular curvature or a portion of an ellipse.

According to the embodiment illustrated in FIG. 5A, one color filter CF1 or CF2, and one microlens ML1 or ML2 may be provided on one photodiode PD1 or photodiode PD2, respectively.
For example, light incident on the center of microlens ML1 passes through color filter CF1, and only light of a specific wavelength band (e.g., 500 to 600 nm corresponding to green color) is transmitted, and the transmitted light of the specific wavelength band is imaged on photodiode PD1.
Similarly, light incident on the center of the microlens ML2 passes through the color filter CF2, whereby only light of a specific wavelength is transmitted, and the transmitted light of the specific wavelength is imaged on the photodiode PD2.

As illustrated in FIG. 5A, when light incident on one microlens ML1 or microlens ML2 is focused on one photodiode PD1 or photodiode PD2, this is called a single photodiode (single-PD).
A plurality of pixels PX1 and PX2 form a phase detection pixel pair PHDa in pairs, and the image sensor 100 (FIG. 1) uses the phase detection pixel pair PHDa to calculate a disparity due to the phase difference between the photodiode PD1 and the photodiode PD2, thereby determining the distance to the object OBJECT and adjusting the focal length.

Referring to FIG. 5B, the phase detection pixel pair PHDb includes a microlens, a photo-sensing element, and a color filter.
For example, pixel PXx includes two color filters CFa and CFb, as well as photodiodes PDa and PDb corresponding to color filters CFa and CFb, respectively, and similarly, pixel PXy includes two color filters CFc and CFd, as well as photodiodes PDc and PDd corresponding to color filters CFc and CFd, respectively.

According to the embodiment illustrated in FIG. 5B, two color filters CF1 and CF2, and two photodiodes (PDa, PDb) are provided under one microlens MLx.
For example, a first light beam LFx, which is a portion of the light incident on the center of the microlens MLx, passes through a color filter CFa and is imaged on a photodiode PDa, and a second light beam LFy, which is the remaining portion of the light incident on the center of the microlens MLx, passes through a color filter CFb and is imaged on a photodiode PDb.
The same phenomenon as that of pixel PXx may occur in pixel PXy.
As illustrated in FIG. 5B, when light incident on one microlens MLx or microlens MLy is focused on two photodiodes PDa and PDb (or photodiodes PDc and PDd), this is called a dual photodiode (dual-PD).

Referring to FIG. 5C, the phase detection pixel pair PHDc includes a microlens, a photo-sensing element, a color filter, and a metal shield.
For example, pixel PXz includes one color filter CFz, one metal shield MSz, and one photodiode PDz; similarly, pixel PXw includes one color filter CFw, one metal shield MSw, and one photodiode PDw.
The metal shield contains metal as a component and blocks the progression or propagation of light.

According to the embodiment illustrated in FIG. 5C, a third light beam LFz, which is a portion of the light incident on the center of the microlens MLz, passes through the color filter CFz and is also imaged on the photodiode PDz, but the remaining portion of the light incident on the center of the microlens MLz has its optical path blocked by the metal shield MSz, and as a result, is not imaged on the photodiode PDz.
Similarly, a fourth light beam LFw, which is a portion of the light incident on the center of the microlens MLw, passes through the color filter CFw and is also imaged on the photodiode PDw, but the remaining portion of the light incident on the center of the microlens MLw is blocked from being imaged on the photodiode PDw by the metal shield MSw.

In the present invention, for the sake of convenience of explanation, one pixel (e.g., PXz) is illustrated as including one color filter (e.g., CFz) and one metal shield (e.g., MSz). However, due to the characteristics of the metal shield that blocks the passage of light, the metal shield may be provided on the top or bottom of the color filter that is provided sufficiently to be as wide as the microlens (e.g., MLz).
As shown in FIG. 5C, when a part of the light incident on one microlens MLz or MLw is blocked by a metal shield, the photodiode is called a metal shield photodiode (metal shield-PD).

Referring to FIG. 5C, because a portion of the light is blocked by the metal shield, the first light beam LFx (FIG. 5B) and the third light beam LFz are relatively similar for the same object OBJECT, and the disparity for the object OBJECT of phase detection pixel pair PHDb or phase detection pixel pair PHDc is similar.

6A-6C are plan views of the dual photodiode of FIG. 5B, and FIG. 6D shows a vertical cross-sectional view taken along line AA' of FIG. 6A.
Referring to FIG. 6A, a pixel PXx includes a microlens MLx and two sub-pixels, for example a first sub-pixel SPXa and a second sub-pixel SPXb.
The first sub-pixel SPXa and the second sub-pixel SPXb are arranged side by side in a column direction, for example, in the Y-axis direction (second direction).
For example, the first sub-pixel SPXa is disposed on the left side in the pixel PXx, and the second sub-pixel SPXb is disposed on the right side in the pixel PXx.
The first and second sub-pixels SPXa and SPXb include a first and second photodiodes PDa and PDb, respectively.

According to an embodiment, a sensing signal is generated in a first photodiode PDa and a second photodiode PDb.
For example, a first image signal is output from the first subpixel SPXa, and a second image signal is output from the second subpixel SPXb. Based on the first image signal and the second image signal, disparity is calculated by phase difference calculation, thereby adjusting the left-right direction of the focus.

Referring to FIG. 6B, the pixel PXy includes a microlens MLy and two sub-pixels, for example, a third sub-pixel SPXc and a fourth sub-pixel SPXd.
The third sub-pixel SPXc and the fourth sub-pixel SPXd are arranged side by side in the row direction, for example, the X-axis direction (first direction).
For example, the third sub-pixel SPXc is disposed at the upper side within the pixel PXy, and the fourth sub-pixel SPXd is disposed at the lower side within the pixel PXy.
The third sub-pixel SPXc and the fourth sub-pixel SPXd include a third photodiode PDc and a fourth photodiode PDd, respectively.
A third image signal is output from the third subpixel SPXc, and a fourth image signal is output from the fourth subpixel SPXd. Based on the third and fourth image signals, disparity is calculated by phase difference calculation, thereby adjusting the focus in the vertical direction.

Referring to FIG. 6C, the pixel PXxy includes a microlens MLxy and four sub-pixels, for example, a fifth sub-pixel SPXac, a sixth sub-pixel SPXbc, a seventh sub-pixel SPXad, and an eighth sub-pixel SPXbd.
The fifth subpixel SPXac is located at the upper left within pixel PXxy, the sixth subpixel SPXbc is located at the upper right within pixel PXxy, the seventh subpixel SPXad is located at the lower left within pixel PXxy, and the eighth subpixel SPXbd is located at the lower right within pixel PXxy.
In other words, the fifth subpixel SPXac and the sixth subpixel SPXbc, as well as the seventh subpixel SPXad and the eighth subpixel SPXbd, are arranged side by side in the row direction, e.g., the X-axis direction (first direction), and the fifth subpixel SPXac and the seventh subpixel SPXad, as well as the sixth subpixel SPXbc and the eighth subpixel SPXbd, are arranged side by side in the column direction, e.g., the Y-axis direction (second direction).

The fifth sub-pixel SPXac, the sixth sub-pixel SPXbc, the seventh sub-pixel SPXad, and the eighth sub-pixel SPXbd each include one photoelectric conversion element, for example, the fifth photodiode PDac, the sixth photodiode PDbc, the seventh photodiode PDad, and the eighth photodiode PDbd.
Comparing FIG. 6C with FIG. 6A and FIG. 6B, FIG. 6C calculates both the top-bottom and left-right disparity by including four photodiodes (PDac, PDbc, PDad, PDbd) arranged vertically and horizontally under one microlens MLxy.
A structure including four photodiodes (PDac, PDbc, PDad, PDbd) arranged vertically and horizontally under one microlens MLxy is called a quad-cell.

Referring to FIG. 6D, a pixel PXx includes a first layer L1 and a second layer L2 stacked in the Z-axis direction (eg, the third direction).
The first layer L1 is called a photoelectric conversion layer, and includes a color filter CF and a microlens ML formed on a substrate SUB, and two photoelectric conversion elements, for example, a first photodiode PDa and a second photodiode PDb, formed on the substrate SUB.
The second layer L2 is also called a wiring layer, and a plurality of wirings WS are formed in the second layer L2.
The substrate SUB is a silicon wafer, but also an SOI (silicon on insulator) substrate or a semiconductor epitaxial layer.
The substrate SUB includes a first surface Sf and a second surface Sb opposed to each other.
For example, the first surface Sf is the front surface of the substrate SUB, and the second surface Sb is the back surface of the substrate SUB.
Light is incident on the second surface Sb.

The substrate SUB is formed with a plurality of pixel isolation films (SEP1, SEP2) (e.g., deep trench isolation (DTI) or P-type ion implantation regions) extending from the second surface Sb of the substrate SUB toward the first surface Sf. Of the plurality of pixel isolation films (SEP1, SEP2), a pixel region APX in which a pixel PXx is formed is divided by a plurality of relatively long first pixel isolation films SEP1, and the pixel region APX is divided by a relatively short first pixel isolation film SEP1 into a first region A1 and a second region A2 in which a first sub-pixel SPXa and a second sub-pixel SPXb are formed.
In the embodiment, each of the first region A1 and the second region A2 is doped with impurities of a first conductivity type (eg, P type).
A first photodiode PDa and a second photodiode PDb are formed in the first region A1 and the second region A2, respectively.
For example, well regions doped with impurities of a second conductivity type (eg, N-type) are formed as a first photodiode PDa and a second photodiode PDb.

As shown in the figure, the first photodiode PDa and the second photodiode PDb are arranged side by side in a first direction (eg, X direction) or a second direction (eg, Y direction) centered on the optical axis MLXA of the microlens MLx.
A floating diffusion node FD is formed between the first photodiode PDa and the second photodiode PDb.
In one embodiment, transistors are formed between the first photodiode PDa and the first surface Sf, and signals are transmitted to and received from the transistors via a plurality of wirings WS of the wiring layer L2.
This is described in more detail with reference to Figures 11, 12A and 12B.

FIG. 7A is a diagram illustrating phase detection pixels that are not uniformly arranged for each color pixel, and FIGS. 7B and 7C are diagrams illustrating phase detection pixels that are uniformly arranged for each color pixel according to an embodiment of the present invention.

Referring to FIG. 7A, a pixel group PG includes four color channels (eg, a first green pixel, a second green pixel, a red pixel, and a blue pixel).
According to one embodiment, in the pixel group PG, a phase detection pixel pair capable of calculating the up/down disparity is disposed in the first green pixel Gr.
For example, in the nine green sub-pixels constituting the first green pixel Gr, the phase detection pixels P are disposed at the positions where the sixth green sub-pixel Gr6 and the ninth green sub-pixel Gr9 are disposed.
The phase detection pixel P may suffer from crosstalk induced from other adjacent sub-pixels.

For example, in the phase detection pixel pair, the phase detection pixel P arranged at the position of the sixth green subpixel Gr6 is adjacent to the other subpixels (Gr3, Gr5, R4), and in the phase detection pixel pair, the phase detection pixel P arranged at the position of the ninth green subpixel Gr9 is adjacent to the other subpixels (G8, B3, R7).
Therefore, unintended signals from photons accumulated in subpixels adjacent to the phase detection pixel P, or from sensing signals generated from the photons, may be input to the phase detection pixel P, which may act as noise in the phase detection function.

Although the pixel group PG has four color channels, when the phase detection pixel is disposed only in the first green pixel Gr, the crosstalk induction amount differs for each color channel.
For example, referring to FIG. 7A, the first green pixel Gr has three subpixels that induce crosstalk (subpixels (Gr3, Gr5, Gr8)), the red pixel R has two subpixels that induce crosstalk (subpixels (R4, R7)), the blue pixel B has one subpixel that induces crosstalk (subpixel B3), and the second green pixel Gb has very little or substantially no influence of the subpixels that induce crosstalk.
In Figure 7A, for the sake of convenience of explanation, an upper and lower phase detection pixel pair is illustrated, but it will be understood that, taking into account the symmetry of the pixel group PG, a left and right phase detection pixel pair also induces the same phenomenon.

Referring to FIG. 7B, in a pixel group PG, phase detection pixels P are uniformly arranged for each color channel.
According to an embodiment of the present invention, the first green pixel Gr, the red pixel R, the blue pixel B, and the second green pixel Gb each include one phase detection pixel P.
The phase detection pixels P are arranged at specific positions and in the same number for each color channel to make the amount of crosstalk generated from adjacent pixels constant.

According to an embodiment of the present invention, a phase detection pixel P included in one color channel is positioned taking into consideration the position of a phase detection pixel P included in another adjacent color channel.
For example, the phase detection pixel P included in the first green pixel Gr is disposed at the lower right edge of the first green pixel Gr, taking into consideration the position (lower left edge) of the phase detection pixel P of the adjacent red pixel R immediately to the right.
Although not shown in FIG. 7B, the phase detection pixel P included in the first green pixel Gr is also positioned at the lower end of the first green pixel Gr, taking into account the position of the phase detection pixel P of the adjacent blue pixel B immediately below, which will be described in further detail in FIG. 8A.

According to an embodiment of the present invention, a phase detection pixel pair is formed by combining the phase detection pixels P arranged for each color channel above and below or to the left and right.
For example, a phase detection pixel pair for adjusting the left-right focus is formed by a combination of a phase detection pixel P arranged at the lower right end of the first green pixel Gr and a phase detection pixel P arranged at the lower left end of the red pixel.
Although not shown in FIG. 7B, the phase detection pixel P included in the first green pixel Gr can adjust the upper and lower focus by forming an upper and lower phase detection pixel pair with the phase detection pixel P of the adjacent blue pixel B, which will be described in further detail in FIG. 8A.

For ease of explanation, in the present invention, the two phase detection pixels P constituting an upper/lower phase detection pixel pair or a left/right phase detection pixel pair are shown in the figures as being adjacent to each other, but the technical idea of the present invention is not limited thereto, and it will be understood that the two phase detection pixels P constituting an upper/lower phase detection pixel pair or a left/right phase detection pixel pair are separated by one or more sub-pixels.

According to an embodiment of the present invention, a predetermined number of phase detection pixels P are arranged at predetermined specific positions for each color pixel, so that the amount of crosstalk induction for each color channel is constant compared to the case where the phase detection pixels P are arranged non-uniformly for each color pixel.
For example, there are three subpixels (subpixels (Gr6, Gr8, B3)) that induce crosstalk in the phase detection pixel P located at the bottom right end of the first green pixel Gr, and there are three subpixels (subpixels (R4, R8, Gb1)) that induce crosstalk in the phase detection pixel P located at the bottom left end of the red pixel R.

In FIG. 7B, the figure shows that there are two subpixels (subpixels (B4, B8)) that induce crosstalk in the phase detection pixel P located at the bottom left corner of the blue pixel B within a single pixel group PG. However, referring to FIG. 7C as well, the subpixel Gr1 adjacent to the bottom of the blue pixel B and located at the top left corner of the first green pixel Gr of another pixel group may induce crosstalk in the phase detection pixel P of the blue pixel.
Similarly, within a single pixel group PG, the figure shows that there are two subpixels (subpixels (Gb6, Gb8)) that induce crosstalk to the phase detection pixel P located at the bottom right end of the second green pixel Gb, but referring to FIG. 7C, subpixel R3 located at the top right end of the red pixel R of another pixel group may also induce crosstalk to the phase detection pixel P of the second green pixel Gb.

Referring to FIG. 7C together with FIG. 7B, the phase detection pixel P located at the lower left end of the blue pixel is paired with the phase detection pixel P located at the lower right end of the second green pixel Gb included in another pixel group to form a phase detection pixel pair that adjusts the left-right focus, and the phase detection pixel P located at the lower right end of the second green pixel Gb is paired with the phase detection pixel P located at the lower left end of the blue pixel included in another pixel group to form a phase detection pixel pair that adjusts the left-right focus.
For each color channel, the phase detection pixel P arranged at a specific position determined in advance is also arranged at the same position in other pixel groups PG.
For example, the phase detection pixel P located at the bottom right end of the first green pixel Gr is also positioned at the bottom right end of the first green pixel Gr for the other pixel groups PG, and the phase detection pixel P located at the bottom left end of the red pixel R is also positioned at the bottom left end of the red pixel R for the other pixel groups PG.

Referring also to FIG. 7B, the first green pixel Gr has three subpixels that induce crosstalk (subpixels (Gr6, Gr8, Gr1)), the red pixel R has three subpixels that induce crosstalk (subpixels (R4, R8, R3)), the blue pixel B has three subpixels that induce crosstalk (subpixels (B4, B8, B3)), and the second green pixel Gb has three subpixels that induce crosstalk (subpixels (Gb6, Gb8, Gb1)).

According to an embodiment of the present invention, a predetermined number of phase detection pixels P are arranged at specific positions for each color channel, so that the amount of crosstalk induction is substantially the same for each color channel, and constant crosstalk can be easily removed without additional correction.
Therefore, no separate hardware for correction or digital logic implemented by a combination of hardware and software is required.
In addition, according to the technical idea of the present invention, the phase detection pixels P are uniformly arranged at specific positions for each color channel, thereby minimizing the number of phase detection pixels P included in the color channels, thereby improving the signal-to-noise ratio (SNR) of the sensing signal.

8A-8F are diagrams illustrating various examples of uniformly spaced phase detection pixels P according to embodiments of the present invention.
8A-8F, a nonacell is described in which a color pixel includes sub-pixels arranged in a 3×3 matrix.
Moreover, in FIG. 8A to FIG. 8F, repeated explanations will be omitted to the extent that they do not overlap with each other.

Referring to FIG. 8A, the phase detection pixels P included in the pixel group are arranged one for each color channel, and the phase detection pixels P included in the color channels adjacent to each other vertically form an upper and lower phase detection pixel pair.
For example, the phase detection pixel P included in the first green pixel Gr forms an upper and lower phase detection pixel pair (e.g., phase detection pixel pair PHD (FIG. 2)) with the phase detection pixel P of the blue pixel B adjacent to the first green pixel Gr immediately below.
Similarly, the phase detection pixel P included in the red pixel R constitutes an upper and lower phase detection pixel pair with the phase detection pixel P included in the second green pixel Gb in another pixel group immediately adjacent to the upper side of the red pixel R.
The upper and lower phase detection pixel pairs provide upper and lower disparities for the object OBJECT, and the image sensor 100 (FIG. 1) adjusts the upper and lower focus for the object OBJECT based on the calculated disparity.

Referring to FIG. 8B, the phase detection pixel P is located at a middle side location rather than at a corner side of the color channel (eg, FIGS. 7A, 7B, and 8A).
For example, the phase detection pixel P included in the first green pixel Gr constitutes a left-right phase detection pixel pair (e.g., a phase detection pixel pair PHD (FIG. 2)) together with the phase detection pixel P of the red pixel R adjacent to the first green pixel Gr on the immediate right side.
Similarly, the phase detection pixel P included in the blue pixel B forms a left-right phase detection pixel pair with the phase detection pixel P included in the second green pixel Gb in another pixel group adjacent to the left side of the blue pixel B, and the phase detection pixel P included in the second green pixel Gb forms a left-right phase detection pixel pair with the phase detection pixel P included in the blue pixel B in another pixel group adjacent to the right side of the second green pixel Gb.
Thereby, the left and right focus for the object OBJECT is adjusted.

Referring to FIG. 8C together with FIG. 5C, the phase detection pixel P includes a metal shield MSz or a metal shield MSw (FIG. 5C).
As described above, the metal shield MSz or the metal shield MSw (FIG. 5C) is an element containing a metal component and is capable of blocking the progression or propagation of light.
The metal shield MSz or metal shield MSw (FIG. 5C) blocks the path of part of the light incident on the center of the microlens MLz or microlens MLw (FIG. 5C), thereby suppressing imaging onto the photodiode PDz or photodiode PDw (FIG. 5C) at the blocked position.

According to an embodiment of the present invention, the phase detection pixel P included in the first green pixel Gr and the phase detection pixel P included in the blue pixel B form an upper and lower phase detection pixel pair, the phase detection pixel P included in the red pixel R forms an upper and lower phase detection pixel pair with the phase detection pixel P included in the second green pixel Gb in another pixel group, and the phase detection pixel P included in the second green pixel Gb forms an upper and lower phase detection pixel pair with the phase detection pixel P included in the red pixel R in another pixel group.
Of the two phase detection pixels P that constitute an upper and lower phase detection pixel pair, the phase detection pixel P located on the upper side (top) includes, for example, a color filter CFz (Figure 5C) on the upper side and a metal shield MSz (Figure 5C) on the lower side, and the phase detection pixel P located on the lower side (bottom) includes, for example, a color filter CFw (Figure 5C) on the lower side and a metal shield MSw (Figure 5C) on the upper side, thereby detecting upper and lower disparities.

8D, the phase detection pixels P including the metal shield MSz or the metal shield MSw (FIG. 5C) form a left-right phase detection pixel pair.
Although the metal-shield photodiode type phase detection pixel P shown in FIG. 8D is similar to the arrangement of the phase detection pixel P described in FIG. 8B, it is an embodiment in which the metal shield MSz or the metal shield MSw described in FIG. 8C is applied, so that a duplicated description will be omitted.
The phase detection pixel P located on the left side of the two phase detection pixels P that constitute a left-right phase detection pixel pair includes, for example, a color filter CFz (Figure 5C) on the left side and a metal shield MSz (Figure 5C) on the right side, and the phase detection pixel P located on the right side includes, for example, a color filter CFw (Figure 5C) on the right side and a metal shield MSw (Figure 5C) on the left side, thereby detecting left-right disparity.

Referring to FIG. 8E, the phase detection pixels P included in each color channel are adjacent to each other.
For example, the first green pixel Gr includes a phase detection pixel P at its lower right end, the red pixel R includes a phase detection pixel P at its lower left end, the blue pixel B includes a phase detection pixel P at its upper right end, and the second green pixel Gb includes a phase detection pixel P at its upper left end, and as a result, the phase detection pixels P included in one pixel group are all adjacent to each other.
Four adjacent phase detection pixels P can calculate both left/right disparity and top/bottom disparity, and the image sensor 100 (FIG. 1) can adjust both the left/right focus and the top/bottom focus with respect to the object OBJECT.

Referring to FIG. 8F, the phase detection pixels P included in the pixel group are arranged one or more for each color channel.
For example, each of the first green pixel Gr, the red pixel R, the blue pixel B, and the second green pixel Gb includes two phase detection pixels P.
The arrangement of the phase detection pixels P shown in Figure 8F is a pattern that includes two phase detection pixels P in each of the left and right directions compared to the arrangement of the phase detection pixels P shown in Figure 8E, but the technical idea of the present invention is not limited thereto.

According to an embodiment of the present invention, the position of the phase detection pixel P included in one color filter is symmetrical with respect to the phase detection pixel P included in the other color filter.
For example, referring to FIG. 8A , the phase detection pixel P included in the second green pixel Gb is symmetrical to the phase detection pixel P included in the red pixel R with respect to a horizontal line that crosses the color filter, the phase detection pixel P included in the first green pixel Gr is symmetrical to the phase detection pixel P included in the blue pixel with respect to a horizontal line that crosses the pixel group (so-called X-axis symmetry), and the phase detection pixel P included in the first green pixel Gr is positioned at the same position as the phase detection pixel P included in the second green pixel Gb.

For example, referring to FIG. 8E, the phase detection pixel P included in the first green pixel Gr is symmetrical to the phase detection pixel P included in the red pixel R, the phase detection pixel P included in the blue pixel, and the phase detection pixel P included in the second green pixel Gb based on the horizontal and vertical lines that cross the pixel group (so-called origin symmetry).
Furthermore, the technical idea of the present invention may be implemented using various geometrically symmetrical structures that make the amount of crosstalk induced for each color channel constant, other than the embodiments shown in FIGS. 8A to 8F.
The upper and lower phase detection pixel pairs or the left and right phase detection pixel pairs shown in Figures 8A to 8D are referred to as a 1x2 array or a 2x1 array microlens, Figure 8E is also referred to as a 2x2 matrix microlens, and Figure 8F is referred to as a 2x4 matrix microlens.

9A-9F are diagrams illustrating various examples of uniformly spaced phase detection pixels P according to embodiments of the present invention.
9A-9F, a hexadeca-cell is described in which the color pixel includes sub-pixels arranged in a 4×4 matrix.
Moreover, in Figures 9A to 9F, repeated explanations will be omitted to the extent that they do not overlap with each other.

Referring to FIG. 9A, the phase detection pixels P included in the pixel group are arranged one for each color channel, and the phase detection pixels P included in the color channels adjacent to each other on the left and right form a left and right phase detection pixel pair.
For example, the phase detection pixel P included in the first green pixel Gr forms a left-right phase detection pixel pair with the phase detection pixel P included in the red pixel R, the phase detection pixel P included in the blue pixel B forms a left-right phase detection pixel pair with the phase detection pixel P included in the second green pixel Gb in another pixel group adjacent to the left side of the blue pixel B, and the phase detection pixel P included in the second green pixel Gb forms a left-right phase detection pixel pair with the phase detection pixel P included in the blue pixel B in another pixel group adjacent to the right side of the phase detection pixel P included in the second green pixel Gb.

Referring to FIG. 9B, phase detection pixels P included in vertically adjacent color channels form an upper and lower phase detection pixel pair.
For example, the phase detection pixel P included in the first green pixel Gr constitutes an upper and lower phase detection pixel pair with the phase detection pixel P of the blue pixel B adjacent to the first green pixel Gr immediately below.
Similarly, the phase detection pixel P included in the red pixel R forms an upper and lower phase detection pixel pair with the phase detection pixel P included in the second green pixel Gb in another pixel group adjacent to the upper side of the red pixel R, and the phase detection pixel P included in the second green pixel Gb forms an upper and lower phase detection pixel pair with the phase detection pixel P included in the red pixel R in another pixel group immediately adjacent to the lower side of the second green pixel Gb.

9C in conjunction with FIGS. 5C and 8D, the phase detection pixel P includes a metal shield MSz or a metal shield MSw (FIG. 5C).
According to an embodiment of the present invention, the phase detection pixel P included in the first green pixel Gr and the phase detection pixel P included in the red pixel R form a left-right phase detection pixel pair, the phase detection pixel P included in the blue pixel B forms a left-right phase detection pixel pair with the phase detection pixel P included in the second green pixel Gb in a pixel group adjacent to the left side of the blue pixel B, and the phase detection pixel P included in the second green pixel Gb forms a left-right phase detection pixel pair with the phase detection pixel P included in the blue pixel B in another pixel group immediately adjacent to the right side of the second green pixel Gb.

Referring to FIG. 9D, at least two phase detection pixels P are arranged for each color channel.
According to an embodiment of the present invention, the phase detection pixel P is placed in the middle of a color channel, rather than at a corner (e.g., FIGS. 7A, 7B, and 9A) or a side (e.g., FIG. 8B) of the color channel.
For example, the two phase detection pixels P included in the first green pixel Gr are located inside the first green pixel Gr rather than on the frame (or edge) of the first green pixel Gr, and the two phase detection pixels P are arranged above and below each other, thereby forming an upper and lower phase detection pixel pair.

According to an embodiment of the present invention, each color channel of a pixel group has the same arrangement pattern with the phase detection pixel pairs arranged in any one of the color channels.
For example, the red pixel R, the blue pixel B, and the second green pixel Gb each include two phase detection pixels P, and have the same arrangement pattern as the two phase detection pixels P arranged in the first green pixel Gr.
By uniformly disposing the phase detection pixels P for each color channel, the amount of crosstalk induced from adjacent pixels is made constant for each color channel, and the constant crosstalk for each color channel can be easily eliminated.

Referring to FIG. 9E, each color channel includes two or more phase detection pixels P, and the phase detection pixels P of two different color channels are arranged adjacent to each other.
For example, the first green pixel Gr includes two phase detection pixels P arranged consecutively in the vertical direction at the bottom right edge, and the first red pixel R includes two phase detection pixels P arranged consecutively in the vertical direction at the bottom left edge.
Four adjacent phase detection pixels P included in the first green pixel Gr and the first red pixel R constitute an upper/lower phase detection pixel pair and a left/right phase detection pixel pair.
Similarly, the blue pixel B includes two phase detection pixels P arranged consecutively in the vertical direction at its lower left end, and the second green pixel Gb includes two phase detection pixels P arranged consecutively in the vertical direction at its lower right end, so that the four adjacent phase detection pixels P form an upper-lower phase detection pixel pair and a left-right phase detection pixel pair.

Referring to FIG. 9F, each color channel includes two or more phase detection pixels P, and the phase detection pixels P of the four different color channels are arranged adjacent to each other.
For example, the first green pixel Gr includes a phase detection pixel P at the upper left and lower right ends, the red pixel R includes a phase detection pixel P at the lower left and upper right ends, the blue pixel B includes a phase detection pixel P at the upper right and lower left ends, and the second green pixel Gb includes a phase detection pixel P at the upper left and lower right ends, and as a result, the phase detection pixels P included in one pixel group are all adjacent to each other.

FIG. 10 is a diagram illustrating different data outputs for each mode according to an embodiment of the present invention.
In FIG. 10 , the technical concept of the present invention will be described using a tetracell, which is a set of four subpixels arranged in a 2×2 matrix, as an example. However, as described above, the present invention can also be expanded to a nonacell, a hexadecacell, or subpixels arranged in an M×N matrix.

Referring to FIG. 10, the pixel array 110a or the pixel array 110b outputs data differently depending on the mode.
The image sensor 100 (FIG. 1) includes a first mode MD1 or a second mode MD2 as a mode for capturing an image of an object OBJECT, and further includes various other modes according to an imaging environment, an imaging setting, or an imaging scenario.
According to one embodiment, the mode signal MD is provided to the control logic 130 (FIG. 1), which controls the image sensor 100 to sense the object OBJECT in a first mode MD1 or a second mode MD2.

According to an embodiment of the present invention, the first mode MD1 is an imaging mode corresponding to a scenario in which a high-resolution image is required, the illuminance of the imaging environment is high and sufficient light can be secured, or accurate image processing is required (e.g., capture video, etc.), and the second mode MD2 is an imaging mode corresponding to a scenario in which a high-resolution image is not required, the illuminance of the imaging environment is low and sufficient light cannot be secured, or rapid image processing is required (e.g., preview video, etc.).

According to an embodiment, the pixel array 110a generates paired sensing signals or sensing data for each of a plurality of sub-pixels included in a color pixel in a first mode MD1.
For example, the pixel array 110a includes a first green pixel Gr, a second green pixel Gb, a red pixel R, and a blue pixel B, and each color pixel includes four sub-pixels.
A plurality of sub-pixels included in each color channel generate sensing signals or sensing data as a result of sensing an object OBJECT.
For example, each of the four sub-pixels (Gr1, Gr2, Gr3, Gr4) included in the first green pixel Gr generates a sensing signal or sensing data having a data expression depth (e.g., resolution or data depth) of the corresponding sub-pixel as a maximum bandwidth, and outputs the sensing signal or sensing data to the readout circuit 150 (FIG. 1).

According to an embodiment, the pixel array 110b sums and outputs sensing signals or sensing data related to each of a plurality of sub-pixels included in a color pixel in the second mode MD2.
For example, the pixel array 110b sums up the sensing signals or sensing data generated in each of the four sub-pixels (Gr1, Gr2, Gr3, Gr4) included in the first green pixel Gr, and then outputs the sum to the readout circuit 150 (FIG. 1).
For example, the sum Gr' of the sensing signals or sensing data of the first green pixel Gr corresponds to the sum (Gr1'+Gr2'+Gr3'+Gr4') of the sensing signals or sensing data of each of the four sub-pixels (Gr1, Gr2, Gr3, Gr4) included in the first green pixel Gr (Gr=Gr1+Gr2+Gr3+Gr4).

Similarly, the sum Gb' of the sensing signals or sensing data of the second green pixel Gb corresponds to the sum (Gb1'+Gb2'+Gb3'+Gb4') of the sensing signals or sensing data of the four sub-pixels (Gb1, Gb2, Gb3, Gb4) included in the first green pixel Gb, the sum R' of the sensing signals or sensing data of the red pixel R corresponds to the sum (R1'+R2'+R3'+R4') of the sensing signals or sensing data of the four sub-pixels (R1, R2, R3, R4) included in the first red pixel R, and the sum B' of the sensing signals or sensing data of the blue pixel B corresponds to the sum (B1'+B2'+B3'+B4') of the sensing signals or sensing data of the four sub-pixels (B1, B2, B3, B4) included in the first blue pixel B.

According to an embodiment of the present invention, when the pixel array 110a or the pixel array 110b includes a phase detection pixel P (FIG. 4), only the remaining sensing pixels excluding the phase detection pixel P among the sub-pixels present in the color channels are summed.
The signal or data of the summed sensing pixel may contain more information (eg, resolution, contrast, sensitivity, etc.) than the signals or data of each of the multiple sub-pixels.
That is, according to the second mode MD2, even if a sufficient amount of light is not secured, the brightness is increased and the noise is reduced compared to the sensing results of each individual sub-pixel, thereby improving the image quality of the captured image.

FIG. 11 is an equivalent circuit diagram of the pixel PX.
Referring to FIG. 11, a pixel PX includes a photoelectric conversion element and a number of transistors, for example, a transmission transistor TX, a reset transistor RX, a driving transistor DX, and a selection transistor SX.
The photoelectric conversion element is, for example, a photodiode PD.
The photoelectric conversion element is at least one of a phototransistor, a photogate, a pinned photodiode (PPD), and a combination thereof.

The photodiode PD is a PN junction diode that generates electric charges, i.e., negatively charged electrons and positively charged holes, in proportion to the amount of incident light, generating photocharges that vary depending on the intensity of the incident light.
The transfer transistor TX transfers the photocharges to the floating diffusion node FD in response to a transfer control signal TG provided by the row decoder 120 (FIG. 1).
The floating diffusion node FD (or the floating diffusion region) is modeled by a capacitor _CH for storing photocharges.
Due to the potential due to the photocharges stored in the floating diffusion node FD, the drive transistor DX amplifies the photocharges and transmits them to the selection transistor SX.

The drive transistor DX operates as a source follower.
The drive transistor DX receives a signal based on the charge amount of the floating diffusion node FD, in other words, the potential of the floating diffusion node FD, at its gate terminal, and buffers and outputs the received signal.
The selection transistor SX is turned on in response to a selection signal SEL applied to the gate terminal.
The drain node of the selection transistor SX is connected to the source node of the driving transistor DX, and when the selection transistor SX is turned on in response to a selection signal SEL output from the row decoder 120, a pixel signal VOUT having a level corresponding to the voltage level of the floating diffusion node FD is output to the column line CL connected to the pixel PX.

The reset transistor RX resets the floating diffusion node FD based on the power supply voltage VDD in response to a reset signal RS provided from the row decoder 120 .
The reset transistor RX periodically resets the charge stored in the floating diffusion node FD.
The source electrode of the reset transistor RX is connected to the floating diffusion node FD, and the drain electrode of the reset transistor RX is connected to the power supply voltage VDD.
When the reset transistor RX is turned on in response to a reset control signal RS applied to the gate terminal, the power supply voltage VDD connected to the drain electrode of the reset transistor RX is transferred to the floating diffusion node FD.
When the reset transistor RX is turned on, the charge stored in the floating diffusion node FD is discharged, and the floating diffusion node FD is reset.

12A and 12B are circuit diagrams of a pixel PX that sums sensing signals according to an embodiment of the present invention.
To the extent that it does not conflict with FIG. 11, a duplicated description will be omitted.
FIG. 12A shows, for example, a case where pixel PXx is formed by adding up two sub-pixels SPXa and SPXb (FIG. 6A), and FIG. 12B shows, for example, a case where pixel PXxy is formed by adding up four sub-pixels (SPXac, SPXbc, SPXad, SPXbd) (FIG. 6C).

12A, a pixel PXa includes a first photodiode PD1a, a second photodiode PD2a, a first transfer transistor TX1a, a second transfer transistor TX2a, a reset transistor RXa, a drive transistor DXa, and a selection transistor SXa.
The floating diffusion node FDa is shared by the first and second photodiodes PD1a and PD2a, and the first and second transfer transistors TX1a and TX2a.
The first photodiode PD1a and the first transfer transistor TX1a are referred to as a first sub-pixel SPXa (FIG. 6A), and the second photodiode PD2a and the second transfer transistor TX2a are referred to as a second sub-pixel SPXb (FIG. 6A).

Each of the first photodiode PD1a and the second photodiode PD2a generates photocharges that vary depending on the intensity of light.
When the first transmission transistor TX1a is turned on in response to a first transmission control signal TG1a applied to the gate terminal, charges (e.g., photocharges) generated in the first photodiode PD1a are transferred to and stored in the floating diffusion node FDa.
When the second transfer transistor TX2a is turned on in response to a second transfer control signal TG2a applied to the gate terminal, the charges generated in the first photodiode PD1a are transferred to and stored in the floating diffusion node FDa.
The charge stored in the floating diffusion node FDa is output as an output voltage VOUTa.

The first transmission control signal TG1a and the second transmission control signal TG2a are separate signals, and therefore the turn-on times of the first transmission transistor TX1a and the second transmission transistor TX2a are independently controlled by the first transmission control signal TG1a and the second transmission control signal TG2a, respectively.
According to one embodiment, in the first mode MD1, the first transmission control signal TG1a and the second transmission signal TG2a are applied at different time periods, and the first transmission transistor TX1a and the second transmission transistor TX2a are turned on at different time periods, thereby storing respective photocharges in the floating diffusion node FDa.
After any one of the first transmission control signal TG1a or the second transmission signal TG2a is applied, a reset signal RSa for resetting the floating diffusion node FDa is applied, and then the other signal of the first transmission control signal TG1a or the second transmission signal TG2a is applied.
According to one embodiment, in the second mode MD2, the first transmission control signal TG1a and the second transmission signal TG2a are applied at substantially the same time, and the first transmission transistor TX1a and the second transmission transistor TX2a are turned on at substantially the same time, thereby storing their respective photocharges together in the floating diffusion node FDa.

Referring to FIG. 12B, pixel PXb includes a plurality of photoelectric conversion elements (e.g., four photodiodes, a first photodiode PD1b to a fourth photodiode PD4b), a first transmission transistor TX1b to a fourth transmission transistor TX4b, a reset transistor RXb, a driving transistor DXb, and a selection transistor SXb.
A microlens is disposed above each of the first photodiode PD1b to the fourth photodiode PD4b.
Therefore, a combination of a microlens and a photoelectric conversion element is referred to as one pixel, so that the pixel PXb in FIG. 12B can also be considered as a combination of four sub-pixels PXxy (FIG. 6C).

The floating diffusion node FDb is shared by four photoelectric conversion elements (for example, a first photodiode PD1b to a fourth photodiode PD4b) and four transfer transistors (TX1b, TX2b, TX3b, TX4b).
The transmission transistors (TX1b, TX2b, TX3b, TX4b) connect or cut off each of the four photoelectric conversion elements (e.g., the first photodiode PD1b to the fourth photodiode PD4b) to the floating diffusion node FDb depending on the voltage of the transmission control signal (TG1b, TG2b, TG3b, TG4b).

Light incident on the photoelectric conversion elements (for example, the first photodiode PD1b to the fourth photodiode PD4b) is converted into electric charges and stored.
When the charges stored in the first to fourth photodiodes PD1b to PD4b are transferred to the floating diffusion node FDb, they are output to the outside as an output voltage VOUTb via the driving transistor DXb and the selection transistor SXb.
An output voltage VOUTb corresponding to a voltage change at the floating diffusion node FDb is transmitted to, for example, an external readout circuit 150 (FIG. 1).

FIG. 13 is a block diagram showing a schematic configuration of an electronic device including a multi-camera module to which an image sensor according to an embodiment of the present invention is applied, and FIG. 14 is a block diagram showing a detailed configuration of the multi-camera module of FIG. 13.

Referring to FIG. 13, an electronic device 1000 includes a camera module group 1100, an application processor 1200, a power management integrated circuit (PMIC) 1300, and an external memory 1400.
The camera module group 1100 includes multiple camera modules (1100a, 1100b, 1100c).
For example, although the figures show an embodiment in which three camera modules (1100a, 1100b, 1100c) are arranged, the embodiment is not so limited.
In some embodiments, camera module group 1100 may be implemented in a modified form to include only two camera modules.
Also, in some embodiments, the camera module group 1100 may be modified to include n camera modules (n is a natural number equal to or greater than 4).

The detailed configuration of the camera module 1100b will be described in more detail below with reference to FIG. 14, but the following description also applies equally to the other camera modules (1100a, 1100c) according to this embodiment.
Referring to FIG. 14, a camera module 1100 b includes a prism 1105 , an optical path folding element (OPFE) 1110 , an actuator 1130 , an image sensing device 1140 , and a storage 1150 .

The prism 1105 includes a reflecting surface 1107 made of a light reflecting material, and changes the path of light L incident from the outside.
In some embodiments, the prism 1105 redirects light L incident in a first direction X to a second direction Y that is perpendicular to the first direction X.
In addition, the prism 1105 rotates the reflecting surface 1107 of the light-reflecting material around the central axis 1106 in the direction A, or rotates the central axis 1106 in the direction B, thereby changing the path of the light L incident in the first direction X to a second direction Y that is perpendicular to the first direction X.
At this time, the OPFE 1110 also moves in a third direction Z perpendicular to the first direction X and the second direction Y.

In some embodiments, as shown in the figures, the maximum rotation angle of prism 1105 in the A direction is less than or equal to 15° in the positive (+) A direction and greater than 15° in the negative (-) A direction, although the embodiments are not so limited.
In some embodiments, prism 1105 can move around 20° in the plus (+) B or minus (-) B direction, between 10° and 20°, or between 15° and 20°, where the angle of movement can be the same angle in the plus (+) B or minus (-) B direction, or can move to a similar angle, within a range of around 1°.

In some embodiments, the prism 1105 moves the reflective surface 1107 of the light-reflective material in a third direction (eg, the Z direction) parallel to the extension direction of the central axis 1106 .
The OPFE 1110 includes, for example, m groups of optical lenses (where m is a natural number).
The m lenses move in a second direction Y to change the optical zoom ratio of the camera module 1100b.
For example, when the basic optical zoom magnification of the camera module 1100b is Z, when m optical lenses included in the OPFE 1110 are moved, the optical zoom magnification of the camera module 1100b is changed to an optical zoom magnification of 3Z or 5Z, or greater than 5Z.

The actuator 1130 moves the OPFE 1110 or an optical lens (hereinafter referred to as the optical lens) to a specific position.
For example, the actuator 1130 adjusts the position of the optical lens so that the image sensor 1142 is located at the focal length of the optical lens for accurate sensing.
The image sensing device 1140 includes an image sensor 1142 , control logic 1144 , and a memory 1146 .
The image sensor 1142 senses an image of a sensing target using light L provided through an optical lens.

Control logic 1144 controls the overall operation of camera module 1100b.
For example, control logic 1144 controls the operation of camera module 1100b via control signals provided over control signal line CSLb.
Memory 1146 stores information necessary for the operation of camera module 1100b, such as calibration data 1147.
The calibration data 1147 includes information necessary for the camera module 1100b to utilize the externally provided light L to generate image data.
Calibration data 1147 may include, for example, information regarding the degree of rotation, information regarding the focal length, information regarding the optical axis, and the like.
If the camera module 1100b is implemented as a multi-state camera in which the focal length changes depending on the position of the optical lens, the calibration data 1147 may include focal length values for each position (or state) of the optical lens and information regarding autofocusing.

The storage 1150 stores image data sensed via the image sensor 1142 .
The storage 1150 is disposed outside the image sensing device 1140 and is implemented in a stacked form with a sensor chip constituting the image sensing device 1140 .
In some embodiments, storage 1150 is embodied by an electrically erasable programmable read only memory (EEPROM), although the present embodiments are not limited thereto.

13 and 14, in some embodiments, each of the multiple camera modules (1100a, 1100b, 1100c) includes an actuator 1130.
Thereby, each of the multiple camera modules (1100a, 1100b, 1100c) includes calibration data 1147 that may be identical or different from one another due to the operation of the actuator 1130 contained therein.
In some embodiments, among the multiple camera modules (1100a, 1100b, 1100c), for example, one camera module 1100b is a folded lens type camera module including the above-mentioned prism 1105 and OPFE 1110, and for example, the remaining camera modules (1100a, 1100c) are vertical type camera modules that do not include the prism 1105 and OPFE 1110, but this embodiment is not limited thereto.

In some embodiments, one of the camera modules (1100a, 1100b, 1100c), for example the camera module 1100c, is a vertical depth camera that uses, for example, infrared (IR) to extract depth information.
In that case, the application processor 1200 merges image data provided from such a depth camera with image data provided from a different camera module, for example, camera module 1100a or 1100b, to generate a three-dimensional depth image.

In some embodiments, at least two camera modules of the plurality of camera modules (1100a, 1100b, 1100c), for example camera modules (1100a, 1100b), have different fields of view.
In that case, for example, the optical lenses of at least two of the multiple camera modules (1100a, 1100b, 1100c), for example camera modules (1100a, 1100b), are different from each other, but are not limited thereto.
Also, in some embodiments, the viewing angles of each of the multiple camera modules (1100a, 1100b, 1100c) may be different from each other.
In this case, the optical lenses included in each of the multiple camera modules (1100a, 1100b, 1100c) may be different from each other, but are not limited thereto.

In some embodiments, each of the multiple camera modules (1100a, 1100b, 1100c) is positioned physically separate from one another.
In other words, rather than dividing the sensing area of a single image sensor 1142 among multiple camera modules (1100a, 1100b, 1100c), an independent image sensor 1142 is disposed inside each of the multiple camera modules (1100a, 1100b, 1100c).

Referring again to FIG. 13, the application processor 1200 includes an image processing unit 1210, a memory controller 1220, and an internal memory 1230.
The application processor 1200 is implemented separately from the multiple camera modules 1100a, 1100b, and 1100c.
For example, the application processor 1200 and the camera modules 1100a, 1100b, and 1100c may be implemented separately from each other on separate semiconductor chips.
The image processing device 1210 includes a number of sub-image processors ( 1212 a , 1212 b , 1212 c ), an image generator 1214 , and a camera module controller 1216 .

The image processing device 1210 includes a number of sub-image processors (1212a, 1212b, 1212c) corresponding to the number of camera modules (1100a, 1100b, 1100c).
Image data generated from each camera module (1100a, 1100b, 1100c) is provided to a corresponding sub-image processor (1212a, 1212b, 1212c) via mutually separated image signal lines (ISLa, ISLb, ISLc).

For example, image data generated from camera module 1100a is provided to sub-image processor 1212a via image signal line ISLa, image data generated from camera module 1100b is provided to sub-image processor 1212b via image signal line ISLb, and image data generated from camera module 1100c is provided to sub-image processor 1212c via image signal line ISLc.
Such image data transmission is performed, for example, using a camera serial interface (CSI) based on the mobile industry processor interface (MIPI), but the present embodiment is not limited thereto.

In some embodiments, one sub-image processor is arranged to correspond to a plurality of camera modules.
For example, sub-image processor 1212a and sub-image processor 1212c are not embodied separately from each other as shown in the figure, but are integrated into a single sub-image processor, and image data provided from camera module 1100a and camera module 1100c is selected via a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor.
The image data provided to each of the sub-image processors (1212a, 1212b, 1212c) is provided to an image generator 1214.
The image generator 1214 generates an output image using image data provided from each of the sub-image processors 1212a, 1212b, and 1212c in response to image generating information or a mode signal MD.

Specifically, the image generator 1214 combines at least a portion of the image data generated from the camera modules 1100a, 1100b, and 1100c having different viewing angles according to the image generation information or mode signal MD to generate an output image.
In addition, the image generator 1214 selects one of the image data generated from the camera modules 1100a, 1100b, and 1100c having different viewing angles according to image generation information or a mode signal, and generates an output image.

According to an embodiment of the present invention, and still referring to FIG. 4, the image generator 1214 sums (digitally sums) each of the image data obtained by converting the sensing signals output from each of the sub-pixels.
For example, FIG. 4 illustrates a nona cell that is a collection of nine sub-pixels, and the image generator 1214 receives image data that is the result of individual analog-to-digital conversion of multiple sub-pixels included in each color channel, and can then sum image data corresponding to the number of sensing pixels (eight) among the sub-pixels.

In some embodiments, the image generation information includes a zoom signal or zoom factor.
Also, in some embodiments, the mode signal is a signal based on a mode selected by a user, for example.
When the image generation information is a zoom signal (zoom factor) and each camera module (1100a, 1100b, 1100c) has a different observation field of view (viewing angle), the image generator 1214 performs different operations depending on the type of zoom signal.

For example, when the zoom signal is a first signal, the image data output from camera module 1100a and the image data output from camera module 1100c are merged, and then an output image is generated using the merged image signal and the image data output from camera module 1100b that is not used in the merging.
If the zoom signal is a second signal different from the first signal, the image generator 1214 does not perform such image data merging, but instead selects one of the image data output from each camera module (1100a, 1100b, 1100c) to generate an output image.
However, the present embodiment is not limited thereto, and the method of processing image data may be modified in any manner as necessary.

In some embodiments, the image generator 1214 receives multiple image data with different exposure times from at least one of the multiple sub-image processors (1212a, 1212b, 1212c) and performs HDR (high dynamic range) processing on the multiple image data to generate merged image data with an increased dynamic range.
The camera module controller 1216 provides control signals to each of the camera modules (1100a, 1100b, 1100c).
The control signals generated by the camera module controller 1216 are provided to the corresponding camera modules (1100a, 1100b, 1100c) via mutually separated control signal lines (CSLa, CSLb, CSLc).

One of the multiple camera modules (1100a, 1100b, 1100c) is designated as a master camera (e.g., camera module 1100b) by image generation information including a zoom signal or a mode signal, and the remaining camera modules (e.g., camera modules 1100a and 1100c) are designated as slave cameras.
Such information is included in control signals and provided to corresponding camera modules (1100a, 1100b, 1100c) via separate control signal lines (CSLa, CSLb, CSLc).
The zoom factor or operating mode signal changes which camera modules act as master and slave.
For example, if the viewing angle of the camera module 1100a is wider than that of the camera module 1100b and the zoom factor indicates a lower zoom magnification, the camera module 1100b acts as the master and the camera module 1100a acts as the slave.
Conversely, when the zoom factor indicates a high zoom magnification, camera module 1100a acts as the master and camera module 1100b acts as the slave.

In some embodiments, the control signals provided from the camera module controller 1216 to each camera module (1100a, 1100b, 1100c) include a sync enable signal.
For example, if the camera module 1100b is the master camera and the camera modules (1100a, 1100c) are slave cameras, the camera module controller 1216 transmits a sync enable signal to the camera module 1100b.
The camera module 1100b, which has received such a sync enable signal, generates a sync signal based on the received sync enable signal and provides the generated sync signal to the camera modules (1100a, 1100c) via the sync signal line SSL.
The camera module 1100 b and the camera modules 1100 a and 1100 c are synchronized with such a sync signal and transmit image data to the application processor 1200 .

In some embodiments, the control signals provided from the camera module controller 1216 to the multiple camera modules (1100a, 1100b, 1100c) include mode information via a mode signal.
Based on such mode information, the multiple camera modules (1100a, 1100b, 1100c) operate in a first operation mode and a second operation mode related to the sensing speed.
In a first operating mode, the multiple camera modules (1100a, 1100b, 1100c) generate image signals at a first rate (e.g., generate image signals at a first frame rate), encode them at a second rate faster than the first rate (e.g., encode image signals at a second frame rate higher than the first frame rate), and transmit the encoded image signals to the application processor 1200.
At this time, the second speed is 30 times or less than the first speed.

The application processor 1200 stores the received image signal, in other words, the encoded image signal, in an internal memory 1230 or an external memory 1400 of the application processor 1200, and then reads and decodes the encoded image signal from the memory 1230 or the external memory 1400, and displays image data generated based on the decoded image signal.
For example, a corresponding sub-processor among a plurality of sub-processors (1212a, 1212b, 1212c) of the image processing device 1210 performs decoding and image processing on the decoded image signal.

In a second operating mode, the multiple camera modules (1100a, 1100b, 1100c) generate image signals at a third rate slower than the first rate (e.g., generate image signals at a third frame rate lower than the first frame rate) and transmit the image signals to the application processor 1200.
The image signal provided to the application processor 1200 is an unencoded signal.
The application processor 1200 performs image processing on the received image signal or stores the image signal in the memory 1230 or the external memory 1400 .

The PMIC 1300 supplies power, for example, a power supply voltage, to each of the multiple camera modules (1100a, 1100b, 1100c).
For example, under the control of the application processor 1200, the PMIC 1300 supplies a first power to the camera module 1100a via a power supply signal line PSLa, supplies a second power to the camera module 1100b via a power supply signal line PSLb, and supplies a third power to the camera module 1100c via a power supply signal line PSLc.
The PMIC 1300 responds to a power control signal PCON from the application processor 1200 to generate power corresponding to each of the multiple camera modules (1100a, 1100b, 1100c) and adjust the power level.
The power control signal PCON includes a power adjustment signal for each operation mode of the camera modules 1100a, 1100b, and 1100c.
For example, the operating mode may include a low power mode, where the power control signal PCON includes information regarding the camera module operating in the low power mode and the power level to be set.
The levels of power provided to each of the multiple camera modules (1100a, 1100b, 1100c) may be the same or different from one another.
The power level can also be changed dynamically.

FIG. 15 is a block diagram showing a schematic configuration of an electronic device 30 according to an embodiment of the present invention.
Referring to FIG. 15, an electronic device 30 includes a processor 31, a memory 32, a storage device 33, an image sensor 34, an input/output device 35, and a power supply 36, and each component communicates with each other via a bus.
The image sensor 100 in FIG. 1 is applied as the image sensor 34 in FIG. 15, but a duplicated description will be omitted.

Processor 31 performs specific calculations or tasks necessary for the operation of electronic device 30 .
The memory 32 and storage device 33 store data necessary for the operation of the electronic device 30 .
For example, the processor 31 includes a microprocessor, a CPU (central processing unit), an AP (application processor), etc., the memory 32 includes a volatile memory and/or a non-volatile memory, and the storage device 33 includes an SSD (solid static driver), an HDD (hard disk drive), a CD-ROM (compact disc read only memory), etc.
The input/output device 35 includes input means such as a keyboard, a keypad, and a mouse, and output means such as a printer and a display.
The power supply 36 provides the operating voltage required for the operation of the electronic device 30 .

FIG. 16 is a block diagram showing a schematic configuration of an electronic device 1a according to an embodiment of the present invention.
Referring to FIG. 16, an electronic device 1a according to an embodiment of the present invention includes an image sensor 10a, an image signal processor (ISP) 20a, an application processor (AP) 30a, a display device 50a, a working memory 40a, a storage device 60a, a user interface 70a, and a wireless transceiver unit 80a.
The image sensor 100 in FIG. 1 operates as the image sensor 10a in FIG. 17, but a duplicated description will be omitted.

The image sensor 10a generates image data, for example, raw image data, based on the received optical signal, and provides the binary data to an image signal processor (ISP) 20a.
The image signal processor (ISP) 20a performs image processing to change the data format of the image data IDAT (FIG. 1), which is digital data related to an image (e.g., converting the image data IDAT of a Bayer pattern into a YUV format or an RGB format), and performs image processing to improve image quality, such as noise removal, brightness adjustment, and sharpness adjustment.

In an embodiment, the image signal processor (ISP) 20a performs pre-processing operations to remove distortions from the image data IDAT and improve algorithm performance, such as white balancing, denoising, demosaicking, lens shading, gamma correction, edge detection, sharpness improvement, noise reduction, gain adjustment, waveform shaping, interpolation, edge enhancement, and binning.

The image signal processor (ISP) 20a performs pre-processing, thereby improving the aftercare speed of the image data IDAT.
Image sensor 10a and image signal processor (ISP) 20a are referred to as a camera module 15a.
In one embodiment, an image signal processor (ISP) 20a is included both external to image sensor 10a for space efficiency and internal to image sensor 10a for increased processing speed.
For the sake of convenience, the present invention has been described assuming that the image signal processor (ISP) 20a is provided separately from the application processor 30a, but the present invention is not limited thereto.
For example, the image signal processor (ISP) 20a may exist as a sub-component of the application processor 30a, rather than being configured as separate hardware or a combination of hardware and software.

The application processor 30a is provided as a system-on-chip (SoC) that controls the overall operation of the electronic device 1a and runs application programs, an operation system, and the like.
The application processor 30a controls the operation of the image signal processor (ISP) 20a, and provides converted image data generated by the image signal processor (ISP) 20a to the display device 50a or stores it in the storage device 60a.

The working memory 40a stores programs and/or data that are processed or executed by the application processor 30a.
The storage device 60a may be implemented as a non-volatile memory device such as a NAND flash memory or a resistive memory. For example, the storage device 60a may be provided as a memory card (such as a multimedia card (MMC), embedded multimedia card (eMMC), secure digital (SD) card, microSD card, etc.).

The storage device 60a stores data and/or programs relating to the execution algorithms that control the image processing operations of the image signal processor (ISP) 20a, and the data and/or programs are loaded into the working memory 40a when the image processing operations are performed.
For example, the working memory 40a or the storage device 60a may include, as non-volatile memory, a read only memory (ROM), a flash memory, a phase change random access memory (PRAM), a magnetic random access memory (MRAM), a resistive random access memory (RRAM), a ferroelectric random access memory (FeRAM), etc., and as volatile memory, a static random access memory (SRAM), a dynamic random access memory (DRAM), etc., but are not limited to the above examples.

The user interface 70a may be implemented by a variety of devices capable of receiving user input, such as a keyboard, a curtain key panel, a touch panel, a fingerprint sensor, or a microphone.
The user interface 70a receives user input and provides signals corresponding to the received user input to the application processor 30a.
The wireless transmitting/receiving unit 80a includes a modem 81a, a transceiver 82a, and an antenna 83a.

The present invention is not limited to the above-described embodiment. Various modifications can be made without departing from the technical scope of the present invention.

30, 1000 Electronic device 31 Processor 32, 1146 Memory 33 Storage device 34, 100, 1142 Image sensor 35 Input/output device 36 Power supply 110 Pixel array 120 Row decoder 130, 1144 Control logic 140 Ramp generator 150 Readout circuit 151 Correlated double sampling circuit 153 Analog-to-digital converter circuit 155 Buffer 1100a, 1100b, 1100c Camera module 1130 Actuator 1140 Image sensing device 1147 Calibration data 1150 Storage

Claims (17)

In a pixel array including a plurality of pixel groups,
Each of the plurality of pixel groups includes a plurality of color pixels arranged in a 2×2 matrix ;
Each of the plurality of color pixels includes a plurality of sub-pixels arranged in an (m)×(n) matrix (m and n are natural numbers equal to or greater than 2) and sensing light of the same wavelength band;
The plurality of sub-pixels include a phase detection pixel that detects a phase and a sensing pixel that senses an image,
The plurality of color pixels include a first color pixel and a second color pixel adjacently arranged in a first diagonal direction,
A pixel array, characterized in that a position of a first phase detection pixel included in the first color pixel within the first color pixel is the same as a position of a second phase detection pixel included in the second color pixel within the second color pixel . The pixel array of claim 1, wherein the phase detection pixels are disposed at predetermined positions of each of the plurality of color pixels. The plurality of color pixels further includes a third color pixel and a fourth color pixel adjacent to each other in a second diagonal direction ,
2. The pixel array of claim 1, wherein the first phase detection pixel included in the first color pixel is adjacent to a third phase detection pixel included in the third color pixel. The pixel array of claim 3 , wherein the first phase detection pixel and the third phase detection pixel share a single microlens. The plurality of color pixels further includes a third color pixel and a fourth color pixel adjacent to each other in a second diagonal direction,
The plurality of pixel groups includes a first pixel group and a second pixel group adjacent to each other in a first direction,
2. The pixel array of claim 1, wherein the second phase detection pixel included in the second color pixel included in the first pixel group is adjacent to a fourth phase detection pixel included in the fourth color pixel included in the second pixel group. The plurality of color pixels further includes a third color pixel and a fourth color pixel adjacent to each other in a second diagonal direction,
2. The pixel array of claim 1 , wherein a position of the first phase detection pixel included in the first color pixel is symmetrical to a position of a third phase detection pixel included in the third color pixel in the third color pixel. 2. The pixel array of claim 1, wherein an amount of crosstalk induced to the phase detection pixel from the sub-pixels adjacent to the phase detection pixel is the same for each color pixel. The pixel array of claim 1, wherein the plurality of color pixels includes a first green pixel, a red pixel, a blue pixel, and a second green pixel. The plurality of color pixels further includes a third color pixel and a fourth color pixel adjacent to each other in a second diagonal direction,
Each of the plurality of color pixels includes at least two of the phase detection pixels arranged symmetrically within each color pixel;
the first color pixels include the first phase detection pixel and a fifth phase detection pixel ;
the second color pixels include the second phase detection pixel and a sixth phase detection pixel ;
the third color pixel includes a third phase detection pixel and a seventh phase detection pixel ;
the fourth color pixel includes a fourth phase detection pixel and an eighth phase detection pixel;
The pixel array of claim 1 , wherein the first phase detection pixel, the sixth phase detection pixel, the third phase detection pixel, and the eighth phase detection pixel are adjacent to each other. The pixel array of claim 9 , wherein the first phase detection pixel, the sixth phase detection pixel, the third phase detection pixel, and the eighth phase detection pixel share one microlens. The pixel array of claim 1, characterized in that the phase detection pixel is a metal shield type. a pixel array including a plurality of color pixels arranged in a matrix , each of the color pixels including a plurality of sub-pixels sensing light of the same wavelength band ;
a readout circuit configured to convert sensing signals received from the pixel array via a plurality of column lines into binary data;
a row decoder configured to generate a row selection signal for controlling the pixel array to output the sensing signal on a row-by-row basis via a plurality of row lines;
and control logic configured to control the row decoder and the readout circuit ;
The plurality of sub-pixels included in each of the plurality of color pixels include a phase detection pixel that detects a phase and a sensing pixel that senses an image,
The plurality of color pixels include a first color pixel and a second color pixel adjacently arranged in a first diagonal direction,
a position of a first phase detection pixel included in the first color pixel within the first color pixel is the same as a position of a second phase detection pixel included in the second color pixel within the second color pixel . The plurality of color pixels further includes a third color pixel and a fourth color pixel adjacent to each other in a second diagonal direction,
13. The image sensor of claim 12 , wherein the first phase detection pixel is adjacent to a third phase detection pixel included in the third color pixel . The image sensor of claim 12 , wherein the phase detection pixels are disposed one for each of the plurality of color pixels. a pixel array including a plurality of color pixels arranged in a matrix , each of the color pixels including a plurality of sub-pixels sensing light of the same wavelength band , the color pixels each including a phase detection pixel disposed at a predetermined position;
a readout circuit configured to convert sensing signals received from the pixel array via a plurality of column lines into binary data;
a row decoder configured to generate a row selection signal for controlling the pixel array to output the sensing signal on a row-by-row basis via a plurality of row lines;
a control logic configured to control the row decoder and the readout circuit and change an output method of the sensing signal based on a mode signal ;
The plurality of color pixels include a first color pixel and a second color pixel arranged in a first diagonal direction,
a position of a first phase detection pixel included in the first color pixel within the first color pixel is the same as a position of a second phase detection pixel included in the second color pixel within the second color pixel . the mode signal indicates a first mode or a second mode;
16. The image sensor of claim 15, wherein the pixel array outputs the sensing signal from each of the plurality of sub-pixels according to the first mode, and outputs a sum of sensing results of the plurality of sub-pixels according to the second mode . The plurality of color pixels further includes a third color pixel and a fourth color pixel adjacent to each other in a second diagonal direction,
16. The image sensor of claim 15 , wherein the first phase detection pixel is adjacent to a third phase detection pixel included in the third color pixel .

Images