JP3667098B2 - Imaging sensor, image signal processing method, image signal processing system, imaging apparatus, and storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging sensor, an image signal processing method, an image signal processing system, an imaging apparatus, and a storage medium.
[0002]
[Prior art]
In an image signal processing circuit that handles color image signals, various image signal processes are performed on image signals output from an image sensor. As one of the image signal processes, there is known an apparatus that performs a bare color determination for identifying a person.
[0003]
FIG. 10 is a block diagram showing a schematic configuration of a conventional color determination device. As shown in FIG. 10, this naked color determination device converts an image signal output as an analog signal from a CCD (imaging sensor) 101 into a digital signal by an A / D conversion circuit 102 and stores it in a line memory 103. Is stored (for example, data for two lines).
[0004]
Then, the image data accumulated in the line memory 103 is subjected to image processing in the image processing circuit 104 in units of “2 × 2” pixel blocks, for example, to generate a luminance signal Y and color difference signals U and V. Then, the color difference signals U and V are supplied to the bare color determination circuit 105 to perform the bare color determination based on the signal components of the color difference signals U and V.
[0005]
[Problems to be solved by the invention]
In the case where the color determination is performed as described above, since the image signal output from the CCD (imaging sensor) 101 is output in a line-sequential serial, the line memory 103 is subjected to digital conversion by the A / D conversion circuit 102. The image processing could not be performed in units of a predetermined pixel block unless it was stored in the pixel block.
[0006]
For this reason, conventionally, when performing image processing including color determination, a large-capacity memory is required, which not only has a poor cost performance, but also has a problem that the speed of signal processing becomes slow.
[0007]
In view of the above-described problems, it is a first object of the present invention to provide an image sensor that can internally perform color determination processing for an arbitrary pixel.
In addition, it is possible to simplify the configuration of the image signal processing circuit that performs various processes on the image signal output from the imaging sensor and to improve the processing speed of the image signal processing circuit. The purpose of 2.
[0008]
[Means for Solving the Problems]
An image sensor of the present invention includes a block reading unit that reads out a pixel signal output from a color solid-state image sensor for each pixel block of a predetermined unit, and a luminance signal and a color difference signal from the pixel signal of the pixel block output from the block reading unit. And a color detection means for detecting a color for each pixel block based on a color difference signal output from the signal processing means.
Another feature of the image sensor of the present invention is that the pixel block of the predetermined unit is a basic unit block of a color filter arranged in the color solid-state image sensor.
Another feature of the image sensor according to the present invention is that the color detecting means detects human skin color.
Another feature of the image sensor of the present invention is that the detection result of the color detection means is output in synchronization with a luminance signal and a color difference signal output from the signal processing means. .
Another feature of the imaging sensor according to the present invention is that the color detection unit includes a first threshold value and a second threshold value for the (R−Y) component in the color difference signal output from the signal processing unit. , The third threshold value and the fourth threshold value are set for the (B−Y) component, and the relationship in which the first threshold value <(R−Y) component <the second threshold value is satisfied, and A color detection signal indicating that a target color has been detected is output when both of the relationship of the third threshold value <(RY) component <the fourth threshold value are satisfied. Yes.
Another feature of the image sensor of the present invention is that the first to fourth threshold values are variable.
Another feature of the imaging sensor according to the present invention is that the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the imaging device.
[0009]
The image signal processing method of the present invention includes a block readout unit that reads out a pixel signal output from a color solid-state imaging device for each pixel block of a predetermined unit, a luminance signal from the pixel signal of the pixel block output from the block readout unit, and A signal processing unit that calculates a color difference signal; and a color detection unit that detects a color for each pixel block based on the color difference signal output from the signal processing unit. A compression process for performing a process of compressing the information amount of the image signal using the output of the color detection means without passing through a color processing process for performing at least white balance correction or γ correction for color information; Performing a decompression process for decompressing the information-compressed image signal, and performing the color processing process after the decompression process is completed. It is set to.
Another feature of the image signal processing method according to the present invention is that the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the imaging device. It is characterized by.
Another feature of the image signal processing method according to the present invention is that the pixel block of the predetermined unit is a basic unit block of a color filter arranged in the color solid-state imaging device. .
Another feature of the image signal processing method of the present invention is that the color detection means detects human skin color.
Another feature of the image signal processing method of the present invention is that the detection result of the color detection means is output in synchronization with a luminance signal and a color difference signal output from the signal processing means. It is said.
Another feature of the image signal processing method of the present invention is that the color detection means includes a first threshold value for the (R−Y) component of the color difference signal output from the signal processing means, and The second threshold value is set, and the third threshold value and the fourth threshold value are set for the (B−Y) component, and the relationship is such that the first threshold value <(R−Y) component <the second threshold value. And when the relationship of the third threshold value <(R−Y) component <the fourth threshold value is satisfied, a color detection signal indicating that the target color has been detected is output. It is a feature.
Another feature of the image signal processing method according to the present invention is that the compression step and the expansion step are a vector quantization step and a vector decoding step performed by a codebook method.
Another feature of the image signal processing method of the present invention is that the compression step and the expansion step are performed by a compression / expansion method that performs DCT, quantization, and variable length coding.
[0010]
An image signal processing system according to the present invention includes a block reading unit that reads out a pixel signal output from a color solid-state imaging device for each pixel block of a predetermined unit, a luminance signal from the pixel signal of the pixel block output from the block reading unit, and A signal processing unit that calculates a color difference signal; and a color detection unit that detects a color of the pixel block based on the color difference signal output from the signal processing unit. An image signal processing system comprising an image signal supply side for performing signal processing and outputting and an image signal input side using an image signal obtained from the image signal supply side, wherein at least white balance correction or color information is performed The compression means for performing information compression processing on the image signal using the output of the color detection means before performing the γ correction. A decompression unit and a color processing unit that are provided on the signal supply side and perform an information decompression process on the information-compressed image signal are provided on the image signal input side. The information compression process by the compression unit and the decompression unit After the information expansion processing is completed, at least white balance correction or γ correction is performed on the color information by the color processing means.
Another feature of the image signal processing system according to the present invention is that the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the image sensor. It is characterized by.
Another feature of the image signal processing system according to the present invention is that the pixel block of the predetermined unit is a basic unit block of a color filter arranged in the color solid-state imaging device. .
Another feature of the image signal processing system of the present invention is that the color detecting means detects human skin color.
Another feature of the image signal processing system of the present invention is that the detection result of the color detection means is output in synchronization with a luminance signal and a color difference signal output from the signal processing means. It is said.
Another feature of the image signal processing system according to the present invention is that the color detection means includes a first threshold value for the (R−Y) component in the color difference signal output from the signal processing means, and The second threshold value is set, and the third threshold value and the fourth threshold value are set for the (B−Y) component, and the relationship is such that the first threshold value <(R−Y) component <the second threshold value. And when the relationship of the third threshold value <(R−Y) component <the fourth threshold value is satisfied, a color detection signal indicating that the target color has been detected is output. It is a feature.
Another feature of the image signal processing system according to the present invention is that the compression step and the expansion step are a vector quantization step and a vector decoding step performed by a codebook method.
Another feature of the image signal processing system according to the present invention is that the compression step and the expansion step are performed by a compression / expansion method that performs DCT, quantization, and variable length coding.
[0011]
An image pickup apparatus according to the present invention includes a block reading unit that reads out a pixel signal output from a color solid-state image pickup device for each pixel block of a predetermined unit, and a luminance signal and a color difference signal from the pixel signal of the pixel block output from the block reading unit. An image pickup means having a signal processing means for calculating the color difference signal output from the signal processing means and a color detection means for detecting the color of the pixel block, and an image signal output from the image pickup means. Compression means for generating a compressed image signal by performing an information compression process using the output of the color detection means without performing at least white balance correction or γ correction on color information.
Another feature of the imaging apparatus according to the present invention is that the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the imaging device. It is said.
Another feature of the image pickup apparatus of the present invention is that it has a writing means for writing the compressed image signal output from the compression means to a storage medium.
Another feature of the imaging apparatus according to the present invention is that a reading unit that reads a compressed image signal stored in the storage medium, and a decompressing unit that performs an information expansion process on the compressed image signal read by the reading unit. And color processing means for performing at least one of white balance correction and γ correction on the color information of the image signal reproduced by the decompression means.
Another feature of the imaging apparatus according to the present invention is that the predetermined unit pixel block is a basic unit block of a color filter disposed in the color solid-state imaging device.
Another feature of the imaging apparatus according to the present invention is that the color detecting means detects human skin color.
Another feature of the imaging apparatus of the present invention is that the detection result of the color detection means is output in synchronization with the luminance signal and the color difference signal output from the signal processing means. .
Another feature of the imaging apparatus according to the present invention is that the color detection means includes a first threshold value and a second threshold value for the (R−Y) component of the color difference signal output from the signal processing means. In addition to setting a threshold value, a third threshold value and a fourth threshold value are set for the (B−Y) component, and the relationship in which the first threshold value <(R−Y) component <the second threshold value is satisfied, and the first threshold value is set. A color detection signal indicating that a target color has been detected is output when both of the relationship of threshold value 3 <(RY) component <fourth threshold value are satisfied.
Another feature of the imaging apparatus according to the present invention is that the compression process and the expansion process are a vector quantization process and a vector decoding process performed by a codebook method.
Another feature of the imaging apparatus according to the present invention is that the compression step and the expansion step are performed by a compression / expansion method that performs DCT, quantization, and variable length coding.
[0012]
A storage medium according to the present invention stores a program for causing a computer to execute the procedure of the image signal processing method.
Another feature of the storage medium of the present invention is that a program for causing a computer to function as each of the above means is stored.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the imaging sensor, the image signal processing method, the image signal processing system, the imaging apparatus, and the storage medium of the present invention will be described.
FIG. 1 is a block diagram showing a configuration of an image sensor of the present invention. As shown in FIG. 1, the image sensor 10 includes a two-dimensional image sensor (photoelectric conversion unit) 11, a two-line image memory 12, a block readout circuit 13, a luminance color difference signal calculation unit 14, and a bare color determination circuit 15. It is formed on an IC chip.
[0014]
In the image sensor 10 of the present embodiment configured as described above, a color filter is disposed on the two-dimensional image sensor 11 in order to generate the luminance signal Y and the color difference signals U and V.
[0015]
FIG. 2 shows an example of a basic arrangement of color filters arranged on the two-dimensional image sensor 11, and FIG. 2A shows a Bayer arrangement (RGGB) in a primary color filter. In this case, the luminance signal Y ₁ Is Y ₁ = 0.3R + 0.59G + 0.11B. The color difference signal U is U = R ₁ -Y ₁ The color difference signal V is V = B ₁ -Y ₁ It becomes.
[0016]
FIG. 2B shows an example of a complementary color system color filter. In this case, the luminance signal Y is Y = Mg + G + Cy + Ye = 2B + 3G + 2R. The color difference signal U is U = (Mg + Ye) − (G + Cy) = 2R−G, and the color difference signal V is V = (G + Ye) − (Mg + Cy) = 2B−G.
[0017]
The pixel signal output from the two-dimensional image sensor 11 provided with such a basic unit color filter is supplied to the two-line image memory 12, and signals for two lines are accumulated. Then, it is read out for each basic unit (2 × 2) of the color filter by the block readout circuit 13 and given to the luminance / color difference signal calculation unit 14.
[0018]
As shown in step P1 of the flowchart of FIG. 3, the luminance / chrominance signal calculation unit 14 performs a matrix calculation on the input pixel signal of the basic unit block using the equations as shown in FIGS. 2 (a) and 2 (b). The luminance signal Y and the color difference signals U and V are generated and output to the outside, and the color difference signals U and V are output to the bare color determination circuit 15.
[0019]
As shown in step P2 of the flowchart of FIG. 3, the bare color determination circuit 15 determines whether or not the color difference signals U and V are the bare colors in units of (2 × 2) pixels that are input. In the fifteenth embodiment, a threshold value for performing the color determination is set in advance, and the color determination is performed based on the threshold value.
[0020]
That is, as shown in FIG. 4, the first threshold value (A) and the second threshold value (B) are set for the (R−Y) component as the determination threshold value. A third threshold (-D) and a fourth threshold (-C) are set for the (B-Y) component.
[0021]
Then, the input color difference signals U and V satisfy the first threshold (A) <color difference signal U <second threshold (B), and the third threshold (−D) <color difference signal V <fourth. When the threshold value (−C) is reached, it is determined that the color of the input pixel block is a bare color. Alternatively, it is determined whether or not the region is in the area indicated by hatching in FIG.
[0022]
FIG. 5 is a circuit diagram showing a configuration example of the two-dimensional image sensor 11 and an example in which a primary color filter is provided. In the circuit of FIG. ₁ When the line becomes active, pixel R ₁ Is the vertical output line V ₁ Forwarded to G _1A Is the vertical output line V ₂ Forwarded to R _Four Is the vertical output line V _Three Forwarded to G _4A Is the vertical output line V _Four Forwarded to
[0023]
Next, the gate line S of the reading transistor ₁ Is active, each vertical output line V ₁ ~ V _Four The upper pixel data is written and held in the memory capacitor C. That is, the pixel R ₁ Is the memory capacitor C ₁ And pixel G _1A Is the memory capacitor C _Three And pixel R _Four Is the memory capacitor C _Five And pixel G _4A Is the memory capacitor C ₇ Written to each.
[0024]
Next, S ₁ Is turned off and then V ₁ ~ V _Four Reset, then H ₂ When the line becomes active, pixel G _1B Is the vertical output line V ₁ Forwarded to B ₁ Is the vertical output line V ₂ Forwarded to G _4B Is the vertical output line V _Three Forwarded to B _Four Is the vertical output line V _Four Forwarded to
[0025]
Next, the gate line S of the reading transistor ₂ Activates each vertical output line V ₁ ~ V _Four The upper pixel data is the pixel G _1B Is the memory capacitor C ₂ And pixel B ₁ Is the memory capacitor C _Four And pixel G _4B Is the memory capacitor C ₆ And pixel B _Four Is the memory capacitor C ₈ Written to each.
[0026]
As described above, when pixel data for two lines is written in the two-line image memory 12, reading is performed for each (2 × 2) pixel block. That is, the gate line B of the read transistor ₁ Is activated, the memory capacitor C ₁ ~ C _Four Pixel data R held in ₁ , G _1B , G _1A , B ₁ Are output in parallel.
[0027]
Next, the gate line B of the reading transistor ₂ Is activated, the memory capacitor C _Five ~ C ₈ Pixel data R held in _Four , G _4B , G _4A , B _Four Are output in parallel. As described above, sequential reading is performed for each (2 × 2) pixel block.
[0028]
FIG. 6 is a circuit diagram illustrating a specific configuration example of the luminance / color difference signal calculation unit 14 and the bare color determination circuit 15. Pixel data R input to the luminance / chrominance signal calculation unit 14 ₁ , G _1B , G _1A , B ₁ Is the pixel data G _1B And pixel data G _1A Are added and then multiplied by a coefficient of “× 0.5” to obtain an average value. Also, pixel data R ₁ Has a coefficient of “× 0.3” and pixel data B ₁ Are multiplied by a coefficient of “× 0.11” and added to generate a luminance signal Y. Next, the calculation of (R−Y) and the calculation of (B−Y) are performed to obtain the color difference signals U and V.
[0029]
As described above, the obtained luminance signal Y and color difference signals U and V are output to the outside of the image sensor 10, and the color difference signals U and V are given to the color determination circuit 15. As described above, the threshold values (A), (B), (−D), and (−C) for determining the naked color are set in the naked color determination circuit 15, and the color difference signal U and the color difference signal V are set. Are compared to these thresholds.
[0030]
As described above, the first threshold value (A) <the color difference signal U <the second threshold value (B), and the third threshold value (−D) <the color difference signal V <the fourth threshold value (−C). When this happens, it is determined that the color of the input pixel block is a bare color.
[0031]
FIG. 7 is a circuit diagram showing another configuration example of the two-dimensional image sensor 11 and an example in which a complementary color filter is provided. FIG. 8 is a circuit diagram showing a configuration example of the luminance / color difference signal calculation unit 14 and the bare color determination circuit 15 when a complementary color filter is provided. 7 is the same as the operation of the circuit shown in FIG. 5, and the operation of the circuit of FIG. 8 is the same as the operation of the circuit shown in FIG. .
[0032]
FIG. 9 is a block diagram illustrating an example of an image processing apparatus using a bare color detection signal, and illustrates an example of a face image extraction apparatus.
As shown in FIG. 9, in the reproduced one screen, since the dark color flag is output from the dark color determination circuit 15 for the dark color region, the above-mentioned predetermined color including the dark color region is output. By setting the range as a face clipping frame, a face image can be easily and reliably clipped on one screen.
[0033]
In the above-described embodiment, the example of detecting the bare color has been described. However, any color other than the bare color can be detected by changing the threshold set in the bare color determination circuit 15. Can do.
[0034]
In the above-described embodiment, an example is shown in which color detection is performed for each basic unit of the color filter, but the average value of the pixels of the “n × m” (n and m are natural numbers) blocks is calculated. Then, it is possible to perform the color detection based on the color difference signals U and V generated based on the average value.
[0035]
Next, a second embodiment using the image sensor 10 of the present embodiment will be described with reference to the block diagram of FIG.
FIG. 11 is a block diagram illustrating an example of an image signal processing system configured using the imaging unit 10 of the present invention. As shown in FIG. 11, in this image signal processing system, an image signal output side is configured by an image sensor IC chip unit (imaging unit) 10, a codebook compression device 40, and a code number output device 41. The code number input device 50, the code book type expansion device 60, the color processing device 100, and the image display or storage device 90 constitute an image signal input side.
[0036]
The image sensor 10 includes an image sensor (light receiving element) 11, a two-line image memory 12, a block readout circuit 13, a luminance / color difference signal calculation unit 14, and an image color determination circuit 15. As described above, the imaging sensor 10 outputs an imaging unit output signal S10 such as a Y signal, a U signal, and a V signal, and a bare color detection signal (flag).
[0037]
The imaging unit output signal S10 output from the imaging sensor 10 is input to the codebook compression apparatus 40. As described above, the code book type compression device 40 is stored in advance in the pattern of the imaging unit output signal S10 for the predetermined number of pixels input from the imaging sensor 10 and a code book storage device (not shown). Compare multiple codes (patterns). That is, when the output of the imaging unit output signal S10 is a signal for 1000 × 1000 pixels, this is compared with the code pattern in the codebook in units of 4 × 4 pixel blocks for compression, for example. This code pattern is also composed of 4 × 4 pixel blocks.
[0038]
The code book storage device (not shown) of the present embodiment stores a plurality of codes in a pattern corresponding to the compressed block size from the imaging unit output signal S10 output from the imaging sensor 10, and the code book In order to find the most similar pattern among them, the system compression apparatus 40 first selects only the code group of the bare color from the code book. For example, a mark for the color code may be added in advance to the code for the code book, or the color for each code in the code book is calculated and only the skin color is selected. Also good. Then, the code group with the highest correlation is selected from the narrowed down code group, and the code number of the pattern is output. The code number output from the code book type compression device 40 is transmitted to the code number input device side by the code number output device 41 via a medium such as a communication line.
[0039]
The code number sent via the communication line is input by the code number input device 50 and supplied to the code book type expansion device 60. The code book type expansion device 60 reads a pattern corresponding to the input code number from a code book storage device (not shown), and reproduces the image data compressed by the code book type compression device 40.
[0040]
The imaging unit output signal S10 reproduced by the code book type expansion device 60 is then given to the color processing device 100. The color processing apparatus 100 is a color that performs various processes necessary for obtaining good image quality, such as color correction processes such as white balance correction and γ correction, regarding color information in the input image pickup unit output signal S10. A processing unit 82 is included.
[0041]
Therefore, the original signals Ye, Cy, Mg, and Gr input from the codebook type expansion device 60 are subjected to predetermined color processing in the color processing device 100, and the final luminance signal y and color difference signals u and v are processed. Is generated and output.
[0042]
The luminance signal y and the color difference signals u and v output from the color processing apparatus 100 are given to an image display or storage device 90, where the image is displayed or stored in a storage medium. In this embodiment, an example in which the codebook method is used as the information compression / decompression method has been described. However, a compression / decompression method that performs DCT, quantization, and variable-length coding may be used.
[0043]
As described above, in the image signal processing system according to the present embodiment, the color correction processing that is performed to obtain high-quality image quality is performed after the information expansion processing without being performed before the information compression processing. Therefore, it is possible to minimize image quality degradation due to block noise and high-frequency noise that accompanies information compression processing → information decompression processing, greatly reduce the amount of information transmitted over a line, and color It is possible to prevent the image signal from being deteriorated after processing, and to obtain a high-quality image.
[0044]
Next, a third embodiment will be described with reference to FIG. In the second embodiment described above, an example in which a signal subjected to compression processing is output to the outside via a medium such as a communication line is shown. However, in this example, a signal after compression is temporarily stored. An example is shown in which the present invention is applied to an imaging apparatus that records on a recording medium, reproduces it, and performs color processing after compression / decompression processing.
[0045]
That is, the configuration different from that in FIG. 11 is provided with a writing device 130, a storage medium 131, and a reading device 132 between the codebook compression device 40 and the codebook expansion device 60.
[0046]
With this configuration, the storage capacity necessary for storing the image signal output from the imaging sensor 10 in the storage medium 131 can be significantly reduced, and the image signal read from the storage medium 131 is encoded. Since the color processing is performed in the color processing apparatus 100 after being reproduced by the book type decompression apparatus 60, the image quality is hardly deteriorated, and a high quality image can be displayed on the image display apparatus 133.
[0047]
Note that the imaging apparatus of the present embodiment can be limited to the configuration up to the medium 131. Further, the reading device 132, the code book type decompression device 60, the color processing device 100, and the image display device 133 can be included in a playback device (for example, a personal computer).
[0048]
In this embodiment, an example in which the codebook method is used as the information compression / decompression method has been described. However, a compression / decompression method that performs DCT, quantization, and variable-length coding may be used.
[0049]
A storage medium storing software program codes for realizing the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU, neither of which is shown) of the system or apparatus. Needless to say, this can also be achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the embodiment, and the storage medium storing the program code constitutes the present invention.
[0050]
A ROM, floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, or the like can be used as a storage medium for supplying the program code.
[0051]
Further, by executing the program code read by the computer, not only the functions of the embodiment are realized, but also the OS or the like running on the computer performs the actual processing based on the instruction of the program code. A case where the function of the embodiment is realized by performing part or all of the processing is also included.
[0052]
Further, after the program code read from the storage medium is written to the memory provided in the extension function board inserted in the computer or the function extension unit connected to the computer, the function extension is performed based on the instruction of the program code. This includes a case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the embodiment are realized by the processing.
[0053]
【The invention's effect】
As described above, according to the present invention, color detection can be performed for a predetermined pixel block, so that processing for detecting an arbitrary color in an image signal output from an image sensor can be easily and reliably performed. it can.
[0054]
According to another feature of the present invention, an image signal in a format suitable for image compression can be output from the imaging sensor, so that the accuracy and processing speed when compressing the image signal output from the imaging sensor are increased. Can be greatly improved.
[0055]
In addition, according to another feature of the present invention, luminance color difference calculation or the like, which is general preprocessing when color image signal processing is performed, can be performed on the imaging sensor, so that the calculation speed can be increased. In addition, it is possible to reduce the memory capacity required for color signal processing to be performed later.
[0056]
According to another feature of the present invention, feature data for each block from the image sensor is calculated without performing color correction such as white balance correction or γ correction at least on color information, and the feature data is used. When performing a compression process for performing an information compression process on an image signal and a decompression process for performing an information decompression process on the image signal subjected to the information compression process, the color process is performed after performing the information compression process and the information decompression process. Therefore, it is possible to greatly reduce the amount of data when transmitting an image signal or storing it in a storage medium, and to prevent image quality deterioration after color processing. High quality images.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of an imaging sensor according to the present invention.
FIG. 2 is a diagram illustrating an example of a color filter array.
FIG. 3 is a flowchart showing an outline of a process for performing a bare color determination.
FIG. 4 is a diagram showing color components for performing a bare color determination.
FIG. 5 is a circuit diagram showing a configuration example of a two-dimensional image sensor and a two-line image memory provided with primary color filters.
6 is a circuit diagram illustrating a configuration example of a luminance / color difference signal calculation unit and a bare color determination circuit corresponding to the circuit of FIG. 5;
FIG. 7 is a circuit diagram showing a configuration example of a two-dimensional image sensor and a two-line image memory provided with complementary color filters.
8 is a circuit diagram illustrating a configuration example of a luminance / color difference signal calculation unit and a bare color determination circuit corresponding to the circuit of FIG. 7;
FIG. 9 is a block diagram showing an example in which the image sensor of the present embodiment is applied to a face image extraction device.
FIG. 10 is a block diagram showing a schematic configuration of a conventional marbled color detection device.
FIG. 11 is a block diagram showing a second embodiment of an image signal processing system using an image sensor of the present embodiment.
FIG. 12 is a block diagram illustrating an example in which the imaging sensor according to the present embodiment is applied to an imaging apparatus.
[Explanation of symbols]
10 Imaging sensor (imaging unit)
11 Two-dimensional image sensor
12 2-line image memory
13 block readout circuit
14 Luminance color difference signal calculation unit
15 Bare color judgment circuit

Claims (35)

Block readout means for reading out pixel signals output from the color solid-state imaging device for each pixel block of a predetermined unit;
Signal processing means for calculating a luminance signal and a color difference signal from the pixel signal of the pixel block output from the block reading means;
An image sensor comprising: a color detection unit configured to detect a color of each pixel block based on a color difference signal output from the signal processing unit. The image sensor according to claim 1, wherein the pixel block of the predetermined unit is a basic unit block of a color filter disposed in the color solid-state image sensor. The image sensor according to claim 1, wherein the color detection unit detects a human skin color. The imaging sensor according to claim 1, wherein the detection result of the color detection unit is output in synchronization with a luminance signal and a color difference signal output from the signal processing unit. In the color detection means, a first threshold value and a second threshold value are set for the (R−Y) component in the color difference signal output from the signal processing means, and the first value for the (B−Y) component is set. The threshold value 3 and the fourth threshold value are set, and the relationship in which the first threshold value <(R−Y) component <the second threshold value and the third threshold value <(R−Y) component <the fourth value is set. 5. The image sensor according to claim 1, wherein a color detection signal indicating that a target color has been detected is output when both of the relations serving as threshold values are satisfied. . The imaging sensor according to claim 5, wherein the first threshold value to the fourth threshold value are variable. 2. The imaging sensor according to claim 1, wherein the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the imaging device. Block readout means for reading out pixel signals output from the color solid-state imaging device for each pixel block of a predetermined unit; signal processing means for calculating luminance signals and color difference signals from the pixel signals of the pixel blocks output from the block readout means; A method for processing an image signal output from an image sensor, comprising color detection means for detecting a color for each pixel block based on a color difference signal output from the signal processing means,
A compression process for compressing the information amount of the image signal using the output of the color detection means without passing through a color processing process for performing at least white balance correction or γ correction for color information, and the image subjected to the information compression process An image signal processing method comprising: performing an expansion process for performing an expansion process on a signal, and performing the color processing process after the expansion process is completed. 9. The image signal processing method according to claim 8, wherein the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the imaging device. 10. The image signal processing method according to claim 8, wherein the pixel block of the predetermined unit is a basic unit block of a color filter disposed in the color solid-state imaging device. The image signal processing method according to claim 8, wherein the color detection unit detects a human skin color. The image signal processing according to any one of claims 8 to 11, wherein the detection result of the color detection means is outputted in synchronization with a luminance signal and a color difference signal outputted from the signal processing means. Method. In the color detection means, a first threshold value and a second threshold value are set for the (R−Y) component in the color difference signal output from the signal processing means, and the first value for the (B−Y) component is set. The threshold value 3 and the fourth threshold value are set, and the relationship in which the first threshold value <(R−Y) component <the second threshold value and the third threshold value <(R−Y) component <the fourth value is set. The image signal processing method according to any one of 8 to 17, wherein a color detection signal indicating that a target color has been detected is output when both of the relations serving as threshold values are satisfied. . 14. The image signal processing method according to claim 8, wherein the compression step and the expansion step are a vector quantization step and a vector decoding step performed by a codebook method. 14. The image signal processing method according to any one of 8 to 13, wherein the compression step and the expansion step are performed by a compression / expansion method that performs DCT, quantization, and variable length coding. Block readout means for reading out pixel signals output from the color solid-state imaging device for each pixel block of a predetermined unit; signal processing means for calculating luminance signals and color difference signals from the pixel signals of the pixel blocks output from the block readout means; An image output by performing predetermined signal processing on the image signal output from the image sensor, and having color detection means for detecting the color of the pixel block based on the color difference signal output from the signal processing means An image signal processing system comprising a signal supply side and an image signal input side that uses an image signal obtained from the image signal supply side,
A compression means for performing information compression processing on the image signal using the output of the color detection means before performing at least white balance correction or γ correction on color information is provided on the image signal supply side, and the information compressed image An expansion means and a color processing means for performing information expansion processing on the signal are provided on the image signal input side,
An image signal processing system, wherein after the information compression processing by the compression unit and the information expansion processing by the expansion unit are finished, at least white balance correction or γ correction is performed on the color information by the color processing unit. The image signal processing system according to claim 16, wherein the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the image pickup device. 18. The image signal processing system according to claim 16, wherein the pixel block of the predetermined unit is a basic unit block of a color filter disposed in the color solid-state imaging device. The image signal processing system according to any one of claims 16 to 18, wherein the color detecting means detects a human skin color. The image signal processing according to any one of claims 16 to 19, wherein the detection result of the color detection means is output in synchronization with a luminance signal and a color difference signal output from the signal processing means. system. In the color detection means, a first threshold value and a second threshold value are set for the (R−Y) component in the color difference signal output from the signal processing means, and the first value for the (B−Y) component is set. The threshold value 3 and the fourth threshold value are set, and the relationship in which the first threshold value <(R−Y) component <the second threshold value and the third threshold value <(R−Y) component <the fourth value is set. The image signal processing system according to any one of claims 16 to 20, wherein a color detection signal indicating that a target color has been detected is output when both of the relations serving as threshold values are satisfied. . The image signal processing system according to any one of claims 16 to 21, wherein the compression step and the expansion step are a vector quantization step and a vector decoding step performed by a codebook method. The image signal processing system according to any one of claims 16 to 21, wherein the compression step and the expansion step are performed by a compression / expansion method that performs DCT, quantization, and variable length coding. Block readout means for reading out pixel signals output from the color solid-state imaging device for each pixel block of a predetermined unit; signal processing means for calculating luminance signals and color difference signals from the pixel signals of the pixel blocks output from the block readout means; Imaging means having color detection means for detecting the color of the pixel block based on the color difference signal output from the signal processing means;
Compression means for generating a compressed image signal by performing an information compression process on the color information of the image signal output from the imaging means without using at least white balance correction or γ correction using the output of the color detection means. An imaging apparatus comprising: 25. The imaging apparatus according to claim 24, wherein the block reading unit, the signal processing unit, and the color detection unit are provided on the same IC chip as the imaging device. 25. The imaging apparatus according to claim 24, further comprising a writing unit that writes a compressed image signal output from the compression unit to a storage medium. Reading means for reading out the compressed image signal stored in the storage medium;
Decompression means for performing information decompression processing on the compressed image signal read by the readout means;
25. The imaging apparatus according to claim 24, further comprising color processing means for performing at least one of white balance correction and γ correction on the color information of the image signal reproduced by the expansion means. The imaging apparatus according to any one of claims 24 to 27, wherein the pixel block of the predetermined unit is a basic unit block of a color filter disposed in the color solid-state imaging device. The image pickup apparatus according to any one of claims 24 to 28, wherein the color detection means detects a human skin color. 30. The imaging apparatus according to claim 24, wherein the detection result of the color detection unit is output in synchronization with a luminance signal and a color difference signal output from the signal processing unit. The color detection means sets a first threshold value and a second threshold value for the (R−Y) component in the color difference signal output from the signal processing means, and a third value for the (B−Y) component. A threshold value and a fourth threshold value are set, and the relationship of the first threshold value <(R−Y) component <the second threshold value and the third threshold value <(R−Y) component <the fourth threshold value are set. 31. The imaging apparatus according to claim 24, wherein a color detection signal indicating that a target color is detected is output when both of the following relationships are satisfied. 32. The imaging apparatus according to claim 24, wherein the compression process and the expansion process are a vector quantization process and a vector decoding process performed by a codebook method. The imaging apparatus according to any one of claims 24 to 32, wherein the compression step and the expansion step are performed by a compression / expansion method that performs DCT, quantization, and variable length coding. A storage medium storing a program for causing a computer to execute the procedure of the image signal processing method according to any one of claims 8 to 15. A storage medium storing a program for causing a computer to function as each means according to any one of claims 16 to 33.

Images