JP4515546B2 - Image signal processing apparatus and electronic still camera equipped with the apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image signal processing apparatus and an electronic still camera equipped with the apparatus, and more particularly, a high resolution imaged by a high resolution electronic still camera using a solid-state imaging device having a large number of pixels such as about 1.3 million pixels. The effect of defective pixels of the solid-state image pickup device during this shooting, which is suitable for application in a system that receives a signal of a subject image and obtains a signal for reproducing the video in real time on a standard television system monitor such as NTSC The present invention relates to an image signal processing apparatus for reducing a false signal generated in step 1 and an electronic still camera including the apparatus.
[0002]
[Prior art]
In general, a solid-state imaging device formed of a semiconductor such as a CCD (Charge Coupled Device) produces defective pixels that output a specific level of signal in a state where light is not incident due to a local crystal defect of the semiconductor. It is known that defective pixels cause image quality degradation.
[0003]
In order to eliminate the image quality deterioration due to the defective pixels, generally, defect correction processing (false signal reduction processing) is performed on the output signal of the solid-state imaging device.
[0004]
Conventionally, as such a defect correction processing technique, for example, there is one disclosed in JP-A-4-235472. To correct a data error based on a pixel defect of the solid-state imaging device disclosed therein, the median value of the signal values of all the pixels in the n × n pixel region centered on the target pixel is obtained. A median filter processing circuit using this as the signal value of the pixel of interest is used.
[0005]
[Problems to be solved by the invention]
However, the above-described defect correction processing method using the median filter processing circuit requires a storage circuit for storing data of a plurality of scanning lines (lines), and thus the problem of an increase in circuit scale and data of a plurality of lines. As a result, the time required for the processing becomes longer.
[0006]
Therefore, when the above-described defect correction processing method using the median filter processing circuit is used to reproduce a high-resolution subject image captured by a high-resolution electronic still camera in real time on a standard television system monitor such as NTSC. It was difficult to apply to the image signal processing apparatus used for the above.
[0007]
The present invention eliminates the disadvantages of the prior art, receives a signal of a high-resolution subject image photographed by a high-resolution electronic still camera using a solid-state image sensor having a large number of pixels, and monitors the image of the standard-resolution monitor. An image signal processing apparatus that can be applied to a system for obtaining a signal to be reproduced in real time, and that can reduce false signals with a short circuit time and a small circuit scale, and an electronic still camera equipped with this apparatus are provided. For the purpose.
[0008]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, the present invention provides a color comprising a solid-state imaging device having a color filter of a plurality of colors on the front face and obtaining color image information corresponding to these color filters in a dot-sequential manner from each pixel. In an image signal processing apparatus which receives a color image signal picked up by an image pickup means in the form of a digital signal in the order in which it is obtained and processes this signal, this apparatus is based on the pixels sent in the order obtained above. Pixel interpolation means for receiving color image data in the form of digital signals and obtaining red R, green G, and blue B primary color signal data synchronized for each pixel from the received color image data, and pixel interpolation means for each pixel A predetermined number of the same color pixels adjacent to the front and rear of the horizontal scanning line centered on each pixel for each received color element are received by the synchronized red R, green G, and blue B primary color signal data. A median filter processing unit that performs median filter processing on a continuous n-pixel region including the output and outputs color signal data indicating a median value by this processing, and the median filter processing unit has a defective pixel in the solid-state imaging device; It is characterized in that median filter processing is performed on all pixels regardless of whether or not.
[0009]
In order to solve the above-described problem, the same apparatus as the above-described premise is obtained on the premise, storage means for storing position data indicating a defective pixel position among a plurality of pixels of the solid-state image sensor. Pixel interpolating means for receiving color image data in the form of digital signals based on the pixels sent in the order and obtaining red R, green G, and blue B primary color signal data synchronized for each pixel from the received color image data And three primary color signal data of red R, green G, and blue B from the pixel interpolating means at the timing corresponding to each pixel having no defect among the plurality of pixels of the solid-state imaging device based on the position data read from the storage means Are output as they are, and the three primary colors of red R, green G, and blue B from the pixel interpolating means at the timing corresponding to each defective pixel among the plurality of pixels of the solid-state imaging device With respect to the signal data, median filter processing is performed on a continuous n-pixel region including a predetermined number of the same color elements adjacent to the front and rear of the horizontal scanning line centering on each color element corresponding to each defective pixel. Median filter processing means for outputting color signal data indicating values, and when there is a defective pixel in the solid-state imaging device, the median filter processing means performs horizontal scanning centering on a color element corresponding to the defective pixel. It is characterized in that the median filter process is performed only on a continuous n pixel region including a predetermined number of the same color elements adjacent to each other on the line.
[0010]
Furthermore, in order to solve the above-described problems, the present invention has a high-resolution solid-state imaging that has color filters of a plurality of colors on the front face and obtains color image information corresponding to these color filters in a dot-sequential manner from each pixel. In an electronic still camera provided with an image signal processing apparatus that receives a color image signal picked up by a color image pickup means composed of elements in the form of a digital signal in the order in which it is obtained, the image signal processing apparatus includes the above-mentioned image signal processing apparatus. The color image data in the form of digital signals based on the pixels sent in the order obtained are received, and red R, green G, and blue are synchronized for each pixel from the color image data received based on a predetermined pixel interpolation control signal. B interpolating means for obtaining B primary color signal data, red R, green G, and blue B primary color signal data synchronized for each pixel from the pixel interpolating means. For each color element received based on the Dian filter processing control signal, median filter processing is performed on a continuous n-pixel region including a predetermined number of same-color pixels adjacent to the front and rear of the horizontal scanning line centered on each pixel. Generating a median filter processing means for outputting color signal data indicating a median value by the control means, a control signal including a pixel interpolation control signal to the pixel interpolation means, and a control signal including a median filter processing control signal to the median filter processing means. Control means for outputting, and this control means causes median filter processing to be performed on all pixels regardless of whether or not there is a defective pixel in the solid-state imaging device by a median filter processing control signal to the median filter processing means. It is characterized by.
[0011]
Furthermore, in order to solve the above-described problem, the present invention provides an electronic still camera having the same image signal processing device as that described above, wherein the image signal processing device has a defect among a plurality of pixels of the solid-state imaging device. Storage means for storing position data indicating pixel positions, and color image data in the form of digital signals based on pixels sent in the order obtained in the premise, and color image data received based on a predetermined pixel interpolation control signal Pixel interpolating means for obtaining red R, green G, and blue B primary color signal data synchronized for each pixel, and a predetermined median filter processing control signal based on position data read from the storage means based on a predetermined read control signal Red R, green G from the pixel interpolation means at a timing corresponding to each pixel having no defect among the plurality of pixels of the solid-state imaging device The three primary color signal data of color B are output as they are, and the three primary color signals of red R, green G, and blue B from the pixel interpolating means at the timing corresponding to each defective pixel among the plurality of pixels of the solid-state imaging device. For the data, median filter processing is performed on a continuous n-pixel region including a predetermined number of the same color elements adjacent to each other on the horizontal scanning line centering on each color element corresponding to each defective pixel, and the median value by this processing Median filter processing means for outputting color signal data indicating a control signal, a control signal including a read control signal to the storage means, a control signal including a pixel interpolation control signal to the pixel interpolation means, and a median filter process to the median filter processing means And a control means for generating and outputting a control signal including the control signal. When there is a defective pixel in the solid-state imaging device by a median filter processing control signal to the processing means, a continuous number including a predetermined number of the same color elements adjacent to each other on the horizontal scanning line around the color element corresponding to the defective pixel. The median filter process is performed only for the n pixel region.
[0012]
In this specification, the term “high resolution” refers to a high resolution that exceeds standard television systems such as NTSC.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Next, an image signal processing apparatus according to the present invention and an embodiment of an electronic still camera equipped with the apparatus will be described in detail with reference to the accompanying drawings.
[0014]
FIG. 1 shows a first embodiment of an electronic still camera to which an image signal processing apparatus according to the present invention is applied. In the figure, an electronic still camera 1 reproduces an electric signal of a subject image obtained by imaging with an imaging lens 10 as a visible image by a reproducing unit 26 and compresses high-resolution image data represented by the electric signal. This is a device for storing in the memory card 36. The memory card 36 is detachably attached to the main body of the electronic still camera 1 and can store digital image data transmitted from the memory card 36 in a state where it can be stored and held, and can output the stored digital image data upon request. This is a card-like semiconductor memory device.
[0015]
In particular, the camera 1 performs a reduction process for the false signal generated by the movie processing unit 20 due to the defective pixels of the image sensor 12 included in the high-resolution signal representing the electrical signal of the subject image. Since the high-resolution signal is thinned out and converted to the low-resolution signal, the video of the high-resolution subject image can be displayed on the monitor device 29 via the playback unit 26 in real time.
[0016]
Referring to FIG. 1, a camera 1 includes an imaging lens 10, an imaging device 12, a preprocessing circuit 14, an analog / digital (A / D) converter 16, a YC processing unit 18, a movie processing unit 20, a post processing circuit 22, and a digital analog. (D / A) Converters 24 and 40, playback unit 26, selector 28, monitor device 29, compression processing unit 30, memory 32, interface (I / F) 34, memory card 36, control unit 38, timing signal generation circuit 42 and a drive signal generation circuit 44.
[0017]
The imaging lens 10 forms an optical image of the subject on the imaging surface of the image sensor 12.
[0018]
The image pickup device 12 is a solid-state image pickup device that converts an image formed by the image pickup lens 10 into an electrical signal and outputs it. In this embodiment, in the case of a high-resolution camera, about 1.3 million pixels (number of display dots, horizontal 1280 × vertical 1024 dots) ), And in the case of a standard resolution camera, a CCD with approximately 400,000 pixels (number of display dots, horizontal 640 x vertical 512 dots) is used.
[0019]
A color filter (not shown) is mounted on the imaging cell array of the solid-state imaging device, and pixel signals color-modulated in response to the drive signal 130 from the drive signal generation circuit 44 are output to the output 100 in a dot sequence. An appropriate color segment array is used for the color filter. The output 100 is connected to the input of the preprocessing circuit 14.
[0020]
The preprocessing circuit 14 is a processing circuit that amplifies the image signal input to the input 100 to a predetermined level, and further performs processing such as black level correction, white balance correction, and gamma correction on the amplified image signal. Is output to the output 102. Output 102 is connected to the input of analog-to-digital converter 16.
[0021]
The analog-digital converter 16 has a conversion circuit for converting the analog image signal appearing at the input 102 into a digital value, and at least for one frame in which high-resolution digital image data for one frame converted into a digital value is stored. A frame memory having a capacity of The image data stored in the frame memory is read out to the output 104 under the control 120 of the control unit 38. The output 104 is connected to the inputs of the YC processing unit 18 and the movie processing unit 20.
[0022]
In this embodiment, the YC processing unit 18 is a conversion circuit that converts high resolution or standard resolution R, G, and B image data appearing at the input 104 into data formats of luminance signal data Y and color difference signal data RY, BY. It is. The YC processing unit 18 has two outputs, output 106 and output 112. The YC processing unit 18 outputs the converted standard resolution luminance signal data Y and color difference signal data RY, BY to the output 106. The YC processing unit 18 outputs the converted standard resolution or high resolution luminance signal data Y and color difference signal data RY and BY to the output 112 for storage in the memory card 36. Output 106 is connected to the input of post-processing circuit 22, and output 112 is connected to the input of selector 28.
[0023]
The post-processing circuit 22 is a correction circuit that performs contour correction on the standard-resolution luminance signal data Y input to the input 106 and color correction on the standard-resolution color difference signal data RY and BY. Are output to the output 108. Output 108 is connected to the input of digital to analog converter 24.
[0024]
The digital / analog converter 24 converts the image signal input to the input 108 into an image signal represented by an analog value, and outputs it to the output 110. The output 110 is connected to the input of the playback unit 26.
[0025]
In this embodiment, the reproduction unit 26 converts the image signal input to the input 110 into, for example, an NTSC video signal. The reproduction unit 26 has an output 27, to which a monitor device 29 is connected. The monitor device 29 is an image display device that displays the NTSC video signal as a visible image on a screen.
[0026]
If the monitor device is a PAL or SECAM system, the playback unit 26 may be a conversion circuit that converts an image signal input to the input 110 into a PAL or SECAM video signal. The monitor device 29 functions as a viewfinder of the camera 1, and the connection 27 with the playback unit 26 may be either wired or wireless, and may be either a fixed connection or a detachable connection. Of course, the camera 1 may also include an optical viewfinder.
[0027]
Referring to FIG. 1, the selector 28 is a control circuit that controls the transmission path of image data under the control of the control unit 38 and transfers the image data to a desired circuit. The selector 28 inputs the image data converted into a predetermined data format by the YC processing unit 18 via the connection line 112 and transfers the input image data to the memory 32 via the connection line 114.
[0028]
The memory 32 is constituted by a video RAM or the like, and is a frame memory that accumulates at least one frame corresponding to one screen of high-resolution image data appearing at the input 114. Further, the memory 32 outputs the image data accumulated under the control of the control unit 38 to the output 114 and transfers it to the compression processing unit 30 connected to the output 116 via the selector 28.
[0029]
The compression processing unit 30 is a processing circuit that performs compression processing such as two-dimensional orthogonal transformation, normalization, and Huffman coding on standard resolution or high resolution image data input to the input 116 under the control of the control unit 38. is there. The compression processing unit 30 also outputs the image data compressed under the control of the control unit 38 to the interface 34 connected to the output 118 via the selector 28. The interface 34 transfers the data such as the compressed image data appearing at the input 118 and the control signal supplied from the control unit 38 to the memory card 36, and the data such as the compressed image data output from the memory card 36. Transfer to selector 28.
[0030]
Referring to FIG. 1, the control unit 38 forms a first control signal for controlling and managing each functional unit described above based on various timing signals sent from the timing signal generating circuit 42 via the connection line 126. In addition to having a control circuit (not shown), the control unit 38 also has, in particular, a second control circuit (not shown) for generating control signals for controlling and managing the movie processing unit 20.
[0031]
In this embodiment, the second control circuit is a high-resolution image stored in the frame memory of the analog-digital converter 16 described above, particularly when a subject image from a high-resolution CCD is monitored by, for example, an NTSC monitor device. A read control signal for reading the image data is generated, and a control signal for operating the movie processing unit 20 that performs image processing on the image data read from the frame memory is also generated.
[0032]
When the control unit 38 receives a shutter release signal by pressing the button from a release button circuit (not shown), it forms an activation signal for starting shooting and sends it to the timing signal generation circuit 42 via the control line 124. send.
[0033]
Upon receiving the activation signal 124 from the control unit 38, the timing signal generation circuit 42 forms various timing signals for operating the drive signal generation circuit 44 synchronized with the activation signal 124, and drives them via the signal line 128. This is a synchronization signal generation circuit that outputs to the signal generation circuit 44 and forms various timing signals for operating the control unit 38 synchronized with the activation signal 124 and outputs them to the control unit 38 via the signal line 126. . These various timing signals are formed based on a clock signal output from a reference signal oscillator (not shown) in the timing signal generation circuit.
[0034]
The drive signal generation circuit 44 is a circuit that forms various drive signals for the image sensor 12 based on various timing signals input from the input 128. Various drive signals formed by this circuit are sent to the image sensor 12 via the signal line 130.
[0035]
FIG. 2 shows an internal configuration of the movie processing unit 20 of the camera 1.
[0036]
In this embodiment, the movie processing unit 20 first performs interpolation processing on the pixel data R, G, and B from the high-resolution CCD appearing at the input 104, and synchronizes the pixel data R, G, and B for each pixel. B is obtained, and then the pixel data R, G, and B obtained by reducing the false signal based on the defective pixels of the CCD included in the synchronized pixel data R, G, and B are obtained, and then reduced. The processed pixel data R, G, and B are thinned out to be converted into pixel data R, G, and B having the number of pixels that can be displayed on the NTSC monitor device, and then the converted pixel data R, G And B are output to the output 107, or YC conversion is performed on the converted pixel data R, G and B to obtain the luminance signal data Y and the color difference signal data RY and BY and output to the output 106. is there.
[0037]
As shown in FIG. 2, the movie processing unit 20 includes a selector 200, a first multiplier 210, a first adder 220, a first median filter processing circuit 230, a first memory 238, an input control unit 240, An output control unit 242, a YC converter 244, a second multiplier 250, a second adder 260, a second median filter processing circuit 270 and a second memory 278 are provided.
[0038]
The selector 200 converts the pixel data input to the input 104 into a first multiplier 210, a first adder 220, a first median filter processing circuit 230, a first memory 238 and a YC conversion circuit 244. Or a second conversion route comprising a second multiplier 250, a second adder 260, a second median filter processing circuit 270, a second memory 278, and a YC conversion circuit 244. This switch is used to select whether to convert. In this case, the selection signal is supplied from the second control circuit of the control unit 38.
[0039]
In the present embodiment, pixel data of 1280 × 1024 pixels stored in the frame memory in the analog-digital converter 16 based on the read control signal 120 from the second control circuit of the control unit 38 is converted into 1280 × 1280 pixels. The dots are read as they are, and the vertical 1024 dots are read every other line and input to the input 104 of the selector 200. The control signal 120 read from the frame memory may correspond to the resolution of the monitor device.
[0040]
When the first conversion route is used, the selector 200 receives the selection signal 122 from the second control circuit of the control unit 38, outputs the pixel data input to the input 104 to the output 300, and also outputs the second conversion. When the route is used, the pixel data input to the input 104 is output to the output 302. The output 300 is connected to the input of the first multiplier 210 and the output 302 is connected to the input of the second multiplier 250.
[0041]
The first multiplier 210 receives pixel data R, G, and B from an input 300, multiplies each of the input pixel data R, G, and B by a predetermined coefficient to obtain a result of the multiplication. This is a multiplication circuit using a ROM or the like, and comprises an R multiplier 212, a G multiplier 214, and a B multiplier 216.
[0042]
The R multiplier 212 multiplies the input pixel data R by a predetermined coefficient and outputs the result of the multiplication to the output 304, and the G multiplier 214 multiplies the input pixel data G by the predetermined coefficient and performs the multiplication. The result is output to the output 306, and the B multiplier 216 multiplies the input pixel data B by a predetermined coefficient and outputs the result of the multiplication to the output 308. The output 304 of the R multiplier 212 is input to the R adder 222 of the first adder 220, the output 306 of the G multiplier 214 is input to the G adder 224 of the first adder 220, and the B multiplier 216. Are connected to the inputs of the B adder 226 of the first adder 220, respectively.
[0043]
The R adder 222 is a circuit that adds pixel data R at a predetermined sample point from the input 304 and pixel data R at a predetermined sample point different from the predetermined sample point, and outputs the result of the addition. Output to 310. Similarly, the G adder 224 and the B adder 226 also output the addition results of the pixel data G and B to the outputs 312 and 314, respectively. In this case, the number of sample points may be any number and determined by the system.
[0044]
As can be seen from the above description, the R adder 222, the G adder 224, and the B adder 226 each perform pixel interpolation, and thus output R, G, and B pixel data at each sample point 310, respectively. , 312 and 314. The output 310 is the corresponding input of the R median filtering circuit 232 of the first median filtering circuit 230, and the output 312 is the corresponding input of the R median filtering circuit 234 of the first median filtering circuit 230, and the output 314. Are respectively connected to corresponding inputs of the R median filter processing circuit 236 of the first median filter processing circuit 230.
[0045]
The three R, G, and B pixel data interpolated by the first adder 220 are subjected to a false signal reduction process based on the defect of the CCD 12 pixel in the next first median filter processing circuit 230. It is.
[0046]
In FIG. 3A, the median filter processing means that all pixels in the left and right n pixel regions on the same line centering on the pixel 420 are focused on 420 pixels (shown by diagonal lines in the figure). This is a process for obtaining the median value of the signal value of the pixel and using it as the signal value of the pixel 420. In the first embodiment, this process is performed for all pixels of one screen (image). . FIGS. 3B and 3C show median filter processing for the pixel 430 when the region size is 1 × 3 (see 450).
[0047]
The median filter processing in the first median filter processing circuit 230 uses the 1 × 3 region size as described above. Accordingly, each of the three median filter processing circuits R, G, and B includes a storage circuit that stores at least one pixel of interest and one pixel on the left and right of the pixel of interest, and three pixels stored in the storage circuit. A comparison circuit that compares the level values and arranges them in descending order of the level values to obtain a median value may be provided.
[0048]
In this embodiment, such processing is performed by the median filter processing control formed by the input control unit 240 based on the control signal 122 for generating the median filter processing control signal from the second control circuit of the control unit 38. Based on signal 318.
[0049]
The R median filter processing circuit 232 performs such processing on the pixel data R, the G median filter processing circuit 234 performs on the pixel data G, and the B median filter processing circuit 236 performs on the pixel data B. It is. From the output 316 of the R, G and B median filter processing circuits 232, 234 and 236, R, G and B pixel data subjected to the median filter processing are output. Output 316 is connected to a corresponding input of first memory 238.
[0050]
The first memory 238 is composed of a video RAM or the like. In this embodiment, the R, G, and B pixel data (horizontal 1280 × vertical 512 × color type 3 dots) appearing at the input 316 are stored. It is a memory circuit having a capacity to accumulate.
[0051]
In this embodiment, the R, G, and B pixel data appearing at the input 316 are input to the input controller 240 based on the control signal 122 for generating the write control signal from the second control circuit of the controller 38. The pixel data R, G, and B stored in the memory 238 by the write control signal formed in this manner and the control data for generating the read control signal from the second control circuit of the control unit 38 are stored in the memory 238. Based on 122, the data is read from the memory 238 by the interlace read control signal formed by the output control unit 242 and output to the output 320.
[0052]
Specifically, the interlace readout control signal described above is a control signal for reading out pixel data every other pixel for every odd line or every even line. Therefore, from the output 320 of the first memory 238, R, G, and B pixel data of 640 × 512 dots are output as one frame. The output 320 of the first memory 238 is connected to the corresponding input of the YC converter 244.
[0053]
In this example, the first memory 238 is a storage circuit capable of storing R, G, and B pixel data of 1280 horizontal x 512 vertical dots each. However, each of the first memory 238 is R of 640 horizontal x 512 dots vertical. , G and B pixel data may be stored in the memory circuit. In this case, the write control signal sent from the input control unit 240 may be a control signal for writing pixel data every other pixel for each line.
[0054]
The YC converter 244 includes a selection circuit and a YC conversion circuit, and the selection circuit selects whether to use the pixel data from the first memory 238 or the pixel data from the second memory 278. One of them is selected based on the switching signal 122 from the control unit 38. The selected pixel data R, G, and B from the first memory 238 or the second memory 278 are sent to the YC conversion circuit and output from the output 107 thereof.
[0055]
The YC conversion circuit is composed of, for example, a multiplier, an adder, a subtracter, etc., and receives pixel data R, G, and B from the selection circuit, and uses the input pixel data R, G, and B, respectively, to Luminance signal data Y (horizontal 640 × 512 dots) and color difference based on the YC conversion control signal formed by the output control unit 340 based on the control signal 122 for generating the YC conversion control signal from the second control circuit. This is a circuit that converts the data format to signal data RY and BY (each 640 horizontal by 512 vertical dots). The converted data is output from the output 106.
[0056]
Referring to FIG. 2, the input control unit 240 is based on the control signal 122 for generating the write control signal from the second control circuit of the control unit 38 shown in FIG. This is a control circuit for generating write control signals for the first memory 238 and the second memory 278.
[0057]
The input controller 240 also medians to the first and second median filter processing circuits 230 and 270 based on the control signal 122 for generating the median filter processing control signal from the second control circuit of the control unit 38. A control circuit for generating a filter processing control signal 318. In this first embodiment, the median filter processing control signal 318 is a control signal for performing reduction processing for all pixels of one screen.
[0058]
In addition, the output control unit 242 has the first memory 238 and the second memory 238 based on the control signal 122 for generating the read control signal from the second control circuit of the control unit 38, as mentioned before. This is a control circuit for forming an interlaced or non-interlaced read control signal for the memory 278. The output control unit 242 is a control circuit that forms a YC conversion control signal to the YC converter 360 based on the control signal 122 for generating the YC conversion control signal from the second control circuit of the control unit 38. is there.
[0059]
As shown in FIG. 1, the digital-to-analog converter 40 converts the digital image data R, G and B from the input 107 into image signals represented by analog values. 525 lines, 525/60 RGB monitor device with a field frequency of 60 Hz, 625 lines, 625/50 RGB monitor device with a field frequency of 50 Hz, or a low resolution consisting of a smaller number of lines. Output to RGB monitor device (neither shown).
[0060]
Returning to FIG. 2, the second multiplier 250 has the same function as the first multiplier 210 described above, and the second adder 260 has the same function as the first adder 220 described above. The second median filter processing circuit 270 has the same function as the first median filter processing circuit 230 described above, and the second memory 278 has the same function as the first memory 238 described above. Therefore, the description of each part is omitted.
[0061]
Since FIG. 2 has the first and second conversion routes as described above, basically, when the subject image is displayed on the monitor by the first conversion route, the second conversion route is used. This conversion route performs subject image conversion processing, and when the subject conversion image is displayed on the monitor by the second conversion route, the first conversion route can perform subject image conversion processing. .
[0062]
2 does not include, for example, the second multiplier 250, the second adder 260, and the second median filter processing circuit 270 of the second conversion route, and outputs 316 of the first median filter processing circuit 230. Is connected to the input of the second memory 278, basically, as in FIG. 2, display of the subject image on the monitor and conversion processing of the subject image can be performed independently. .
[0063]
In FIG. 2, for example, only the first conversion route which does not include the second multiplier 250, the second adder 260, the second median filter processing circuit 270 and the second memory 278 of the second conversion route. When the subject image is displayed on the monitor, the subject image conversion processing is not performed. When the subject image conversion processing is performed, the subject image display is not displayed on the monitor. You may not do it.
[0064]
1 and 2, the pixel data may be sent directly from the analog / digital converter 16 to the first multiplier 210 and the second multiplier 250 without going through the frame memory of the analog / digital converter 16. .
[0065]
Next, pixel data appearing in each part of FIG. 2 are shown in FIGS. 4 to 7, and a signal processing method of each part of the movie processing unit 20 will be described in detail. FIG. 4 shows a display dot from a high-resolution CCD with a G-stripe RB complete checkered filter array, that is, a horizontal 1280 × vertical 1024 dot configuration, for the convenience of explanation, the display dots are reduced to a horizontal 8 × vertical 4 dot configuration. The pixel data is processed and displayed by the respective units of the movie processing unit 20.
[0066]
FIG. 5 (a) shows an example of the level value at each pixel position in FIG. 4 (a), FIG. 5 (b) shows the level value at each pixel position in FIG. 4 (b), and FIG. 4 (c) shows the level value at each pixel position, and FIG. 5 (d) shows the level value when the false signal reduction process is performed in the state of FIG. 5 (c).
[0067]
The level value “0” of the display dot numbers 4 and 5 of the NO1 line in FIG. 5A is the level output from the display dot numbers 4 and 5 of the CCD NO1 line because they are defective pixels in this example. The value is “0”.
[0068]
Also, FIG. 6 shows a display dot from a high-resolution CCD with an RGB stripe color filter array, that is, a horizontal 1280 × vertical 1024 dot configuration, for the convenience of explanation, the display dots are reduced to a horizontal 6 × vertical 4 dot configuration. The pixel data is the pixel data that appears after the pixel data is processed by each unit of the movie processing unit 20.
[0069]
FIG. 7 (a) shows an example of the level value at each pixel position in FIG. 6 (a), FIG. 7 (b) shows the level value at each pixel position in FIG. 6 (b), and FIG. 6 (c) shows the level value at each pixel position, and FIG. 7 (d) shows the level value when the false signal reduction process is performed in the state shown in FIG. 7 (c).
[0070]
The level value “0” of the display dot numbers 4 and 5 of the NO1 line in FIG. 7 (a) is the level output from them because the display dot numbers 4 and 5 of the NO1 line of the CCD are defective pixels in this example. The value is “0”.
[0071]
4 to 7, the horizontal numbers indicate the numbers of display dots in the line direction, and the vertical numbers indicate the line numbers.
[0072]
First, a signal processing method of pixel data output by a high-resolution CCD having a G stripe RB complete checkered filter arrangement will be described with reference to FIGS.
[0073]
The pixel signal including the defective pixel at the above-described pixel position output from the high-resolution CCD (imaging device 12) having the G-stripe RB complete checkered filter array shown in FIG. 1 is subjected to processing such as gamma correction by the preprocessing circuit 14. The analog image signal pre-processed by the analog-digital converter 16 is converted into a digital value, and the pixel data converted into the digital value is based on the write control signal of the control unit 38. 4A is stored in a frame memory included in the analog-digital converter 16 as shown in FIG. As described above, the level value at each pixel position shown in FIG. 4 (a) in this case is shown as an example in FIG. 5 (a).
[0074]
The pixel data stored in the frame memory is read out every other line by a read control signal from the control unit 38, and the pixel data R, G, and B read out every other line (see FIG. 4 (b)). Reference, which is a diagram in which the pixel data of the NO1 line and NO3 line in FIG. 4A is read out) is sent to the first multiplier 210 via the selector 200. The level value at each pixel position shown in FIG. 4 (b) in this case is shown in FIG. 5 (b).
[0075]
The R multiplier 212 of the first multiplier 210 is inputted pixel data R in this embodiment. _n Is multiplied by one of the coefficients 1, 3/4, 1/2, and 1/4, and the result is output to the R adder 222 of the first adder 220. Similarly, the G multiplier 214 and the B multiplier 216 of the first multiplier 210 also input pixel data G in this embodiment. _n, Pixel data B _n Is multiplied by one of the coefficients 1, 3/4, 1/2, and 1/4, and the result is output to the G adder 224 and B adder 226 of the first adder 220, respectively.
[0076]
The R adder 222 of the first adder 220 is, for example, first R ₁ Add 0 to R ₁₁ And then R ₁ 3/4 to R _Five 1/4 plus R _{twenty two} And then R ₁ 1/2 to R _Five 1/2 plus R ₃₃ And then R ₁ 1/4 to R _Five 3/4 plus R ₄₄ And then R _Five Add 0 to R ₅₅ And in this way sequentially R _nn Is output to the R median filter processing circuit 232 of the first median filter processing circuit 230.
[0077]
The G adder 224 of the first adder 220 is, for example, firstly G ₀ 1/2 (pixel data G in this case ₀ Is R in this example ₁ The level value is set to “1” in this example) ₂ 1/2 plus G ₁₁ And then G ₂ + 0 to G _{twenty two} And then G ₂ 1/2 to G _Four 1/2 plus G ₃₃ And then G _Four + 0 to G ₄₄ And by this way G _nn Are sequentially generated and output to the G median filter processing circuit 234 of the first median filter processing circuit 230.
[0078]
The B adder 226 of the first adder 220 is, for example, first B ₀ 1/2 (pixel data B in this case ₀ Is R in this example ₁ And the level value is set to “1” in this example) _Three 1/2 and B ₁₁ And then B ₀ 1/4 to B _Three 3/4 plus B _{twenty two} Create then B _Three 0 to B ₃₃ And then B _Three 3/4 to B ₇ 1/4 plus B ₄₄ Create then B _Three 1/2 to B ₇ 1/2 and B ₅₅ Create then B _Three 1/4 to B ₇ 3/4 plus B ₆₆ Create then B ₇ 0 to B ₇₇ And in this way sequentially B _nn Is output to the B median filter processing circuit 236 of the first median filter processing circuit 230.
[0079]
The pixel data R output from the first adder 220 in this way. _nn , G _nn , B _nn For example, as shown in FIG. 4 (c), the level value in this case is as shown in FIG. 5 (c).
[0080]
As can be seen from the above description and FIGS. 4B and 4C, for example, for pixel data R, R ₁₁ , R ₅₅ Other than R _{twenty two} , R ₃₃ , R ₄₄ , R ₆₆ , R ₇₇ , R ₈₈ Are interpolation pixels, and pixel data G _{twenty two} , G ₄₄ , G ₆₆ , G ₈₈ G other than ₁₁ , G ₃₃ , G ₅₅ , G ₇₇ Are interpolated pixels, and for pixel data B, B ₃₃ , B ₇₇ Other than B ₁₁ , B _{twenty two} , B ₄₄ , B ₅₅ , B ₆₆ , B ₈₈ Are interpolated pixels. A method for creating such an interpolation pixel may be determined in advance by the system.
[0081]
Regarding the processing in the first median filter processing circuit 230 to be described next, in this example, the level value of the left pixel (not shown) of the left pixel level value shown in FIG. .
[0082]
The R pixel will be described. First, when the pixel of interest is the NO1 display dot of the NO1 line, the storage circuit of the R median filter processing circuit 232 stores the display dot on the left of the NO1 display dot, the NO1 display dot, and the NO1. Three pieces of data of the NO2 display dot on the right side of the display dot are stored. The three data stored in the storage circuit are read out and sent to the comparison circuit.
[0083]
The comparison circuit compares the level value “1” of the display dot on the left side of the NO1 display dot with the level value “1” of the NO1 display dot and the level value “3/4” of the NO2 display dot. In other words, “1”, “1”, “3/4” are arranged in this order, and the median value “1” is obtained, and the median data obtained is subjected to median filtering on the target pixel. The output is output to 316.
[0084]
When the process using the NO1 display dot as the target pixel is completed in this manner, the processing circuit 232 then enters a process using the NO2 display dot as the target pixel. Also in this process, the process as described above is performed. However, the target display dots are NO1 to NO3 display dots. When the processing using the NO3 to NO4 display dots as the target pixel is completed in the same processing, the processing circuit 232 next enters the processing using the NO5 display dot based on the defective pixel of the CCD as the target pixel.
[0085]
The process of setting the NO5 display dot based on the defective pixel as the pixel of interest will be described. The storage circuit of the processing circuit 232 stores three data of the NO4 display dot, the NO5 display dot, and the NO6 display dot. The three data stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level value “1/4” of the NO4 display dot with the level value “0” of the NO5 display dot and the level value “1/4” of the NO6 display dot. 4 ”,“ 1/4 ”, and“ 0 ”are arranged in this order, and the median“ 1/4 ”data is output to the output 316 as the median filter data for the pixel of interest.
[0086]
Thus, the level value “0” at the target pixel becomes the level value “1/4” by the median filter processing, that is, the false signal reduction processing is performed by this circuit. Data obtained by performing median filter processing on the target pixel in the R pixel sequentially by such a method is obtained and output to the first memory 238.
[0087]
Similarly, the G and B median filter processing circuits 234 and 236 obtain data obtained by performing median filter processing on the target pixel in the G and B pixels by the above-described method, and output the data to the first memory 238. As a result, the pixel data of each pixel arrangement shown in FIG. 4 (c) is stored in the first memory 238 at the level value shown in FIG. 5 (d).
[0088]
The pixel data R stored in the first memory 238 in this way. _nn , G _nn , B _nn In this embodiment, is read from the first memory 238 based on the read control signal of the output control unit 242 and sent to the YC converter 244. This YC converter 244 is used for the pixel data R from the first memory 238. _nn , G _nn , B _nn For example, 0.3R _nn + 0.59G _nn + 0.11B _nn Luminance signal Y from the calculation of _nn And also 0.7R _nn -0.59G _nn -0.11B _nn From the calculation of color difference signal R _nn -Y _nn Create -0.3R _nn -0.59G _nn + 0.89B _nn From the calculation of color difference signal B _nn -Y _nn And the generated signals are output to the output 106.
[0089]
Next, a signal processing method for pixel data output by a high-resolution CCD having an RGB stripe color filter array will be described with reference to FIGS. 1 to 3 and FIGS.
[0090]
The pixel signal including the defective pixel at the above-described pixel position output by the high-resolution CCD (imaging device 12) having the RGB stripe color filter array shown in FIG. 1 is subjected to processing such as gamma correction by the pre-processing circuit 14 and analog. The analog image signal sent to the digital converter 16 and preprocessed by the analog-digital converter 16 is converted into a digital value, and the pixel data converted into the digital value is converted into an analog-digital converter based on the write control signal of the control unit 38. The frame memory included in the device 16 is stored as shown in FIG. As described above, the level value at each pixel position shown in FIG. 6A in this case is shown as an example in FIG. 7A.
[0091]
The pixel data stored in the frame memory is read every other line by the read control signal of the control unit 38, and the RGB pixel data read every other line (see FIG. 6B). (a) in which the pixel data of the NO1 line and the NO3 line are read out) is sent to the first multiplier 210 via the selector 200. The level value at each pixel position shown in FIG. 6 (b) in this case is shown in FIG. 7 (b).
[0092]
The R multiplier 212 of the first multiplier 210 is inputted pixel data R in this embodiment. _n Is multiplied by one of the coefficients 1, 2/3, or 1/3, and the result is output to the R adder 222 of the first adder 220. Similarly, the G multiplier 214 and the B multiplier 216 of the first multiplier 210 also input pixel data G in this embodiment. _n Pixel data B _n Is multiplied by one of the coefficients 1, 2/3, and 1/3, and the result is output to the G adder 224 and the B adder 226 of the first adder 220, respectively.
[0093]
The R adder 222 of the first adder 220 is, for example, first R ₁ Add 0 to R ₁₁ Create and R ₁ R to 2/3 _Four 1/3 plus R _{twenty two} And then R ₁ 1/3 to R _Four 2/3 plus R ₃₃ And then R _Four Add 0 to R ₄₄ And in this way sequentially R _nn Is output to the R median filter processing circuit 232 of the first median filter processing circuit 230.
[0094]
The G adder 224 of the first adder 220 is, for example, firstly G ₀ 1/3 (pixel data G in this case ₀ Is R in this example ₁ And the level value is set to “1” in this example) ₂ 2/3 plus G ₁₁ And then G ₂ + 0 to G _{twenty two} And then G ₂ G to 2/3 _Five 1/3 plus G ₃₃ And then G ₂ 1/3 to G _Five 2/3 plus G ₄₄ And then G _Five + 0 to G ₅₅ And in this way sequentially G _nn Is output to the G median filter processing circuit 234 of the first median filter processing circuit 230.
[0095]
The B adder 226 of the first adder 220 is also, for example, first B ₀ 2/3 (pixel data B in this case ₀ Is R in this example ₁ And the level value is set to “1” in this example) _Three 1/3 plus B ₁₁ Create then B ₀ 1/3 to B _Three 2/3 plus B _{twenty two} Create then B _Three 0 to B ₃₃ Create then B _Three 2/3 to B ₆ 1/3 plus B ₄₄ Create then B _Three 1/3 to B ₆ 2/3 plus B ₅₅ Create then B ₆ 0 to B ₆₆ And in this way sequentially B _nn Is output to the B median filter processing circuit 236 of the first median filter processing circuit 230.
[0096]
As a result, the pixel data R output from the first adder 220 is displayed. _nn , G _nn , B _nn For example, as shown in FIG. 6 (c), the level value in this case is as shown in FIG. 7 (c).
[0097]
As can be seen from the above description and FIGS. 6C and 6D, for example, for pixel data R, R ₁₁ , R ₄₄ Other than R _{twenty two} , R ₃₃ , R ₅₅ , R ₆₆ Are interpolation pixels, and pixel data G _{twenty two} , G ₅₅ G other than ₁₁ , G ₃₃ , G ₄₄ , G ₆₆ Are interpolated pixels, and for pixel data B, B ₃₃ , B ₆₆ Other than B ₁₁ , B _{twenty two} , B ₄₄ , B ₅₅ Are interpolated pixels. A method for creating such an interpolation pixel may be determined in advance by the system.
[0098]
In the processing in the first median filter processing circuit 230 to be described next, in this example, the level value of the left pixel (not shown) of the left pixel level value shown in FIG. .
[0099]
The R pixel will be described. First, when the pixel of interest is the NO1 display dot of the NO1 line, the storage circuit of the R median filter processing circuit 232 stores the display dot on the left of the NO1 display dot, the NO1 display dot, and the NO1. Three pieces of data of the NO2 display dot on the right side of the display dot are stored. The three data stored in the storage circuit are read out and sent to the comparison circuit.
[0100]
The comparison circuit compares the level value “1” of the display dot to the left of the NO1 display dot with the level value “1” of the NO1 display dot and the level value “2/3” of the NO2 display dot, and the level value is large. In order, that is, “1”, “1”, and “2/3” are arranged in this order, and the median “1” data is output to the output 316 as median filter processing on the pixel of interest.
[0101]
When the process using the NO1 display dot as the target pixel is completed in this manner, the processing circuit 232 then enters a process using the NO2 display dot as the target pixel. Also in this process, the process as described above is performed. However, the target display dots are NO1 to NO3 display dots. When the processing using the NO2 to NO3 display dots as the target pixel is completed in the same processing, the processing circuit 232 then starts the processing using the NO4 display dot based on the defective pixel of the CCD as the target pixel.
[0102]
The process of setting the NO4 display dot based on the defective pixel as the pixel of interest will be described. The storage circuit of the R median filter processing circuit 232 stores three data of the NO3 display dot, the NO4 display dot, and the NO5 display dot. The three data stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level value “1/3” of the NO3 display dot with the level value “0” of the NO4 display dot and the level value “1/3” of the NO5 display dot. ”,“ 1/3 ”, and“ 0 ”are arranged in this order, and the median“ 1/3 ”data is output to the output 316 as data subjected to median filter processing on the pixel of interest.
[0103]
Thus, the level value “0” at the target pixel becomes the level value “1/3” by the median filter processing, that is, the false signal reduction processing is performed by this circuit. In this way, data obtained by performing median filter processing on the target pixel in the R pixel in sequence is generated and output to the first memory 238.
[0104]
Similarly, the G and B median filter processing circuits 234 and 236 obtain data obtained by performing median filter processing on the target pixel in the G and B pixels, and output the data to the first memory 238. As a result, the pixel data of each pixel arrangement shown in FIG. 6 (c) is stored in the first memory 238 at the level value shown in FIG. 7 (d).
[0105]
The pixel data R stored in the first memory 238 in this way. _nn , G _nn , B _nn Are read from the first memory 238 based on the read control signal of the output control unit 242 and sent to the YC converter 244. The YC converter 244 stores the pixel data R from the first memory 238. _nn , G _nn , B _nn For example, 0.3R _nn + 0.59G _nn + 0.11B _nn Luminance signal Y from the calculation of _nn And also 0.7R _nn -0.59G _nn -0.11B _nn From the calculation of color difference signal R _nn -Y _nn Create -0.3R _nn -0.59G _nn + 0.89B _nn From the calculation of color difference signal B _nn -Y _nn And the generated signals are output to the output 106.
[0106]
The operation of the first embodiment will be described.
[0107]
First, the image sensor 12 of the camera 1 is a standard resolution CCD (number of display dots, horizontal 640 x vertical 512 dots) until the pixel signal output from the CCD 12 is converted to an NTSC video signal and displayed on the monitor. Will be described.
[0108]
The optical image of the subject by the imaging lens 10 is formed on the imaging surface of the standard resolution CCD 12. The CCD 12 scans a pixel of horizontal 640 × 512 dots of the image and sends the pixel signal to the preprocessing circuit 14. The preprocessing circuit 14 amplifies the input image signal to a predetermined level, further performs processing such as black level correction, white balance correction, and gamma correction on the amplified image signal, and sends the processed signal to the analog-digital converter 16.
[0109]
The analog-digital converter 16 converts the input analog image signal into a digital value, and further stores the image data converted into the digital value in a frame memory in the analog-digital converter 16. The image data stored in the frame memory is read by an inlaced read control signal from the control unit 38 and sent to the YC processing unit 18.
[0110]
The YC processing unit 18 converts the input image data R, G and B into data formats of luminance signal data Y and color difference signal data RY and BY and sends them to the post-processing circuit 22. The post-processing circuit 22 performs contour correction on the input luminance signal data Y, performs color correction on the input color difference signal data RY and BY, and sends the corrected image signal to the digital-analog converter 24.
[0111]
The digital-analog converter 24 converts the input image signal into an image signal represented by an analog value and sends the image signal to the reproduction unit 26. The playback unit 26 converts the input image signal into an NTSC video signal and sends it to an NTSC monitor device. The monitor device displays an image of the subject from the camera 1.
[0112]
Next, the image sensor 12 of the camera 1 is a high-resolution CCD (number of display dots, horizontal 1280 x vertical 1024 dots) of G stripe RB perfect checkered filter, and the pixel signal output from this CCD12 is an NTSC video signal The operation from when it is converted to 1 and displayed on the monitor will be described. The position of the defective pixel of the CCD 12 is the same as described above.
[0113]
The optical image of the subject by the imaging lens 10 is formed on the imaging surface of the CCD 12 with high resolution. The CCD 12 scans a pixel of horizontal 1280 × vertical 1024 dots of the image and sends the pixel signal to the preprocessing circuit 14. The preprocessing circuit 14 amplifies the input image signal to a predetermined level, and further performs processing such as black level correction, white balance correction, and gamma correction on the amplified image signal, and sends the processed signal to the analog-digital converter 16.
[0114]
The analog-digital converter 16 converts the input analog image signal into a digital value, and further stores the image data converted into the digital value in the frame memory in the analog-digital converter 16 in a time of 120 ms in this embodiment. The pixel data of horizontal 1280 × vertical 1024 dots from the CCD 12 is stored in the frame memory in a pixel arrangement as shown in FIG. The level value of each pixel shown in FIG. 4 (a) in this case is shown in FIG. 5 (a).
[0115]
The image data accumulated in the frame memory in this manner is read every other line by the read control signal of the control unit 38, and the read pixel data R, G, and B (FIG. 4B, FIG. 5 (b)) is sent to the first multiplier 210 via the selector 200. The R multiplier 212 of the first multiplier 210 receives the input pixel data R _n Is multiplied by a coefficient of 1, 3/4, 1/2, or 1/4 and sent to the R adder 222 of the first adder 220. Similarly, the G multiplier 214 and the B multiplier 216 of the first multiplier 210 also input pixel data G. _n Pixel data B _n Is multiplied by a coefficient of 1, 3/4, 1/2, or 1/4 and sent to the G adder 224 and B adder 226 of the first adder 220, respectively.
[0116]
R adder 222 receives R from R multiplier 212. ₁ And R adder 222 add 0 inside R ₁₁ And then R from the R multiplier 212 ₁ 3/4 and R _Five 1/4 and R _{twenty two} And then R from the R multiplier 212 ₁ 1/2 and R _Five Add 1/2 and R ₃₃ And then R from the R multiplier 212 ₁ 1/4 and R _Five Add 3/4 and R ₄₄ And then R from the R multiplier 212 _Five And R adder 222 add 0 inside R ₅₅ And then sequentially R _nn Is sent to the R median filter processing circuit 232 of the first median filter processing circuit 230.
[0117]
The G adder 224 is connected to the G multiplier 214 from the G multiplier 214. ₂ 1/2 and G adder 224 G inside ₀ Add 1/2 and G ₁₁ And then G from G multiplier 214 ₂ And G adder 224 add 0 inside G _{twenty two} And then G from G multiplier 214 ₂ 1/2 and G _Four Add 1/2 and G ₃₃ And then G from G multiplier 214 _Four And G adder 224 add 0 inside G ₄₄ Create G like this _nn Are sequentially generated and sent to the G median filter processing circuit 234 of the first median filter processing circuit 230.
[0118]
The B adder 226 is also connected to the B adder 216 from the B multiplier 216. _Three 1/2 and B adder 226 B inside ₀ Add 1/2 and B ₁₁ And then B from the B multiplier 216 _Three 3/4 and B adder 226 B inside ₀ 1/4 plus B _{twenty two} And then B from the B multiplier 216 _Three And B adder 226 add 0 inside B ₃₃ And then B from B multiplier 216 _Three 3/4 and B ₇ 1/4 plus B ₄₄ And then B from B multiplier 216 _Three 1/2 and B ₇ Add 1/2 and B ₅₅ And then B from B multiplier 216 _Three 1/4 and B ₇ 3/4 plus B ₆₆ And then B from B multiplier 216 ₇ And B adder 226 add 0 inside B ₇₇ And then in order B _nn Is sent to the B median filter processing circuit 236.
[0119]
As a result, the pixel data R output from the first adder 220 is displayed. _nn , G _nn , B _nn 4 (c), and the level value in this case is as shown in FIG. 5 (c).
[0120]
In the case of the R pixel, the first pixel of interest is the NO1 display dot of the NO1 line. Three data of dots are stored. In this case, the display dot data to the left of the NO1 display dot (not shown) is created inside the processing circuit 232, and the other two data are sent from the R adder 222. The three data stored in the storage circuit are read out and sent to the comparison circuit of the processing circuit 232.
[0121]
The comparison circuit compares the level value “1” of the display dot to the left of the NO1 display dot with the level value “1” of the NO1 display dot and the level value “3/4” of the NO2 display dot. In order, that is, in order of “1”, “1”, “3/4”, the median “1” data is output to the output 316 as data subjected to median filter processing in the pixel of interest.
[0122]
When the process using the NO1 display dot as the target pixel is completed in this manner, the processing circuit 232 then enters a process using the NO2 display dot as the target pixel. Also in this process, the process as described above is performed. However, the target display dots are NO1 to NO3 display dots, and these data are sent from the R adder 222. When the processing using the NO3 display dot and the NO4 display dot as the target pixel is completed by the same processing method, the processing circuit 232 then starts the processing using the NO5 display dot based on the defective pixel of the CCD as the target pixel.
[0123]
The storage circuit of the processing circuit 232 stores three data of NO4 display dots, NO5 display dots, and NO6 display dots. The three data stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level value “1/4” of the NO4 display dot with the level value “0” of the NO5 display dot and the level value “1/4” of the NO6 display dot. 4 ”,“ 1/4 ”, and“ 0 ”are arranged in this order, and the median“ 1/4 ”data is output to the output 316 as the median filter data for the pixel of interest.
[0124]
Thus, the level value “0” at the target pixel becomes the level value “1/4” by the median filter processing, that is, the false signal reduction processing is performed by this circuit. In this way, data obtained by performing median filter processing on the target pixel in the R pixel sequentially is created and sent to the first memory 238.
[0125]
In parallel, the G and B median filter processing circuits 234 and 236 also create data obtained by performing median filter processing on the pixel of interest in the G and B pixels by the processing method described above, and output it to the first memory 238. To do.
[0126]
Pixel data R from the first median filter processing circuit 230 _nn , G _nn , B _nn Is stored in the first memory 238 by the write control signal of the input control unit 240. In this case, since the R, G, and B pixel data at each sample point is sent from the processing circuit 230, the first memory 238 is composed of 1280 horizontal x 512 vertical dots. The data of each pixel arrangement shown in FIG. 5 is stored at the level value shown in FIG.
[0127]
In this embodiment, pixel data is read from the frame memory of the analog-digital converter 16 and the pixel data R is read within 120 ms. _nn , G _nn , B _nn Are stored in the first memory 238. Accordingly, the pixel data stored in the first memory 238 can be displayed on the NTSC system monitor device 29 via the reproducing unit 26 in real time. Further, when the monitor device 29 is a monitor having a resolution lower than that of the NTSC system, the pixel data can be thinned out, so that the pixel data is read from the frame memory of the analog-digital converter 16 and the pixel data R _nn , G _nn , B _nn It is possible to further shorten the time until the first memory 238 is stored in the first memory 238.
[0128]
The pixel data R stored in the first memory 238 in this way. _nn , G _nn , B _nn Is read from the first memory 238 by an interlaced readout control signal for reading out pixel data every other pixel for every odd line or every even line from the output control unit 242 and sent to the YC converter 244. Therefore, from the output 320 of the first memory 238, R, G, and B pixel data of 640 × 512 dots are output as one frame.
[0129]
The YC converter 244 receives the pixel data R from the first memory 238. _nn , G _nn , B _nn And the luminance signal Y using the above equation _nn , Color difference signal R _nn -Y _nn And color difference signal B _nn -Y _nn Is sent to the post-processing circuit 22. Since the operation after the post-processing circuit 22 is the same as that in the case of the standard resolution described above, description thereof is omitted.
[0130]
Next, the image sensor 12 of the camera 1 is a high resolution CCD (number of display dots, horizontal 1280 x vertical 1024 dots) with an RGB stripe color filter array, and the pixel signal output from the CCD 12 is converted into an NTSC video signal. The operation until conversion and display on the monitor will be described. The position of the defective pixel of the CCD 12 is the same as described above.
[0131]
The optical image of the subject by the imaging lens 10 is formed on the imaging surface of the CCD 12 with high resolution. The CCD 12 scans a pixel of horizontal 1280 × vertical 1024 dots of the image and sends the pixel signal to the preprocessing circuit 14. The preprocessing circuit 14 amplifies the input image signal to a predetermined level, and further performs processing such as black level correction, white balance correction, and gamma correction on the amplified image signal, and sends the processed signal to the analog-digital converter 16.
[0132]
The analog-digital converter 16 converts the input analog image signal into a digital value, and further stores the image data converted into the digital value in the frame memory in the analog-digital converter 16 in a time of 120 ms in this embodiment. Pixel data of horizontal 1280 × vertical 1024 dots from the CCD 12 is stored in the frame memory in a pixel array as shown in FIG. The level value of each pixel shown in FIG. 6 (a) in this case is shown as an example in FIG. 7 (a).
[0133]
The image data stored in the frame memory is read every other line by the read control signal of the control unit 38, and the read pixel data R, G, and B (FIG. 6B, FIG. 7). b) is sent to the first multiplier 210 via the selector 200. The R multiplier 212 of the first multiplier 210 receives the input pixel data R _n Is multiplied by a coefficient of 1, 2/3, or 1/3 and sent to the R adder 222 of the first adder 220. Similarly, the G multiplier 214 and the B multiplier 216 of the first multiplier 210 also input pixel data G. _n Pixel data B _n Is multiplied by a coefficient of 1, 2/3 or 1/3 and sent to the G adder 224 and B adder 226 of the first adder 220, respectively.
[0134]
R adder 222 receives R from R multiplier 212. ₁ And R adder 222 add 0 inside R ₁₁ And then R from the R multiplier 212 ₁ 2/3 and R _Four 1/3 plus R _{twenty two} And then R from the R multiplier 212 ₁ 1/3 and R _Four 2/3 plus R ₃₃ And then R from the R multiplier 212 _Four And R adder 222 add 0 inside R ₄₄ And then sequentially R _nn Is sent to the R median filter processing circuit 232 of the first median filter processing circuit 230.
[0135]
The G adder 224 is connected to the G multiplier 214 from the G multiplier 214. ₂ 2/3 and G adder 224 G inside ₀ 1/3 plus G ₁₁ And then G from G multiplier 214 ₂ And G adder 224 add 0 inside G _{twenty two} And then G from G multiplier 214 ₂ 2/3 and G _Five 1/3 plus G ₃₃ And then G from G multiplier 214 ₂ 1/3 and G _Five 2/3 plus G ₄₄ And then G from G multiplier 214 _Five And G adder 224 add 0 inside G ₅₅ And then sequentially G _nn Is sent to the G median filter processing circuit 234 of the first median filter processing circuit 230.
[0136]
The B adder 226 is also connected to the B adder 216 from the B multiplier 216. _Three 1/3 and B adder 226 B inside ₀ 2/3 plus B ₁₁ And then B from the B multiplier 216 _Three 2/3 and B adder 226 B inside ₀ 1/3 plus B _{twenty two} And then B from B multiplier 216 _Three And B adder 226 add 0 inside B ₃₃ And then B from the B multiplier 216 _Three 2/3 and B ₆ 1/3 plus B ₄₄ And then B from B multiplier 216 _Three 1/3 and B ₆ 2/3 plus B ₅₅ And then B from B multiplier 216 ₆ Add 0 inside B adder 226 to B ₆₆ And then in order B _nn Is sent to the B median filter processing circuit 236 of the first median filter processing circuit 230.
[0137]
As a result, the pixel data R output from the first adder 220 is displayed. _nn , G _nn , B _nn Is as shown in FIG. 6 (c), and the level value in this case is as shown in FIG. 7 (c).
[0138]
In the case of the R pixel, the first pixel of interest is the NO1 display dot of the NO1 line. Three data of dots are stored. In this case, the display dot data to the left of the NO1 display dot (not shown) is created inside the processing circuit 232, and the other two data are sent from the R adder 222. The three data stored in the storage circuit are read out and sent to the comparison circuit of the processing circuit 232.
[0139]
The comparison circuit compares the level value “1” of the display dot to the left of the NO1 display dot with the level value “1” of the NO1 display dot and the level value “2/3” of the NO2 display dot, and the level value is large. In order, that is, “1”, “1”, and “2/3” are arranged in this order, and the median “1” data is output to the output 316 as median filter processing on the pixel of interest.
[0140]
When the process using the NO1 display dot as the target pixel is completed in this manner, the processing circuit 232 then enters a process using the NO2 display dot as the target pixel. Also in this process, the process as described above is performed. However, the target display dots are NO1 to NO3 display dots. When the process using the NO3 display dot as the pixel of interest is completed in the same process, the processing circuit 232 then enters the process of setting the NO4 display dot based on the defective pixel of the CCD as the pixel of interest.
[0141]
The storage circuit of the processing circuit 232 stores three data of NO3 display dots, NO4 display dots, and NO5 display dots. The three data stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level value “1/3” of the NO3 display dot with the level value “0” of the NO4 display dot and the level value “1/3” of the NO5 display dot. 3 ”,“ 1/3 ”, and“ 0 ”are arranged in this order, and the median“ 1/3 ”data is output to the output 316 as the median filter processed at the target pixel.
[0142]
Thus, the level value “0” at the target pixel becomes the level value “1/3” by the median filter processing, that is, the false signal reduction processing is performed by this circuit. In this way, data obtained by performing median filter processing on the target pixel in the R pixel in sequence is generated and output to the first memory 238.
[0143]
At the same time, the G and B median filter processing circuits 234 and 236 also create data obtained by performing median filter processing on the target pixel in the G and B pixels by the method described above, and output the data to the first memory 238. .
[0144]
Pixel data R from the first median filter processing circuit 230 _nn , G _nn , B _nn Is stored in the first memory 238 by the write control signal of the input control unit 240. In this case, the R, G, and B pixel data at each sample point is sent from the processing circuit 230, so that the first memory 238 has 1280 horizontal x 512 vertical dots as shown in FIG. Data of each pixel arrangement shown is stored as a level value shown in FIG.
[0145]
In this embodiment, pixel data is read from the frame memory of the analog-digital converter 16 and the pixel data R is read within 120 ms. _nn , G _nn , B _nn Are stored in the first memory 238. Accordingly, the pixel data stored in the first memory 238 can be displayed on the NTSC system monitor device 29 via the reproducing unit 26 in real time. Further, when the monitor device 29 is a monitor having a resolution lower than that of the NTSC system, the pixel data can be thinned out, so that the pixel data is read from the frame memory of the analog-digital converter 16 and the pixel data R _nn , G _nn , B _nn It is possible to further shorten the time until the first memory 238 is stored in the first memory 238.
[0146]
Pixel data R read out from the first memory 238 by an interlaced readout control signal for reading out pixel data every other pixel every odd line or every even line from the output control unit 242. _nn , G _nn , B _nn The operation until the image signal is converted to the NTSC video signal is the same as that of the G-striped RB perfect checkered filter, and will not be described.
[0147]
According to the first embodiment, the movie processing unit 20 performs interpolation processing on the pixel data R, G, and B from the high-resolution CCD 12, and the pixel data R, G, and B is obtained and then reduced by performing a reduction process on the false signal based on the defective pixel of CCD12 included in the synchronized pixel data R, G and B, and also on the pixel signal which is not a defective pixel of CCD12. The processed pixel data R, G and B are obtained, and then the pixel data R, G having the number of pixels which can be displayed on the NTSC monitor device by thinning the reduced pixel data R, G and B. And B, and then YC conversion is performed on the converted pixel data R, G, and B to obtain luminance signal data Y and color difference signal data RY, BY, so that a high-resolution subject image has good image quality. Replay video in real time There is an effect that can be displayed on the monitor device 29 via the section 26.
[0148]
FIG. 8 shows a second embodiment of an electronic still camera to which the image signal processing apparatus according to the present invention is applied. In the electronic still camera 2 shown in the figure, the difference from the above-described electronic still camera 1 is that only the false signal generated due to the defective pixel of the image sensor 12 included in the high resolution signal representing the electric signal of the subject image is reduced. The point is that processing is performed.
[0149]
As shown in FIG. 8, the camera 2 includes an imaging lens 10, an imaging device 12, a preprocessing circuit 14, an analog / digital (A / D) converter 16, a YC processing unit 18, a post processing circuit 22, a digital analog (D / D). A) Converters 24 and 40, playback unit 26, selector 28, monitor device 29, compression processing unit 30, memory 32, interface (I / F) 34, memory card 36, drive signal generation circuit 44, defect position data storage circuit 46, a movie processing unit 48, a control unit 50, and a timing signal generating circuit 52.
[0150]
8 differs from FIG. 1 in that a defect position data storage circuit 46 is added, the movie processing unit 20 in FIG. 1 is changed to a movie processing unit 48 in FIG. The control unit 38 is changed to the control unit 50 in FIG. 8, the timing signal generation circuit 42 in FIG. 1 is changed to the timing signal generation circuit 52 in FIG. 8, the timing signal generation circuit 52 and the defect position data. The control line 132 is added between the storage circuit 46 and the signal line 134 is added between the defect position data storage circuit 46 and the control unit 50. In FIG. 8, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and redundant description is omitted.
[0151]
The defect position data storage circuit 46 is composed of a ROM or the like. In this embodiment, the defect position data storage circuit 46 is a storage circuit that accumulates data indicating the defective pixel position of the high resolution CCD 12. The accumulated defective pixel position data is read by a read control signal 132 from the timing signal generation circuit 52 and is output from the output 134 to the control unit 50. The defective pixel position data is composed of data indicating the line position where the defective pixel exists and address data indicating the pixel position in the line at the line position.
[0152]
The control unit 50 forms first control signals for controlling and managing the respective functional units similar to those shown in FIG. 1 based on various timing signals sent from the timing signal generating circuit 52 via the connection line 126. In addition to having a circuit (not shown), the control unit 52 also has a second control circuit (not shown) for generating a control signal for controlling and managing the movie processing unit 48 in particular.
[0153]
In the present embodiment, the second control circuit is a high-resolution image stored in the frame memory of the analog-digital converter 16 described above, particularly when an object image from a high-resolution CCD is monitored by, for example, an NTSC monitor device. A read control signal for reading the image data is generated, and an operation signal for the movie processing unit 48 that performs image processing on the image data read from the frame memory is also generated. In particular, the control signal to the movie processing unit 48 is generated based on data indicating the defective pixel position from the defective position data storage circuit 46.
[0154]
When the control unit 50 receives a shutter release signal by pressing the button from a release button circuit (not shown), the control unit 50 forms an activation signal for starting photographing, and sends it to the timing signal generation circuit 52 via the control line 124. send.
[0155]
Upon receiving the activation signal 124 from the control unit 50, the timing signal generation circuit 52 forms various timing signals for operating the drive signal generation circuit 44 synchronized with the activation signal 124, and drives them via the signal line 128. Various timing signals are output to the signal generating circuit 44 and synchronized with the start signal 124 to operate the control unit 50, and are output to the control unit 50 through the signal line 126. This is a synchronization signal generating circuit that forms various timing signals for operating the synchronized defect position data storage circuit 46 and outputs them to the control unit 50 via the control line 132. These various timing signals are formed based on a clock signal output from a reference signal oscillator (not shown) in the timing signal generation circuit 52.
[0156]
FIG. 9 shows the internal configuration of the movie processing unit 48 of the camera 2.
[0157]
In this embodiment, the movie processing unit 48 first performs interpolating processing on the pixel data R, G and B from the high resolution CCD appearing at the input 104 to synchronize the pixel data R, G and B is obtained, and then the reduced pixel data R, G, and B are obtained by performing the reduction process only on the false signal based on the defective pixel of the CCD included in the synchronized pixel data R, G, and B. The pixel data R, G, and the number of pixels that can be displayed on the NTSC monitor device by thinning the synchronized pixel data R, G, and B including the pixel data R, G, and B that have been reduced The converted pixel data R, G and B are output to the output 107, or YC conversion is performed on the converted pixel data R, G and B to obtain the luminance signal data Y and the color difference signal data RY. , BY got An image processing circuit for outputting the force 106.
[0158]
As shown in FIG. 9, the movie processing unit 48 includes a selector 200, a first multiplier 210, a first adder 220, a first median filter processing circuit 280, a first memory 238, an input control unit 288, An output control unit 242, a YC converter 244, a second multiplier 250, a second adder 260, a second median filter processing circuit 290, and a second memory 278 are provided.
[0159]
9 differs from FIG. 2 in that the first median filter processing circuit 230 in FIG. 2 is changed to the first median filter processing circuit 280 in FIG. 9, and the input control unit 240 in FIG. 9 is changed to the input control unit 288, and the second median filter processing circuit 270 in FIG. 2 is changed to the second median filter processing circuit 290 in FIG. 9, parts corresponding to those in FIG. 2 are denoted by the same reference numerals, and redundant description is omitted.
[0160]
The pixel data of the three systems R, G, and B interpolated by the first adder 220 is reduced only for the false signal based on the defect of the CCD12 pixel in the first median filter processing circuit 280. Done.
[0161]
The first median filter processing circuit 280 is composed of an R median filter processing circuit 282, a G median filter processing circuit 284, and a B median filter processing circuit 286, and the second median filter processing circuit 290 is an R median filter processing circuit. 292, a G median filter processing circuit 294, and a B median filter processing circuit 296.
[0162]
Each of the processing circuits 282, 284, 286, 292, 294, 296 is composed of the same circuit, and this circuit converts the input signal when the input signal is not a false signal based on the defective pixel of the CCD12. If the signal is a fake signal, the fake signal and a signal around one pixel of the fake signal are input, the level values of the three input pixels are compared, and then arranged in descending order of the level value. Then, the median value is obtained, and a signal of the obtained median is output.
[0163]
Here, the R median filter processing circuit 282 will be described as an example. In this example, the processing circuit 282 includes a first switch circuit, a second switch circuit, a memory circuit, and a comparison circuit. The first switch circuit receives the pixel data R from the R adder 222 at its input terminal, and when the received pixel data R is not a false signal based on a defective pixel of the CCD 12, outputs it to its output terminal 1. Further, in the case of a false signal, the switching circuit outputs a false signal and pixel data R around one pixel of the false signal to its output terminal 2.
[0164]
The storage circuit is a storage circuit that temporarily stores a false signal from the output terminal 2 of the first switch circuit and pixel data R of around 1 pixel of the false signal. These three pixel data R temporarily stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level values of the three pixels sent, arranges them in descending order of the level values, then obtains their median (median), and the obtained median is input to the second switch circuit. This is a circuit to be sent to the terminal 2.
[0165]
The second switch circuit has an input terminal 1 that receives pixel data R that is not a false signal from the output terminal 1 of the first switch circuit, and this circuit has a case where the input pixel data R is not a false signal based on a defective pixel. Is a switching circuit that outputs pixel data R input to the input terminal 1 to the output terminal, or pixel data R indicating the median input to the input terminal 2 to the output terminal when it is a false signal. The output terminal of the second switch circuit is connected to the signal line 316.
[0166]
In this embodiment, such processing is performed by the median filter processing control formed by the input control unit 288 based on the control signal 122 for generating the median filter processing control signal from the second control circuit of the control unit 50. Based on signal 318.
[0167]
Referring to FIG. 9, the input control unit 288 is based on the control signal 122 for generating the write control signal from the second control circuit of the control unit 50 shown in FIG. This is a control circuit for generating write control signals for the first memory 238 and the second memory 278.
[0168]
The input control unit 288 also includes first and second median filters based on the control signal 122 for generating the median filter processing control signal from the second control circuit of the control unit 50, as mentioned briefly above. A control circuit that forms a median filter processing control signal 318 to processing circuits 280 and 290. In this second embodiment, the median filter processing control signal 318 is a control signal for performing reduction processing only for defective pixels on one screen.
[0169]
The operation of the second embodiment will be described.
[0170]
First, the image sensor 12 of the camera 2 is a high resolution CCD (number of display dots, horizontal 1280 x vertical 1024 dots) of G stripe RB perfect checkered filter, and the pixel signal output from this CCD12 is converted to an NTSC video signal. The operation until conversion and display on the monitor will be described. The position of the defective pixel of the CCD 12 is the same position as in FIG.
[0171]
Analog pixel signals from the high-resolution CCD 12 are sent to the analog-digital converter 16 via the pre-processing circuit 14, converted into digital pixel signals, and stored, and the stored pixel data is read every other line. Pixel data R sent to the first adder 220 via the selector 200 and the first multiplier 210 of the movie processor 48 and synchronized for each pixel. _nn , G _nn , B _nn Since the operation up to the point where the above is obtained is the same as that of the high resolution CCD 12 having the G stripe RB complete checkered filter of the first embodiment, the description is omitted.
[0172]
As a result, the pixel data R output from the first adder 220 is displayed. _nn , G _nn , B _nn 4 (c), and the level value in this case is as shown in FIG. 5 (c).
[0173]
The R pixel will be described. The first pixel is the NO1 display dot of the NO1 line, which is not a defective pixel. Therefore, the input terminal is connected to the output terminal 1 of the first switch circuit of the R median filter processing circuit 282. A control signal for connection is sent from the input control unit 288, and a control signal for connecting the input terminal 1 to the output terminal is sent from the input control unit 288 to the second switch circuit. Therefore, data of level value “1” is output from the output 316 of the processing circuit 282.
[0174]
When the processing using the NO1 display dot as the target pixel is completed as described above, the processing circuit 282 next enters the processing using the NO2 display dot as the target pixel. Also in this process, since the second pixel is not a defective pixel, the above process is performed. Similarly, since the third and fourth pixels are not defective pixels, the above-described processing is performed. Therefore, from the output 316 of the processing circuit 282, the data of the level value “3/4” for the second pixel, the data of the level value “1/2” for the third pixel, the data for the fourth pixel Data with level value “1/4” is output.
[0175]
When the process using the NO4 display dot as the target pixel is completed, the processing circuit 282 next enters a process using the NO5 display dot as the target pixel. In this case, a control signal for connecting the input terminal to the output terminal 2 is sent from the input control unit 288 to the first switch circuit of the processing circuit 282, and the input terminal 2 is sent to the second switch circuit. Is sent from the input control unit 288 to connect to the output terminal.
[0176]
As a result, three data of NO4 display dots, NO5 display dots, and NO6 display dots are stored in the storage circuit of the processing circuit 282 via the first switch circuit. The three data stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level value “1/4” of the NO4 display dot with the level value “0” of the NO5 display dot and the level value “1/4” of the NO6 display dot. ”,“ 1/4 ”,“ 0 ”are arranged in this order, and the median“ 1/4 ”data is output as the data subjected to the median filter processing in the target pixel through the second switch circuit 316. Output to.
[0177]
That is, in the processing circuit 282, when the input pixel data R is not a false signal based on the defective pixel, the input pixel data R is sent to the first memory 238 as it is, and when it is a false signal, Pixel data R indicating the median is obtained and sent to the first memory 238. Such processing is also performed in the processing circuits 284 and 286 for the pixel data G and B.
[0178]
Pixel data R from the first median filter processing circuit 280 _nn , G _nn , B _nn Is stored in the first memory 238 by the write control signal of the input control unit 288. In this case, since the R, G, and B pixel data at each sample point is sent from the processing circuit 280, the first memory 238 has 1280 horizontal x 512 vertical dots as shown in FIG. Pixel data of the pixel arrangement shown is stored at the level value shown in FIG.
[0179]
Similarly to the first embodiment, the second embodiment reads pixel data from the frame memory of the analog-digital converter 16 and the pixel data R within 120 ms. _nn , G _nn , B _nn Are stored in the first memory 238. Accordingly, the pixel data stored in the first memory 238 can be displayed on the NTSC system monitor device 29 via the reproducing unit 26 in real time.
[0180]
Further, when the monitor device 29 is a monitor having a resolution lower than that of the NTSC system, the pixel data can be thinned out, so that the pixel data is read from the frame memory of the analog-digital converter 16 and the pixel data R _nn , G _nn , B _nn It is possible to further shorten the time until the first memory 238 is stored in the first memory 238. Since the operation after the first memory 238 is the same as that in the first embodiment, the description thereof is omitted.
[0181]
Next, the image sensor 12 of the camera 2 is a high resolution CCD (number of display dots, horizontal 1280 x vertical 1024 dots) with an RGB stripe color filter array, and the pixel signal output from the CCD 12 is converted into an NTSC video signal. The operation until conversion and display on the monitor will be described. The position of the defective pixel of the CCD 12 is the same position as in FIG.
[0182]
Analog pixel signals from the high-resolution CCD 12 are sent to the analog-digital converter 16 via the pre-processing circuit 14, converted into digital pixel signals, and stored, and the stored pixel data is read every other line. Pixel data R sent to the first adder 220 via the selector 200 and the first multiplier 210 of the movie processor 48 and synchronized for each pixel. _nn , G _nn , B _nn Since the operation up to the point where the above is obtained is the same as that of the high resolution CCD 12 having the RGB stripe color filter of the first embodiment, the description thereof is omitted.
[0183]
As a result, the pixel data R output from the first adder 220 is displayed. _nn , G _nn , B _nn Is as shown in FIG. 6 (c), and the level value in this case is as shown in FIG. 7 (c).
[0184]
The R pixel will be described. The first pixel is the NO1 display dot of the NO1 line, which is not a defective pixel. Therefore, the input terminal is connected to the output terminal 1 of the first switch circuit of the R median filter processing circuit 282. A control signal for connection is sent from the input control unit 288, and a control signal for connecting the input terminal 1 to the output terminal is sent from the input control unit 288 to the second switch circuit. Therefore, data of level value “1” is output from the output 316 of the processing circuit 282.
[0185]
When the processing using the NO1 display dot as the target pixel is completed as described above, the processing circuit 282 next enters the processing using the NO2 display dot as the target pixel. Also in this process, since the second pixel is not a defective pixel, the above process is performed. Similarly, since the third pixel is not a defective pixel, the above-described processing is performed. Accordingly, from the output 316 of the processing circuit 282, the data of the level value “2/3” is output for the second pixel, and the data of the level value “1/3” is output for the third pixel.
[0186]
When the process using the NO3 display dot as the target pixel is completed, the processing circuit 282 then enters the process using the NO4 display dot as the target pixel. In this case, a control signal for connecting the input terminal to the output terminal 2 is sent from the input control unit 288 to the first switch circuit of the processing circuit 282, and the input terminal 2 is sent to the second switch circuit. Is sent from the input control unit 288 to connect to the output terminal.
[0187]
As a result, three data of NO3 display dot, NO4 display dot, and NO5 display dot are stored in the storage circuit of the processing circuit 282 via the first switch circuit. The three data stored in the storage circuit are read out and sent to the comparison circuit. The comparison circuit compares the level value “1/3” of the NO3 display dot with the level value “0” of the NO4 display dot and the level value “1/3” of the NO5 display dot. ”,“ 1/3 ”,“ 0 ”are arranged in this order, and the median“ 1/3 ”data is output through the second switch circuit 316 as data subjected to median filter processing at the target pixel. Output to.
[0188]
That is, in the processing circuit 282, when the input pixel data R is not a false signal based on the defective pixel, the input pixel data R is sent to the first memory 238 as it is, and when it is a false signal, Pixel data R indicating the median is obtained and sent to the first memory 238. Such processing is also performed in the processing circuits 284 and 286 for the pixel data G and B.
[0189]
Pixel data R from the first median filter processing circuit 280 _nn , G _nn , B _nn Is stored in the first memory 238 by the write control signal of the input control unit 288. In this case, since the R, G, and B pixel data at each sample point is sent from the processing circuit 280, the first memory 238 is shown in FIG. Pixel data of the pixel arrangement shown is stored at the level value shown in FIG.
[0190]
Similarly to the first embodiment, the second embodiment reads pixel data from the frame memory of the analog-digital converter 16 and the pixel data R within 120 ms. _nn , G _nn , B _nn Are stored in the first memory 238. Accordingly, the pixel data stored in the first memory 238 can be displayed on the NTSC system monitor device 29 via the reproducing unit 26 in real time.
[0191]
Further, when the monitor device 29 is a monitor having a resolution lower than that of the NTSC system, the pixel data can be thinned out, so that the pixel data is read from the frame memory of the analog-digital converter 16 and the pixel data R _nn , G _nn , B _nn It is possible to further shorten the time until the first memory 238 is stored in the first memory 238. Since the operation after the first memory 238 is the same as that in the first embodiment, the description thereof is omitted.
[0192]
According to the second embodiment, the movie processing unit 48 performs interpolation processing on the pixel data R, G, and B from the high-resolution CCD 12, and the pixel data R, G, and B is obtained, and then the pixel data R, G, and B obtained by performing the reduction process only on the false signal for the defective pixel of the CCD12 included in the synchronized pixel data R, G, and B are obtained. For pixel signals that are not defective pixels, pixel data R, G, and B are output as they are, and then the reduced pixel data R, G, and B and the pixel data R, G, and B that are output as they are mixed The pixel data is thinned to be converted into pixel data R, G and B having the number of pixels which can be displayed on the NTSC system monitor, and then the YC conversion is performed on the converted pixel data R, G and B. line Thus, since the luminance signal data Y and the color difference signal data RY and BY are obtained, a high-resolution subject image can be displayed on the monitor device 29 via the playback unit 26 in real time. effective.
[0193]
Although the signal processing has been described for pixel data in the horizontal scanning line direction, similar signal processing can be performed for pixel data in the vertical direction.
[0194]
In the first and second embodiments, the median filter process is performed on the interpolated pixel data, and the pixel data subjected to the median filter process is thinned out and stored, or is thinned after the storage. However, the interpolated pixel data may not be subjected to median filter processing, and the interpolated pixel data may be thinned out and stored, or may be thinned out after storage.
[0195]
As described above, in this embodiment, when the imaging device 12 of the cameras 1 and 2 is a high-resolution CCD, the number of pixels of the high-resolution subject image formed on the CCD can be displayed on the NTSC monitor device. Movie processing units 20 and 48 for converting the number of pixels are provided.
[0196]
The present invention is applied to an electronic still camera, and an image formed on a high-resolution CCD having a horizontal 1280 × vertical 1024-dot configuration composed of a G-strip RB complete checkered filter array or an RGB stripe-color filter array is displayed on an NTSC monitor device. Although a specific processing circuit that can be used has been described, the present invention provides an image formed on a high-resolution CCD such as a Bayer color filter array, an interline color filter array, a G stripe RB checkered filter array, or a diagonal stripe color filter array. , PAL system monitor device, SECAM system monitor device, 525/60 system RGB monitor device, 625/50 system RGB monitor device, or a monitor device with lower resolution than the above monitor devices can do.
[0197]
【The invention's effect】
As described above, according to the present invention, the image signal processing device and the image signal processing device of the electronic still camera have a plurality of color filters on the front surface, and color image information corresponding to these color filters is obtained from each pixel. A color image signal picked up by a color image pickup means comprising a solid-state image pickup device obtained in a dot-sequential manner is obtained as color image data in the form of a digital signal in the order in which it is obtained, and is synchronized for each pixel from the received color image data. Pixel interpolating means for obtaining red R, green G, and blue B primary color signal data, and receiving each primary color signal data of red R, green G, and blue B synchronized for each pixel from the pixel interpolating means. A median filter process is performed on a continuous n-pixel region including a predetermined number of pixels of the same color adjacent to each other on the horizontal scanning line centered on each pixel. From the median filter processing means for outputting the color signal data indicating the median, or the pixel interpolation means at the timing corresponding to each pixel having no defect among the plurality of pixels of the solid-state image sensor based on the position data read from the storage means The three primary color signal data of red R, green G, and blue B are output as they are, and the red R, green G from the pixel interpolating means at the timing corresponding to each defective pixel among the plurality of pixels of the solid-state imaging device. With respect to the three primary color signal data of blue B, median filter processing is performed on consecutive n pixel regions including a predetermined number of the same color elements adjacent to each other on the horizontal scanning line around the color elements corresponding to the defective pixels. And a median filter processing means for outputting color signal data indicating a median value by processing.
[0198]
Since these median filter processing means is a processing circuit in an n pixel region on one horizontal scanning line, it can process a false signal based on a defective pixel of the solid-state imaging device in a short processing time and the circuit scale is small. There is an effect that can be done.
[0199]
Therefore, these median filter processing means can be used when a high-resolution subject image captured by an electronic still camera with a high-resolution solid-state imaging device is reproduced in real time on a standard television system resolution monitor such as NTSC. There is an effect that can be done.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a first embodiment of an electronic still camera to which an image signal processing apparatus according to the present invention is applied.
FIG. 2 is a block diagram showing an example of a movie processing unit applied to the electronic still camera shown in FIG.
3 is an explanatory diagram illustrating an example of median filter processing performed by the movie processing unit illustrated in FIGS. 2 and 9. FIG.
FIG. 4 is an explanatory diagram showing an example of signal processing performed by the movie processing unit shown in FIGS. 2 and 9 that receives pixel data from a high-resolution CCD with a G-strip RB complete checkered filter arrangement.
FIG. 5 is a diagram illustrating an example of level values in the explanatory diagram illustrated in FIG. 4;
6 is an explanatory diagram showing an example of signal processing performed by the movie processing unit shown in FIGS. 2 and 9 that receives pixel data from a high-resolution CCD with an RGB stripe color filter array. FIG.
7 is a diagram showing an example of level values in the explanatory diagram shown in FIG. 6. FIG.
FIG. 8 is a block diagram showing a second embodiment of an electronic still camera to which the image signal processing apparatus according to the present invention is applied.
9 is a block diagram showing an example of a movie processing unit applied to the electronic still camera shown in FIG.
[Explanation of symbols]
1, 2 Electronic still camera
10 Imaging lens
12 Image sensor
14 Pre-processing circuit
16 Analog-to-digital converter (A / D)
18 YC processing section
20, 48 Movie processing part
22 Post-processing circuit
24, 40 Digital analog converter (D / A)
26 Playback section
28, 200 selector
30 Compression processing section
32 memory
34 Interface (I / F)
36 Memory card
38, 50 Control unit
42, 52 Timing signal generator
44 Drive signal generation circuit
46 Defect position data storage circuit
210 First multiplier
220 First adder
230, 280 First median filter processing circuit
238 First memory
240, 288 input controller
242 Output controller
244 YC converter
250 Second multiplier
260 Second adder
270, 290 Second median filter processing circuit
278 Second memory

Claims (12)

A color image signal picked up by a color image pickup means comprising a solid-state image pickup device having a color filter of a plurality of colors on the front face and obtaining color image information corresponding to these color filters from each pixel in a dot-sequential manner can be obtained. In an image signal processing apparatus that receives and processes the signal in the form of a digital signal in order, the apparatus monitors the high-resolution pixel data,
Receives color image data in the form of a digital signal based on the pixels sent in the order obtained, and synchronizes the received color image data for each color pixel to produce three primary color signal data of red R, green G, and blue B Pixel interpolation means for obtaining
The three primary color signal data of the red R, green G, and blue B are received from the pixel interpolating means, and a predetermined color among the pixels of the same color as the adjacent pixels on the horizontal scanning line centering on each pixel of the data. Median filter processing means for performing median filter processing for each color element of the three primary color signal data for continuous n pixel regions including a number of pixels, and outputting color signal data indicating a median value by the processing;
The median filter processing means, rows that have a median filtering process for all pixels regardless of whether there is a defective pixel on the solid-state imaging device,
The apparatus further receives, from the median filter processing means, three primary color signal data of red R, green G, and blue B synchronized for each pixel subjected to median filter processing, and based on a predetermined thinning control signal, the median filter An image signal processing apparatus comprising pixel thinning means for thinning out three primary color signal data from a processing means and converting them into low resolution three primary color signal data . A color image signal picked up by a color image pickup means comprising a solid-state image pickup device having a color filter of a plurality of colors on the front face and obtaining color image information corresponding to these color filters from each pixel in a dot-sequential manner can be obtained. In an image signal processing apparatus that receives and processes the signal in the form of a digital signal in order, the apparatus monitors the high-resolution pixel data,
Storage means for storing position data indicating a defective pixel position among a plurality of pixels of the solid-state imaging device;
Receives color image data in the form of a digital signal based on the pixels sent in the order obtained, and synchronizes the received color image data for each color pixel to produce three primary color signal data of red R, green G, and blue B Pixel interpolation means for obtaining
Based on the position data read from the storage means, the three primary color signals of red R, green G, and blue B from the pixel interpolation means at the timing corresponding to each pixel having no defect among the plurality of pixels of the solid-state imaging device The data is output as it is, and the three primary color signal data of red R, green G, and blue B from the pixel interpolating means at the timing corresponding to each defective pixel among the plurality of pixels of the solid-state imaging device The median filter processing is performed on the consecutive n pixel regions including a predetermined number of pixels of the same color as the pixels adjacent to the front and rear of the horizontal scanning line with the defective pixels as the center. A median filter processing means for performing each color element and outputting color signal data indicating a median value by the processing;
The median filter processing means includes a predetermined number of elements among pixels of the same color as the adjacent pixels on the horizontal scanning line around the defective pixel when the solid-state imaging device has a defective pixel. Median filter processing is performed for each color element of the three primary color signal data only for continuous n pixel regions,
The apparatus further receives, from the median filter processing means, three primary color signal data of red R, green G, and blue B synchronized for each pixel subjected to median filter processing, and based on a predetermined thinning control signal, the median filter An image signal processing apparatus comprising pixel thinning means for thinning out three primary color signal data from a processing means and converting them into low resolution three primary color signal data.   3. The image signal processing apparatus according to claim 1, wherein the solid-state imaging device is a high-resolution CCD. 4. A color image signal picked up by a color image pickup means comprising a high-resolution solid-state image pickup device having a color filter of a plurality of colors on the front face and obtaining color image information corresponding to these color filters in a dot-sequential manner from each pixel. In an electronic still camera equipped with an image signal processing device that receives and processes the signals in the form of digital signals in the order obtained,
When the image signal processing device monitors high-resolution pixel data,
The color image data in the form of digital signals based on the pixels sent in the order obtained is received, and the received color image data is synchronized for each color pixel based on a predetermined pixel interpolation control signal, and red R, green G Pixel interpolating means for obtaining the three primary color signal data of blue B,
The three primary color signal data of red R, green G, and blue B are received from the pixel interpolating means, and before and after the horizontal scanning line centered on each pixel of the three primary color signal data based on a predetermined median filter processing control signal. Median filter processing is performed for each color element of the three primary color signal data for consecutive n pixel regions including a predetermined number of pixels of the same color as the adjacent pixels, and color signal data indicating a median value by the processing is obtained. Median filter processing means for outputting;
Control means for generating and outputting a control signal including a pixel interpolation control signal to the pixel interpolation means and a median filter processing control signal to the median filter processing means, and
The control means causes median filter processing to be performed for all pixels regardless of whether or not the solid-state imaging device has a defective pixel by a median filter processing control signal to the median filter processing means ,
The image signal processing device further receives red R, green G, and blue B primary color signal data synchronized for each pixel subjected to median filter processing from the median filter processing means, and based on a predetermined thinning control signal. Pixel thinning means for thinning out the three primary color signal data from the median filter processing means and converting the data into low resolution three primary color signal data;
The electronic still camera provided with the image signal processing device, wherein the control means further includes a thinning control signal generating means for generating and outputting a control signal including a thinning control signal to the pixel thinning means . An electronic still camera comprising the image signal processing device according to claim 4 .
The image signal processing apparatus further includes:
A first output terminal for outputting red R, green G, and blue B primary color signal data in real time for each low resolution pixel from the pixel thinning means;
The camera equipped with the image signal processing device further includes:
A first viewfinder of a video monitor that displays video of three primary color signal data of red R, green G, and blue B synchronized for each low-resolution pixel from the first output terminal. An electronic still camera equipped with an image signal processing device. An electronic still camera comprising the image signal processing device according to claim 4 .
The image signal processing apparatus further includes:
Three primary color signal data of red R, green G, and blue B, which are synchronized for each low-resolution pixel, are received from the pixel decimation unit, and 3 for each pixel from the pixel decimation unit is synchronized based on a predetermined conversion control signal. Including color signal conversion means for generating synchronized luminance signal data Y, color difference signal data RY and color difference signal data BY for each pixel from the primary color signal data;
The electronic still camera provided with an image signal processing device, wherein the control means further includes conversion control signal generation means for generating and outputting a control signal including a conversion control signal to the color signal conversion means. An electronic still camera comprising the image signal processing device according to claim 6 .
The image signal processing apparatus further includes:
A second output terminal for outputting luminance signal data Y, color difference signal data RY and color difference signal data BY synchronized for each low resolution pixel from the color signal conversion means in real time;
The camera equipped with the image signal processing device further includes:
And a second view finder of a video monitor for displaying a video of luminance signal data Y, color difference signal data RY and color difference signal data BY synchronized for each low resolution pixel from the second output terminal. An electronic still camera provided with an image signal processing device. A color image signal picked up by a color image pickup means comprising a high-resolution solid-state image pickup device having a color filter of a plurality of colors on the front face and obtaining color image information corresponding to these color filters in a dot-sequential manner from each pixel. In an electronic still camera equipped with an image signal processing device that receives and processes the signals in the form of digital signals in the order obtained,
When the image signal processing device monitors high-resolution pixel data,
Storage means for storing position data indicating a defective pixel position among a plurality of pixels of the solid-state imaging device;
The color image data in the form of digital signals based on the pixels sent in the order obtained is received, and the received color image data is synchronized for each color pixel based on a predetermined pixel interpolation control signal, and red R, green G Pixel interpolating means for obtaining the three primary color signal data of blue B,
The pixel interpolating means at a timing corresponding to each pixel having no defect among a plurality of pixels of the solid-state imaging device based on a predetermined median filter processing control signal based on position data read from the storage means based on a predetermined reading control signal The three primary color signal data of red R, green G, and blue B are output as they are, and the red R from the pixel interpolating means at the timing corresponding to each defective pixel among the plurality of pixels of the solid-state image sensor. For the three primary color signal data of green G and blue B, a continuous number including a predetermined number of pixels among the pixels of the same color as the adjacent pixels on the horizontal scanning line around the defective pixels as the center Median filter processing is performed for each color element of the three primary color signal data for n pixel regions, and color signal data indicating the median value by the processing is output. And the median filter processing means that,
Generate and output a control signal including a read control signal to the storage means, a control signal including a pixel interpolation control signal to the pixel interpolation means, and a median filter processing control signal to the median filter processing means Control means for
When there is a defective pixel in the solid-state imaging device by the median filter processing control signal to the median filter processing means, the control means has the same color as the adjacent pixels on the horizontal scanning line around the defective pixel. of the pixels, rows that are in each color component of the three primary color signal data a median filtering process only for the n pixel region contiguous including a predetermined number of pixels,
The image signal processing device further receives red R, green G, and blue B primary color signal data synchronized for each pixel subjected to median filter processing from the median filter processing means, and based on a predetermined thinning control signal. Pixel thinning means for thinning out the three primary color signal data from the median filter processing means and converting the data into low resolution three primary color signal data;
The electronic still camera provided with the image signal processing device, wherein the control means further includes a thinning control signal generating means for generating and outputting a control signal including a thinning control signal to the pixel thinning means. An electronic still camera comprising the image signal processing device according to claim 8 .
The image signal processing apparatus further includes:
A first output terminal for outputting red R, green G, and blue B primary color signal data in real time for each low resolution pixel from the pixel thinning means;
The camera equipped with the image signal processing device further includes:
A first viewfinder of a video monitor that displays video of three primary color signal data of red R, green G, and blue B synchronized for each low-resolution pixel from the first output terminal. An electronic still camera equipped with an image signal processing device. An electronic still camera comprising the image signal processing device according to claim 8 .
The image signal processing apparatus further includes:
Three primary color signal data of red R, green G, and blue B, which are synchronized for each low-resolution pixel, are received from the pixel decimation unit, and 3 for each pixel from the pixel decimation unit is synchronized based on a predetermined conversion control signal. Including color signal conversion means for generating synchronized luminance signal data Y, color difference signal data RY and color difference signal data BY for each pixel from the primary color signal data;
The electronic still camera provided with an image signal processing device, wherein the control means further includes conversion control signal generation means for generating and outputting a control signal including a conversion control signal to the color signal conversion means. An electronic still camera comprising the image signal processing device according to claim 10 .
The image signal processing apparatus further includes:
A second output terminal for outputting luminance signal data Y, color difference signal data RY and color difference signal data BY synchronized for each low resolution pixel from the color signal conversion means in real time;
The camera equipped with the image signal processing device further includes:
And a second view finder of a video monitor for displaying a video of luminance signal data Y, color difference signal data RY and color difference signal data BY synchronized for each low resolution pixel from the second output terminal. An electronic still camera provided with an image signal processing device. In an electronic still camera provided with an image signal processing apparatus according to any one of claims 4 to 11, wherein the solid-state imaging device, an electronic still camera provided with an image signal processing apparatus which is a high resolution CCD .

Images