JP6437766B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus having a function of converting and outputting a video signal from an imaging element.

In recent years, with the advancement of digital technology of television camera systems, the number of pixels of the imaging device has changed from 1920 × 1080 pixels (abbreviated as 2K) to approximately 4000 × 2000 pixels (abbreviated as 4K) and approximately 8000 × 4000 pixels (abbreviated as 8K). Is required.
Therefore, in order to increase the resolution by shifting pixels, in a multi-panel color camera with shifted spatial pixels, the R signal and G signal whose sampling phases are inverted with each other by shifting the spatial pixels are switched at opposite phases of the sampling frequency. Then, a signal in which the R signal and the G signal are alternately switched is generated, and after the R signal is mixed, the band is limited by a band pass filter, and then combined with the luminance signal output from the matrix circuit. (See Patent Document 1).
Furthermore, in a multi-panel color camera with spatial pixel shifting, the luminance signal (Y) is generated from the R, G, B signals by a matrix calculation of a double-speed clock every half the sampling period (1 / fs) of the solid-state imaging device. Output at double speed sampling, and before the matrix calculation, the high frequencies of the R and B signals are amplified from the rated values, and the high frequency phase of the G signal is set to a predetermined cycle (1 / fs) from the phase of the R and B signals. Delay half (see Patent Document 2).
By the way, the G (green) 1 image sensor is arranged with a 1/2 pixel pitch shift in the horizontal direction and a 1/2 pixel pitch shift in the vertical direction with respect to the G2 image sensor. Then, high frequency components in the vertical direction and the horizontal direction are calculated from the obtained G1 and G2 signals and added to the G signal to improve the vertical and horizontal resolutions (see Patent Document 3). .
In addition, the imaging apparatus has a four-plate configuration with B (blue), R (red), G (green) 1 and G2 imaging elements. The B and R image sensors are arranged with a 1/2 pixel pitch shift in the vertical direction with respect to the G2 image sensor. Then, the high frequency component in the vertical direction is calculated from the obtained B and R signals and added to the G signal to improve the vertical resolution. Further, the G1 and G2 image sensors are arranged with a ½ pixel pitch shift in the horizontal direction to improve the resolution in the horizontal direction (see Patent Document 4).
In addition, the imaging apparatus has a four-plate configuration with B (blue), R (red), G (green) 1 and G2 imaging elements. The R image sensor is arranged with a 1/2 pixel pitch shift in the vertical direction with respect to the G2 image sensor. The B image sensor is arranged with a one-pixel pitch shift in the vertical direction with respect to the R image sensor. Then, a high frequency component in the vertical direction is calculated from the obtained B and R signals and added to the G signal to improve the resolution in the vertical direction. Since a high frequency component can be obtained from signals in the same field, generation of false signals is prevented. (See Patent Document 5).

In a television camera using three color separation optical systems and three image sensors, the axial positions of the red image sensor and the blue image sensor are set to green for the typical value of axial chromatic aberration in order to facilitate the production of a zoom lens. The image is offset from the axial position of the image sensor and bonded to the color separation optical system.
For this reason, when a zoom lens for three lenses is combined with a single-plate color image sensor via an adapter with a dummy glass for the color separation optical system, the red image sensor and the blue image sensor are equivalent to the typical value of axial chromatic aberration. The on-axis position is different from the on-axis position of the green image sensor, and the red and blue modulation degrees deteriorate.
Also, a CMOS image sensor that reads video signal charges all at once from photodiodes called gyroval shutters is expensive. CMOS image sensors that sequentially read out video signal charges from photodiodes called focal plane shutters or rolling shutters can be read at high speed with low power consumption and low cost. However, when the subject moves, video signal charges are read sequentially from the photodiodes. The reading time differs depending on the screen position of the subject, and the moving subject is deformed.

Japanese Patent Laid-Open No. 11-355794 JP 2000-209601 A JP-A-6-339146 JP 7-327233 A JP-A-8-98190

  The present invention has been made in view of the above-described conventional circumstances, and is combined with a three-lens zoom lens to provide a camera system with low moiré and high resolution and modulation degree (CMOS imaging that facilitates high-speed readout). The purpose is to use a device to achieve a simple structure.

In the present invention, in an imaging apparatus having a four-plate configuration with a color separation optical system and imaging elements of R (red), G (green) 1, G2, and B (blue), imaging of G2 with respect to the imaging element of G1 The elements are arranged with a 1/2 pixel pitch in the vertical direction and a 1/2 pixel pitch in the horizontal direction,
The R image sensor is arranged at the same position as the G1 image sensor in the vertical direction at the same position as the G2 image sensor in the vertical direction, and the B image sensor is arranged at the same position as the G2 image sensor in the vertical direction at the same position as the G1 image sensor. Arrange the image sensor at the same position as the image sensor, or arrange the R image sensor in the same position as the G2 image sensor in the vertical direction and the same position as the G1 image sensor in the horizontal direction, and image the B image sensor in the vertical direction. Either the same position as the element or the same position as the G2 image sensor in the horizontal direction,
The low frequency component of the R output video signal is the signal of the R image sensor, the low frequency component of the B output video signal is the signal of the B image sensor, and the low frequency component of the G output video signal is the signal of the G1 image sensor. Let it be the average value with the signal of the image sensor of G2,
Low sensitivity setting at least (-3dB, -6dB, -9dB, -12dB, etc. for studio use, both RB high S / N), electrically high color temperature (6800K, 5600K, 4800K, etc.) or B (ambient) If the average ratio of / (G1 + G2) is not small (such as 0.5 or more), the signal of the G2 image sensor and the signal of the B image sensor are used as the high-frequency signal of the even-numbered scanning line, and the surroundings) When the average ratio of R / (G1 + G2) is not small (such as 0.5 or more), the signal of the G1 image sensor and the signal of the R image sensor are alternately set as the high frequency signal of the odd scanning line, and at least High sensitivity setting (for monitoring applications such as +18 dB, +24 dB, +36 dB, +48 dB, +60 dB), electrically low color temperature (such as 3200K, 2800K) or average ratio of (ambient) B / (G1 + G2) (0.5 Less than If it is small, the alternating signal of the G1 image sensor signal and the G2 image sensor signal is the high-frequency signal of the even scanning line, and the average ratio of R / (G1 + G2) is small (such as 0.5 or less). Alternatively, in the case of the left and right ends of the screen, the imaging method is characterized in that an alternating signal of a signal from the G1 imaging device and a signal from the G2 imaging device is used as a high-frequency signal of an odd scanning line.

  Further, in the imaging method, the R, G1, G2, and B imaging elements are sequentially scanned, and an addition signal of the high-frequency signal of the even-numbered scanning line and the high-frequency signal of the odd-numbered scanning line is used for the interlaced scanning. An imaging method characterized in that a high frequency signal of an output video signal is used.

In addition, in an imaging apparatus having a four-plate configuration with a color separation optical system and imaging elements of R (red), G (green) 1, G2, and B (blue),
The G1 image sensor and the G1 image sensor are arranged with a ½ pixel pitch shift in the vertical direction and a ½ pixel pitch in the horizontal direction, the G1 image sensor signal, and the G2 image sensor. A first selector that alternately selects a signal (by a 1/2 frequency-divided clock of the image sensor pixel clock) and a first low-pass filter that passes a low-frequency component of the output signal of the first selector. A first determination unit that binarizes the output signal of the first LPF, and a first delay unit that delays the output signal of the first selector,
R image sensor arranged at the same position in the vertical direction with respect to the G1 image sensor and shifted by 1/2 pixel pitch in the horizontal direction, and the same position in the horizontal direction with 1/2 pixel pitch in the vertical direction relative to the G1 image sensor. A second selector that alternately selects a signal from the G image sensor and a signal from the R image sensor, and a second delay that delays the output signal from the second selector. A second selector that alternately selects a signal from the G2 image sensor and a signal from the B image sensor, and a third delayer that delays the output signal of the second selector Alternatively, the B image sensor disposed at the same position in the vertical direction with respect to the G1 image sensor and shifted by ½ pixel pitch in the horizontal direction is the same as the G1 image sensor in the ½ pixel pitch horizontal direction in the vertical direction. R image sensor arranged at the position, G1 image sensor signal and B A second selector that alternately selects a signal of the image pickup device, a second delay device that delays an output signal of the second selector, a signal of the image pickup device of G2, and a signal of the image pickup device of R A second selector that alternately selects and a third delayer that delays the output signal of the second selector, and
The second LPF that passes the low-frequency component of the signal of the R image sensor, the third LPF that passes the low-frequency component of the signal of the B image sensor, the output of the first LPF, and the second LPF A first subtractor for calculating a difference from the output, and an R / (G1 + G2) average ratio offset signal for offsetting selection at an average ratio of R / (G1 + G2) to the output of the first subtractor. A first adder,
A second subtractor that calculates the difference between the output of the first LPF and the output of the third LPF and the output of the second subtractor that offsets the selection at an average ratio of B / (G1 + G2) B / A combination of a second adder that adds an average ratio offset signal of (G1 + G2), or a B gain control signal that controls the gain of the B image sensor signal or the gain of the B video signal and the gain of the G image sensor signal or G A G gain control signal that controls the gain of the video signal, a subtractor that calculates the difference between the B gain control signal and the G gain control signal, and a gain offset signal that offsets the selection in gain to the subtractor output are added. A combination of second adders that perform or a combination of third adders that add the screen left and right edge signals to the output of the second adder, and
A fifth selector for selecting the (G1 + G2) output signal of the first delay device and the (G1 + R) output signal of the second delay device according to the output signal of the first adder; A sixth selection for selecting the (G1 + G2) output signal of the first delay device and the (G2 + B) output signal of the third delay device by the output signal of the adder or the output signal of the third adder A seventh selector for selecting the output signal of the fifth selector and the output signal of the sixth selector by the scanning line odd / even flag signal, and the high frequency component of the output signal of the seventh selector And a high-pass filter (HPF) that passes the
A fourth adder, a fifth adder, and a sixth adder that add the output signal of the HPF to the output signal of the first LPF, the output signal of the second LPF, and the output signal of the third LPF. And an adder.

  Furthermore, in the imaging apparatus, means for sequentially scanning the imaging elements of R, G1, G2, and B, means for adding the high-frequency signal of the even-numbered scanning line and the high-frequency signal of the odd-numbered scanning line, An image pickup apparatus comprising means for adding the output signal of the adding means to the output video signal of the interlaced scanning.

  Here, the progressive video signal of 1125 total scanning lines of 59.94 Hz (about 60 Hz) indicates a video signal in a format generally expressed as 1080 / 60p (or 60P). That is, it is a progressive video signal of 1080 effective scanning lines (1125 total scanning lines), 60 frames per second (about).

  The progressive video signal of 1125 total scanning lines of 23.98 Hz (about 24 Hz) refers to a video signal in a format generally expressed as 1080 / 24p (or 24P). That is, it is a progressive video signal with 1080 effective scanning lines (total scanning line 1125), 24 (about) frames per second.

Here, the interlaced video signal of 1125 total scanning lines of 59.94 Hz (about 60 Hz) refers to a video signal in a format generally expressed as 1080 / 60i (or 60I). That is, 1080 effective scanning lines (1125 total scanning lines), 60 (about) frames per second, and an interlaced video signal.
The 74.175 MHz clock corresponds to the timing of one pixel in the 1080 / 60i video signal.
The 29.97 MHz frame pulse corresponds to the timing of two frames in the 1080 / 60p video signal and one frame in the 1080 / 60i video signal. That is, the start timings of the 1080 / 60p and 1080 / 60i frames can be synchronized by the 29.97 MHz frame pulse.

Here, the interlaced video signal of 525 total scanning lines of 59.94 Hz (about 60 Hz) indicates a video signal in a format generally expressed as 480 / 60i (or 60I). That is, 480 effective scanning lines (total scanning line 525), 60 (about) frames per second, and an interlaced video signal.
The 27 MHz clock corresponds to the timing of one pixel in the 480 / 60i video signal.

  According to the present invention, a camera having a high resolution and modulation degree (4K television) with little moire combined with a three-lens zoom lens (for various 2K HDTVs such as a high-magnification super telephoto purchased by a broadcasting station) The system can be realized with a simple configuration.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention. The block diagram (sequential scanning output) which shows the example of the signal processing which concerns on one Example of this invention Block diagram showing an example of signal processing according to an embodiment of the present invention (interlaced scanning output) FIG. 6 is a block diagram (sequential scan output) showing an example of signal processing according to an embodiment of the present invention (if the B gain is equal to or less than the B gain limit signal, the G2 + B signal is converted into a high frequency signal of an even scan line regardless of the G gain. In Fig. 2B, gain offset signal, G gain control signal, B gain control signal, and B gain limit signal are added. (Chromatic aberration is corrected at 3 wavelengths and spherical aberration coma at 2 wavelengths. To avoid the purple g-line 435nm coma that remains on the lens (abpo-named apochromat) that has been corrected for aberrations, do not use B + G2 at the left and right edges of the screen, but increase G1 + R and G1 + G2. (The left and right edge signals of the screen are added to Fig. 2B to make the area signal.) The flowchart (sequential scanning output) which shows the example of the signal processing which concerns on one Example of this invention The flowchart (interlaced scanning output) which shows the example of the signal processing based on one Example of this invention Flowchart showing an example of signal processing according to an embodiment of the present invention (sequential scanning output) (a lens whose chromatic aberration is corrected at three wavelengths and spherical aberration and coma aberration are corrected at two wavelengths (named by Abbe) To avoid the purple g-line 435nm coma that remains in the apochromat, do not use B + G2 on the left and right edges of the screen, but instead use G1 + R and G1 + G2 as high-frequency signals. Added "Is the left and right edges of the screen?" Schematic diagram showing the bonding position of the image pickup device of the image pickup apparatus according to one embodiment of the present invention ((a) the overlapping state of each pixel of the four-plate image pickup device, (b) the center of each pixel of the four-plate image pickup device. relative position)

An imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.
In the following description, each numerical value indicating the frequency of the clock or pulse is expressed by rounding a decimal fraction to a predetermined digit, and it is not required to exactly match these numerical values.

FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention.
The image pickup apparatus 2 selects a 1080 / 60p image pickup signal obtained by operating an image pickup device having 1920 × 1080 effective pixels by progressive scanning of four frames of R, G1, G2, and B at 60 frames per second and a frequency component. By selecting and adding, a 2160 / 60p or 2160 / 60i video signal is generated.

FIG. 2A is a block diagram showing an example of signal processing according to an embodiment of the present invention of 2160 / 60p (sequential scanning output).
FIG. 2B is a block diagram showing an example of signal processing according to an embodiment of the present invention of 2160 / 60i (interlaced scanning output).

FIG. 3A is a flowchart showing an example of signal processing according to an embodiment of the present invention of 2160 / 60p (sequential scanning output).
FIG. 3B is a flowchart showing an example of signal processing according to an embodiment of the present invention of 2160 / 60i (interlaced scanning output).

FIG. 4 is a schematic diagram showing the bonding position of the image pickup element of the image pickup apparatus according to the embodiment of the present invention. (a) is the degree of overlap of each pixel of the four-plate image sensor, and (b) is the relative position of each pixel center of the four-plate image sensor. This is equivalent to the Bayer arrangement of the single-plate relative positions at the center of each pixel of the four-plate image sensor in (b), and the difference is that the four plates overlap by half pixels.
Note that (c), (d), (e), (f), and (g) are the degree of overlap of the pixels of the four-plate image sensor.

  FIG. 1 is a block diagram illustrating an example of the configuration of an image pickup apparatus according to an embodiment of the present invention. Reference numeral 1 denotes a lens, 2 denotes an image pickup apparatus, 3G1 denotes a first green (G1) image pickup element (a CMOS in which peripheral circuits are integrated). CCD image sensor integrated with image sensor or peripheral circuit), 3G2 is a second green (G2) image sensor, 3R is a red (R) image sensor, 3B is a blue (B) image sensor, 4 is A video signal processing circuit including an interpolation processing circuit constituted by an FPGA, 5 is a color separation optical system (for four plates), and 7 is a CPU.

  FIG. 2A is a block diagram showing an example of signal processing according to an embodiment of the present invention of 2160 / 60p (sequential scanning output) and signal processing according to an embodiment of the present invention of 2160 / 60i (interlaced scanning output). In FIG. 2B, which is an example block diagram, 11 is an interpolation processing circuit, 21, 22 and 23 are low-pass filters (LPF), 24 is a high-pass filter (HPF), 26, 27, 28, 29, 30 are adders, 31, 32, 33 are subtractors, 34, 35, 36, 37, 38 are pixel delay units, 39 is a bit shift unit, 41, Reference numerals 42, 43, 44, 45, 46 and 47 are selectors.

  The detailed configuration and operation will be described below. In FIG. 1 of the block diagram illustrating the configuration example of the imaging apparatus according to an embodiment of the present invention, incident light converged by the lens 1 is converted to R (red) by the color separation optical system 5 for four plates of the imaging apparatus 2. A first green (G1) image pickup device 3G1 of a CMOS image pickup device or a CCD image pickup device integrated with peripheral circuits which are color-separated into G (green) 1, G2 and B (blue) and integrated peripheral circuits; The second green (G2) image sensor 3G2, red (R) image sensor 3R, and blue (B) image sensor 3B form four sets of 1080 / 60p image signals that are controlled by the CPU 7. It is converted into 2160 / 60p (sequential scanning output) or FIG. 2160 / 60i (interlaced scanning output) by the video signal processing circuit 4 including an interpolation processing circuit constituted by an FPGA.

FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention, FIG. 4 is a schematic diagram illustrating a bonding position of an imaging element of the imaging apparatus according to an embodiment of the present invention, and 2160 / FIG. 2A is a block diagram showing an example of signal processing according to an embodiment of the present invention of 60p (sequential scanning output), and an example of signal processing according to an embodiment of the present invention of 2160 / 60i (interlaced scanning output). In the block diagram shown in FIG. 2B, in the interpolation processing circuit 11, reference numeral 40 denotes a 1/2 frequency divider, which divides the imaging element pixel clock.
In the interpolation processing circuit 11, the G2 image sensor 3G2 and the G1 image sensor 3G1 are arranged with a ½ pixel pitch shift in the horizontal direction and a G1 image sensor 3G2, and the G1 image sensor 3G2. The first selector 41 and the output signal of the first selector 41 that alternately select the image sensor signal and the G2 image sensor signal (by a 1/2 frequency-divided clock of the image sensor pixel clock). A first low-pass filter (hereinafter referred to as LPF) that passes a low-frequency component of the first LPF, a first determiner that binarizes 21 output signals of the first LPF, and 41 of the first selector A first pixel delay 34 for delaying the output signal.

R imaging element 3R arranged at the same position in the vertical direction and shifted by 1/2 pixel pitch in the vertical direction with respect to G1 imaging element 3G1 and G1 imaging element in the vertical direction by 1/2 pixel pitch in the horizontal direction B image sensor 3B arranged at the same position, 42 of the second selector for alternately selecting the signal of the G image sensor 3G1 and the signal of the R image sensor 3R, and 42 of the second selector A second pixel delay device 35 for delaying the output signal, a second selector 42 for alternately selecting a signal of the G2 image sensor 3G2 and a signal of the B image sensor 3B, and a second selector. Or a third pixel delay device 36 for delaying the output signal of 42, or a B pixel disposed at the same position in the vertical direction and shifted by a ½ pixel pitch in the horizontal direction with respect to the image pickup device 3G1 of G1. Perpendicular to the image sensors 3B and G1 42 of the 2nd selector which selects alternately the image pick-up element 3R of the R image sensor 3R arrange | positioned in the horizontal direction at the same position in the horizontal direction, the signal of the image pick-up element 3G1 of G1, and the signal of the image pick-up element 3B of B The second selector 35 for delaying the output signal of the second selector 42, and the second selector 35 for alternately selecting the signal of the G2 image sensor 3G2 and the signal of the R image sensor 3R. 42 and a third pixel delay 37 that delays the output signal of the second selector.
The second LPF 22 that passes the low-frequency component of the signal of the R image sensor 3R, the third LPF 23 that passes the low-frequency component of the signal of the B image sensor 3B, and the first LPF 21 The first subtractor 31 for calculating the difference between the output of the second LPF 22 and the output of the second LPF 22 and the selection of the R / (G1 + G2) average ratio offset to the output of the first subtractor 31 And R / (G1 + G2) average ratio offset signal to be added.
Here, when the same processing FPGA is installed as in LPF 21, 22 and 23, low-speed parallel processing that reduces power consumption or time-sharing processing that reduces the number of gates may be used.

The second subtractor 32 that calculates the difference between the output of the first LPF and the output of the third LPF and the output of the second subtractor 32 are selected with an average ratio of B / (G1 + G2). The combination of the second adder 26 that adds the B / (G1 + G2) average ratio offset signal to be offset, or the B gain control signal for controlling the gain of the B image sensor signal or the gain of the B video signal and the G imaging G gain control signal that controls the gain of the element signal or G video signal, the subtractor that calculates the difference between the B gain control signal and the G gain control signal, and the selection of the gain offset to the subtractor output Having a combination of at least one of 26 combinations of second adders for adding gain offset signals to
A fifth selector for selecting the (G1 + G2) output signal of the first delay unit and the (G1 + R) output signal of the second delay unit according to the output signal of the first adder; 45, and the output signal of 26 of the second adder selects the (G1 + G2) output signal of the first delay and the (G2 + B) output signal of 36 of the third delay. The selector 46, the seventh selector 47 for selecting the output signal of the fifth selector 46 and the output signal of the sixth selector 46 by the scanning line odd / even flag signal, A high-pass filter (hereinafter referred to as HPF) 24 that passes the high-frequency component of the output signal of 47 of the selector,
The fourth adder 28 and the fifth adder add the output signal of the HPF to the output signal of the first LPF, the output signal of the second LPF, and the output signal of the third LPF. 29 and 30 of the sixth adder.

Further, in the low-pass filter (LPF) of 21, 22 and 23 in the interpolation processing circuit 11 of FIG. 2A and FIG. 2B of the block diagram showing an example of signal processing according to an embodiment of the present invention, the LPF 21 Similarly, the low-pass filtering is realized by the pixel delay unit 38 and the bit shift unit 39 of the adder 26.
Further, in the block diagram of the high-pass filter (HPF) 24 in the interpolation processing circuit 11 of FIGS. 2A and 2B, which shows an example of signal processing according to an embodiment of the present invention, the pixel delay unit 36 and the subtraction are subtracted. High-pass filtering is realized at 33 of the vessel.

  2A of the block diagram showing an example of signal processing according to an embodiment of the present invention of 2160 / 60p (sequential scan output), 33 and 50 are subtracters, 51 and 52 are comparators, 53 and 54 are logical sums. If the B gain is equal to or less than the B gain limit signal, the G2 + B signal is used as a high-frequency signal for even-numbered scanning lines regardless of the G gain.

  Furthermore, the R, G1, G2, and B imaging elements bonded together are sequentially scanned as shown in FIG. 4A, which is a schematic diagram showing the bonding position of the imaging elements of the imaging apparatus according to an embodiment of the present invention. . 2B of the block diagram illustrating an example of signal processing according to an embodiment of the present invention of 2160 / 60i (interlaced scanning output), and the high-frequency signal of the even-numbered scanning line and the high-frequency signal of the odd-numbered scanning line. 47, a high-pass filter (HPF) 24, and adders 27, 28, and 29 of the means for adding the output signal of the adding means to the output video signal of the interlaced scanning. Have

3A and 3B of the flowchart showing an example of signal processing according to an embodiment of the present invention, 60 is “start”, 61 is “B gain is sufficiently low”, and 62 is “(color temperature)”. Is 4800K or more) B gain is not much different from G gain ", 63 is" whether the average R level is not very different from the average G level ", 64 is" G2 and B `` Make the high-frequency signal of the alternate signal the high-frequency signal of the even-numbered scanning line '', 65 is `` Make the high-frequency signal of the alternating signal of G1 and G2 the high-frequency signal of the even-numbered scanning line '' Yes, 66 is “whether the average R level is not significantly different from the average G level” (for example, it is not lawn), and 67 is “the high-frequency signal of the odd scan line is the high-frequency signal alternated with G1 and R” 68 is “to make the high-frequency signal alternated with G1 and R the high-frequency signal of the odd scanning line”, and 72 is “end”.
Here, in FIGS. 3A and 3B of the flowchart, either the processing of 61, 62, 63, 64, 65 or the processing of 66, 67, 68 may be performed first.

  In FIG. 3A of the flowchart showing an example of signal processing according to an embodiment of the present invention of 2160 / 60p (sequential scan output), 69 is “odd scan line” and 70 is “odd for RGB low-frequency signal? “A signal obtained by adding the high-frequency signal of the scanning line is converted into an RGB video signal”, and 71 is “a signal obtained by adding the high-frequency signal of the even-numbered scanning line to the RGB low-frequency signal is converted into an RGB video signal”. is there.

  In FIG. 3B of the flowchart showing an example of signal processing according to one embodiment of the present invention of 2160 / 60i (interlaced scanning output), 73 is “the high-frequency signal of the even-numbered scanning line and the odd-numbered scanning line of the RGB low-frequency signal. The signal obtained by adding the high-frequency signal is used as an RGB video signal. "

Only differences from the first embodiment will be described.
In the second embodiment, the high-frequency luminance signal generation formula is switched between G1R, BG2 and G1G2 based on optical design information or actual measurement values of the RGB residual aberration of the lens. Specifically, in order to avoid the purple g-line 435nm coma that remains on a lens (apochromat named by Abbe) whose chromatic aberration has been corrected at three wavelengths and spherical aberration and coma corrected at two wavelengths. In Example 2, at the left and right edges of the screen, B + G2 is not used for the high frequency signal, and G1 + R and G1 + G2 are used for the high frequency signal. FIG. 3C to which “right or left end” is added is a flowchart showing an example of signal processing according to an embodiment of the present invention.
FIG. 2C, which is a block diagram illustrating an example of signal processing according to an embodiment of the present invention, is common to Embodiment 1 because a screen left and right edge signal is added to FIG. 2B.

Specifically, the second subtractor for calculating the difference between the output of the first LPF and the output of the third LPF and the output of the second subtractor are selected with an average ratio of B / (G1 + G2). A combination of a second adder that adds an average ratio offset signal of B / (G1 + G2) that offsets the B, or a B gain control signal that controls the gain of the B image sensor signal or the gain of the B video signal and the G image sensor G gain control signal for controlling signal gain or G video signal gain, subtractor for calculating the difference between the B gain control signal and the G gain control signal, and offsetting the selection by gain to the subtractor output It has at least one combination of a second adder combination for adding the gain offset signal or a third adder combination for adding the screen left and right edge signals to the output of the second adder.
A fifth selector for selecting the (G1 + G2) output signal of the first delay device and the (G1 + R) output signal of the second delay device according to the output signal of the first adder; A sixth (G1 + G2) output signal of the first delay device and a (G2 + B) output signal of the third delay device are selected by the output signal of the second adder or the output signal of the third adder. , A seventh selector that selects the output signal of the fifth selector and the output signal of the sixth selector by the scanning line odd / even flag signal, and the output signal of the seventh selector And a high-pass filter (hereinafter referred to as HPF) that passes the band component.

Even if the arrangement itself can be easily inferred from the Bayer arrangement, the typical value of the axial chromatic aberration of the three-lens zoom lens is offset on the axis with respect to G, and the image pickup device is attached to the color separation optical system. By correcting the lateral chromatic aberration and the bonding error in the pixel plane, it is possible to utilize not only G1G2 but also RB with high resolution and degree of modulation even in a lens in which axial chromatic aberration and lateral chromatic aberration remain. The present invention becomes more effective.
In addition, for low-sensitivity settings for studio use, in the case of electrically high color temperature or high-brightness pixels, horizontal and vertical resolutions, resolutions, and modulations are used as signals alternating with G1 image sensor signals and R image sensor signals. In addition to increasing the degree, it is possible to make the invention more effective by increasing the oblique resolution, resolution, and modulation degree.

According to the present invention, for example, a camera system having a high resolution and modulation degree (4K television) with little moire, combined with various 3K zoom lenses for 2K HDTV such as a high-magnification super telephoto purchased by a broadcasting station. Can be realized with a simple configuration. In particular, axial chromatic aberration is reduced by axial offset bonding of the image sensor, axial chromatic aberration is reduced by correcting chromatic aberration of magnification and bonding error, and it becomes easy to increase resolution and modulation with less moire. .
On the other hand, a single-plate color camera using a super 35 mm size 4K resolution on-chip color filter-equipped imaging device cannot correct axial chromatic aberration and lateral chromatic aberration unlike the present invention. A large and expensive dedicated lens that optically corrects axial chromatic aberration and lateral chromatic aberration is required.
Therefore, according to the present invention, using a B4 mount lens widely used for HDTV cameras using three 1920 × 1080 pixel (2K) 2/3 type image sensors and a color separation optical system, 1920 × 1080 pixels are used. Using four (2K) 2/3 type image sensors and a color separation optical system leads to higher resolution and higher modulation degree of a 4K camera such as 3840 × 2160 pixels. In addition, using four 4K 2/3 type image pickup devices such as 3840 × 2160 pixels and a color separation optical system leads to higher resolution and higher modulation degree of an 8K camera such as 7680 × 4320 pixels.
Furthermore, the former type 1 mount lens for three HDTV cameras can be applied to the former 1 + 1/4 type mount lens for three tube HDTV cameras.
Further, according to the present invention, G1R and BG2 can be high-frequency components of the video with low sensitivity, and the lawn can be a camera with G1G2 as the high-frequency components of the video.

1: lens, 2: imaging device,
3G1: first green (G1) image pickup device (a CMOS image pickup device in which peripheral circuits are integrated or a CCD image pickup device in which peripheral circuits are integrated),
3G2: second green (G2) image sensor, 3R: red (R) image sensor,
3B: Blue (B) image sensor,
4: Video signal processing circuit including interpolation processing circuit composed of FPGA, 5: Color separation optical system,
7: CPU, 11: interpolation processing circuit,
21, 22, 23: Low-pass filter (LPF), 24: High-pass filter (HPF),
25, 26, 27, 28, 29, 30, 47: adder,
31, 32, 33, 50: subtractor,
34, 35, 36, 37, 38: pixel delay device,
39: Bit shift part,
41, 42, 43, 44, 45, 46: selector,
51, 52: comparator, 53, 54: logical OR

Claims (4)

In an image pickup apparatus having a four-plate configuration with a color separation optical system and R (red), G (green) 1, G2, and B (blue) image sensors, the G2 image sensor is perpendicular to the G1 image sensor. ½ pixel pitch horizontally and shifted by ½ pixel pitch,
The R image sensor is arranged at the same position as the G1 image sensor in the vertical direction at the same position as the G2 image sensor in the vertical direction, and the B image sensor is arranged at the same position as the G2 image sensor in the vertical direction at the same position as the G1 image sensor. Arrange the image sensor at the same position as the image sensor, or arrange the R image sensor in the same position as the G2 image sensor in the vertical direction and the same position as the G1 image sensor in the horizontal direction, and image the B image sensor in the vertical direction. Either the same position as the element or the same position as the G2 image sensor in the horizontal direction,
If at least the low sensitivity setting and the electrically high color temperature or the average ratio of B / (G1 + G2) are not small, the signal of the G2 image sensor and the signal of the B image sensor are alternately used as the high-frequency signal of the even scanning line. When the average ratio of R / (G1 + G2) is not small, the signal of the G1 image sensor and the signal of the R image sensor are alternately set as the high-frequency signal of the odd scanning line, and at least high sensitivity setting, electrical In the case of the low color temperature, the average ratio of B / (G1 + G2) is small, or the left and right edges of the screen, the alternating signal of the G1 image sensor signal and the G2 image sensor signal is set as the high-frequency signal of the even scanning line, An imaging method characterized in that when the average ratio of R / (G1 + G2) is small, an alternating signal of a signal of the G1 image sensor and a signal of the G2 image sensor is used as a high-frequency signal of an odd scanning line. The imaging method according to claim 1, wherein the R, G1, G2, and B imaging elements are sequentially scanned,
An imaging method, wherein an addition signal of the high-frequency signal of the even-numbered scanning line and the high-frequency signal of the odd-numbered scanning line is used as a high-frequency signal of an output video signal of interlaced scanning. In an image pickup apparatus having a four-plate configuration with a color separation optical system and R (red), G (green) 1, G2, and B (blue) image sensors, the G2 image sensor is perpendicular to the G1 image sensor. Means for shifting by a 1/2 pixel pitch in the horizontal direction,
The R image sensor is arranged at the same position as the G1 image sensor in the vertical direction at the same position as the G2 image sensor in the vertical direction, and the B image sensor is arranged at the same position as the G2 image sensor in the vertical direction at the same position as the G1 image sensor. Arrange the image sensor at the same position as the image sensor, or arrange the R image sensor in the same position as the G2 image sensor in the vertical direction and the same position as the G1 image sensor in the horizontal direction, and image the B image sensor in the vertical direction. Means that is arranged at the same position as the G2 image pickup element in the horizontal direction at the same position as the element;
If at least the low sensitivity setting and the electrically high color temperature or the average ratio of B / (G1 + G2) are not small, the signal of the G2 image sensor and the signal of the B image sensor are alternately used as the high-frequency signal of the even scanning line. When the average ratio of R / (G1 + G2) is not small, the signal of the G1 image sensor and the signal of the R image sensor are alternately set as the high-frequency signal of the odd scanning line, and at least high sensitivity setting, electrical In the case of the low color temperature, the average ratio of B / (G1 + G2) is small, or the left and right edges of the screen, the alternating signal of the G1 image sensor signal and the G2 image sensor signal is the high-frequency signal of the even scanning line An image pickup apparatus comprising means for converting an alternating signal of a G1 image pickup element signal and a G2 image pickup element signal into a high-frequency signal of an odd scanning line when the average ratio of R / (G1 + G2) is small . The imaging device according to claim 3, wherein the R, G1, G2, and B imaging elements are sequentially scanned,
An imaging apparatus comprising: means for converting a sum signal of the high-frequency signal of the even-numbered scanning line and the high-frequency signal of the odd-numbered scanning line into a high-frequency signal of an output video signal of interlaced scanning.

Images