JP6570122B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

In recent years, with the advancement of digital technology of television camera systems, the number of pixels of the imaging device has changed from 1920 × 1080 pixels (abbreviated as 2K) to approximately 4000 × 2000 pixels (abbreviated as 4K) and approximately 8000 × 4000 pixels (abbreviated as 8K). Is required.
Therefore, as a resolution increase by pixel shifting, in a multi-plate color camera with spatial pixel shifting, an R (Red) signal and a G (Green) signal, whose sampling phases are reversed with each other by spatial pixel shifting, are obtained at the above sampling frequency. The signals are switched in opposite phases to generate a signal in which the R signal and the G signal are alternately switched. After the R signal is mixed, the band is limited by the band pass filter, and then combined with the luminance signal output from the matrix circuit. (For example, refer to Patent Document 1).

  Furthermore, in a multi-plate color camera with spatial pixel shifting, the luminance signal (Y) is a matrix of double speed clocks every half of the sampling period (1 / fs) of the solid-state imaging device from R, G, B (Blue) signals. Generated by calculation, output by double speed sampling, and before matrix calculation, the high frequency range of the R and B signals is amplified from the rated value, and the high frequency phase of the G signal is determined by a predetermined period (1 / fs) from the phase of the R and B signals. ) (See, for example, Patent Document 2).

  By the way, the G1 (Green1) image sensor is arranged with a horizontal pixel pitch shift of 1/2 pixel pitch and a vertical pixel pitch shift with respect to the G2 (Green2) image sensor. Then, the high frequency components in the vertical direction and the horizontal direction are calculated from the obtained G1 and G2 signals and added to the G signal to improve the vertical and horizontal resolutions (for example, Patent Document 3). reference.).

  In addition, the imaging apparatus has a four-plate configuration with B (blue), R (red), G1 (green), and G2 imaging elements. The B and R image sensors are arranged with a 1/2 pixel pitch shift in the vertical direction with respect to the G2 image sensor. Then, the high frequency component in the vertical direction is calculated from the obtained B and R signals and added to the G signal to improve the vertical resolution. Further, the G1 and G2 image sensors are arranged with a ½ pixel pitch shift in the horizontal direction to improve the resolution in the horizontal direction (see, for example, Patent Document 4).

  In addition, the imaging apparatus has a four-plate configuration with B (blue), R (red), G1 (green), and G2 imaging elements. The R image sensor is arranged with a 1/2 pixel pitch shift in the vertical direction with respect to the G2 image sensor. The B image sensor is arranged with a one-pixel pitch shift in the vertical direction with respect to the R image sensor. Then, a high frequency component in the vertical direction is calculated from the obtained B and R signals and added to the G signal to improve the resolution in the vertical direction. Since a high frequency component is obtained from signals in the same field, generation of false signals is prevented (see, for example, Patent Document 5).

At this time, the light received by the imaging elements G1 and G2 is split by the half mirror in the prism, but the half mirror has a characteristic that the reflectance changes depending on the incident angle of the light. There may be a difference in the amount of light received by the imaging elements G1 and G2 depending on the angle of incident light. In the dual green method in which G1 and G2 are combined and interpolated to generate a high-definition image, if there is a difference in the amount of light received by the G1 and G2 image sensors, an oblique grid pattern is generated in the generated image. It should be noted that when a fine subject whose image level is not flat is imaged, the oblique grid pattern is difficult to see, but appears as fixed pattern noise in the image.
Further, the image sensor has temperature characteristics, and when the temperature of the image sensor changes, the level of the output signal may increase or decrease even with the same amount of received light. For this reason, when a temperature difference occurs between the G1 and G2 image sensors, a level difference occurs between the output signals of G1 and G2, and in this case as well, noise in an oblique grid pattern or fixed pattern is generated in the video.

JP-A-11-355794 JP 2000-209601 A JP-A-6-339146 JP 7-327233 A JP-A-8-98190

  SUMMARY OF THE INVENTION An object of the present invention is to prevent the occurrence of oblique grid pattern or fixed pattern noise in an image in a dual green color image pickup apparatus.

  The image pickup apparatus of the present invention is an image pickup apparatus having two green image pickup elements, and includes a level difference correction unit that corrects a level difference between the G1 signal and the G2 signal output from the two green image pickup elements. The unit includes a flatness determination unit, a level difference determination unit, and a level adjustment unit. The flatness determination unit determines flatness between the target pixel of the G1 signal and the G2 signal and the surrounding pixels, and the level difference determination unit is a flatness determination unit. Determines whether there is a level difference between the average values of the target pixel and the peripheral pixels of the G1 signal and G2 signal determined to be flat, and the level adjustment unit determines that the level difference determination unit determines that there is a level difference. A gain ratio of the signal is calculated, and level adjustment is performed on the G1 signal or the G2 signal with the gain ratio.

  The imaging apparatus of the present invention is the imaging apparatus described above, wherein the level difference correction unit further includes an integration average unit and a memory unit, and the integration average unit is a gain of the G1 signal and the G2 signal calculated by the level adjustment unit. The memory unit stores the gain ratio obtained by integrating and averaging the integration unit, and the level adjustment unit applies the pixel determined by the flatness determination unit to be not a flat image in the next frame in the previous frame. The level adjustment is performed on the G1 signal or the G2 signal with the gain ratio stored in the memory unit.

  Furthermore, the imaging apparatus according to the present invention is the imaging apparatus described above, wherein the level difference correction unit further includes a linear interpolation unit, and the integrating average unit divides the screen into a plurality of areas, and gains of the G1 signal and the G2 signal The ratio is integrated and averaged for each area, the memory unit stores the gain ratio that the integrated average unit integrated and averaged for each area, the linear interpolator calculates the gain ratio by linear interpolation over the entire screen, and the level adjuster is flat For the pixel determined by the determination unit as not being a flat image in the next frame, level adjustment is performed on the G1 signal or the G2 signal with the gain ratio stored in the memory unit in the previous frame.

  ADVANTAGE OF THE INVENTION According to this invention, it can prevent that the noise of a diagonal grid | lattice-like pattern or a fixed pattern generate | occur | produces in an image | video in a dual green type color imaging device.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention. It is a block diagram of a video signal processing unit according to an embodiment of the present invention. It is a block diagram which shows the structure of the level difference correction | amendment part which concerns on one Example of this invention. It is a block diagram which shows the structure of the level difference correction | amendment part which concerns on another one Example of this invention. It is a block diagram which shows the structure of the level difference correction | amendment part which concerns on another one Example of this invention. It is a schematic diagram which shows the overlapping condition of each pixel of the bonding position of the 4 image pick-up element of the imaging device which concerns on one Example of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention.
In FIG. 1, an imaging device 110 includes a color separation optical system 112, a first green (G1, Green1) imaging element 111G1, a second green (G2, Green2) imaging element 111G2, a red (R, Red) imaging element 111R, and a blue color. (B, Blue) An image sensor 111B, a video signal processor 113, an image sensor driver 114, and a CPU (Central Processing Unit) 115 are configured.

In the imaging device 110, incident light 100 is imaged by a lens 101, and color-separated for four plates by a color separation optical system 112, and a first green (G1) imaging element 111G1 and a second green (G2) imaging element 111G2 The red (R) image sensor 111R and the blue (B) image sensor 111B perform photoelectric conversion, and the video signal processing unit 113 performs various signal processing to output an HD-SDI (High Definition Serial Digital Interface) signal.
Note that the video signal output from the imaging device 110 of the present invention is not limited to the HD-SDI of one embodiment, and compression, encryption, etc. are not an issue.

FIG. 4 is a schematic diagram showing the overlapping state of each pixel at the bonding position of the four image pickup elements of the image pickup apparatus according to the embodiment of the present invention.
FIG. 4A is an example of a pixel bonding position of the first green (G1) image sensor 111G1 and the second green (G2) image sensor 111G2.
FIG. 4B is an example of a bonding position of pixels corresponding to the Bayer array.
Note that the imaging apparatus 110 according to an embodiment of the present invention is not limited to the bonding position of the four-plate imaging device in FIG.

FIG. 2 is a block diagram of a video signal processing unit according to an embodiment of the present invention.
In FIG. 2, the video signal processing unit 113 includes a level correction unit 210, a selection unit 220, and a video signal output unit 230.
The level correction unit 210 includes a level difference correction unit 211 and delay units 212 and 213.

The level difference correction unit 211 performs a correction process for eliminating the level difference between the G1 signal and the G2 signal, a process for adjusting the G1 signal level to the G2 signal level, or a process for adjusting the G2 signal level to the G1 signal level. I do.
The level difference correction unit 211 may be an FPGA (Field Programmable Gate Array) or a hybrid of a CPU and an FPGA.
The selection unit 220 alternately outputs the G1 ′ signal and the G2 ′ signal, or synthesizes the G1 ′ signal and the G2 ′ signal and outputs it as a G video signal.

The delay unit 212 is a delay process performed to convert the R signal into an R video signal that matches the timing of the G video signal.
The delay unit 213 is a delay process performed to convert the B signal into a B video signal that matches the timing of the G video signal.
Note that the delay units 212 and 213 may be LPFs (Low Pass Filters). The delay units 212 and 213 may be unnecessary when the image signals can be read from the R image sensor 111R and the B image sensor 111B with a predetermined time delay by the driving method of the image sensor drive unit 114.

  The video signal output unit 230 generates and outputs, for example, an HD-SDI signal from the G video signal, the R video signal, and the B video signal.

Example 1
FIG. 3A is a block diagram illustrating a configuration of a level difference correction unit according to an embodiment of the present invention.
3A, the level difference correction unit 211 includes a delay unit 311, a flatness determination unit 312, a level difference determination unit 313, and a level adjustment unit 314.
Note that the delay unit 311 is a delay process performed to change the G1 signal into a G1 ′ signal that matches the timing of the G2 ′ signal. When the level of the G1 signal is adjusted, the delay unit 311 delays the G2 signal.
The CPU unit 115 controls the flatness determination unit 312 and the level difference determination unit 313. Note that the CPU unit 115 may perform the processes of the flatness determination unit 312 and the level difference determination unit 313.

  The flatness determination unit 312 calculates the difference between the maximum value and the minimum value of the level of the target pixel and the surrounding pixels in each of the G1 signal and the G2 signal, and the target pixel if the difference between the maximum value and the minimum value is equal to or less than a predetermined threshold value. Is determined to be a flat portion.

  When the flatness determination unit 312 determines that both the G1 signal and the G2 signal are flat portions, the level difference determination unit 313 determines the average value of the levels of the target pixel and peripheral pixels of the G1 signal, and the target pixel of the G2 signal and the surrounding area. When the average value of the pixel level is compared and there is a difference between the average values and the difference is equal to or less than a predetermined upper limit value, the G1 signal and the G2 signal capturing the flat portion due to some influence It is determined that there is a difference in the video level.

  In the level difference determination unit 313, the upper limit value is provided for the difference between the average values. When a high-frequency striped pattern or the like is projected, a dark part of the striped pattern appears in the target pixel and surrounding pixels of the G1 signal. This is to prevent erroneous determination that a bright portion with a striped pattern appears in the target pixel and the peripheral pixels of the G2 signal and that the level difference is generated between the flat portions.

  When the level difference determination unit 313 determines that a level difference has occurred between the G1 signal and the G2 signal, the level adjustment unit 314 determines the gain ratio between the G1 signal and the G2 signal from the average value and the level difference. Is calculated, the gain of the G2 signal (or the G1 signal, or both the G1 signal and the G2 signal) is adjusted, and the levels of the G1 signal and the G2 signal are matched.

  As described above, according to one embodiment of the present invention, when a level difference occurs between the G1 signal and the G2 signal in the pixels of the flat portion of the video signal, the level difference is detected, and the G1 signal and the G2 signal are detected. By performing the correction to match the level difference, it is possible to prevent an oblique grid pattern from occurring in the image.

(Example 2)
FIG. 3B is a block diagram illustrating a configuration of a level difference correction unit according to another embodiment of the present invention.
3B, the level difference correction unit 211 includes a delay unit 311, a flatness determination unit 312, a level difference determination unit 313, a level adjustment unit 314, an integration average unit 325, and a memory unit 326.
Note that the delay unit 311 is a delay process performed to change the G1 signal into a G1 ′ signal that matches the timing of the G2 ′ signal.
The CPU unit 115 controls the flatness determination unit 312, the level difference determination unit 313, the integration average unit 325, and the memory unit 326. Note that the CPU unit 115 may perform processing of the flatness determination unit 312, the level difference determination unit 313, the integration average unit 325, and the memory unit 326.

  The flatness determination unit 312 calculates the difference between the maximum value and the minimum value of the level of the target pixel and the surrounding pixels in each of the G1 signal and the G2 signal, and the target pixel if the difference between the maximum value and the minimum value is equal to or less than a predetermined threshold value. Is determined to be a flat portion.

  When the flatness determination unit 312 determines that both the G1 signal and the G2 signal are flat portions, the level difference determination unit 313 determines the average value of the levels of the target pixel and peripheral pixels of the G1 signal, and the target pixel of the G2 signal and the surrounding area. When the average value of the pixel level is compared and there is a difference between the average values and the difference is less than or equal to a predetermined upper limit value, the G1 signal and G2 signal capturing the flat portion due to some influence It is determined that there is a difference in the video level.

  When the level difference determination unit 313 determines that a level difference has occurred between the G1 signal and the G2 signal, the level adjustment unit 314 determines the gain ratio between the G1 signal and the G2 signal from the average value and the level difference. Is calculated.

When the level difference determination unit 313 has a level difference between G1 and G2 in the level difference determination unit 313 and the level adjustment unit 314 calculates the gain ratio between the G1 signal and the G2 signal, the integration average unit 325 stores the gain ratio in the memory. Using the unit 326, the average is performed.
Since the range in which the gain ratio is integrated and averaged is within the same frame, the integration average unit 325 stores the gain ratio that has been integrated and averaged before moving to the processing of the next frame in the memory unit 326.

  In the next frame, when the flatness determination unit 312 determines that the target pixel is not a flat part, the level adjustment unit 314 uses the G2 signal (with the gain averaged over the previous frame stored in the memory unit 326). Alternatively, the level of the G1 signal or both the G1 signal and the G2 signal) is adjusted.

  As described above, according to another embodiment of the present invention, the G1 signal and the G2 signal are applied by applying the gain ratio of the G1 signal and the G2 signal that are integrated and averaged in the previous frame in pixels that are not flat portions of the video signal. By correcting the level difference, it is possible to prevent the occurrence of fixed pattern noise in the video.

(Example 3)
FIG. 3C is a block diagram showing a configuration of a level difference correction unit according to another embodiment of the present invention.
3C, the level difference correction unit 211 includes a delay unit 311, a flatness determination unit 312, a level difference determination unit 313, a level adjustment unit 314, an integration average unit 325, a memory unit 326, and a linear interpolation unit 337.
Note that the delay unit 311 is a delay process performed to change the G1 signal into a G1 ′ signal that matches the timing of the G2 ′ signal.
The CPU unit 115 controls the flatness determination unit 312, the level difference determination unit 313, the integration average unit 325, the memory unit 326, and the linear interpolation unit 337. Note that the CPU unit 115 may perform processing of the flatness determination unit 312, the level difference determination unit 313, the integration average unit 325, the memory unit 326, and the linear interpolation unit 337.

  The flatness determination unit 312 calculates the difference between the maximum value and the minimum value of the level of the target pixel and the surrounding pixels in each of the G1 signal and the G2 signal, and the target pixel if the difference between the maximum value and the minimum value is equal to or less than a predetermined threshold value. Is determined to be a flat portion.

  When the flatness determination unit 312 determines that both the G1 signal and the G2 signal are flat portions, the level difference determination unit 313 determines the average value of the levels of the target pixel and peripheral pixels of the G1 signal, and the target pixel of the G2 signal and the surrounding area. When the average value of the pixel level is compared and there is a difference between the average values and the difference is less than or equal to a predetermined upper limit value, the G1 signal and G2 signal capturing the flat portion due to some influence It is determined that there is a difference in the video level.

  When the level difference determination unit 313 determines that a level difference has occurred between the G1 signal and the G2 signal, the level adjustment unit 314 determines the gain ratio between the G1 signal and the G2 signal from the average value and the level difference. Is calculated.

The integrating average unit 325 divides the screen into a plurality of vertical and horizontal areas, and there is a level difference between the G1 signal and the G2 signal in the flat portion of the video of the divided area, and the level adjustment unit 314 calculates the gain ratio between the G1 signal and the G2 signal. Then, the gain ratio is integrated and averaged for each area using the memory unit 326.
When the integration averaging process for each area of one frame is completed, the integration averaging unit 325 stores the gain ratio obtained by integration averaging for each area in the memory unit 326.

The linear interpolation unit 337 uses the gain ratio of the G1 signal and G2 signal obtained by integration averaging for each area stored in the memory unit 326 as the gain ratio of the center pixel of the area, and linearly interpolates the gain ratio of the pixels in between. calculate.
Note that the linear interpolation unit 337 performs linear interpolation by skipping the area when there is no flat part in the area and there is no gain ratio information.

  In the next frame, when the flatness determination unit 312 determines that the target pixel is not a flat part, the level adjustment unit 314 uses the G2 signal (or G1 signal or G1 signal as the gain calculated by the linear interpolation unit 337). Adjust the level of both G2 signals).

  As described above, according to another embodiment of the present invention, the G1 signal is obtained by applying the gain ratio of the G1 signal and the G2 signal calculated by linear interpolation over the entire screen in the pixel that is not a flat portion of the video signal. By correcting the level difference between the G1 signal and the G2 signal, it is possible to prevent the occurrence of fixed pattern noise in the video even when the gain ratio between the G1 signal and the G2 signal is not constant in the screen.

  An image pickup apparatus according to an embodiment of the present invention can prevent the occurrence of noise in a diagonal grid pattern or a fixed pattern in an image in a dual green color image pickup apparatus.

  Although one embodiment of the present invention has been described in detail above, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

  By automatically correcting the level difference between the G1 signal and the G2 signal, the present invention can also be applied to a shooting environment with a large temperature fluctuation.

  100: incident light, 101: lens, 110: imaging device, 111G1: G1 imaging device, 111G2: G2 imaging device, 111R: R imaging device, 111B: B imaging device, 112: color separation optical system, 113: video signal processing 114: Image sensor drive unit, 115: CPU unit, 210: Level correction unit, 211: Level difference correction unit, 212, 213: Delay unit, 220: Selection unit, 230: Video signal output unit, 311: Delay unit 312: Flatness determination unit 313: Level difference determination unit 314: Level adjustment unit 325: Integration average unit 326: Memory unit 337: Linear interpolation unit

Claims (3)

In an imaging device having two green imaging elements,
A level difference correction unit that corrects a level difference between the G1 signal and the G2 signal output from the two green image sensors;
The level difference correction unit includes a flatness determination unit, a level difference determination unit, and a level adjustment unit,
The flatness determination unit determines flatness between the target pixel and the surrounding pixels of the G1 signal and the G2 signal,
The level difference determination unit determines whether there is a level difference in the average value of the target pixel and the peripheral pixels of the G1 signal and the G2 signal determined to be flat by the flatness determination unit,
The level adjusting unit calculates a gain ratio of the G1 signal and G2 signal when the level difference determination unit determines that there is a level difference, have row level adjustment by the gain ratio to the G1 signal or G2 signal,
The level difference correction unit further includes an integration average unit and a memory unit,
The integration average unit averages the gain ratio between the G1 signal and the G2 signal calculated by the level adjustment unit,
The memory unit stores the gain ratio obtained by the integration averaging unit,
The level adjustment unit is configured to apply a pixel level determined by the flatness determination unit to the G1 signal or the G2 signal at the gain ratio stored in the memory unit in the previous frame for pixels determined to be not flat images in the next frame. An image pickup apparatus that performs adjustment . The imaging device according to claim 1 ,
The level difference correction unit further includes a linear interpolation unit,
The cumulative average unit divides the screen into a plurality of areas, and averages the gain ratio of the G1 signal and the G2 signal for each area.
The memory unit stores the gain ratio obtained by the integration averaging unit for each area,
The linear interpolation unit calculates a gain ratio by linear interpolation over the entire screen,
The level adjustment unit is configured to apply a pixel level determined by the flatness determination unit to the G1 signal or the G2 signal at the gain ratio stored in the memory unit in the previous frame for pixels determined to be not flat images in the next frame. An image pickup apparatus that performs adjustment. In an imaging method in an imaging apparatus having two green imaging elements,
Level difference correction means for correcting the level difference between the G1 signal and the G2 signal output from the two green image sensors;
The level difference correction means includes flatness determination means, level difference determination means, and level adjustment means,
The flatness determining means determines flatness between the target pixel of the G1 signal and the G2 signal and the surrounding pixels,
The level difference determining means determines whether there is a level difference in the average value of the target pixel and the peripheral pixels of the G1 signal and the G2 signal determined to be flat by the flatness determining means,
The level adjusting means calculates a gain ratio between the G1 signal and the G2 signal when the level difference determining means determines that there is a level difference, performs level adjustment with the gain ratio on the G1 signal or the G2 signal,
The level difference correction means further includes a cumulative average means and a memory means,
The cumulative averaging means cumulatively averages the gain ratio between the G1 signal and the G2 signal calculated by the level adjusting means,
The memory means stores a gain ratio obtained by integrating and averaging the integrating average means,
The level adjusting unit is configured to apply a level to the G1 signal or the G2 signal at the gain ratio stored in the memory unit in the previous frame for a pixel determined by the flatness determining unit to be not a flat image in the next frame. An imaging method characterized by performing adjustment.

Images