JP7025161B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus, for example, an image pickup apparatus having a function of correcting a specific hue for each pixel.

The television camera has a function called 6-color independent masking or 12-color masking, which detects a specific hue for each pixel and corrects a specific hue for each pixel (see Patent Document 1).
In addition, the color printer is used on the monitor in non-linear high-order masking in which the linear matrix and the ink density signal (Y, _M , _C ) are determined by high-order polynomials for the three primary color density signals (DR, DG, _DB ). Colors that humans prefer by optimizing the color conversion parameters using an evaluation function that weights and adds the deviations in lightness, saturation, and hue between the colors in the print and the colors on the print. The reproduction area is compressed (see Patent Document 2).
Furthermore, the television camera uses the first color signal of the primary color gamut in a wider color gamut than the primary color point of BT.709 (standard ITU-R (International Telecommunication Union Radiocommunication sector) BT (Broadcasting service (Television)) .709). , BT.709 is converted to a second color signal based on the primary color and corrected (see Patent Document 3).

By the way, in a high-sensitivity color camera for surveillance, a neutral density filter (hereinafter referred to as a spot ND filter) is mounted only in the center near the aperture of the lens, or two ND filters with opposite shades are stacked in the opposite direction. A rotation variable ND filter that functions as a variable ND filter that changes the amount of incident light with an ND value even if the aperture is constant is used. There is also an electronic variable ND filter that electronically changes the light transmittance, which is mounted on a camcorder (see Non-Patent Document 5).
In broadcasting, monitoring, etc., the primary color signals of red (R), green (G), and blue (B) are often converted into luminance signals and color difference signals to halve the band and color gamut. However, improvements in video signal compression technology have made it possible to achieve both compression ratio and fidelity, and red, green, and blue primary color signals may be compressed for transmission or recording / playback.

Japanese Unexamined Patent Publication No. 9-247701 Japanese Unexamined Patent Publication No. 6-189121 Japanese Unexamined Patent Publication No. 2006-33575

FUJIFILM PHOTO HANDBOOK, Internet (URL: http://fujifilm.jp/support/filmandcamera/download/pack/pdf/ff_opticalfilter_001.pdf) HOYA ND0.1, Internet (URL: https://www.hoyacandeo.co.jp/japanese/products/eo_pdf/ND0.1.pdf) HOYA ND1.0, Internet (URL: https://www.hoyacandeo.co.jp/japanese/products/eo_pdf/ND1.pdf) Tokina Neutral Density Filter, Internet (URL: http://www.tokina.co.jp/io-filters/filters/neutral-density-filter.html) Luminex LCD shutter (electronic variable ND filter), Internet (URL: http://www.luminex.co.jp/pdf/products03/LC-tec_FOS-NIR1100-specification-1602.pdf)

However, monitoring and transmission with ITU / BT.2020 (UHD (ultra-high precision) TV standard), which has a wider color gamut than the red, green, and blue primary color signals or BT.709 (HD (high precision) TV standard) primary color points. When the color gamut becomes wider due to recording and playback, it becomes possible to recognize changes in the color tone due to changes in the aperture of the lens with a variable ND filter or changes in the spectral transmission rate of the variable ND filter due to changes in the ND value of the variable ND filter. It is more difficult to make adjustments to correct the deterioration of color reproduction due to the change in the aperture value of the lens with the variable ND filter or the change in the ND value of the variable ND filter.
An object of the present invention is to provide an image pickup apparatus capable of correcting a change in a color signal due to a change in the aperture value of a lens with a variable ND filter or a change in the ND value of a variable ND filter.

The following is a brief description of a typical example of the present invention.
That is, the image pickup device is a red, green, and blue image sensor of each pixel signal generated from a lens with a variable ND filter or a variable ND filter, an image pickup element with an on-chip color filter or a color separation optical system, and three or more image pickup elements. It has a means for detecting a specific hue for each pixel of a primary color image signal and independently correcting a specific hue for each pixel, a real-time auto-white means, and a hue calculation means for each pixel. In conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter, 6 or more independent color corrections that change the amount and direction of color correction on a pixel-by-pixel basis are performed.

According to the present invention, it is possible to increase the degree of freedom in adjusting the color correction of a color signal and to perform color correction more faithfully and independently in each hue range.

It is a block diagram which shows the television camera which concerns on Example 1. FIG. It is a block diagram which shows the television camera which concerns on modification 1-1. It is a schematic diagram which shows the magnitude relation of R / G / B and the corresponding hue range. It is a block diagram which shows the structure of the hue saturation detection correction part which concerns on Example 2. FIG. It is a block diagram which shows the structure of the hue saturation detection correction part which concerns on modification 2-2. It is explanatory drawing of the hue region in the color tone correction. It is a conceptual diagram of a hue region. It is explanatory drawing of the calculation principle of a primary color component, a complementary color component, and a saturation component. It is explanatory drawing of the color tone correction processing by the 6-color internal / external independent color tone correction method of the hue saturation detection correction part which concerns on Example 2. FIG. It is explanatory drawing of the color tone correction processing by the 6 color inside and outside independent color tone correction method of the hue saturation detection correction part which concerns on modification 2-1. It is explanatory drawing of the color tone correction processing by the 6-color internal / external independent color tone correction method of the hue saturation detection correction part which concerns on Example 4. FIG. It is explanatory drawing of the color tone correction processing by the 6 color inside and outside independent color tone correction method of the hue saturation detection correction part which concerns on modification 2-2. It is explanatory drawing of the color tone correction processing by the 6-color internal / external independent color tone correction method of the hue saturation detection correction part which concerns on Example 5. FIG. It is a correction characteristic diagram of Example 2. It is a block diagram which shows the structure of the hue saturation detection correction part which concerns on Example 3. FIG. It is a block diagram which shows the structure of the hue saturation detection correction part which concerns on Example 4. FIG. It is a block diagram which shows the structure of the hue saturation detection correction part which concerns on Example 5. FIG. It is explanatory drawing of the hue region in the color tone correction which concerns on Example 3. FIG. It is a conceptual diagram of the hue region which concerns on Example 3. FIG. It is explanatory drawing of the calculation principle of the primary color component and the complementary color component which concerns on Example 3. FIG. It is a correction characteristic diagram of the modification 3-1. It is a schematic diagram which shows the operation of the conventional 12-color independent color tone correction. It is a schematic diagram which shows the operation on the color vector waveform of 6-color independent color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of a 12-color independent color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of 16-color independent color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of 24 color independent color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of 6 color inside and outside independent color tone correction corresponding to the saturation of 6 color points of a vector chart, (a) is an example of a film-based ND filter, (b) is a glass-based. This is an example of the ND filter. It is a schematic diagram which shows the operation on the color vector waveform of the 12-color internal / external independent color tone correction corresponding to the saturation of 6 color points of a vector chart, (a) is an example of a film-based ND filter, and (b) is a glass-based. This is an example of the ND filter. It is a schematic diagram which shows the operation on the color vector waveform of the 12-color internal / external independent color tone correction corresponding to the saturation of 6 color points of a vector chart, (a) is an example of a film-based ND filter, and (b) is a glass-based. This is an example of the ND filter. There is a schematic diagram showing the operation on the color vector waveform of 24-color internal / external independent color tone correction corresponding to the saturation of 6 color points of the vector chart, (a) is an example of a film-based ND filter, and (b) is a glass-based This is an example of an ND filter. It is a schematic diagram which shows the operation on the color vector waveform of 6-color independent saturation continuous variable color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of a 12-color independent saturation continuously variable color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of 16-color saturation independent continuous variable color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram which shows the operation on the color vector waveform of 24 color saturation independent continuous variable color tone correction, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram of the spectral transmittance of an ND filter, (a) is an example of a film-based ND filter, and (b) is an example of a glass-based ND filter. It is a schematic diagram of the spot ND filter and the aperture value of a lens. It is a schematic diagram of a rotation variable ND filter.

First, the ND filter will be described with reference to FIGS. 18 to 20. FIG. 18 is a schematic diagram of the spectral transmittance of the ND filter, FIG. 18 (a) is FUJIFILM ND 1.0 (10%), which is an example of a film-based ND filter, and FIG. 18 (b) is a glass-based filter. HOYA ND 1.0 (1%), which is an example of an ND filter. FIG. 19 is a schematic diagram of an example of a spot ND filter of a variable ND filter and an aperture value of a lens, and is an example of an equivalent aperture value F16 of a spot ND filter with an open aperture value F2 of the lens. FIG. 20 is a schematic diagram of a rotation variable ND filter, which is an example of a variable ND filter.

In a high-sensitivity color camera for surveillance, as shown in FIG. 19, a spot ND filter is mounted only in the central portion (corresponding aperture value F16 in FIG. 19) near the aperture of the lens. In the vicinity of the opening of the aperture of the lens, the transmittance around the spot ND filter is dominated by 100%, and the transmittance is almost 100%. The transmittance at the center of the spot ND filter (for example, 1%) becomes dominant near the aperture cutoff of the lens aperture (equivalent aperture value F16 in FIG. 19), and the transmittance is almost the same as the transmittance of the spot ND filter (for example, 1%). Become. As a result, in the direct sunlight of snow, it can be handled near the aperture of the lens aperture (equivalent aperture value F16 in Fig. 19), and in the star light, it can be handled near the opening of the lens aperture (F2 in Fig. 19), and it corresponds to a wide range of illuminance. can do.

In addition, although the spectral transmittance of the ND filter is almost uniform on a film basis, the transmittance at the red end is 2 to 3 times higher than the green average transmittance, and the green is green compared to the green long wavelength side transmittance. The transmittance on the short wavelength side of the above is 20% lower, and the transmittance at the blue end is 10% lower than the transmittance of the green average (see FIG. 18A and Non-Patent Document 1). However, if the ND filter is film-based, aberrations may occur due to poor thickness uniformity, resulting in a decrease in the MTF (Modulation Transfer Function) of the lens.

Furthermore, in the ND filter, the spectral transmittance on a glass basis is as high as 3.3 to 7 times the transmittance at the red end compared to the green average transmittance, and the green short wavelength compared to the green center wavelength side transmittance. The side transmittance is 20% lower, the green long wavelength side transmittance is 20% to 30% lower than the green center wavelength side transmittance, and the blue edge transmittance is 1/3 to 1/3 compared to the green average transmittance. Some are as low as 8 (see FIG. 18 (b) and Non-Patent Documents 2, 3 and 4). However, when the ND filter is glass-based, the thickness uniformity is good, aberration does not occur, and the MTF of the lens does not decrease.

On a glass base, the transmittance at the red end is 3.3 to 7 times higher than the green average transmittance, and the green short-wavelength side transmittance is 20% compared to the green center wavelength side transmittance. The transmittance on the long wavelength side of green is 20% to 30% lower than the transmittance on the center wavelength side of green, and the transmittance at the blue end is 1/3 to 1/8 lower than the transmittance on the green average (non-patented). See Documents 2, 3 and 4).

Further, as shown in FIG. 20, the rotation variable ND filter functions as a variable ND filter in which two ND filters having opposite shades are stacked and rotated in the opposite direction to change the incident light amount by the ND value even if the aperture is constant. Although the shades are shown on the stairs in FIG. 20, the shades are actually continuously variable.

As mentioned above, when the color gamut becomes wider due to monitoring, transmission, recording and playback in ITU / BT.2020, which has a wider color gamut than the red, green, and blue primary color signals or the primary color point of BT.709, a lens with a variable ND filter. It becomes possible to recognize the change in color gamut due to the change in the spectral transmission of the variable ND filter due to the change in the aperture of the variable ND filter or the change in the ND value of the variable ND filter. It is even more difficult to make adjustments to compensate for the deterioration of color reproduction due to changes in the values. The variable ND filter includes a spot ND filter, a rotation variable ND filter, an electronic variable ND filter, and the like.

Therefore, in the embodiment, the configuration is as follows.

(1) The image pickup device is
(A) Red, green, and blue primary color video signals of each pixel signal generated from a lens or variable ND filter with a variable ND filter, an image sensor with an on-chip color filter or a color separation optical system, and three or more image sensors. It has a means for detecting a specific hue for each pixel and independently correcting a specific hue for each pixel, and (b) a real-time auto white means. The means (a) has a hue calculation means for each pixel, and changes the amount and direction of color correction for each pixel in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter 6 Performs independent color correction beyond color.

(2) In the image pickup apparatus of (1) above,
In conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter, 6 or more independent color corrections are performed to change at least one hue in the direction toward the short wavelength direction.

(3) In the image pickup apparatus of (2) above,
In conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter, 12 or more independent color corrections are performed to change the hue of green in the direction toward the center or both ends.

(4) In the image pickup apparatus of (2) above,
In conjunction with the aperture value of a lens with a variable ND filter or the ND value of a variable ND filter, the hues of at least one or more colors on the short wavelength side and the center hue are varied in the direction toward the short wavelength direction, or at least one or more. The long wavelength end of the color of is subjected to 16 or more independent color corrections that are variable in the direction toward the long wavelength direction.

(5) In the image pickup apparatus of (2) above,
In conjunction with the aperture value of the lens with variable ND filter or the ND value of the variable ND filter, the hue is changed in the direction of moving the hue toward the short wavelength direction only at the blue end, or the hue is moved toward the short wavelength direction only at the long wavelength end of red. 24 or more independent color corrections are performed to change the hue at the wavelength end in the short wavelength direction, such as by changing with.

(6) The image pickup device includes a lens with a spot ND filter and an image pickup element with an on-chip color filter having a high cross point of blue and green spectral transmission, or a color separation optical system having a high cross point of blue and green spectral transmission. A specific hue is detected for each pixel of 6 or more independent color corrections of the primary color video signal of red, green, and blue of each pixel signal generated from three or more image pickup elements, and a specific hue is specified for each pixel. It has a means for independently correcting hue and saturation. This correction means performs independent color correction of 6 or more colors in which the amount and direction of color correction of a specific hue and saturation are changed in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter.

According to the embodiment, among the color changes due to the variation in the spectral transmittance of the ND filter, the color change that cannot be corrected by the red-blue gain adjustment of the auto white is the aperture value of the lens with the variable ND filter or the ND of the variable ND filter. Increases the degree of freedom in adjusting the color correction of the color signal in response to changes in color due to the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter according to the value, making it more faithful and independent in each hue range. More faithful color correction can be achieved with easy adjustment.

Furthermore, if the direction and amount of color correction are adjusted independently in the range of each hue and saturation in conjunction with saturation, the cross point of the spectral transmission of blue and green is not for BT.2020, which is low, but the conventional one. An image pickup device with an on-chip color filter with a high cross point of blue and green spectral transmission or a color separation optical system for BT.2020 with a low cross point of blue and green spectral transmission, but with blue for BT709, etc. Increased the degree of freedom in adjusting the color correction of the red, green, and blue primary color video signals of each pixel signal generated from a color separation optical system with a high crosspoint of green spectral transmission and three or more image pickup elements. Therefore, it is possible to realize more faithful color correction independently and more faithfully in each hue range by easy adjustment.

Hereinafter, examples and modifications will be described with reference to the drawings. However, in the following description, the same components may be designated by the same reference numerals and repeated description may be omitted.

(Outline of independent color correction that changes the color correction direction according to saturation)
The image pickup apparatus according to the first embodiment will be described with reference to FIGS. 1A, 2, 15A to 15D, and 18.

FIG. 1A is a block diagram showing the configuration of the television camera according to the first embodiment, and is a pre-gamma matrix using a prism and three image pickup elements. FIG. 2 is a schematic diagram showing a hue range corresponding to the magnitude relationship of R / G / B. FIG. 15A is a schematic diagram showing the operation on the color vector waveform of the ND filter and the 6-color independent color tone correction. FIG. 15B is a schematic diagram showing the operation on the color vector waveform of the ND filter and the 12-color independent color tone correction. FIG. 15C is a schematic diagram showing the operation on the color vector waveform of the ND filter and the 16-color independent color tone correction. FIG. 15D is a schematic diagram showing the operation on the color vector waveform of the ND filter and the 24-color independent color tone correction. In FIGS. 15A-15D, (a) corresponds to FUJIFILM ND1.0 (10%), which is a film-based ND filter, and (b) corresponds to HOYA ND1.0 (1%), which is a glass-based ND filter. do.

As shown in FIG. 1A, the television camera main body 30A according to the first embodiment includes a prism 32, three image pickup elements 33R, 33G, 33B, a video signal processing unit 35A, and a CPU (Central Processing Unit) unit. It is composed of 39 and a parallel-serial conversion unit (P / S) 37. Each of the three image pickup elements 33R, 33G, and 33B is a CCD (charge-coupled device) image pickup element and an AFE (analog front-end processor) or a CMOS (Complementary Metal Oxide Semiconductor) image pickup element having a built-in AFE. The CPU unit 39 controls each unit of the television camera body 30A.

The television camera body 30A is combined with a lens 31 with a variable ND filter and an image display unit 40 such as a viewfinder or a monitor display to form the television camera (imaging device) 100A according to the first embodiment. The variable ND filter of the lens 31 with a variable ND filter may be a film-based spot ND filter or a glass-based spot ND filter provided near the aperture, or may be a rotation variable ND filter or an electronic variable ND filter. In FIG. 1A, a lens 31 with a variable ND filter is used, but after a lens without a variable ND filter (between the lens and the prism 32 which is a color separation optical system), a rotation variable ND filter, an electronic variable ND filter, or the like is used. A variable ND filter may be provided.

The incident light from the subject is imaged by the lens 31 with a variable ND filter, and the imaged incident light is decomposed into red light, green light, and blue light by the prism 32 of the television camera body 30A, respectively, and the image sensors 33R and 33G, respectively. , 33B photoelectrically converted, correlated double sampling, gain correction, and analog-digital conversion are performed, and sent to the video signal processing unit 35A for various video signal processing such as color correction, contour correction, gamma correction, and knee correction. It will be done.

The video signal processing unit 35A is composed of a hue saturation detection correction unit 38A and a matrix (MATRIX) unit 36.

The hue saturation detection correction unit 38A is composed of adders 12, 13, 14, a gamma correction unit 381, and a color tone correction unit 382, and performs color tone correction before gamma correction.

The color tone correction unit 382 detects the color difference between RG, RB, and GB and the saturation, determines the hue region, determines the saturation, and calculates the correction value. Based on the aperture value of the lens 31 with a variable ND filter or the ND value of the variable ND filter obtained via the CPU unit 39, the video signal processing unit 35A performs color tone correction, and the three image pickup elements 33R, 33G, 33B The red-blue gain is adjusted with. Auto white balance is performed by adjusting the red and blue gains.

After various video signal processing is performed in the video signal processing unit 35A, the matrix unit 36 outputs the video signal of BT.709 Y = 0.2126R + 0.7152G + 0.0722B.
Pb = 0.5389 (BY)
Pr = 0.6350 (RY)
The R / G / B is converted into a luminance signal (Y) and a color difference signal (Pb / Pr) according to the calculation formula of. Then, it is converted into a serial video signal by the parallel-serial conversion unit 37 and output to the outside.

In addition, Y = 0.2627R + 0.6780G + 0.0593B of the output of the video signal in ITU / BT.2020, which has a wider color gamut than the primary color point of BT.709.
Pb = 0.5315 (BY)
Pr = 0.6782 (RY)
There is also a video signal output based on the formula. In addition, there are outputs of video signals of the primary colors of red, green, and blue.

The image display unit 40 displays a setting menu of the television camera main body 30 and an arbitrary hue saturation region within a specific hue.

The hue saturation detection correction unit 38 in the video signal processing unit 35A is further linked with the saturation based on the aperture value of the lens 31 with a variable ND filter or the ND value of the variable ND filter obtained via the CPU unit 39. The direction and amount of color correction are calculated, and the hue range (hue region) of the subject is detected from the magnitude relationship of each signal level of R / G / B as shown in FIG.

Here, the hue is displayed in 6 divisions, but if the magnitude relationship of each signal level of R / G / B is further subdivided, 12 colors are independent, 16 colors are independent, 18 colors are independent, 24 colors are independent, etc. It is also possible to redifferentiate the hue more.

The CPU 9 outputs information (signal a) in an arbitrary hue range set by the user to the hue saturation detection correction unit 38A in the video signal processing unit 35A. The hue saturation detection correction unit 38A in the video signal processing unit 35A outputs the hue information of the pixels and the saturation information of the pixels that match the hue range set by the user to the CPU 9. In the CPU 9, based on the hue information of the pixel, the saturation information of the pixel, the aperture value of the lens 31 with the variable ND filter, or the ND value of the variable ND filter, the hue saturation in the video signal processing unit 35A is further linked with the saturation. Control the correction calculation of. The hue saturation detection correction unit 38A more faithfully corrects the color change that cannot be corrected by the red and blue gain adjustment of auto white among the color change due to the variation in the spectral transmission rate of the ND filter, and independently corrects the color in each hue range. Then, color correction is performed independently in each hue range according to the aperture value of the lens with the variable ND filter or the ND value of the variable ND filter.

The hue saturation detection correction unit 38A has, for example, 6-color independent color tone correction for moving red and blue in the short wavelength direction as shown in FIG. 15A, and 12-color independent color tone correction for moving the green hue to the center as shown in FIG. 15B. 16 color independent color tone correction when corresponding to an inclination of 560 nm to 580 nm as shown in FIG. 15C (a) and when corresponding to a concave portion of 560 nm to 600 nm as shown in FIG. 15C (b), FIG. 15D (a). When it corresponds to a convex of 700 nm and a concave of 420 nm as shown in FIG. 15D, and when it corresponds to an inclination of 530 nm to 540 nm and when it corresponds to an inclination of 460 nm to 480 nm as shown in FIG. 15D (b). Performs 24-color independent color tone correction.

The image display unit 40 superimposes the menu screen on the image of the subject, and the user sets the hue range, the saturation range, and the luminance signal level while looking at the menu screen. Further, the hue range and the saturation range set by superimposing the subject image on the image display unit 40 are matched so that it can be confirmed whether the hue range set by the user matches the color of the target subject. A marker may be displayed in the area of the place.

<Modification 1-1>
A modified example of the first embodiment (modified 1-1) will be described with reference to FIG. 1B.

FIG. 1B is a block diagram showing the configuration of the television camera according to the modified example 1-1, and is a post-gamma matrix using an image sensor with an on-chip color filter.

As shown in FIG. 1B, the television camera main body 30B according to the second embodiment includes an image sensor 34, a video signal processing unit 35B, a CPU unit 39, and a parallel-serial conversion unit (P / S) 37. It is configured. The image sensor 34 is a CCD image sensor with an on-chip color filter and an AFE, or a CMOS image sensor with a built-in AFE. The CPU unit 39 controls each unit of the television camera body 30B.

The television camera body 30B is combined with a lens 31 with a variable ND filter and an image display unit 40 such as a viewfinder or a monitor display to form the television camera (imaging device) 100B according to the modified example 1-1.

The incident light from the subject is imaged by the lens 31 with a variable ND filter, and the imaged incident light is photoelectrically converted by the image sensor 34 of the television camera body 30B for correlation double sampling, gain correction, and analog-digital. It is converted and sent to the video signal processing unit 35B, where various video signal processing such as color correction, contour correction, gamma correction, and knee correction is performed.

The video signal processing unit 35B is composed of a hue saturation detection correction unit 38B and a matrix (MATRIX) unit 36.

The hue saturation detection correction unit 38B is composed of adders 12, 13, 14, a gamma correction unit 381, a color tone correction unit 382, and a color tone separation unit 383, and performs color tone correction after gamma correction.

The color tone correction unit 382 performs calculations for RG, RB, GB and saturation detection, hue region determination, and saturation determination. Based on the aperture value of the lens 31 with a variable ND filter or the ND value of the variable ND filter obtained via the CPU unit 39, the image signal processing unit 35B performs color tone correction, and the image sensor 34 or the color tone separation unit 383 red-blue. Gain adjustment is performed.

The difference between the color tone correction of the image pickup apparatus according to Example 1 and the modification 1-1 and the conventional color tone correction is that the color tone correction is performed based on the aperture value of the lens 31 with a variable ND filter or the ND value of the variable ND filter. The red-blue gain adjustment (auto white balance) is performed based on the aperture value of the lens 31 with the variable ND filter or the ND value of the variable ND filter.

Further, the differences between the color tone correction of the image pickup apparatus according to the first embodiment and the first modification and the conventional image pickup device are as follows.

(1) When corresponding to BT.2020 with a wide color range, the crosspoint of the spectral transmittance of blue and green is low. Instead of the image pickup device for BT.2020, the crosspoint of the conventional blue and green spectral transmittance is An image pickup device with a high on-chip color filter, or a cross point of blue and green spectral transmittance such as for BT.709 instead of a color separation optical system for BT.2020 with a low cross point of spectral transmittance between blue and green. It is to use a high color separation optical system and three or more image pickup elements.

(2) From the red, green, and blue primary color video signals of each pixel signal generated from the image pickup element of (1) above, when corresponding to BT.2020 with a wide color gamut, the color is correlated with the saturation for each pixel. The amount and direction of color correction are changed in pixel units such as changing the amount and direction of correction or changing the amount and direction of color correction independently inside and outside the saturation. This provides an independent color correction function for 6 or more colors that changes the amount and direction of color correction in conjunction with the aperture value of a lens with a variable ND filter or the ND value of a variable ND filter for each pixel. To have.

According to the above (1) and (2), it is possible to output a red, green and blue primary color signal or a video signal in ITU / BT.2020 having a wider color gamut than the primary color point of BT.709.

That is, the image pickup apparatus according to Example 1 or Modification 1-1 (this example) includes (a) a lens with a variable ND filter or a variable ND filter, an image sensor with an on-chip color filter, or a color separation optical system. (B) A means for detecting a specific hue for each pixel of the red, green, and blue primary color video signals of each pixel signal generated from one or more image sensors and independently correcting the specific hue for each pixel. It has real-time auto-white means. (A) The means has a hue calculation means for each pixel, and is linked with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter and further linked with the saturation, for example, FIGS. 15A to 15D and FIGS. As shown in, 6 or more independent color corrections are performed in which the amount and direction of color corrections are varied for each pixel. Further, the image pickup apparatus according to the present embodiment has a gain adjusting means for adjusting the red-blue gain based on the aperture value of the lens with spot ND or the ND value of the variable ND filter.

Further, the image pickup apparatus according to the present embodiment obtains at least one or more hues as described below in conjunction with the aperture value of the lens 31 with a variable ND filter or the ND value of the variable ND filter and further with the saturation. Performs independent color correction of 6 or more colors that are variable in the direction closer to the short wavelength direction.
(1) For example, as shown in FIG. 18 (a), for example, FIGS. 15A (a) and 18 (a) correspond to the change in hue due to the transmittance of the red end of the film-based ND filter being higher than the green average. As shown in, red changes the hue in the direction toward the short wavelength direction.
(2) For example, as shown in FIG. 18 (b), for example, FIGS. 15A (b) and 18 (b) correspond to the change in hue due to the transmittance of the red end of the glass-based ND filter being higher than the green average. As shown in, red changes the hue in the direction toward the short wavelength direction.
(3) For example, as shown in FIG. 18A, in response to a change in hue due to a lower transmittance at the blue end than the green average transmittance of a film-based ND filter, for example, FIG. 15A (a), As shown in 18 (a), blue changes the hue in the direction toward the short wavelength direction.
(4) For example, as shown in FIG. 18 (b), the glass-based ND filter also responds to the change in hue due to the lower transmittance at the blue end than the green average transmittance, for example, FIG. 15A (b). As shown in 18 (b), blue changes the hue in the direction toward the short wavelength direction.

Further, the image pickup apparatus according to the present embodiment is linked with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter, and further linked with the saturation, in a direction in which the hue is moved to the center or both ends as shown below. Performs independent color correction of 12 or more colors that can be changed with.
(1) For example, as shown in FIG. 18 (a), in response to a change in hue due to a lower central wavelength side transmittance than the green long wavelength side transmittance and short wavelength side transmittance of a film-based ND filter. For example, as shown in FIGS. 15B (a) and 18 (a), green changes the hue in a direction toward the center wavelength side.
(2) For example, as shown in FIG. 18B, it corresponds to the change in color tone due to the lower transmittance on the long wavelength side and the transmittance on the short wavelength side than the green central long wavelength side transmittance of the glass-based ND filter. For example, as shown in FIGS. 15B (b) and 18 (b), the hue of green changes in a direction away from the center of green.

Further, the image pickup apparatus according to the present embodiment is linked with the aperture value of the lens with the spot ND filter or the ND value of the variable ND filter, and further linked with the saturation, and has at least one or more color lengths as shown below. 16 or more independent color corrections are performed in which the wavelength side is variable in the direction toward the short wavelength direction, or the long wavelength end of at least one color is variable in the direction toward the long wavelength direction.
(1) For example, as shown in FIG. 18 (a), the change in hue due to the lower transmittance on the center wavelength side (the transmittance from 560 nm to 580 nm is inclined) than the transmittance on the long wavelength side of green of the film-based ND filter. Corresponding to, for example, as shown in FIGS. 15C (a) and 18 (a), the long wavelength side of green changes the hue in the direction toward the short wavelength side.
(2) For example, as shown in FIG. 18B, the transmittance at the green long wavelength end is lower than the transmittance at the center side of the green long wavelength side of the glass-based ND filter (transmittance from 580 nm to 600 nm). Corresponds to the change in color tone due to the concave), for example, as shown in FIGS. 15C (b) and 18 (b), the long wavelength end of green changes the hue in a direction away from the center of green.

Further, the image pickup apparatus according to the present embodiment interlocks with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter, and further interlocks with the saturation, and changes the hue at the wavelength end in the short wavelength direction as shown below. Performs independent color correction of 24 or more colors that can be changed to.
(1) For example, as shown in FIG. 18A, the film-based ND filter has a spectral transmittance equivalent to the average spectral transmittance of green at the center of blue and the long wavelength side, and the blue edge compared to the average spectral transmittance of green. Corresponding to the change in color tone due to the low transmittance of the chisel (the spectral transmittance at 420 nm is concave), for example, as shown in FIGS. 15D (a) and 18 (a), only the blue end is shifted in the short wavelength direction. Variable in direction.
(2) For example, as shown in FIG. 18A, the film-based ND filter has a spectral transmittance higher than the average spectral transmittance of green at the center of red and the short wavelength side, and the transmittance is remarkably high only at the long wavelength end of red. Corresponding to the change in color tone due to the fact (the spectral transmittance at 700 nm is convex), for example, as shown in FIGS. Let me.
(3) For example, as shown in FIG. 18 (b), it corresponds to the change in hue due to the lower transmittance on the short wavelength side (inclination from 530 nm to 540 nm) than the transmittance on the green center wavelength side of the green of the glass-based ND filter. Then, for example, as shown in FIGS. 15D (b) and 18 (b), the hue on the center wavelength side of green is varied in the direction toward the short wavelength direction.
(4) For example, as shown in FIG. 18 (b), the change in hue due to the lower transmittance at the long wavelength end side (inclination from 460 nm to 480 nm) compared to the long wavelength central side transmittance of blue of the glass-based ND filter. Correspondingly, as shown in FIGS. 15D (b) and 18 (b), for example, the hue on the center side of the long wavelength of blue is varied in the direction toward the long wavelength direction.

Further, the independent color correction method for 6 or more colors for each pixel signal is linked to the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter for each pixel, and is shown in FIGS. 15A to 15D and FIG. Variables the amount and direction of color correction for a particular hue and saturation.

Further, in the above color correction method, at least one hue is shortened as described below by interlocking with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter and further interlocking with the saturation in pixel units. It is variable in the direction closer to the wavelength direction.
(1) As shown in FIG. 18 (a), the transmittance of the red end of the film-based ND filter is higher than the green average, and as shown in FIG. 18 (b), the transmittance of the red end of the glass-based ND filter is higher. As shown in FIGS. 15A to 15D and 18, red changes the hue in the direction toward the short wavelength direction in response to the change in hue due to being higher than the green average.
(2) As shown in FIG. 18A, FIGS. 15A to 15D, 18 (a) correspond to the change in color tone due to the lower transmittance at the blue end than the green average transmittance of the film-based ND filter. ), Blue changes the hue in the direction toward the short wavelength direction.
(3) As shown in FIG. 18 (b), the glass-based ND filter also corresponds to the change in color tone due to the lower transmittance at the blue end than the green average transmittance, and FIGS. 15A to 15D, 18 (b). ), Blue changes the hue in the direction toward the short wavelength direction.

As described above, the image pickup apparatus of this embodiment is variable by adjusting the direction and amount of color correction in conjunction with the saturation according to the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter. When correcting the color change of the color signal due to the aperture value of the lens with ND filter or the ND value of the variable ND filter, the amount and direction of color correction is changed for each pixel as shown below, or the color is corrected independently inside and outside the saturation. 6 or more independent color corrections are performed to change the amount and direction of color correction for each pixel, such as changing the amount and direction of.
(1) Red has a hue corresponding to the change in hue due to the transmittance of the red end of the film-based ND filter being higher than the green average and the transmittance of the red end of the glass-based ND filter being higher than the green average. It can be changed in the direction toward the short wavelength direction, or (2) both the film-based ND filter and the glass-based ND filter respond to the change in hue due to the lower transmittance at the blue end than the green average transmittance. Blue changes the hue in the direction toward the short wavelength direction.

As a result, it is possible to more faithfully correct the color change of the color signal due to the aperture value of the lens with the variable ND filter or the ND value of the variable ND filter in conjunction with the saturation by easy adjustment.

(Details of independent color correction that changes the color correction direction according to saturation)
Example 2 which is an example of the hue saturation detection correction unit 38 of Example 1 will be described with reference to FIGS. 3A, 4 to 6, 7A, 8, 17A to 17D.

FIG. 3A is a block diagram showing a configuration of a hue saturation detection correction unit according to the second embodiment. FIG. 4 is an explanatory diagram of a hue region in the color tone correction according to the second embodiment. FIG. 5 is a conceptual diagram of a hue region according to the second embodiment. FIG. 6 is an explanatory diagram of the calculation principle of the primary color component, the complementary color component, and the saturation component according to the second embodiment, and is a diagram showing that the saturation component is determined in addition to the primary color component and the complementary color component.

FIG. 7A is an explanatory diagram of a color tone correction process by a six-color internal / external independent color tone correction method, and is a case where the amount and direction of the color correction are linearly correlated with the saturation in FIG. 17A. FIG. 8 is a correction characteristic diagram of the second embodiment.

FIG. 17A is a schematic diagram showing the operation on the color vector waveform of the 6-color independent saturation continuously variable color tone correction. FIG. 17B is a schematic diagram showing the operation on the color vector waveform of the 12-color independent saturation continuously variable color tone correction. FIG. 17C is a schematic diagram showing the operation on the color vector waveform of 16 color saturation independent continuously variable color tone correction. FIG. 17D is a schematic diagram showing the operation on the color vector waveform of the 24-color saturation independent continuously variable color tone correction. In FIGS. 17A to 17D, (a) corresponds to the example of FUJIFILM ND1.0 (10%), and (b) corresponds to the example of HOYA ND1.0 (1%).

As shown in FIG. 3A, the hue saturation detection correction unit 38A includes a color tone correction unit 382A and adders 12, 13, and 14. The color correction unit 382A includes subtractors 1, 2, and 3, a hue region determination circuit 4, a component amount determination circuit 5, a constant selection circuit 6, a multiplier 7, 8, 16 and an adder (-1 times multiplication). Instrument) 9, 10 and a data selection addition circuit 11.

The hue correction unit 382A first calculates the color difference signal (RG) from the video signal (R, G) input by the subtractor 1, and then the video signal (R, B) input by the subtractor 2. ), The color difference signal (RB) is calculated, the subtractor 3 calculates the color difference signal (GB) from the input video signals (G, B), and the result is the hue region determination circuit 4. To the component amount determination circuit 5, the saturation component amount, the primary color component amount, and the complementary color component amount are determined.

Therefore, based on the calculation results of the subtractors 1, 2 and 3, the hue region determination circuit 4 first determines the hue region as shown in FIG. FIG. 5 is a conceptual diagram of this hue region, in which a straight line from the center point toward each color direction of the primary colors R, G, B and the complementary colors Ye, Cy, and Mg is used as a reference line, thereby the first region (region 1). , 2nd region (region 2), 3rd region (region 3), 4th region (region 4), 5th region (region 5) and 6th region (region 6) divided into 6 hue regions. Is.

Further, in the component amount determination circuit 5, the levels of the video signals (R, G, B) are compared, and the maximum level, the intermediate level, and the minimum level are determined as shown in FIG. Then, in the process of this comparison determination, the level difference between the maximum level and the intermediate level is obtained and used as the primary color component amount, and further, the level difference between the intermediate level and the minimum level is obtained and used as the complementary color component amount. The level difference between the maximum level and the minimum level is obtained, and this is used as the saturation component amount. Here, the maximum level color corresponds to the primary color, and the minimum level component corresponds to the white component. Then, the complementary color can be determined from the information of the maximum level color and the minimum level color, and as a result, as shown in FIG. 4, the primary color component and the complementary color component can be determined.

That is, the color tone correction unit 382A realizes color correction in which the direction is continuously variable with the saturation of FIGS. 17A to 17D showing the operation on the color vector waveform of the independent saturation continuously variable color tone correction.

In the example of FIG. 6, since the maximum level is R and the intermediate level is G, the primary color component is R and the complementary color component is Ye (yellow), which is an intermediate hue between R and G. The primary color component amount is RG, the complementary color component amount is GB, and the minimum level B amount is the white component amount. Therefore, in the case of FIG. 6, the result shown is the second from the bottom of FIG. RB is the amount of saturation component.

The determination result of the hue region by the hue region determination circuit 4 is supplied to the constant selection circuit 6, a specific gain constant is selected by the constant selection circuit 6 according to the determination result, and it is supplied to the multipliers 7 and 8. Therefore, the correction is performed by multiplying the primary color component amount and the complementary color component amount determined by the component amount determination circuit 5, respectively. Therefore, in the constant selection circuit 6, a specific gain constant corresponding to each hue region from the first region (region 1) to the sixth region (region 6) is set in advance.

In this way, the primary color component amount and the complementary color component amount multiplied by the gain constants by the multipliers 7 and 8 are used in the data selection addition circuit 11 for selecting addition / subtraction and connection selection for the video signal (R, G, B). , On the one hand directly and on the other hand via complements (-1 multiplier) 9 and 10, respectively. Then, after the addition destination is selected by the data selection addition circuit 11, the data is supplied to the adders 12, 13, and 14 and added to the video signals (R, G, B). Therefore, if the above processing is shown in a flowchart, it will be as shown in FIG. 7A, for example.

In the flowchart of FIG. 7A, the color difference is calculated, the maximum value and the minimum value are determined, the coefficient is calculated according to the maximum value, the minimum value, and the saturation, and the linear change (the amount and direction of the color correction are linear with the saturation). Correlation) correction. An example shown in FIG. 6 (when the maximum level is R, the intermediate level is G, and the minimum level is B) will be described with reference to FIG. 7A.

Step S101: The arithmetic unit 1 calculates the RG color difference signal, the arithmetic unit 2 calculates the RB color difference signal, and the arithmetic unit 3 calculates the GB color difference signal.

Step S102: The hue region determination circuit 4 determines whether or not BR is negative, and if YES (BR <0), the process proceeds to step S103, and if NO (BR ≧ 0), step S. Move to S104.

Step S103: The hue region determination circuit 4 determines whether or not RG is positive, and if YES (RG> 0, R is the maximum level), the process proceeds to step S105 and NO (RG ≦ 0). , G is the maximum level), the process proceeds to step S106.

Step S105: The hue region determination circuit 4 determines whether GB is negative, and if YES (GB <0, G is the minimum level), the process proceeds to step S108, and NO (RB ≧ 0). , B is the minimum level), the process proceeds to step S115.

The hue region determination in steps S104, S106, and S107 is the same as described above, and the description thereof will be omitted. Further, steps S108 to S114 are processing of the first region (region 1), steps S115 to S121 are processing of the sixth region (region 6), steps S122 to S128 are processing of the fourth region (region 4), and steps S129 to S129. S135 corresponds to the processing of the fifth region (region 5), steps S136 to S142 correspond to the processing of the third region (region 3), and steps S143 to S149 correspond to the processing of the second region (region 2).

In the case of the example of FIG. 6, the component amount determination circuit 5 determines that RB is the saturation component amount, RG is the primary color component amount, and GB is the complementary color component amount, and performs the processing after step S115.

Step S115: The constant selection circuit 6 supplies the constants of L1, L4, L7, and L10 to the multiplier 16, the multiplier 16 multiplies the constant by the saturation component, and L1 (RB), L4 (RB). ), L7 (RB), L10 (RB) Saturation component coefficients are calculated.

Step S116: The constant selection circuit 6 supplies the constants of K1 and K4 to the multiplier 7, the multiplier 7 multiplies the constant by the primary color component, and the primary color component coefficients of K1 (RG) and K4 (RG). Perform the calculation of.

Step S117: The constant selection circuit 6 supplies the constants of K7 and K10 to the multiplier 8, the multiplier 8 multiplies the constants by the primary color components, and the complementary color component coefficients of K7 (GB) and K10 (GB). Perform the calculation of. Steps 115, S116, and S117 may be performed in parallel.

Step S118: The data selection addition circuit 11 selects the inputs from the multipliers 7 and 16 and adds L1 (RB) K1 (RG) to R (adds to the register for R, and after step S121, The contents of the R register are supplied to the adder 12 and added to R).

Step S119: The data selection addition circuit 11 selects the inputs from the multipliers 7 and 16 and adds L4 (RB) K4 (RG) to B (adds to the register for B, and after step S121, The contents of the register for B are supplied to the adder 14 and added to B). The data selection addition circuit 11 selects the input from the complement 9 and the multiplier 16 and adds L4 (RB) {-K4 (RG)} to G (L4 (RB) K4 (R). -G) is subtracted from G) (adds to the G register, and after step S121, the contents of the G register are supplied to the adder 13 and added to G).

Step S120: The data selection addition circuit 11 selects the inputs from the multipliers 8 and 16 and adds L7 (RB) K7 (GB) to R (adds to the R register, and after step S121, The contents of the R register are supplied to the adder 12 and added to R). The data selection addition circuit 11 selects the inputs from the multipliers 8 and 16 and adds L7 (RB) K7 (GB) to G (adds to the register for G, and after step S121, the register for G). Is supplied to the adder 13 and added to G).

Step S121: The data selection addition circuit 11 selects the inputs from the multipliers 8 and 16 and adds L10 (RB) K10 (GB) to R (adds to the R register, and after step S121, The contents of the R register are supplied to the adder 12 and added to R). The data selection addition circuit 11 selects inputs from the complement 10 and the multiplier 16 and adds L10 (RB) {-K10 (GB)} to G (L10 (RB) K10 (G). -B) is subtracted from G) (adds to the G register, and after step S121, the contents of the G register are supplied to the adder 13 and added to G).

Therefore, when correcting the color tone of the video signal (R), for example, in the case of correction in the saturation direction, the video signal (R) is obtained after multiplying the primary color component amount (RG) by a specific constant (Kr). It will be added. At this time, if the ratio by the constant (Kr) is in the range of -1 to 1 times, the level difference between the intermediate level and the minimum level (complementary color component amount) and the minimum level amount (white component amount) are also obtained by this correction. ) Does not change.

Further, when correcting the saturation direction of the video signal (Ye), the complementary color component amount (GB) is multiplied by a specific constant (Ky) and then added to R and G, respectively. Even at this time, if the ratio by the constant (Ky) is in the range of -1 to 1 times, the level difference between the maximum level and the intermediate level (primary color component amount) and the minimum level amount (white component) are also obtained by this correction. Amount) does not change.

Therefore, in this case, by manipulating the constants Kr and Ky, it is possible to independently correct the saturation direction of the primary color R and the complementary color Ye while maintaining the white balance. In the above 6-color independent color tone correction method, the chromaticity direction can be corrected independently, and even when the input video signal is in a different hue, the independent correction can be performed in the same manner. Is omitted.

Further, in this embodiment, the saturation of the difference between the primary color (maximum value) and white (minimum value) after the determination of the maximum value and the minimum value is calculated, and the coefficient is calculated according to the saturation. When the coefficient is calculated according to the saturation as in FIG. 7A described above, 7B described later, and FIGS. 17A to 17D, a linear change occurs. When the coefficient is selected according to the saturation as shown in FIG. 7D described later, it is a step change.

<Modification 2-1>
The first modification (modification example 2-1) of the second embodiment will be described with reference to FIG. 7B.

FIG. 7B is an explanatory diagram of a color tone correction process by a six-color internal / external independent color tone correction method, and is a case where the amount and direction of the color correction are linearly correlated with the saturation in FIG. 17A.

In the flowchart of FIG. 7B, after determining the maximum value and the minimum value, the saturation of the difference between the primary color (maximum value) and white (minimum value) is calculated, the coefficient is calculated according to the saturation, and the linear change (color) is performed. It is a correction in which the amount and direction of color correction are linearly correlated) or the curve changes or the step changes. An example shown in FIG. 6 (when the maximum level is R, the intermediate level is G, and the minimum level is B) will be described with reference to FIG. 7B.

Step S101B: Find the maximum of R, G, B. When R is maximum, the process proceeds to step S102B, when G is maximum, the process proceeds to step S103B, and when B is maximum, the process proceeds to step S104B.

Step S102B: Find the smaller of G and B. When G is small, G is the minimum level and the process proceeds to step S151B, and when B is small, B is the minimum level and the process proceeds to step S152B.

Step S103B: Find the smaller of R and B. When R is small, R is the minimum level and the process proceeds to step S153B, and when B is small, B is the minimum level and the process proceeds to step S154B.

Step S104B: Find the smaller of R and G. When R is small, R is the minimum level and the process proceeds to step S155B, and when G is small, G is the minimum level and the process proceeds to step S156B.

Steps S151B, S108, S109B, S110B, S111 to S114 process the first region (region 1), steps S152B, S115, S109B, S116B, S117B, S118 to S121 process the sixth region (region 6), step S153B. , S122, S123B, S124B, S125 to S128 correspond to the processing of the fourth region (region 4). Further, steps S129 to S135 are processing of the fifth region (region 5), steps S154B, S129, S130B, S131B and S132 to S142 are processing of the third region (region 3), steps S156B, S143, S144B, S145B, S146. ~ S149 corresponds to the processing of the second region (region 2).

In the case of the example of FIG. 6, the processes of steps S152B, S115, S109B, S116B, S117B, and S118 to S121 are performed.

Step S152B: The arithmetic unit 2 calculates the color difference signal (saturation) of RB.

Step S115: The constant selection circuit 6 supplies the constants of L1, L4, L7, and L10 to the multiplier 16, the multiplier 16 multiplies the constant by the saturation component, and L1 (RB), L4 (RB). ), L7 (RB), L10 (RB) Saturation component coefficients are calculated.

Step S116B: The arithmetic unit 1 calculates the RG color difference signal (primary color component), and the arithmetic unit 3 calculates the GB color difference signal (complementary color component).

Step S117: The constant selection circuit 6 supplies the constants of K1 and K4 to the multiplier 7, the multiplier 7 multiplies the constant by the primary color component, and the primary color component coefficients of K1 (RG) and K4 (RG). Perform the calculation of. The constant selection circuit 6 supplies the constants of K7 and K10 to the multiplier 8, and the multiplier 8 multiplies the constants by the primary color components to calculate the complementary color component coefficients of K7 (GB) and K10 (GB). conduct.

The process after step S118 is the same as that of the second embodiment.

As shown in FIGS. 7A described above and 7C to 7E described later, the color difference may be calculated after all the color differences are calculated, and the coefficient may be calculated. It may be calculated, and the order does not matter.

FIG. 14 is a schematic diagram showing the operation of the conventional 12-color independent color tone correction. 16A is a schematic diagram showing the operation on the color vector waveform of 6-color internal / external independent color tone correction, (a) corresponds to FUJIFILM ND1.0 (10%), and (b) is HOYA ND1.0 (1%). ) Is supported. Is.

In the operation on the color vector waveform of the 6-color internal / external independent color tone correction as shown in FIG. 16A and the operation on the color vector waveform of the 6-color independent saturation continuously variable color tone correction as shown in FIG. 17A, the pure color around the color range is used. Since the conversion of the peripheral hue saturation and the retention of the hue saturation around the white in the center of the color range can be adjusted independently, the degree of freedom of color correction at the time of color range conversion is increased.

In the operation on the color vector waveform of the 6-color internal / external independent color tone correction as shown in FIG. 16A, it is better to change the correlation with the internal / external saturation according to the saturation of the 6 color points of the vector chart, which is a wider color gamut. In color correction during conversion of a narrow color gamut, the degree of freedom can be adjusted more naturally by adjusting the conversion of the hue saturation around the pure color around the color gamut and the retention of the hue saturation around the white in the center of the color gamut. Will be higher.

<Modification 2-2>
The second modification (modification 2-2) of the second embodiment will be described with reference to FIGS. 3B, 7D, 16A, and 17A.

FIG. 3B is a block diagram showing the configuration of the hue saturation detection correction unit according to the modification 2-2. FIG. 7D is an explanatory diagram of color tone correction processing by the six-color internal / external independent color tone correction method, and is a case where the amount and direction of color correction change stepwise depending on the saturation of FIG. 15D.

In the color tone correction unit 382B according to the modification 2-2, the color correction in different directions inside and outside of FIG. 16A and the continuous variation of the correlation with the saturation inside and outside of FIG. 17A are the saturation of the six color points of the vector chart. It realizes a correspondingly variable color correction.

In the color tone correction unit 382B, the constant selection circuit 24 is changed from the color tone correction unit 382A. The constant selection circuit 24B realizes a color correction in which the correlation with the saturation inside and outside of FIG. 16A is variable according to the saturation of the six color points of the vector chart.

In the flowchart of FIG. 7D, the color difference is calculated, the maximum value and the minimum value are determined, and then the coefficients are selected according to the maximum value, the minimum value, and the saturation, and the correction is performed to change the steps. An example shown in FIG. 6 (when the maximum level is R, the intermediate level is G, and the minimum level is B) will be described with reference to FIG. 7D.

Steps S101 to S107 are the same as in the second embodiment of FIG. 7A. Further, steps S108D to S114 are processing of the first region (region 1), steps S115D to S121 are processing of the sixth region (region 6), steps S122D to S128 are processing of the fourth region (region 4), and steps S129D to S135 corresponds to the processing of the fifth region (region 5), steps S136D to S142 correspond to the processing of the third region (region 3), and steps S143D to S149 correspond to the processing of the second region (region 2).

In the case of the example of FIG. 6, the component amount determination circuit 5 determines that RB is the saturation component amount, RG is the primary color component amount, and GB is the complementary color component amount, and performs the processing after step S115D.

Step S115D: The constant selection circuit 24 calculates the saturation component coefficients of L1 (RB), L4 (RB), L7 (RB), and L10 (RB), and the multipliers 7 and 8 are calculated. Supply to.

Step S116D: The constant selection circuit 24 supplies the constants of K1 and K4 to the multiplier 7, the multiplier 7 multiplies the constant by the primary color component, and the primary color component coefficients of K1 (RG) and K4 (RG). Perform the calculation of. The multiplier 7 multiplies L1 (RB) and L4 (RB) supplied from the constant selection circuit 24 with K1 (RG) and K4 (RG), and L1 (RB). K1 (RG), L4 (RB) K4 (RG) are obtained.

Step S117D: The constant selection circuit 24 supplies the constants of K7 and K10 to the multiplier 8, the multiplier 8 multiplies the constants by the primary color components, and the complementary color component coefficients of K7 (GB) and K10 (GB). Perform the calculation of. The multiplier 8 multiplies L7 (RB) and L10 (RB) supplied from the constant selection circuit 24 with K7 (GB) and K10 (GB), and L7 (RB). K7 (GB), L10 (RB) K10 (GB) are obtained.

Step S118D: The data selection addition circuit 11 selects the input from the multiplier 7 and adds L1 (RB) K1 (RG) to R (adds to the register for R, and after step S121D, for R). The contents of the register are supplied to the adder 12 and added to R).

Step S119D: The data selection addition circuit 11 selects the input from the multiplier 7 and adds L4 (RB) K4 (RG) to B (adds to the register for B, and after step S121D, for B). The contents of the register are supplied to the adder 14 and added to B). The data selection addition circuit 11 selects the input from the complement 9 and adds L4 (RB) {-K4 (RG)} to G (L4 (RB) K4 (RG). Subtract from G) (add to the G register, and after step S121D, supply the contents of the G register to the adder 13 and add to G).

Step S120D: The data selection addition circuit 11 selects the input from the multiplier 8 and adds L7 (RB) K7 (GB) to R (adds to the register for R, and after step S121D, for R). The contents of the register are supplied to the adder 12 and added to R). The data selection addition circuit 11 selects the input from the multiplier 8 and adds L7 (RB) K7 (GB) to G (adds to the register for G, and after step S121D, the contents of the register for G). Is supplied to the adder 13 and added to G).

Step S121D: The data selection addition circuit 11 selects the input from the multiplier 8 and adds L10 (RB) K10 (GB) to R (adds to the register for R, and after step S121D, for R). The contents of the register are supplied to the adder 12 and added to R). The data selection addition circuit 11 selects the input from the complement 10 and adds L10 (RB) {-K10 (GB)} to G (L10 (RB) K10 (GB). Subtract from G) (add to the G register, and after step S121, supply the contents of the G register to the adder 13 and add to G).

In the stepwise variable operation depending on the saturation, a constant selection circuit 24B may be installed as in the hue saturation detection correction unit 38B of FIG. 3B, which performs color correction in different directions stepwise on the inside and outside.

In the operation that is variable stepwise depending on the saturation, the color difference is calculated and the maximum value and the minimum value are determined, as in the case of the saturation coefficient processing in FIG. 7D, which is the operation of the constant selection circuit 24B in FIG. 3B, and then the maximum value. The coefficient may be selected according to the minimum value and the saturation, and the saturation coefficient may be changed stepwise according to the saturation.

The image pickup apparatus according to the second embodiment and the modified examples 2-1 and 2-2 (the present embodiment) includes a color difference calculation unit, a hue region determination unit, a saturation component amount determination unit, and a primary color component amount determination unit. It has a complementary color component amount determination unit, a constant selection unit, a multiplication unit, a primary color component correction signal inversion unit, a complementary color component correction signal inversion unit, a data selection addition unit, and a correction signal addition unit.

As shown in FIGS. 7A, 7B, and 7D, the color difference calculation unit calculates the color difference signal, and the hue region determination unit determines the positive / negative of the color difference signal and determines the maximum value and the minimum value of the RGB imaging signal to determine the primary color region. The hue region of the and the complementary color region is determined, the saturation component amount determination unit determines the saturation component, the primary color component amount determination unit determines the primary color component, and the complementary color component amount determination unit determines the complementary color component.

As shown in FIGS. 7A, 7B, and 7D, the constant selection unit selects a constant by determining the hue region, and the multiplication unit multiplies the selected constant by the primary color component, the complementary color component, and the saturation component, and inverts the primary color component correction signal. The part inverts the primary color component correction signal, the complementary color component correction signal inversion part inverts the complementary color component correction signal, the data selection addition part selects the data to be added by the hue region determination, and the correction signal addition part goes to the RGB image pickup signal. Add the correction signal. As a result, it is specified in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter for each pixel, such as changing the correlation to the saturation inside and outside for each pixel or changing the threshold inside and outside the saturation. It has an independent color correction function of 6 or more colors that changes the effect method of saturation that changes the amount and direction of color correction of hue and saturation.

That is, the image pickup device according to this embodiment is not for BT.2020, which has a low cross point of blue and green spectral transmission, but is an image pickup device with an on-chip color filter, which has a high cross point of spectral transmission of blue and green. Or, instead of the color-resolving optics for BT.2020, which has a low cross-point of blue and green spectral transmission, but the color-resolving optics for BT.709, etc., which have a high cross-point of blue and green spectral transmission. It corresponds to BT.2020 with a wide color range from the red, green, and blue primary color video signals of each pixel signal generated from more than one image pickup element. In this case, the aperture value of the lens with a variable ND filter or the ND of the variable ND filter can be changed for each pixel, such as changing the amount and direction of color correction for each pixel or changing the amount and direction of color correction independently inside and outside the saturation. The amount and direction of color correction are changed in conjunction with the value and further in conjunction with the saturation to make corrections. At that time, the amount and direction of color correction are changed by saturation in pixel units such as changing the amount and direction of color correction by correlating with the saturation for each pixel or changing the amount and direction of color correction independently inside and outside the saturation. By changing the direction (having an independent color correction function of 6 or more colors), more faithful color gamut conversion can be performed with easy adjustment.

In particular, it is better to change the correlation to the saturation inside and outside according to the saturation of the 6 color points of the vector chart, in the color correction at the time of conversion between the wide color gamut and the narrow color gamut, in the vicinity of the pure color around the color gamut. The degree of freedom is increased because the adjustment between the conversion of hue saturation and the maintenance of hue saturation around white in the center of the color gamut can be adjusted more naturally.

(12 independent colors, 16 independent colors, 18 independent colors, 24 independent colors, etc.)
Example 3 which is another example of the hue saturation detection correction unit 38 of Example 1 will be described with reference to FIGS. 9A, 10 to 12, 16B to 16D, and 18.

FIG. 9A is a block diagram showing a configuration of a hue saturation detection correction unit according to the third embodiment. 10 and 11 are explanatory views of a hue region in the color tone correction of the third embodiment. FIG. 12 is an explanatory diagram of the calculation principle of the primary color component and the complementary color component according to the third embodiment, and is a diagram showing the color correction characteristic in the saturation direction. FIG. 16B is a schematic diagram showing the operation on the color vector waveform of the 12-color internal / external independent color tone correction. FIG. 16C is a schematic diagram showing the operation on the color vector waveform of the 16-color internal / external independent color tone correction. FIG. 16D is a schematic diagram showing the operation on the color vector waveform of the 24-color internal / external independent color tone correction. In FIGS. 16B to 16D, (a) corresponds to FUJIFILM ND1.0 (10%), and (b) corresponds to HOYA ND1.0 (1%).

The hue saturation detection correction unit 38C according to the third embodiment has a linear correlation between the amount and direction of color correction for the saturation such as 12 independent colors in FIG. 16B, 16 independent colors in FIG. 16C, and 24 independent colors in FIG. 16D. handle.

As shown in FIG. 9A, the hue saturation detection correction unit 38C includes a color tone correction unit 382C and adders 12, 13, and 14. The color tone correction unit 382C includes an intermediate hue setting circuit 15, an α / β, β / α calculation circuit 17, a constant selection circuit 18, a multiplier 19, 20, 26, and data selection in the color tone correction unit 382A of the second embodiment. The addition / subtraction circuit 21 and the primary color / complementary color region determination circuit 22 are additionally configured.

The neutral hue setting circuit 15 functions to enable the setting of a new neutral color to be set as a reference color. For example, the skin color (hue (F)), which is an intermediate color between R and Ye, is preset. be. The primary color / complementary color region determination circuit 22 in the hue region determination circuit 4C determines the hue of the input video signal (R, G, B) based on the hue (F) given by the intermediate hue setting circuit 15. It functions to generate a predetermined control signal (S).

The α / β, β / α calculation circuit 17 functions to calculate a predetermined constant (α / β, β / α) from the data given from the intermediate hue setting circuit 15. These constants (α / β, β / α) will be described later. The constant selection circuit 18 functions to select and output one of the constants (α / β, β / α) according to the control signal (S).

The multipliers 19, 20, and 26 are one of the constants (α / β, β / α) selected by the constant selection circuit 18 for the saturation component, the primary color component, and the complementary color component output from the component amount determination circuit 5. Works to multiply. The data selection addition / subtraction circuit 21 selects data according to the determination result by the hue region determination circuit 4C and the control signal (S), and performs predetermined addition / subtraction. The details of the operation of the data selection addition / subtraction circuit 21 will be described later.

Next, the operation of this embodiment will be described with reference to FIGS. 10 and 11. FIGS. 10 and 11 are diagrams showing saturation (color saturation) and chromaticity (hue) in order to explain the operation principle of the third embodiment, and in these figures, the distance from O of the origin (center point) is obtained. The direction represents the saturation, and the direction perpendicular to the saturation (the direction in which the circle is drawn) represents the saturation.

Here, the present invention is used for any intermediate color correction such as an internal / external independent color correction function having more than 6 colors such as skin color independent, 12 color independent, 16 color independent, 18 color independent, or 24 color independent in addition to 6 color independent. Is also applicable, but it is particularly preferable to apply it for skin color correction. Therefore, in this embodiment, the correction of skin color will be mainly described below as an example. Then, since the hue of this flesh color is located in the region between R and Ye, that is, the sixth region (region 6), in these FIGS. 10 and 11, the sixth region from R (red) to Ye (yellow). Only (region 6) is shown, where the hue of the flesh color is represented by the point F.

Therefore, this point is set as an auxiliary reference color (F) as shown in the figure, and the data is set in the intermediate hue setting circuit 15 as described above.

As a result, the sixth region (region 6) becomes two auxiliary regions, that is, the region (6-1) by the axis passing from the center point (O) to the auxiliary reference color (F) point, that is, the auxiliary reference line. It will be divided into the area (6-2). Next, the hue of the input video signal is measured by the primary color / complementary color region determination circuit 22 in a region (6-1) between R and F and a region (6-1) between F and Ye, as shown in FIG. Judgment is made by dividing into 2). Then, first, at this time, since the operation is performed when the determination result of the hue region determination circuit 4C is in the sixth region (region 6), the primary color component amount output from the component amount determination circuit 5 And the amount of complementary color components are as follows.

Primary color component amount = RG = Rc
Complementary color component amount = GB = Yc
Next, whether the hue of the input video signal is in these regions (6-1) or regions (6-2) is identified by the determination of the primary color / complementary color region determination circuit 22, and each is as shown below. In addition, the correction is made separately.

<Correction processing in area (6-1)>
At this time, the output from each circuit is as follows. First, the constant β / α is selected in the constant selection circuit 18, and the constant β / α is output to the multipliers 19 and 20. Next, the data selection addition / subtraction circuit 21 outputs a signal [Rc—Yc × (β / α)], a signal (−Yc), and a signal [Yc × (β / α)]. Further, the constant selection circuit 6 selects constants (Kr, Kf), and these constants (Kr, Kf) are output to the multipliers 7 and 8.

As a result of these, first, the signal [Rc—Yc × (β / α)] × Kr + Kf × [Yc × (β / α)] is output from the data selection addition circuit 11 to the adder 12 and the signal ( It is added to R), and then the signal [(−Yc) × Kf] is output to the adder 14 and added to the signal (B).

Therefore, in FIG. 10, when the point A is used as the coordinates of the input video signal and is represented by the vector A, this vector A is represented by the composition of the R component vector R1 and the skin color component vector F1.

A = R1 + F1
Next, assuming that the gain constant dedicated to the saturation direction adjustment of R is Kr and the gain constant dedicated to the saturation direction adjustment of the skin color is Kf, when performing color correction in the saturation direction of R, | R1 | × Kr may be added in the saturation direction of R, that is, added to R, and | F1 | × Kf may be added in the skin color saturation direction when performing color correction in the skin color saturation direction.

Therefore, the calculation method of these quantities | R1 |, | F1 | and the addition method in the skin color saturation direction will be described. For this purpose, all the corrections should be expressed as corrections to the R, G, and B components. It's fine. Therefore, first, let R be the R component basic vector, F be the skin color component basic vector, Y be the Ye component basic vector, and B be the B component basic vector.
F = α × Y + β × R = α × (-B) + β × R
And.

Next, the coordinate vector A of the input video signal is represented by the composition of the R component and the Ye component. Here, A = Y × Yc + R × Rc and Rc and Yc can be easily obtained as described in the color tone correction method of the second embodiment. In this case, R> G> B, and therefore, as is clear from FIG. 6, Rc = RG and Yc = GB.

Then
A = Y × Yc + R × Rc
= (1 / α) x (F-β x R) x Yc + R x Rc
= F × Yc / α + R × (Rc-β × Yc / α)
So,
｜ R1 ｜ ＝ Rc－β × Yc / α
｜ F1 ｜ ＝ Yc / α
Will be.

Therefore, if F × Yc / α is expressed by vector R and vector B,
F × Yc / α ＝ (α × (−B) ＋ β × R) × Yc / α
= B × (−Yc) + R × (β × Yc / α)
Will be.

Therefore, the above results can be summarized as follows. That is, first, in order to perform color correction in the saturation direction of R, | R1 | Kr = (Rc−β × Yc / α) × Kr may be added to R. Next, in order to perform color correction in the skin color saturation direction, | F1 | × Kf may be added in the skin color saturation direction, which means that −Yc × Kf is added to B and (β. It is equivalent to adding × Yc / α) × Kf to R.

Here, letting θ be the angle between the R vector and the skin color vector.
α × sin (60 ° −θ) = β × sin (θ)
Because it is
β / α = sin (60 ° -θ) / sin (θ)
Will be.

Therefore, when θ = 20 °, β / α = 1.8794, but if this is ≈2.0, the correction at this time is (Rc-) for the color correction in the saturation direction of R. 2 × Yc) × Kr may be added to R, and for color correction in the saturation direction of the skin color, −Yc × Kf may be added to B and 2 × Yc × Kf may be added to R. Then, by changing β / α, the reference axis of the skin color can be adjusted.

The above is a description of the correction in the saturation direction, but since the same concept can be applied to the correction in the chromaticity direction, the description will be omitted.

<Correction processing in area (6-2)>
At this time, the output from each circuit is as follows. First, the constant α / β is selected in the constant selection circuit 18, and the constants α / β are output to the multipliers 19 and 20. Next, the data selection addition / subtraction circuit 21 outputs a signal [Yc—Rc × (α / β)], a signal (Rc), and a signal [−Rc × (α / β)]. Further, the constant selection circuit 6 selects constants (Ky, Kf), and these constants (Ky, Kf) are output to the multipliers 7 and 8.

Then, as a result of these, the signal [Rc × Kf] is first output from the data selection addition circuit 11 to the adder 12 and added to the signal (R), and then the signal {-[Yc-Rc × ( α / β)] × Ky−Kf × [Rc × (α / β)]} is output to the adder 14 and added to the signal (B).

Therefore, in FIG. 11, when point C is used as the coordinates of the input video signal and is represented by the vector C, this vector C is represented by the composition of the Ye component vector Y1 and the skin color component vector F2.

C = Y1 + F2
Next, assuming that the gain constant dedicated to adjusting the saturation direction of Ye is Ky and the gain constant dedicated to adjusting the saturation direction of the skin color is Kf, | Y1 | × Ky is B for color correction in the saturation direction of Ye. For color correction in the skin color saturation direction, | F2 | × Kf may be added in the skin color saturation direction.

Next, the calculation method of | Y1 | and | F2 | and the addition method in the skin color saturation direction are the same as those in the above-mentioned region (6-1), and therefore, are as follows.

C = Y × Yc + R × Rc
= Y x Yc + (1 / β) x (F-α x Y) x Rc
= F × Rc / β + Y × (Yc-α × Rc / β)
Therefore, | Y1 | = Yc-α × Rc / β
| F2 | = Rc / β
Will be.

Here, if F × Rc / β is expressed by a vector R and a vector B,
F × Rc / β = (α × (−B) + β × R) × Rc / β
= -B x (α x Rc / β) + R x Rc
Will be.

Therefore, the above results can be summarized as follows. That is, first, when performing color correction in the saturation direction of Ye, | Y1 | × Ky = (Yc−α × Rc / β) × Ky may be subtracted from B. Next, when performing color correction in the skin color saturation direction, | F2 | × Kf is added in the skin color saturation direction, which means that (-α × Rc / β) × Kf is converted to B. It is equivalent to adding and adding Rc × Kf to R.

Therefore, if the angle θ between the R vector and the skin color vector is 20 °, which is the same as in the above-mentioned region (6-1), α / β = 0.5321, and this is set to ≈0.5. At this time, when performing color correction in the saturation direction of Ye, (Yc-0.5Rc) × Ky may be subtracted from B, and when performing color correction in the saturation direction of skin color, −0. 5 × Rc × Kf may be added to B, and Rc × Kf may be added to R.

The above is a description of the correction in the chromaticity direction, but since the same concept can be applied to the correction in the chromaticity direction, the description will be omitted.

FIG. 12 shows the characteristics obtained by the corrections described in the respective sections of the region (6-1) and the region (6-2). The characteristics of FIG. 12 show the gain characteristics of the color correction in the saturation direction of R, the color correction in the saturation direction of Ye, and the color correction in the saturation direction of the skin color in an superimposed manner, as shown in the figure. In addition, if the saturation direction gain constant (Kf) of the skin color is controlled, the saturation direction of the skin color is irrespective of the saturation direction gain constant (Kr) of R and the gain constant (Ky) of the saturation direction of Ye. It turns out that color correction can be performed.

Therefore, according to this embodiment, it is possible to minimize the influence on R and Ye, perform effective color tone correction for the skin color, and surely eliminate the discomfort when the television camera is switched. be able to.

<Modification 3-1>
Next, a modified example of the third embodiment (modified example 3-1) will be described with reference to FIG. FIG. 13 shows the saturation direction color correction characteristic according to the modification 3-1. In the modification 3-1 a correction function having a gain characteristic centered on the skin color axis F is generated and extracted. This is added to the function of the second embodiment in the modified example 3-1 of FIG. 13, and according to this method, the region that cannot be corrected by the second embodiment can be corrected. ..

Further, in this embodiment, the saturation of the difference between the primary color (maximum value) and white (minimum value) after the determination of the maximum value and the minimum value is calculated, and the coefficient is calculated according to the saturation. When the coefficient is calculated according to the saturation as shown in FIGS. 7A, 7B and 7E, it is a linear change. When the coefficient is selected according to the saturation as shown in FIG. 7D, it is a step change.

The image pickup apparatus according to this embodiment has a color difference calculation unit between RG, RB, and GB, a hue region determination unit between a primary color region and a complementary color region, an intermediate hue setting unit, and a saturation component amount. It has a determination unit, a primary color component amount determination unit, a complementary color component amount determination unit, a first multiplication unit, a second multiplication unit, a data selection addition unit, a data selection addition / subtraction unit, and an addition unit.
The chromaticity calculation unit calculates the chromaticity signal, and the hue region determination unit determines the positive / negative of the chromaticity signal and determines the maximum and minimum values of the RGB imaging signal to determine the hue region between the primary color region and the complementary color region. The saturation component amount determination unit determines the saturation component, the primary color component amount determination unit determines the primary color component, and the complementary color component amount determination unit determines the complementary color component.
The first multiplication unit selects and multiplies the constants before data selection addition and subtraction by the hue area judgment and the intermediate hue judgment, and the data selection addition and subtraction part selects the ones that are not multiplied by the constants and performs addition and subtraction, and the first multiplication unit. Selects and multiplies the constants after data selection addition and subtraction by determining the hue area. The data selection addition unit selectively adds the inversion / non-inversion of the primary color component correction signal and the inversion / non-inversion of the complementary color component correction signal, which are the results of the second multiplication unit, and the addition unit adds the correction signal to the RGB image pickup signal. I do. As a result, it is specified in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter for each pixel, such as changing the correlation to the saturation inside and outside for each pixel or changing the threshold inside and outside the saturation. It has more independent color correction functions than 6 colors that change the effect method of saturation that changes the amount and direction of color correction of hue and saturation.

That is, the image pickup apparatus according to this embodiment is linked with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter in pixel units, and further linked with the saturation, and the hue is centered or both ends as shown below. It has an independent color correction function of 12 or more colors that can be changed in the direction toward the lens. In the imaging method according to this embodiment, the hue is moved to the center or both ends as shown below in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter and further with the saturation in pixel units. Performs independent color correction of 12 or more colors that can be changed in the direction.
(1) As shown in FIG. 18A, FIG. 16B (1) corresponds to the change in hue due to the lower transmittance of the green long wavelength side of the film-based ND filter than the green short wavelength side transmittance. As shown in a) and 18 (a), green changes the hue in the direction toward the short wavelength side.
(2) As shown in FIG. 18 (b), corresponding to the change in color tone due to the lower transmittance on the long wavelength side and the transmittance on the short wavelength side than the green central long wavelength side transmittance of the glass-based ND filter. , As shown in FIGS. 16B (b) and 18 (b), green changes the hue in a direction away from the center of green.

Further, in the image pickup apparatus according to the present embodiment, at least one or more as described below is linked with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter and further linked with the saturation in pixel units. It has 16 or more independent color correction functions that change the long wavelength side of a color in the direction toward the short wavelength direction, or change the long wavelength end of at least one color in the direction toward the long wavelength direction. The imaging method according to the present embodiment is linked to the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter for each pixel, and further linked to the saturation, and has at least one or more colors as described below. 16 or more independent color corrections are performed so that the long wavelength side is variable in the direction toward the short wavelength direction, or the long wavelength end of at least one color is variable in the direction toward the long wavelength direction.
(1) For example, as shown in FIG. 18 (a), for example, in response to a change in hue due to a lower central wavelength side transmittance than the green long wavelength side transmittance of a film-based ND filter, for example, FIG. 16C (a). ) And 18 (a), the long wavelength side of green changes the hue in the direction toward the short wavelength side.
(2) As shown in FIG. 18 (b), the transmittance at the green long wavelength end is lower than the transmittance at the center side of the green long wavelength side of the glass-based ND filter (transmittance from 580 nm to 600 nm is high). Corresponding to the change in color tone due to the concave), as shown in FIGS. 16C (b) and 18 (b), the long wavelength end of green changes the hue in a direction away from the center of green.

Further, the image pickup apparatus according to the present embodiment interlocks with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter in pixel units, and further interlocks with the saturation to obtain the hue at the wavelength end as shown below. It has an independent color correction function of 24 or more colors that can be changed in the short wavelength direction. In the imaging method according to this embodiment, the hue at the wavelength end is set to a short wavelength as shown below in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter and further with the saturation in pixel units. Performs independent color correction of 24 or more colors that are variable in the direction.
(1) As shown in FIG. 18A, the film-based ND filter has a spectral transmittance equivalent to the average spectral transmittance of green at the center of blue and the long wavelength side, and only the blue end compared to the average spectral transmittance of green. In response to the change in color tone due to the low transmittance (the spectral transmittance at 420 nm is concave), as shown in FIGS. 16D (a) and 18 (a), the hue is shifted toward the short wavelength direction only at the blue end. Make it variable. Or,
(2) As shown in FIG. 18 (a), the film-based ND filter has a spectral transmittance higher than the average spectral transmittance of green at the center of red and the short wavelength side, and the transmittance is remarkably high only at the long wavelength end of red. Corresponding to the change in color tone due to (the spectral transmittance at 700 nm is convex), as shown in FIGS. 16D (a) and 18 (a), the hue is changed in the direction toward the short wavelength direction only at the long wavelength end of red.
(3) For example, as shown in FIG. 18 (b), it corresponds to the change in hue due to the lower transmittance on the short wavelength side (inclination from 530 nm to 540 nm) than the transmittance on the green center wavelength side of the green of the glass-based ND filter. Therefore, for example, as shown in FIGS. 16D (b) and 18 (b), the hue on the center wavelength side of green is varied in the direction toward the short wavelength direction.
(4) For example, as shown in FIG. 18 (b), the change in hue due to the lower transmittance at the long wavelength end side (inclination from 460 nm to 480 nm) compared to the long wavelength central side transmittance of blue of the glass-based ND filter. Correspondingly, as shown in FIGS. 16D (b) and 18 (b), for example, the hue on the center side of the long wavelength of blue is varied in the direction toward the long wavelength direction.

In summary, in addition to 6 colors, skin color independent, 12 color independent, 16 color independent, 18 color independent, 24 color independent, etc., more than 6 independent color corrections, aperture value of lens with variable ND filter or variable ND It is better to change the direction and amount of color correction in conjunction with the ND value of the filter and further in conjunction with the saturation. In the correction, the degree of freedom is increased because the adjustment between the conversion of the hue saturation around the pure color around the color range and the maintenance of the hue saturation around the white in the center of the color range can be adjusted more naturally.

(Correlated to the power of the saturation component)
Example 4 which is another example of the hue saturation detection correction unit 38 of the first embodiment will be described with reference to FIGS. 9B and 7C.

FIG. 9B is a block diagram showing the configuration of the hue saturation detection correction unit according to the fourth embodiment. FIG. 7C is an explanatory diagram of the color tone correction process by the six-color internal / external independent color tone correction method, and the amount and direction of the color correction correlate with the power of saturation.

In the hue saturation detection correction unit 38D according to the fourth embodiment, multipliers 27, 22, 23 are added to the hue saturation detection correction unit 38C of the third embodiment. The hue saturation detection / correction unit 38D powers the saturation component and supplies it to the data selection addition / subtraction circuit 21 in order to correspond to a curve-like variable operation depending on the saturation. In the color tone correction unit 382D of the hue saturation detection correction unit 38D, the fourth power of the saturation component is calculated by the three multipliers 27, 22, and 23, but even if the multiplier is two cubes, the multiplier is used. May be 5 to the 6th power.

The difference between FIGS. 7C and 7A is the addition of a process in which the selected saturation component is raised to the nth power. Here, n is a natural number. In FIG. 9B, (n-1) multipliers in FIG. 7C will be added.

In the flowchart of FIG. 7C, the color difference is calculated, the maximum value and the minimum value are determined, the coefficient is calculated according to the maximum value, the minimum value, and the saturation, and the linear change (the amount and direction of the color correction are linear with the saturation). Correlation) correction. An example shown in FIG. 6 (when the maximum level is R, the intermediate level is G, and the minimum level is B) will be described with reference to FIG. 7C.

Steps S101 to S107 are the same as in FIG. 7A. Further, steps S151 and S108C to S114C are processing of the first region (region 1), steps S152 and S115C to S121C are processing of the sixth region (region 6), and steps S153 and S122C to S128C are processing of the fourth region (region 4). Processing, steps S154 and S129C to S135C are processing of the fifth region (region 5), steps S155 and S136C to S142C are processing of the third region (region 3), and steps S156 and S143C to S149C are processing of the second region (region 2). ), Corresponds to.

In the case of the example of FIG. 6, the component amount determination circuit 5 determines that RB is the saturation component amount, RG is the primary color component amount, and GB is the complementary color component amount, and performs the processing after step S152.

Step S152: Multipliers 27, 22, 23, 26 calculate the nth root (5th root) of the saturation component (RB).

Step S115C: The constant selection circuit 6 supplies the constants of L1, L4, L7, and L10 to the multiplier 16, and the data selection addition / subtraction circuit 21 supplies the power of the saturation component (RB) ⁵ to the multiplier 16. The multiplier 16 multiplies the constant by (RB) ⁵ and colors L1 (RB) ⁵ , L4 (RB) ⁵ , L7 (RB) ⁵ , and L10 (RB) ⁵ . Calculate the power component coefficient.

Step S116C: The constant selection circuit 6 supplies the constants of K1 and K4 to the multiplier 7. The data selection addition / subtraction circuit 21 adds / subtracts the primary color component multiplied by the selected constant or not multiplied by the constant and the complementary color component multiplied by the selected constant or not multiplied by the constant (this is R-). Let G be.) Supply to the multiplier 7. The multiplier 7 multiplies the constant by the primary color components and calculates the primary color component coefficients of K1 (RG) and K4 (RG).

Step S117C: The constant selection circuit 6 supplies the constants of K7 and K10 to the multiplier 8. The data selection addition / subtraction circuit 21 adds / subtracts the primary color component multiplied by the selected constant or not multiplied by the constant and the complementary color component multiplied by the selected constant or not multiplied by the constant (this is G-). Let B be.) Supply to the multiplier 8. The multiplier 8 multiplies the constant and the complementary color component, and calculates the complementary color component coefficients of K7 (GB) and K10 (GB). Steps 115C, S116C and S117C may be performed in parallel.

Step S118C: The data selection addition circuit 11 selects the inputs from the multipliers 7 and 16 and adds L1 (RB) ⁵ K1 (RG) to R (adds to the register for R, and after step S121C. , The contents of the R register are supplied to the adder 12 and added to R).

Step S119C: The data selection addition circuit 11 selects the inputs from the multipliers 7 and 16 and adds L4 (RB) ⁵ K4 (RG) to B (adds to the register for B, after step S121C). , The contents of the register for B are supplied to the adder 14 and added to B). The data selection addition circuit 11 selects the inputs from the complement 9 and the multiplier 16 and adds L4 (RB) ⁵ {-K4 (RG)} to G (L4 (RB) ⁵ K4. (RG) is subtracted from G) (adds to the G register, and after step S121C, the contents of the G register are supplied to the adder 13 and added to G).

Step S120C: The data selection addition circuit 11 selects the inputs from the multipliers 8 and 16 and adds L7 (RB) ⁵ K7 (GB) to R (adds to the register for R, and after step S121C. , The contents of the R register are supplied to the adder 12 and added to R). The data selection addition circuit 11 selects the inputs from the multipliers 8 and 16 and adds L7 (RB) ⁵ K7 (GB) to G (adds to the register for G, and after step S121, for G). The contents of the register are supplied to the adder 13 and added to G).

Step S121C: The data selection addition circuit 11 selects the inputs from the multipliers 8 and 16 and adds L10 (RB) ⁵ K10 (GB) to R (adds to the register for R, and after step S121). , The contents of the R register are supplied to the adder 12 and added to R). The data selection addition circuit 11 selects inputs from the complement 10 and the multiplier 16 and adds L10 (RB) ⁵ {-K10 (GB)} to G (L10 (RB) ⁵ K10. (GB) is subtracted from G) (adds to the G register, and after step S121C, the contents of the G register are supplied to the adder 13 and added to G).

By color correction with variable direction by saturation, in color correction at the time of conversion between wide color gamut and narrow color gamut, conversion of hue saturation around pure color around color gamut and hue saturation around white in the center of color gamut The degree of freedom is increased because the adjustment with holding can be adjusted more naturally.

FIGS. 17A to 17D show a continuously variable operation depending on the saturation among the color corrections whose direction is variable by the saturation, and FIGS. 16A to 16D are operations having a threshold value in the saturation. The operation is not limited to the color correction operation that linearly correlates with the saturation and is variable in direction, and may be a stepwise variable operation depending on the saturation (not shown) or a curved curve operation depending on the saturation.

(Corresponds to the saturation of 6 color points)
Example 5, which is another example of the hue saturation detection correction unit 38 of Example 1, will be described with reference to FIGS. 9C, 7E, 16A, and 16B.

FIG. 9C is a block diagram showing the configuration of the hue saturation detection correction unit according to the fifth embodiment. FIG. 7E is an explanatory diagram of a color tone correction process by a six-color internal / external independent color tone correction method, and is a diagram in which the coefficient of linear correlation between the amount and direction of color correction for saturation changes depending on the hue.

As shown in FIG. 9C, in the hue correction unit 382E of the hue saturation detection correction unit 38E according to the fifth embodiment, the constant selection circuit 25 is changed from the color tone correction unit 382C. The constant selection circuit 25 realizes a color correction in which the correlation with the saturation inside and outside of FIGS. 16A and 16B is variable according to the saturation of the six color points of the vector chart.

In the flowchart of FIG. 7E, the coefficient of linear correlation between the amount and direction of color correction for saturation corresponds to the saturation of the six color points of R, G, B, Cy, Ye, and Mg in the vector chart, depending on the hue. Change. This is done by changing the coefficient of linear correlation between the amount and direction of color correction in saturation in the constant selection circuit 6 or the constant selection circuit 18 due to the hue of the hue region determination circuit 4 of the color tone correction unit 382E. It is a correction that calculates the color difference, calculates the coefficient according to the maximum value, the minimum value, and the saturation after determining the maximum value and the minimum value, and linearly changes (the amount and direction of the color correction are linearly correlated with the saturation). .. An example shown in FIG. 6 (when the maximum level is R, the intermediate level is G, and the minimum level is B) will be described with reference to FIG. 7E.

Steps S101 to S107 are the same as in FIG. 7A. Steps S108E to S114 are processing of the first region (region 1), steps S115E to S121 are processing of the sixth region (region 6), steps S122E to S128 are processing of the fourth region (region 4), and steps S129E to S135 are processing. The processing of the fifth region (region 5), the steps S136E to S142 correspond to the processing of the third region (region 3), and the steps S143E to S149 correspond to the processing of the second region (region 2).

In the case of the example of FIG. 6, the component amount determination circuit 5 determines that RB is the saturation component amount, RG is the primary color component amount, and GB is the complementary color component amount, and performs the processing after step S115E.

Step S115E: The constant selection circuit 6 supplies the constants of L1, L4, L7, and L10 to the multiplier 16, and the data selection addition / subtraction circuit 21 corresponds to the primary color component (RG) and the complementary color component (GB). The saturation component (RB) is supplied to the multiplier 16, which multiplies the constant by (RB), L1 (RB), L4 (RB), L7 (R). -B), L10 (RB) saturation component coefficients are calculated.

Step S116C: The constant selection circuit 6 supplies the constants of K1 and K4 to the multiplier 7. The data selection addition / subtraction circuit 21 adds / subtracts the primary color component multiplied by the selected constant or not multiplied by the constant and the complementary color component multiplied by the selected constant or not multiplied by the constant (this is R-). Let G be.) Supply to the multiplier 7. The multiplier 7 multiplies the constant by the primary color components and calculates the primary color component coefficients of K1 (RG) and K4 (RG).

Step S117C: The constant selection circuit 6 supplies the constants of K7 and K10 to the multiplier 8. The data selection addition / subtraction circuit 21 adds / subtracts the primary color component multiplied by the selected constant or not multiplied by the constant and the complementary color component multiplied by the selected constant or not multiplied by the constant (this is G-). Let B be.) Supply to the multiplier 8. The multiplier 8 multiplies the constant and the complementary color component, and calculates the complementary color component coefficients of K7 (GB) and K10 (GB). Steps 115C, S116C and S117C may be performed in parallel.

The procedure after step S118 is the same as that of the second embodiment of FIG. 7A.

16A and 16B correspond to the saturation of the six color points of R, G, B, Cy, Ye, and Mg in the vector chart, and the correlation with the saturation inside and outside is changed for each pixel, or the threshold value inside and outside the saturation is changed. The operation on the color vector waveform of the independent color correction function of 6 or more colors is shown, in which the amount and direction of color correction are changed in pixel units such as variable, and the effect method of saturation is changed. By corresponding to the saturation of 6 color points of the vector chart, in the color correction at the time of conversion between wide color gamut and narrow color gamut, conversion of hue saturation around pure color around color gamut and around white around the center of color gamut. The degree of freedom is increased because the adjustment with the maintenance of the hue saturation of can be adjusted more naturally.

In FIGS. 16A and 16B, the threshold values inside and outside the saturation are changed according to the saturation of the six color points of the vector chart, but the correlation with the saturation inside and outside may be changed for each pixel.

Further, in FIGS. 16A and 16B, the 6 color points of the vector chart are fixedly displayed, but the saturation inside and outside corresponds to the movement of the 6 color points of the vector chart at the time of conversion between the wide color gamut and the narrow color gamut. The threshold value of may be changed, or the correlation with the saturation inside and outside may be changed for each pixel.

By color correction with variable direction by saturation, in color correction at the time of conversion between wide color gamut and narrow color gamut, conversion of hue saturation around pure color around color gamut and hue saturation around white in the center of color gamut The degree of freedom is increased because the adjustment with holding can be adjusted more naturally.

In the above-described embodiment and modification, the range of hue has been limited to R and Ye, but it goes without saying that the present invention can be applied to any hue, and the type of reference color and the like. Needless to say, the number can be set arbitrarily.

Limiting the hue range to R and Ye is an operation of color tone correction that is independent of skin color in addition to 6 colors inside and outside, and is not limited to the operation of color tone correction that is independent of skin color in addition to 6 colors inside and outside, and 12 color saturation independent color tone correction. Alternatively, the operation of 16-color internal / external independent color tone correction, the operation of 18-color internal / external independent color tone correction, or the operation of 24-color internal / external independent color tone correction may be performed.

In other words, in addition to 6 colors independent, skin color independent, 12 color independent, 16 color independent, 18 color independent, 24 color independent, etc. can be used for more independent color correction than 6 colors, and in particular, the correlation with internal and external saturation is the vector chart. It is better to change the color according to the saturation of 6 color points in the color correction at the time of conversion between wide color range and narrow color range. The degree of freedom is increased because the adjustment with the retention of the hue and saturation around white can be adjusted more naturally.

In the image pickup apparatus according to the fifth embodiment, the hue region is determined by the hue region determination unit, and the constant selection unit interlocks with the aperture value of the lens with the variable ND filter or the ND value of the variable ND filter and further interlocks with the saturation. By changing the coefficient for calculating the constant, the correlation with the internal and external saturation is changed for each pixel according to the saturation of the 6 color points of R, G, B, Cy, Ye, and Mg in the vector chart. Or, a color that changes the amount and direction of color correction of a specific hue and saturation in conjunction with the aperture value of a lens with a variable ND filter or the ND value of a variable ND filter, such as changing the threshold inside and outside the saturation. It has an independent color correction function of 6 or more colors that changes the influence method of the degree.

According to the above-mentioned Examples and Modifications, among the color changes due to the variation in the spectral transmission rate of the spot ND filter, the color change that cannot be corrected by the red-blue gain adjustment of the auto white is the aperture value of the lens with the variable ND filter. Or, increase the degree of freedom in adjusting the color correction of the color signal in response to the color change due to the aperture value of the lens with the spot ND filter according to the ND value of the variable ND filter, and more faithfully independently in each hue range. Since faithful color correction can be achieved with easy adjustment, it is possible to achieve both wide illuminance range support for lenses with variable ND filters and faithful color reproduction, and both surveillance and broadcasting applications are widespread.

Furthermore, by adjusting the direction and amount of color correction independently in the range of each hue and saturation in conjunction with saturation, it is not for BT.2020, which has a low crosspoint of spectral transmission between blue and green, but in the past. Image element with on-chip color filter with high cross point of blue and green spectral transmission or for BT.709, etc. instead of color separation optical system for BT.2020 with low cross point of blue and green spectral transmission Increased the degree of freedom in adjusting the color correction of the red, green, and blue primary color video signals of each pixel signal generated from the color separation optical system with a high cross point of the spectral transmission of blue and green and three or more image pickup elements. It is possible to achieve more faithful color correction independently and more faithfully in each hue range with easy adjustment, and an image pickup device with an on-chip color filter or BT with a high crosspoint of low-cost blue and green spectral transmission. A color separation optical system with a high cross-point of spectral transmission between blue and green, such as for .709, and faithful color reproduction can be achieved at the same time, and both surveillance and broadcasting applications are widespread.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

1,2,3: subtractor, 4: hue area determination circuit, 5: component amount determination circuit, 6: constant selection circuit, 7,8: multiplier, 9,10: multiplier (-1 times multiplier), 11: Data selection adder circuit, 12, 13, 14: Adder, 15: Intermediate hue setting circuit, 17: α / β, α / β calculation circuit, 18, 24, 25: Constant selection circuit, 16, 19, 20 , 22, 23, 26, 27: Multiplier, 21: Data selection addition / subtraction circuit, 30A, 30B: Television camera body, 31: Lens with variable ND filter, 32: Prism, 33R, 33G, 33B: Image sensor, 34 : Image sensor, 35A, 35B ,: Video signal processing unit, 36: Matrix unit, 37: Parallel-serial conversion unit, 38A, 38B, 38C, 38D, 38E: Phosphorus saturation detection correction unit, 381: Gamma correction unit, 382A, 382B, 382C, 382D, 382E: color tone correction unit, 383: color tone separation unit, 39: CPU, 40: image display unit, 100A, 100B: television camera (image sensor).

Claims (2)

(A) Red, green, and blue of each pixel signal generated from a lens with a variable ND filter or a variable ND filter, an image sensor with an on-chip color filter or a color separation optical system, and three or more image sensors. A means having a first means for detecting a specific hue for each pixel of the primary color image signal of the above , and a second means for independently correcting the specific hue for each pixel.
(B) Auto-white balance is performed by adjusting the red-blue gain of the image sensor with an on-chip color filter or the three or more image sensors based on the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter. Real-time auto white means and
Have,
The first means include a hue calculation means for calculating hue and saturation based on the color difference of red, green, and blue video signals for each pixel.
Based on the aperture value of the lens with variable ND filter or the ND value of the variable ND filter, the amount and direction of color correction are calculated for each pixel in conjunction with the saturation, and the hue region of which color of the subject is. Has a means to detect if it is in
The second means interlocks with the aperture value of the lens with the variable ND filter or the ND value of the variable ND filter, and the amount of color correction of the specific hue and the saturation detected by the first means. It has a correction means that changes the direction,
The correction means performs independent color correction of 12 or more colors in which the hue of green is changed in a direction toward the center or both ends in conjunction with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter. Imaging device. (A) Red, green, and blue of each pixel signal generated from a lens with a variable ND filter or a variable ND filter, an image sensor with an on-chip color filter or a color separation optical system, and three or more image sensors. A means having a first means for detecting a specific hue for each pixel of the primary color image signal of the above , and a second means for independently correcting the specific hue for each pixel.
(B) Auto-white balance is performed by adjusting the red-blue gain of the image sensor with an on-chip color filter or the three or more image sensors based on the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter. Real-time auto white means and
Have,
The first means include a hue calculation means for calculating hue and saturation based on the color difference of red, green, and blue video signals for each pixel.
Based on the aperture value of the lens with variable ND filter or the ND value of the variable ND filter, the amount and direction of color correction are calculated for each pixel in conjunction with the saturation, and the hue region of which color of the subject is. Has a means to detect if it is in
The second means interlocks with the aperture value of the lens with a variable ND filter or the ND value of the variable ND filter to determine the specific hue detected by the first means and the amount and direction of the color correction. It has a variable correction means and has
The correction means interlocks with the aperture value of the lens with the variable ND filter or the ND value of the variable ND filter to change the hue only at the blue end in the direction toward the short wavelength direction, or the hue only at the long wavelength end of red. An image pickup device that performs independent color correction of 24 or more colors that changes the hue at the wavelength end in the short wavelength direction.

Images