JP5095294B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method provided with a white balance correction unit.

  Conventionally, in an imaging apparatus such as a digital camera, a method of setting the same white balance correction amount for the entire screen has been used for white balance correction on a captured image. However, when the above method is used, if the light source is uniform, the light source can be estimated and the same white balance correction can be performed on the entire screen.However, if there are irradiation areas of different types of light sources in the screen, the entire screen is displayed. It is difficult to perform appropriate white balance correction. In order to solve this problem, a method is widely used in which the entire screen is divided into small areas, and white balance correction is performed for each small area (area) so that no color shift occurs in the entire screen. (For example, Patent Document 1)

  However, in the method of performing white balance with different correction amounts for each area as in Patent Document 1, a difference in color tone (color shift) inevitably occurs at the boundary between adjacent areas, resulting in image quality. There was a problem of lowering.

  In order to solve the color shift at the area boundary, there is a method of suppressing the color shift at the area boundary by performing white balance correction for each pixel. (For example, Patent Document 2)

  However, in the above method, it is necessary to calculate the correction coefficient from the result of white balance evaluation in all areas, and there is a problem that the time required for photographing becomes long because the calculation amount increases. In addition, when calculating the correction coefficient in units of pixels, there is no description of a correction method in a case where the four surrounding pixels of interest are not satisfied, and there is a problem that it is necessary to study in mounting.

JP 2002-271638 A JP 2005-347811 A

  The present invention has been made in view of the above problems in the prior art, and provides an imaging apparatus and an imaging method provided with a white balance correction unit that does not cause color shift in the entire image and does not require time for imaging. With the goal.

  As a result of intensive studies in order to solve the above problems, the inventors of the present invention have at least the imaging device and the imaging method according to the present invention, specifically, the technical features described in (1) to (10) below. It has been found that the above problems can be solved by having the present invention, and the present invention has been completed.

(1) : an imaging element that converts light incident from the optical system into an electrical signal and outputs it as an imaging signal; evaluation value acquisition means that extracts a white balance evaluation value from the imaging signal; and In the imaging apparatus including a dividing unit that divides into blocks, the grid-like block includes an acquisition block and an interpolation block that are not adjacent to each other, and the acquisition block and the interpolation block include a target pixel and a non-target pixel. And a first correction coefficient acquisition means for acquiring a first correction coefficient for white balance set for the target pixel, and a second correction for acquiring a second correction coefficient for white balance set for the non-target pixel Coefficient obtaining means; white balance correction is performed on the target pixel based on the first correction coefficient; and the non-target pixel is based on the second correction coefficient. White balance correction means for performing white balance correction, wherein the first correction coefficient acquisition means acquires a first correction coefficient of the acquisition block based on the white balance evaluation value, and first correction of the interpolation block A coefficient is acquired based on a first correction coefficient of an acquisition block adjacent to the interpolation block, and the second correction coefficient acquisition unit includes the second correction coefficient as a non-target pixel for acquiring the second correction coefficient and a surrounding area. The image pickup apparatus is obtained based on interpolation based on a distance to the target pixel of the first pixel and a first correction coefficient of the peripheral target pixel.

(2) The imaging device according to (1) , wherein the target pixel includes two or more pixels.

(3) The above-mentioned (1) or (2 ), wherein the second correction coefficient acquisition means acquires the correction coefficient of the target pixel as the second correction coefficient when there is one peripheral target pixel. ) .

(4) : The imaging device according to any one of (1) to (3) , wherein the division unit switches the number of divided blocks according to a zoom position.
(5) The imaging apparatus according to (4) , wherein the dividing unit reduces the number of divided blocks as the zoom magnification increases.

(6) : The line segment connecting the surrounding target pixel and the non-target pixel for acquiring the second correction coefficient does not intersect with a line segment connecting all the target pixels in a grid pattern. The imaging apparatus according to any one of 1) to (6).
However, the line segment connecting the target pixel and the non-target pixel is not longer than the length of the diagonal line segment of one block.

(7) : If the non-target pixel is on any one of the line segments that connect all the target pixels in a lattice shape, the peripheral target pixel is the line segment and the target pixel. The imaging apparatus according to any one of (1) to (6) , wherein the pixel of interest is four target pixels that form a minimum square among quadrangles formed by line segments connecting each other.

(8) : The surrounding target pixel is arranged at both ends of the non-target pixel when the non-target pixel is present on any one of the line segments connecting all the target pixels in a grid pattern. The imaging apparatus according to any one of the above (1) to (7) , wherein the two target pixels are selected.

(9) : an image sensor that converts light incident from the optical system into an electrical signal and outputs the signal as an image signal; an evaluation value acquisition unit that extracts a white balance evaluation value from the image signal; and In the imaging method of an imaging apparatus including a dividing unit that divides a block, the grid-like block includes an acquisition block and an interpolation block that are not adjacent to each other, and the acquisition block and the interpolation block include a target pixel and a non-target A first correction coefficient acquisition step of acquiring a first white balance correction coefficient set for the target pixel, and then acquiring a second white balance correction coefficient set for the non-target pixel A second correction coefficient acquisition step, performing white balance correction on the target pixel based on the first correction coefficient, and applying the white balance correction to the non-target pixel A white balance correction step of performing white balance correction based on a second correction coefficient, wherein the first correction coefficient acquisition step acquires the first correction coefficient of the acquisition block based on the white balance evaluation value, and A first correction coefficient of the interpolation block is acquired based on a first correction coefficient of an acquisition block adjacent to the interpolation block, and the second correction coefficient acquisition step includes the second correction coefficient as the second correction coefficient. In this imaging method, the acquisition is performed based on interpolation based on a distance between a non-target pixel to be acquired and a peripheral target pixel, and a first correction coefficient of the peripheral target pixel.

  According to the present invention, it is possible to provide an imaging apparatus and an imaging method including a white balance correction unit that does not cause color shift in the entire image and does not require time for imaging.

The imaging apparatus of the present invention will be described below.
The embodiments described below are preferred embodiments of the present invention, and thus various technically preferable limitations are given. However, the scope of the present invention is particularly limited in the following description. Unless otherwise stated, the present invention is not limited to these embodiments.

  An imaging apparatus according to the present invention includes an imaging element that converts light incident from an optical system into an electrical signal and outputs the electrical signal, an evaluation value acquisition unit that extracts a white balance evaluation value from the imaging signal, and the imaging signal. In an imaging apparatus including a dividing unit that divides into grid blocks, the imaging signal divided into the grid blocks includes a target pixel and a non-target pixel, and the target pixel includes two or more pixels. And a second correction coefficient for acquiring a second correction coefficient for white balance set for the non-target pixel, and a first correction coefficient acquisition means for acquiring a first correction coefficient for white balance set for the target pixel. And obtaining means for performing white balance correction on the target pixel based on the first correction coefficient and performing white balance correction on the non-target pixel based on the second correction coefficient. White balance correction means, wherein the first correction coefficient acquisition means acquires the first correction coefficient based on the white balance evaluation value, and the second correction coefficient acquisition means acquires the second correction coefficient. The second correction coefficient is acquired based on the interpolation based on the distance between the non-target pixel and the peripheral target pixel, and the first correction coefficient of the peripheral target pixel.

  The imaging apparatus of the present invention includes an imaging element that converts light incident from an optical system into an electrical signal and outputs the electrical signal, an evaluation value acquisition unit that extracts a white balance evaluation value from the imaging signal, and the imaging In an imaging apparatus including a dividing unit that divides a signal into grid-like blocks, the grid-like block includes an acquisition block and an interpolation block that are not adjacent to each other, and the acquisition block and the interpolation block are pixels of interest. First correction coefficient acquisition means for acquiring a first white balance correction coefficient to be set for the target pixel, and a second white balance correction coefficient to be set for the non-target pixel. Second correction coefficient acquisition means for acquiring, white balance correction is performed on the target pixel based on the first correction coefficient, White balance correction means for performing white balance correction based on a second correction coefficient, wherein the first correction coefficient acquisition means acquires the first correction coefficient of the acquisition block based on the white balance evaluation value, and the interpolation The first correction coefficient of the block is acquired based on the first correction coefficient of the acquisition block adjacent to the interpolation block, and the second correction coefficient acquisition unit acquires the second correction coefficient and the second correction coefficient. It is obtained based on the interpolation based on the distance between the non-target pixel and the peripheral target pixel, and the first correction coefficient of the peripheral target pixel.

(Remarked pixel)
The target pixel in the present invention may be an arbitrary point in the block, but is preferably one or more pixels at the center of the block.

(Peripheral attention pixel)
Further, in the present invention, the peripheral target pixel is such that the line segment connecting the peripheral target pixel and the non-target pixel for obtaining the second correction coefficient does not intersect the line segment connecting all the target pixels in a grid pattern. This is a pixel of interest.
However, the line segment connecting the target pixel and the non-target pixel is not longer than the length of the diagonal line of one block.

  In addition, when a non-target pixel exists on any one of the line segments that connect all the target pixels in a lattice shape, the peripheral target pixel is a line segment that connects the line segment and the target pixel. It is preferable that the four pixels of interest that form the smallest square among the squares formed by

  On the other hand, when the non-target pixel exists on any one of the line segments that connect all the target pixels in a lattice shape, the peripheral target pixel is arranged at both ends of the line segment. May be the pixel of interest.

(First embodiment)
Next, the basic configuration of the imaging apparatus according to the present invention will be described in more detail with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of the imaging apparatus according to the first embodiment of the present invention.
Data exposed on the CCD light receiving surface through the lens of FIG. 1 is converted into an electrical signal and output from the CCD as an analog image signal. The output analog image signal is converted into a digital signal by an F / E (front end) composed of CDS, AGC, and A / D. In the CDS, only effective analog image signals are extracted from the output from the CCD. An AGC (analog gain controller) amplifies the analog image signal. In A / D, the analog image signal is converted to digital. These timings are synchronized by a timing signal from a timing signal generator (TG).

  Digital data input from the F / E is temporarily stored in an SDRAM which is an image storage unit and a frame memory.

  The YUV conversion unit, which is an image processing apparatus, includes an image processing unit, a display control unit, a data compression unit, and a data expansion unit. The image processing unit processes the data temporarily stored in the SDRAM based on the image processing parameters set by the control unit, and outputs the processed data to the SDRAM. The display control unit sends the data written in the SDRAM to the display unit and displays the captured image. The data compression unit compresses the data written in the SDRAM and outputs the processing result to a storage unit such as a memory card.

  The control unit is a microcomputer (hereinafter referred to as a microcomputer) including a CPU, ROM, RAM, and the like (not shown). This CPU uses the RAM as a work area according to a program stored in the ROM, and gives instructions from the key operation unit. Or, according to an external operation instruction such as a remote controller (not shown) or a communication operation instruction by communication from an external terminal such as a personal computer, the entire operation of the imaging apparatus is controlled. Specifically, the control unit performs imaging operation control, setting of image processing parameters in the image processing apparatus, memory control, and display control.

  The key operation unit is for instructing the operation of the image pickup apparatus, and includes a release key for instructing shooting, an optical zoom and an electronic zoom, and buttons for performing various other settings from the outside.

  Next, the basic operation of AWB control will be briefly described. The subject image through the lens enters the CCD, and the CCD converts the subject image into an electrical signal (analog image data) and outputs R, G, and B analog image data. The analog image data is converted into R, G, B digital image signals by an A / D converter. The converted digital image data is stored in a frame memory.

  When the signal processing IC captures digital image data, the CCD-I / F reads the integrated values of R, G, and B of a specific portion or the entire screen. The microcomputer, which is the control unit, reads the RGB integrated values, performs feature detection, calculates the WB gain Rg and Bg (feature data) so as to achieve an appropriate white balance (hereinafter also referred to as WB), and sends it to the microcomputer. Send. In the YUV conversion unit, which is an image processing device, R, G, and B data are converted into luminance Y and color difference Cb and Cr data and output to the SDRAM. When performing this image conversion, WB gains Rg and Bg are set from the microcomputer to the image processing apparatus.

  Data stored in the SDRAM is read into a data compression unit in the image processing apparatus and compressed by, for example, JPEG compression. The compressed data is recorded on an external recording device such as a memory card.

Next, in order to describe the configuration of the white balance correction means that is a feature of the first embodiment of the imaging apparatus according to the present invention, the configuration of the conventional imaging apparatus will be described first.
FIG. 2 is a schematic diagram in which the imaging signal is divided into grid-like blocks.
In a conventional imaging apparatus, a correction coefficient is acquired by combining RGB integrated values of one or a plurality of blocks from RGB integrated values acquired for each block as shown in the figure.

The center of each block is set as the target pixel as in the target pixel 10 in FIG. 2, and the correction coefficients R _gain and B _gain acquired in each block are set to each target pixel.
The non-target pixel is obtained by interpolation based on the correction coefficient set for the target pixel of the block where the non-target pixel exists and the peripheral target pixel and the distance from the pixel.

FIG. 3 is a schematic diagram showing a positional relationship between a non-target pixel and a target pixel.
In FIG. 3, the imaging signals divided into the grid-like blocks are the non-target pixel 11 that is a correction coefficient calculation target, and the first correction coefficient R0 (the first pixel in the target pixel arranged in the upper left) of the surrounding target pixel. 1 correction coefficient), R1 (first correction coefficient in the pixel of interest arranged in the upper right), R2 (first correction coefficient in the pixel of interest arranged in the lower left), R3 (first correction coefficient in the pixel of interest arranged in the lower right) Correction coefficient). Here, the R1-R3 distance is normalized as 1, and the position of the non-target pixel for which the correction coefficient is to be calculated is represented by x and y. R0 is the origin, the R0 to R1 direction is the x direction, and the R0 to R2 direction is y. As the direction, the second correction coefficient is calculated by the following equations (1) and (2).

R _gain = (1-x) (1-y) R0 + x (1-y) R1 + (1-x) yR2 + xyR3
Formula (1)
B _gain = (1-x) (1-y) B0 + x (1-y) B1 + (1-x) yB2 + xyB3
Formula (2)

On the other hand, the configuration of the white balance correction means in the first embodiment of the imaging apparatus according to the present invention will be described in detail below.
FIG. 4 is a schematic diagram in which the imaging signal in the first embodiment of the imaging apparatus according to the present invention is divided into grid-like blocks.
In the present embodiment, the correction coefficient acquired for each block is set for the surface that is the target pixel 17 including two or more pixels.
For pixels other than the target pixel 17, the second correction coefficient is obtained by interpolation based on the gain (first correction coefficient) set on the surface of each peripheral block and the distance from the pixel to the surface. Specifically, the second correction coefficient is calculated using the above equations (1) and (2).

Except for the above configuration, a configuration similar to a known prior art can be used.
Also, with the above configuration, it is possible to make the color difference due to the correction coefficient difference in the block difficult to see.

(Second Embodiment)
FIG. 5 is a schematic diagram in which the imaging signal in the second embodiment of the imaging apparatus according to the present invention is divided into grid-like blocks.
In the present embodiment, when a non-target pixel exists in the region 12 and there is only one peripheral target pixel, the first correction coefficient of the block having the non-target pixel is set as the second correction coefficient for the non-target pixel. Set as.
Other than the above configuration, the configuration is the same as that of the first embodiment.
According to the above configuration, since no interpolation processing is required in the region 12, the processing time for calculating the correction coefficient can be reduced.

(Third embodiment)
FIG. 6 is a schematic diagram in which the image pickup signal in the third embodiment of the image pickup apparatus according to the present invention is divided into lattice blocks.
In the present embodiment, when there are non-target pixels in the regions 13, 14, 15, and 16 and there are only two peripheral target pixels, the distance between the two first correction coefficients and the non-target pixel. The second correction coefficient in the non-target pixel is obtained by interpolation according to. This calculation is performed using the following formulas (3) to (10).
Other than the above configuration, the configuration is the same as that of the second embodiment.

In the case of region 13, R _gain = (1-x) R2 + xR3 Formula (3)
B _gain = (1-x) B2 + xB3 Formula (4)

In the case of region 14, R _gain = (1-x) R0 + xR1 Formula (5)
B _gain = (1-x) B0 + xB1 Formula (6)

In the case of region 15, R _gain = (1-y) R1 + yR3 Formula (7)
B _gain = (1-y) B1 + yB3 Formula (8)

In the case of region 16, R _gain = (1-y) R0 + yR2 Formula (9)
B _gain = (1-y) B0 + yB2 Formula (10)

  Thereby, since a low-pass filter is applied to the correction coefficient between pixels, it is possible to avoid an unnatural WB from being applied at the boundary between the light sources in an image in which a plurality of light sources are mixed.

(Fourth embodiment)
FIG. 7 is a schematic diagram in which an image pickup signal in the fourth embodiment of the image pickup apparatus according to the present invention is divided into lattice blocks.
In the present embodiment, the number of block divisions is changed depending on the shooting situation.
For example, when the luminance of blocks separated in the horizontal direction is greatly different, or when G / R and B / G of the blocks separated in the horizontal direction are greatly different, the number of blocks in the horizontal direction when dividing the block as shown in FIG. Half the number of vertical blocks.

The luminance, G / R, and G / B of each block are obtained by calculation from output values of a circuit that integrates RGB output from the CCD in the CCD-I / F of FIG.
Here, the first correction coefficient is set with the central pixel of the block as the target pixel. The first correction coefficient for each block is not set individually but is set in units of two as in the region 18.
The second correction coefficient of the non-target pixel in the block is obtained by interpolation based on the first correction coefficient set for the target pixel of the block and the peripheral target pixel and the distance from the non-target pixel.
As a result, the resolution of the horizontal correction coefficient setting interval can be increased, and an optimal WB can be applied to an image in which the light source is frequently switched in the separated horizontal blocks.

In FIG. 7, the number of blocks in the vertical direction is half of the number of blocks in the horizontal direction. However, it is possible to increase the resolution of the correction coefficient setting interval in the vertical direction by reducing the number of blocks in the horizontal direction to half the number of blocks in the vertical direction. It becomes.
Except for the above configuration, the configuration is the same as that of the third embodiment.

(Fifth embodiment)
In the fifth embodiment, the correction coefficient setting method is changed depending on the shooting situation.
For example, a method of halving the number of horizontal / vertical block divisions is conceivable since the display is performed on a display device with a low resolution during monitoring, and there is no need to depict details.
Except for the above configuration, the configuration is the same as that of the third embodiment.

(Sixth embodiment)
FIG. 8 is a schematic diagram in which the imaging signal in the sixth embodiment of the imaging apparatus according to the present invention is divided into grid-like blocks.
In the present invention, it is not necessary to acquire the first correction coefficient from all the divided blocks, and the first correction coefficient may be acquired from only a part of the blocks as shown by the region 19 in FIG. . In the present embodiment, every other first correction coefficient is acquired (acquisition block 19), and the correction coefficient is set for the pixel of interest. For the target pixel of the block from which the first correction coefficient has not been acquired, a method of calculating and setting the first correction coefficient using an average value with the first correction coefficient of the surrounding block (interpolation block) is preferable. It may be a method.

  As a result, even in a monitoring process that requires correction coefficient switching in real time, the processing time required to calculate the correction coefficient can be reduced, and the video after WB processing can be displayed without reducing the frame rate.

  Another method is to reduce the number of horizontal and vertical block divisions according to the zoom magnification. In general, the closer to the tele end, the larger the pattern and the less likely it will be to be irradiated with multiple different light sources, so it is difficult to see the difference in color between small areas by reducing the number of divisions and setting a correction factor in a somewhat large block unit. There is an effect such as.

  According to the first to sixth embodiments of the imaging apparatus according to the present invention, color shift does not occur in the entire image, and shooting that does not require time for shooting is possible.

1 is a block diagram illustrating a configuration in a first embodiment of an imaging apparatus according to the present invention. It is the schematic diagram by which the imaging signal was divided | segmented into the grid | lattice-like block. It is a schematic diagram which shows the positional relationship of a non-focused pixel and a focused pixel. It is the schematic diagram by which the imaging signal in 1st Embodiment of the imaging device which concerns on this invention was divided | segmented into the grid | lattice-like block. It is the schematic diagram by which the imaging signal in 2nd Embodiment of the imaging device which concerns on this invention was divided | segmented into the grid | lattice-like block. It is the schematic diagram by which the imaging signal in 3rd Embodiment of the imaging device which concerns on this invention was divided | segmented into the grid | lattice-like block. It is the schematic diagram by which the imaging signal in 4th Embodiment of the imaging device which concerns on this invention was divided | segmented into the grid | lattice-like block. It is the schematic diagram by which the imaging signal in 6th Embodiment of the imaging device which concerns on this invention was divided | segmented into the grid | lattice-like block.

Explanation of symbols

10 pixel of interest 11 non-pixel of interest 12, 13, 14, 15, 16, 17, 18 area 19 acquisition block

Claims (9)

An image sensor that converts light incident from the optical system into an electrical signal and outputs the signal as an image signal;
Evaluation value acquisition means for extracting a white balance evaluation value from the imaging signal;
In an imaging apparatus comprising: a dividing unit that divides the imaging signal into lattice blocks;
The lattice-shaped block is composed of an acquisition block that is not adjacent to each other and an interpolation block,
The acquisition block and the interpolation block have a target pixel and a non-target pixel,
First correction coefficient acquisition means for acquiring a first correction coefficient for white balance set for the target pixel;
Second correction coefficient acquisition means for acquiring a second correction coefficient for white balance to be set for the non-target pixel;
White balance correction means for performing white balance correction on the target pixel based on the first correction coefficient and performing white balance correction on the non-target pixel based on the second correction coefficient;
The first correction coefficient acquisition means includes
Acquiring a first correction coefficient of the acquisition block based on the white balance evaluation value;
Acquiring a first correction coefficient of the interpolation block based on a first correction coefficient of an acquisition block adjacent to the interpolation block;
The second correction coefficient acquisition means converts the second correction coefficient into the first correction coefficient of the surrounding target pixel and the interpolation based on the distance between the non-target pixel for acquiring the second correction coefficient and the peripheral target pixel. An imaging device characterized in that acquisition is performed on the basis of the above. The imaging apparatus according to claim 1 , wherein the target pixel includes two or more pixels. 3. The imaging apparatus according to claim 1, wherein the second correction coefficient acquisition unit acquires a correction coefficient of the target pixel as the second correction coefficient when the number of the peripheral target pixels is one. The dividing means, the imaging apparatus according to any one of claims 1 to 3, characterized in that switching the number of divided blocks by the zoom position. 5. The imaging apparatus according to claim 4 , wherein the dividing unit reduces the number of divided blocks as the zoom magnification increases. A line connecting the non-target pixel for obtaining the second correction coefficient and the target pixel in the periphery of the above is, of claims 1 to 5, characterized in that does not intersect the line segment connecting all the pixel of interest to each other in a grid pattern The imaging device of any one.
However, the line segment connecting the target pixel and the non-target pixel is not longer than the length of the diagonal line segment of one block. When the non-target pixel exists on any one of the line segments connecting all the target pixels in a grid, the peripheral target pixel is a line connecting the line segment and the target pixel. The imaging apparatus according to any one of claims 1 to 6 , wherein the image pickup device includes four target pixels that form a minimum quadrangular shape among quadrangles composed of minutes. If the non-target pixel is on any one of the line segments that connect all the target pixels in a grid, the peripheral target pixels are arranged at two ends of the line segment. the imaging apparatus according to any one of claims 1 to 6, characterized in that a pixel of interest. An image sensor that converts light incident from the optical system into an electrical signal and outputs the signal as an image signal;
Evaluation value acquisition means for extracting a white balance evaluation value from the imaging signal;
In an imaging method of an imaging apparatus comprising: a dividing unit that divides the imaging signal into grid-like blocks;
The lattice-shaped block is composed of an acquisition block and an interpolation block that are not adjacent to each other,
The acquisition block and the interpolation block have a target pixel and a non-target pixel,
A first correction coefficient acquisition step of acquiring a first correction coefficient for white balance to be set for the target pixel;
Next, a second correction coefficient acquisition step of acquiring a second correction coefficient for white balance to be set for the non-target pixel;
A white balance correction step of performing white balance correction on the target pixel based on the first correction coefficient and performing white balance correction on the non-target pixel based on the second correction coefficient;
The first correction coefficient acquisition step includes
Acquiring a first correction coefficient of the acquisition block based on the white balance evaluation value;
Next, the first correction coefficient of the interpolation block is acquired based on the first correction coefficient of the acquisition block adjacent to the interpolation block,
In the second correction coefficient acquisition step, the second correction coefficient is converted into an interpolation based on a distance between a non-target pixel for acquiring the second correction coefficient and a peripheral target pixel, and a first correction coefficient of the peripheral target pixel. An imaging method characterized in that acquisition is performed based on the above.

Images