JP6196882B2 - Multi-area white balance control device, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing device, multi-area white balance image processing method, multi-area White balance image processing program, computer recording multi-area white balance image processing program, and imaging apparatus provided with multi-area white balance image processing device - Google Patents

Description

  The present invention adjusts and corrects an error in estimating the color temperature of a light source in an image obtained by photographing a scene illuminated with a plurality of light sources, and corrects the white balance of the input image, so that there is no uncomfortable color in the entire image. Multi-area white balance control device, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing device, multi-area white balance The present invention relates to an image processing method, a multi-area white balance image processing program, a computer recording the multi-area white balance image processing program, and an imaging apparatus including the multi-area white balance image processing apparatus.

If the photographing light source is outdoors during the day, its color temperature changes depending on the altitude of the sun and the atmospheric conditions. Also, the color temperature differs between the sun and the shade.
Furthermore, there are various artificial lights such as fluorescent lamps and incandescent lamps indoors, and the color temperature of each light source is different.
In addition, when flash photography is taken outdoors in the daytime, mixed light of flash illumination and external light serves as a photographing light source.
Also, when flash photography is taken outdoors at night, the flash illumination and background artificial illumination are not mixed, but the color temperature of the light source is different in different areas in the same scene.
In addition, when flash photography is performed indoors, light in which flash illumination and artificial illumination are mixed serves as a photographing light source.

  Under the various light sources described above, the human eye adapts to color. That is, the human eye feels a white subject in the attention area white regardless of whether the light source is sunlight, artificial lighting, or a mixed light thereof. The camera, which is an input device in accordance with the characteristics of the human eye, is also configured to adjust so-called white balance by adjusting the RGB gain.

  However, for a camera, the illuminating light source is unknown until shooting, so in general, the type of illuminating illumination (color temperature of the light source) is estimated from the captured image, or the illumination is specified by the user. Correct the white balance of the entire image shot according to the type.

  When the subject scene is illuminated by multiple light sources, the acquired image is divided into multiple areas in order to perform white balance processing with high accuracy regardless of the color temperature of the light source in the area in the scene. However, it is necessary to detect the color temperature of the light source corresponding to each divided area and perform different white balance processing.

Conventionally, for example, Patent Document 1 discloses an example of a method for performing white balance processing on an image obtained by photographing a scene illuminated with a plurality of light sources.
In the method described in Patent Document 1, an input color image is divided into a plurality of small screens, and a color temperature of a photographing light source when the color image is photographed is estimated for each of the divided small screens. , Create a histogram of the estimated color temperature for each small screen, group the small screens, again estimate the color temperature of the photographic light source for each group, and according to the color temperature estimation result for each group, A white balance correction amount is calculated for each group, and white balance correction is performed on each small screen in the group using the white balance correction amount calculated for each group.

  Further, in Patent Document 2, when there are irradiation areas of different types of light sources in the imaging screen, it is difficult to apply an appropriate white balance to the entire imaging screen, resulting in a captured image with color misregistration. In order to solve the problem, the imaging screen is divided into a plurality of areas according to luminance information and color information, different white balance correction coefficients are set for each of the plurality of areas, and each of the plurality of areas is set. It is disclosed that a white balance correction coefficient limit for keeping the difference in white balance correction coefficient set for the above in a predetermined range is provided to generate a white balance correction coefficient for the entire screen.

JP 2002-271638 A JP 2010-213232 A

By the way, in the conventional so-called mixed light source image white balance correction technique in which shooting is performed with a plurality of different light sources, the input image is divided into a plurality of areas prior to the white balance correction, and each divided area is divided into a plurality of areas. For example, the color temperature of the photographing light source when the mixed light source image is photographed is estimated using a light source color temperature estimating method that is generally performed.
However, since each divided area is a part of the scene of the input image, the subject in each area is likely to be a single color subject, and the color temperature of the light source in the divided area is likely to be estimated to the wrong color temperature. .
For example, as shown in FIG. 1, after an input image of a scene with the sun on the front and the shade on the background is divided into 6 × 6 areas, the subject is almost grass in the lower left area corresponding to the foreground lawn. A case arises.
When the color temperature of a light source is estimated by a light source color temperature estimation method that is generally performed on an area of the entire surface of a lawn, the color temperature of the light source is likely to be erroneously estimated as artificial lighting such as a fluorescent lamp. If the white balance is corrected using the corresponding white balance correction gain based on the wrong estimated light source color temperature, the color balance of the area is greatly lost, and the entire image has a very unnatural hue. There is a possibility.

  However, in the method described in Patent Document 1, after estimating the color temperature of the light source for each small screen, the small screens with the estimated color temperature of the light source are sorted into a plurality of groups, and again for each sorted group. Since the color temperature of the light source is estimated, there is an effect of limiting the type of the estimated color temperature of the light source. However, if the estimation of the color temperature of the light source of the small screen is wrong and the estimated color temperature of the light source is different from the original color temperature of the light source, they cannot be sorted into the same group in the first place. For this reason, in the method described in Patent Document 1, when the estimated light source color temperature is incorrect as described above, by correcting the white balance using the white balance correction gain corresponding to the estimated light source color temperature, The color balance of the area is greatly lost, and it is not possible to cope with the problem that the color becomes very unnatural locally.

  The method described in Patent Document 2 sets a white balance correction coefficient (that is, a control value corresponding to the color temperature of the light source) for each of a plurality of divided areas, and estimates (or sets) in the first place. ) It is not assumed that the color temperature of the light source is wrong. For this reason, even in the method described in Patent Document 2, if the estimated light source color temperature is incorrect as described above, the white balance is corrected using the white balance correction gain corresponding to the incorrect estimated light source color temperature. As a result, the color balance of the area is largely lost, and the problem that the entire image has a very unnatural color cannot be dealt with.

  The present invention has been made in view of the above-described problems in the prior art. When the entire captured image is divided into a plurality of areas and the color temperature of the light source in each area is estimated, the estimation is performed in the individual areas. A multi-area white balance control device, multi-area white balance control method, multi-area white balance control program, and multi-area white balance control program that can realize color reproduction without a sense of incongruity even if the color temperature of the light source is wrong. Recorded computer, multi-area white balance image processing apparatus, multi-area white balance image processing method, multi-area white balance image processing program, computer recording multi-area white balance image processing program, and multi-area white balance image And its object is to provide an imaging apparatus having a processing device.

In order to achieve the above object, a multi-area white balance control apparatus according to the present invention estimates area color means for dividing an input image into a plurality of areas and the color temperature of the light source for each area divided by the area division means. The area light source color temperature estimation means and the area light source color temperature estimation means estimated by the area light source color temperature and the area adjacent to the area are compared with the color temperature of the light source. When it is determined whether or not adjustment is necessary for the color temperature of the light source, and based on the estimated color temperature of the light source in the area adjacent to the area, the estimated light source of the area Based on the color temperature of the area estimated light source color temperature adjusting means for adjusting the color temperature, and the light temperature of each area adjusted by the area estimated light source color temperature adjusting means It is characterized by having a white balance correction gain map generating means for generating a white balance correction gain map of the entire input image.

The multi-area white balance control method according to the present invention divides an input image into a plurality of areas, estimates the color temperature of the light source for each divided area, and estimates the color temperature of the light source in the area and the area. Based on the comparison with the color temperature of the light source in the adjacent area, it is determined whether or not the adjustment of the estimated color temperature of the light source in the area is necessary. Adjusting the estimated color temperature of the light source in the area based on the estimated color temperature of the light source, and generating a white balance correction gain map of the entire input image based on the adjusted color temperature of the light source in each area. It is said.

Further, a computer that records a multi-area white balance control program according to the present invention is a computer provided in a multi-area white balance control device, wherein the computer divides an input image into a plurality of areas, the area division Area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the means, the color temperature of the light source in the area estimated by the area light source color temperature estimating means and the light source in the area adjacent to the area Based on the comparison with the color temperature of the area, it is determined whether or not the adjustment of the color temperature of the estimated light source of the area is necessary, and when it is determined that the adjustment is necessary, the estimated light source of the area adjacent to the area Area estimated light source color temperature that adjusts the estimated color temperature of the light source in the area based on the color temperature of And a white balance correction gain map generation unit that generates a white balance correction gain map of the entire input image based on the color temperature of the light source in each area adjusted by the area estimation light source color temperature adjustment unit. A multi-area white balance control program is recorded.

A multi-area white balance control program according to the present invention is a computer-readable multi-area white balance control program provided in a multi-area white balance control device, wherein the computer divides an input image into a plurality of areas. dividing means, the area dividing unit area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by, estimated by the area light source color temperature estimating means, the color temperature and the area of the area light source Based on the comparison with the color temperature of the light source in the area adjacent to the area, it is determined whether adjustment is necessary for the estimated color temperature of the light source in the area. Based on the color temperature of the estimated light source of the area, the estimated light source of the area Area estimated light source color temperature adjusting means for adjusting temperature, white balance correction for generating a white balance correction gain map of the entire input image based on the color temperature of the light source in each area adjusted by the area estimated light source color temperature adjusting means It is characterized by functioning as a gain map generating means.

  According to the present invention, when the entire captured image is divided into a plurality of areas and the color temperature of each area is estimated, even if the color temperature of the light source estimated in the individual area is incorrect, color reproduction without a sense of incongruity Multi-area white balance control device, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing device, multi-area white A balance image processing method, a multi-area white balance image processing program, a computer recording the multi-area white balance image processing program, and an imaging device including the multi-area white balance image processing apparatus are obtained.

It is explanatory drawing which shows an example which divided | segmented the input image into the several area. 1 is a block diagram of a digital camera including a multi-area white balance control device according to an embodiment of the present invention. 3 is a flowchart showing a processing procedure until a gain map for white balance correction is created from an input image in the multi-area white balance control device of FIG. 2. When estimating the color temperature of the light source from the color distribution of the subject in the (R / G, B / G) chromaticity diagram, all the pixels in the image data are plotted on the chromaticity diagram, and the chromaticity diagram in advance. It is a graph which shows an example of a chromaticity diagram and an area determination standard for determining the color temperature of the light source about the plotted point according to the set color temperature area determination standard of various light sources. In the multi-area white balance control device of FIG. 2, the input image is divided into areas and is an explanatory diagram showing an example of the color temperature of the light source estimated for each area by the area light source color temperature estimation means. In the multi-area white balance control device of FIG. 2, the area estimated light source color temperature adjusting means is an explanatory diagram showing an example of the scanning direction when adjusting the estimated color temperature of the light source in the area shown in FIG. is there. Explanation showing the color temperature of the light source in each area after the area estimated light source color temperature adjusting means scans in the direction shown in FIG. 6 and adjusts the estimated color temperature of the light source in the area shown in FIG. FIG. In the multi-area white balance control device of FIG. 2, the input image is divided into areas and is an explanatory diagram showing another example of the color temperature of the light source estimated for each area by the area light source color temperature estimation means. It is explanatory drawing which shows the color temperature of the estimated light source of each area with a certain area. The color temperature of the light source in each area after the area estimated light source color temperature adjusting means scans in the same direction as shown in FIG. 6 and adjusts the estimated color temperature of the light source in the area shown in FIG. It is explanatory drawing which shows.

Prior to the description of the embodiments, the effects of the present invention will be described.
The present invention divides an input image into a plurality of areas, the color temperature of the light source is estimated in each divided area was estimated, and the color temperature of the area of the light source adjacent to the color temperature and the area of the area light source On the basis of the comparison , the necessity of adjustment to the color temperature of the estimated light source of the area is determined, and when it is determined that adjustment is necessary, based on the estimated color temperature of the light source of the area adjacent to the area, The estimated color temperature of the light source in the area is adjusted, and a white balance correction gain map of the entire input image is generated based on the adjusted color temperature of the light source in each area.

As in the present invention it was estimated based on a comparison between the estimated light source color temperature of the area adjacent to the color temperature and the area of the area light source, judging the necessity of adjustment for the color temperature of the light source estimated in the area If the color temperature of the estimated light source in the area is adjusted based on the estimated color temperature of the light source in the area adjacent to the area when it is determined that adjustment is necessary, the estimated color temperature of the area Even if is incorrect, the color temperature of the light source estimated in the area can be adjusted to a more appropriate color temperature. As a result, a white balance correction gain map of the entire input image can be generated based on the color temperature of the light source adjusted to an appropriate color temperature of each area, and the white balance of the entire input image can be generated based on the generated white balance correction map. Is corrected, the color balance of the area in the entire input image is properly maintained, and color reproduction without a sense of incongruity can be realized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention according to the claims.

Embodiment FIG. 2 is a block diagram of a digital camera provided with a multi-area AWB (auto white balance) control device according to an embodiment of the present invention. In FIG. 2, a portion surrounded by a one-dot chain line corresponds to the multi-area white balance control device according to the present embodiment, and a portion surrounded by a two-dot difference line corresponds to the multi-area white balance image processing device. FIG. 3 is a flowchart showing a processing procedure for creating a gain map for white balance correction from an input image in the multi-area white balance control apparatus of FIG.

  As shown in FIG. 2, the multi-area white balance control apparatus 1 of the present embodiment includes an image area dividing unit 11 as an area dividing unit, an area color temperature estimating unit 12 as an area light source color temperature estimating unit, and an area estimation. An area color temperature adjusting unit 13 as a light source color temperature adjusting unit and a WB (white balance) gain map generating unit 14 as a white balance correction gain map generating unit are provided. In FIG. 2, 10 is a multi-area white balance image processing device, 15 is a gain map storage unit that stores a white balance correction gain map generated by a WB (white balance) gain map generation unit 14, and 16 is a white balance processing unit. A WB (white balance) gain correction unit, 21 is a lens, 22 is an image sensor, and 23 is an image storage unit that stores an image captured by the image sensor 22. Note that the digital camera including the multi-area white balance image processing apparatus 10 according to the present embodiment includes a control unit (not shown). The control unit is separately provided with a function of storing an image captured by the image sensor 22 based on a shooting instruction in the image storage unit 23 and a function of inputting an image stored in the image storage unit 23. In the example of FIG. 2, an image formed on the image sensor 22 through the lens 21 is picked up and stored as a digital signal in the image storage unit 23 in accordance with a shooting instruction from a control unit (not shown).

  The image area dividing unit 11 divides the input image into a plurality of blocks having a predetermined size.

  The area color temperature estimation unit 12 estimates the color temperature of the light source for each block divided by the image area division unit 11. A method for estimating the color temperature of the light source will be described later.

The area color temperature adjustment unit 13 estimates the area based on the comparison between the color temperature of the light source of the block estimated by the area color temperature estimation unit 12 and the color temperature of the light source of the adjacent block adjacent to the block. When it is determined that adjustment is necessary for the color temperature of the light source and it is determined that adjustment is necessary, the estimated light source of the block is determined based on the estimated color temperature of the light source adjacent to the area. Adjust the color temperature. A method for adjusting the color temperature of the light source estimated for the block will be described later.

  The WB gain map generation unit 14 generates a white balance correction gain map of the entire input image based on the estimated color temperature of the light source of each block adjusted by the area color temperature adjustment unit 13.

  The WB gain correction unit 16 corrects the white balance of the entire input image based on the white balance correction gain map generated by the WB gain map generation unit 14.

  The flow of processing from the input image using the multi-area white balance control apparatus of this embodiment configured as described above to creating a gain map for white balance correction will be described with reference to FIGS.

First, a control unit of a digital camera (not shown) inputs image data stored in the image storage unit 23 (step S1).
Next, the image area dividing unit 11 divides the input image data into a plurality of blocks having a predetermined size (step S2). For convenience, in the following description, the image data is divided into 6 × 6 blocks.

  Next, the area color temperature estimation unit 12 estimates the color temperature of the light source for each block by the following method, for example, for the image of each block divided by the image area division unit 11 (step S3). ).

  For example, it is assumed that the effective pixels of the image sensor are 4800 × 3600 pixels. The input Bayer pattern RAW image is divided into 6 × 6 blocks by the image area dividing unit 11, and the size of each block is 800 × 600 pixels.

  Prior to the estimation of the color temperature of the light source, for example, a 20 × 20 pixel range is set as one sampling point for the 800 × 600 pixel block, and R, G, By obtaining an average value of the pixel values of B, R, G, and B pixel data having a sampling point of 40 × 30 pixels down-sampled are obtained.

  The chromaticity values (R / G, B / G) are obtained for the sampling points of 40 × 30 pixels, and the chromaticity values at the respective sampling points are plotted on, for example, the chromaticity diagram shown in FIG. Then, for example, using the color temperature area determination criteria of various light sources on the chromaticity diagram as shown in FIG. 4, the area determination of the color temperature of the light source for the points where the chromaticity values of all the sampling points are plotted. And the number of plotted points belonging to the color temperature area of each light source is totaled. The color temperature of the light source of the block is estimated from the order of the number of plotted points belonging to the aggregated color temperature area of each light source, for example, according to the conditions shown in Table 1 below.

  The area color temperature estimation unit 12 estimates the color temperature of the light source by using the above method for all 36 blocks. Thereby, for example, the estimation result of the color temperature of the light source as shown in FIG. 5 is obtained for the entire image.

  When the color temperatures of the light sources of all the blocks are estimated in this way, the color temperatures of the light sources in most blocks estimated by the area color temperature estimation unit 12 are correct compared to the actual visual result. Some blocks can be wrong. For example, in the image of (1, 4) block in the example of FIG. 5, since most of the subject is an outdoor lawn, the color temperature of the light source is estimated by mistake with a fluorescent lamp (“C”). If the block size is small, the subject is likely to be a subject of the same color locally, and as a result, the estimation of the color temperature of the light source by the area color temperature estimation unit 12 is likely to be erroneous.

  After the area color temperature estimation unit 12 finishes estimating the color temperature of the light source for all the blocks, the area color temperature adjustment unit 13 performs, for example, FIG. 6 on the estimated color temperature of the light source shown in FIG. As shown in FIG. 3, by scanning from the upper left to the lower right of the screen, the block is compared with the estimated color temperature of the light source of the block located in the vertical and horizontal 3 × 3 range around the block. The necessity of adjustment for the estimated color temperature of the light source is determined, and when it is determined that the adjustment is necessary, the color of the estimated light source of the block located within the vertical and horizontal 3 × 3 range around the block Based on the temperature, the estimated color temperature of the light source of the block is adjusted (step S4).

The adjustment of the color temperature of the light source estimated for the block by the area color temperature adjustment unit 13 is performed according to the following adjustment criteria, for example.
For example, if the estimated color temperature of the light source in the area is different from any of the estimated color temperatures of the light sources adjacent to the area, adjustment to the estimated color temperature of the light source in the area is necessary. It is determined that there is, and the estimated color temperature of the light source in the area is corrected to a multiple value of the estimated color temperature of the light source in the area adjacent to the area (adjustment criterion 1).
Further, for example, when the estimated color temperature of the light source in the area is different from any of the estimated color temperatures of the light sources adjacent to the area, the adjustment to the estimated color temperature of the light source in the area is performed. It is determined that it is necessary, and the estimated color temperature of the light source in the area is corrected to the multiple values of the estimated color temperature of the light source in the area adjacent to the area (adjustment criterion 2).
Further, for example, when the estimated color temperature of the light source in the area is different from any of the estimated color temperatures of the light sources adjacent to the area in the upper, lower, left, and right areas, the estimated color area of the area is estimated. It is determined that adjustment is necessary for the color temperature of the light source, and the estimated color temperature of the light source in the area is set to be larger than the estimated color temperature of the light source in the upper, lower, left, and right areas adjacent to the area. Correct the value (Adjustment Criteria 3).
In this embodiment, for the sake of convenience, the area color temperature adjustment unit 13 is configured to adjust / correct the color temperature of the light source in which the color temperature of the block is estimated in accordance with the adjustment standard 3. .

In the example of FIG. 6, the area color temperature adjusting unit 13 estimates the color temperature of the estimated light source of the block (4, 1) as “D” (mixed light). Since the color temperature of the light source is “B” (shade) different from “D” (mixed light), it is determined that the estimation of the color temperature of the light source of the block (4, 1) is wrong, It is determined that adjustment to the estimated color temperature of the light source in the block (4, 1) is necessary. Since the estimated multiple color temperature values of the light source in the upper, lower, left and right blocks are “B” (shade), the color temperature “D” (mixed light) of the area (4, 1) is changed to “B” (shade). )
The area color temperature adjustment unit 13 has the estimated color temperature of the light source of the block (2, 3) as “B” (shade), but the estimated color of the light source of the upper, lower, left and right blocks. Since all the temperatures are “A” (Hyuga) different from “B” (shade), it is judged that the estimation of the color temperature of the light source of the block of (2, 3) is wrong. It is determined that the block needs to be adjusted for the estimated light source color temperature. Then, since the majority value of the estimated color temperature of the light source of the upper, lower, left and right blocks is “A” (Hyuga), the color temperature “B” (shade) of the area (2, 3) is set to “A” (Hyuga). To correct.
Further, the area color temperature adjustment unit 13 has the estimated color temperature of the light source of the (0, 4) block as “D” (mixed light). Since all the color temperatures are different from “D”, it is judged that the estimation of the color temperature of the light source of the block (0, 4) is wrong, and the color temperature of the estimated light source of the block (0, 4) It is determined that adjustment is necessary. Then, since the majority value of the estimated color temperature of the light source of the upper, lower, left and right blocks is “A” (Hyuga), the color temperature “D” (mixed light) of the area (0, 4) is set to “A” (Hyuga). )
Further, the area color temperature adjustment unit 13 has the estimated color temperature of the light source of the block (1, 4) as “C” (fluorescent light), but the estimated light source of the upper, lower, left, and right blocks. Since the color temperatures are all different from “C” (fluorescent lamp), it is judged that the color temperature of the light source of the block (1, 4) is wrong, and the block of (1, 4) is estimated. It is determined that adjustment to the color temperature of the light source is necessary. Then, since the majority value of the estimated color temperature of the light source of the upper, lower, left and right blocks is “A” (Hyuga), the color temperature “C” (fluorescent light) of the area (1, 4) is changed to “A” (Hyuga). )
As a result, the color temperature of the light source of the entire image is adjusted as shown in FIG.

By the way, depending on the subject to be photographed, there may be a case where an irradiation area with high saturation locally exists in the photographing screen.
An example of adjustment / correction of the color temperature of the light source in which the color temperature of the block is estimated by the area color temperature adjustment unit 13 in such a case will be described with reference to FIGS.
In the example of FIG. 8, since there is a local area of a highly saturated subject, the color temperature of the area (4, 1) or the area (0, 4) cannot be estimated. In the example of FIG. 8, the area where the color temperature of these light sources cannot be estimated is defined as an unknown color temperature “X”.
Subsequently, as in the example of FIG. 6, the color temperature of the block is compared with the estimated light source color temperature of the block and the four blocks above, below, left, and right so as to scan from the upper left to the lower right of the screen. When it is determined whether or not adjustment is necessary for the estimated color temperature of the light source, and when it is determined that adjustment is necessary, based on the estimated color temperature of the light source of the upper, lower, left, and right blocks of the block, Adjust / correct the estimated color temperature of the light source. As a result, the color temperature of the light source of the entire image is output as shown in FIG.

  After the adjustment of the estimated color temperature of the light source by the area color temperature adjustment unit 13, the WB gain map generation unit 14 is determined in advance based on the adjusted color temperature of the light source of each block shown in FIG. Referring to the R and B gain data corresponding to the color temperature of the light source, a WB correction gain map corresponding to R and B is generated (step S5). The WB correction gain map generated by the WB gain map generation unit 14 is stored (saved) in the gain map storage unit 15 (step S6).

  Then, the WB gain correction unit 16 applies the gain obtained by inputting the gain map storage unit 15 to the pixel value of each color corresponding to the position of the input image, thereby correcting the white balance and correcting the image. Output.

According to the multi-area white balance control device 1 and multi-area white balance control method of the present embodiment, based on the comparison between the estimated color temperature of the light source of the area and the color temperature of the light source of the area adjacent to the area, When it is determined whether or not adjustment is necessary for the estimated color temperature of the light source estimated in the area, the area estimated based on the estimated color temperature of the light source in the area adjacent to the area. Since the color temperature of the light source is adjusted, even if the estimated color temperature of the area is incorrect, the estimated color temperature of the light source of the area can be adjusted to a more appropriate color temperature. As a result, a white balance correction gain map of the entire input image can be generated based on the color temperature of the light source adjusted to an appropriate color temperature of each area, and the white balance of the entire input image can be generated based on the generated white balance correction map. Is corrected, the color balance of the area in the entire input image is properly maintained, and color reproduction without a sense of incongruity can be realized.

  In the multi-area white balance image processing apparatus 10 of the present embodiment, an example in which the image area dividing unit 11 as an area dividing unit divides an input image into blocks having the same shape and area divided by vertical and horizontal straight lines. As described above, the shape and size of each area divided by the area dividing unit in the multi-area white balance image processing apparatus of the present invention is not limited to this embodiment. Such a shape may be sufficient and the size of each area may differ.

In the multi-area white balance control device 1 of the present embodiment, when the area color temperature adjusting unit 13 as the area estimated light source color temperature adjusting unit adjusts the color temperature of the light source whose color temperature is estimated for the block. Although the adjustment standards 1 to 3 are shown as the adjustment standards used in the above, the adjustment standards are not limited to these, and for example, the following adjustment standards 4 to 6 may be used.
For example, if the estimated color temperature of the light source in the area is different from any of the estimated color temperatures of the light sources adjacent to the area, adjustment to the estimated color temperature of the light source in the area is necessary. It is determined that there is, and the estimated color temperature of the light source in the area is corrected to the multiple values of the estimated color temperature of the light source in an area within a predetermined radius range including the area adjacent to the area. (Adjustment criterion 4).
Further, for example, when the estimated color temperature of the light source in the area is different from any of the estimated color temperatures of the light sources adjacent to the area, the adjustment to the estimated color temperature of the light source in the area is performed. Multiple values of the estimated light source color temperature of the area within a predetermined radius range with the area as a reference, including the area adjacent to the area, the color temperature of the estimated light source of the area determined to be necessary (Adjustment criterion 5).
Further, for example, when the estimated color temperature of the light source in the area is different from any of the estimated color temperatures of the light sources adjacent to the area in the upper, lower, left, and right areas, the estimated color area of the area is estimated. It is determined that adjustment to the color temperature of the light source is necessary, and the color temperature of the estimated light source of the area is determined based on the area including the upper, lower, left, and right areas adjacent to the area. Correction is made to the multiple values of the estimated color temperature of the light source in the area within the radius range (adjustment criterion 6).

Further, the multi-area white balance image processing apparatus 10 according to the present embodiment includes a computer with an image area dividing unit 11 (area dividing unit), an area color temperature estimating unit 12 (area light source color temperature estimating unit), and an area color temperature adjusting unit 13. A multi-area white balance image processing program for functioning as (area estimated light source color temperature adjusting means), WB gain map generating section 14 (white balance correction gain map generating means), and WB gain correcting section 16 (white balance processing means). You may comprise with the computer recorded on the internal hard disk.
In addition, the multi-area white balance image processing apparatus 10 according to the present embodiment includes a computer with an image area dividing unit 11 (area dividing unit), an area color temperature estimating unit 12 (area light source color temperature estimating unit), and area color temperature adjustment. Multi-area white balance image processing for functioning as a unit 13 (area estimation light source color temperature adjustment unit), a WB gain map generation unit 14 (white balance correction gain map generation unit), and a WB gain correction unit 16 (white balance processing unit) You may comprise with the computer-readable medium which recorded the program, for example, CD-ROM.

  Multi-area white balance control apparatus, multi-area white balance control method, multi-area white balance control program, computer recording multi-area white balance control program, multi-area white balance image processing apparatus, multi-area white balance image processing method A multi-area white balance image processing program, a computer having a multi-area white balance image processing program recorded thereon, and an imaging apparatus having a multi-area white balance image processing apparatus are so-called mixed light source image white images captured with a plurality of different light sources. This is useful in fields where balance correction is required.

DESCRIPTION OF SYMBOLS 1 Multi area white balance control apparatus 10 Multi area white balance image processing apparatus 11 Area division means (image area division part)
12 Area light source color temperature estimation means (area color temperature estimation unit)
13 Area estimated light source color temperature adjusting means (area color temperature adjusting unit)
14 White balance correction gain map generation means (WB gain map generation unit)
15 Gain map storage unit 16 White balance processing means (WB gain correction unit)
21 Lens 22 Image sensor 23 Image storage unit

Claims (11)

Area dividing means for dividing the input image into a plurality of areas;
Area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means;
Based on the comparison between the color temperature of the light source of the area estimated by the area light source color temperature estimation means and the color temperature of the light source of the area adjacent to the area, the adjustment of the estimated color temperature of the light source of the area is performed. Area estimation light source that adjusts the color temperature of the estimated light source of the area based on the color temperature of the estimated light source of the area adjacent to the area when it is determined whether adjustment is necessary. Color temperature adjusting means;
White balance correction gain map generation means for generating a white balance correction gain map of the entire input image based on the color temperature of the light source of each area adjusted by the area estimation light source color temperature adjustment means. Multi-area white balance control device.   The area estimated light source color temperature adjusting means, when the color temperature of the light source in the area estimated by the area light source color temperature estimating means is different from any of the estimated light temperatures of the areas adjacent to the area In addition, it is determined that the adjustment of the color temperature of the estimated light source of the area is necessary, and the color temperature of the estimated light source of the area is set to a plurality of values of the color temperature of the estimated light source of the area adjacent to the area. The multi-area white balance control device according to claim 1, wherein the multi-area white balance control device is corrected to the following.   The area estimated light source color temperature adjusting means is the estimated light source of the upper, lower, left, and right areas adjacent to the area in which the color temperature of the light source of the area is estimated by the area light source color temperature estimating means. If it is different from any of the color temperatures of the area, it is determined that an adjustment to the color temperature of the estimated light source of the area is necessary, and the color temperature of the estimated light source of the area is adjacent to the area above, below, 2. The multi-area white balance control apparatus according to claim 1, wherein the multi-area white balance control device corrects the estimated color temperature of the light source in the left and right areas to a plurality of values. The area estimated light source color temperature adjusting means includes a color temperature of the light source of the area estimated by the area light source color temperature estimating means and an area adjacent to the area within a predetermined radius range based on the area. Based on the comparison with the color temperature of the light source of the area, it is determined whether or not adjustment is necessary for the estimated color temperature of the light source of the area, and when it is determined that adjustment is necessary, it is adjacent to the area adjacent to the area. The color temperature of the estimated light source in the area is adjusted based on the estimated color temperature of the light source in an area within a predetermined radius range with the area as a reference, including the area to be operated. The multi-area white balance control device described.   The area estimated light source color temperature adjusting means, when the color temperature of the light source in the area estimated by the area light source color temperature estimating means is different from any of the estimated light temperatures of the areas adjacent to the area In addition, it is determined that adjustment to the color temperature of the estimated light source in the area is necessary, and the color temperature of the estimated light source in the area includes a predetermined radius including the area adjacent to the area. 5. The multi-area white balance control apparatus according to claim 4, wherein the multi-area white balance control apparatus corrects the estimated color temperature of the light source in the area within the range to a plurality of values.   The area estimated light source color temperature adjusting means is the estimated light source of the upper, lower, left, and right areas adjacent to the area in which the color temperature of the light source of the area is estimated by the area light source color temperature estimating means. If it is different from any of the color temperatures of the area, it is determined that an adjustment to the color temperature of the estimated light source of the area is necessary, and the color temperature of the estimated light source of the area is adjacent to the area above, below, 5. The multi-area white balance control according to claim 4, wherein the multi-area white balance control is corrected to a multiple value of the estimated color temperature of the light source in an area within a predetermined radius range including the left and right areas as a reference. apparatus. The multi-area white balance control device according to any one of claims 1 to 6,
Based on the white balance correction gain of each area generated by the white balance correction gain map generation unit provided in the multi-area white balance control device, the area division unit of the multi-area white balance control device sets the division target. A multi-area white balance image processing apparatus comprising: white balance processing means for performing white balance processing on the entire input image.   An imaging apparatus comprising the multi-area white balance image processing apparatus according to claim 7. Divide the input image into multiple areas,
Estimate the color temperature of the light source for each divided area,
Estimated, based on a comparison of the color temperature of the area of the light source adjacent to the color temperature and the area of the area light source, and determines the necessity of adjustment for the color temperature of the light source estimated in the area, it is necessary to adjust When it is determined that there is, based on the estimated color temperature of the light source in the area adjacent to the area, adjust the estimated color temperature of the light source in the area,
A multi-area white balance control method that generates a white balance correction gain map of the entire input image based on the adjusted color temperature of the light source in each area. A computer provided in a multi-area white balance control device,
The computer,
Area dividing means for dividing the input image into a plurality of areas;
Area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means,
Based on the comparison between the color temperature of the light source of the area estimated by the area light source color temperature estimation means and the color temperature of the light source of the area adjacent to the area, the adjustment of the estimated color temperature of the light source of the area is performed. Area estimation light source that adjusts the color temperature of the estimated light source of the area based on the color temperature of the estimated light source of the area adjacent to the area when it is determined whether adjustment is necessary. Color temperature adjusting means,
Multi-area white for functioning as white balance correction gain map generation means for generating a white balance correction gain map of the entire input image based on the color temperature of the light source of each area adjusted by the area estimation light source color temperature adjustment means A computer that records a balance control program. A computer-readable multi-area white balance control program provided in the multi-area white balance control device,
The computer,
Area dividing means for dividing the input image into a plurality of areas;
Area light source color temperature estimating means for estimating the color temperature of the light source for each area divided by the area dividing means,
Based on the comparison between the color temperature of the light source of the area estimated by the area light source color temperature estimation means and the color temperature of the light source of the area adjacent to the area, the adjustment of the estimated color temperature of the light source of the area is performed. Area estimation light source that adjusts the color temperature of the estimated light source of the area based on the color temperature of the estimated light source of the area adjacent to the area when it is determined whether adjustment is necessary. Color temperature adjusting means,
Multi-area white for functioning as white balance correction gain map generation means for generating a white balance correction gain map of the entire input image based on the color temperature of the light source of each area adjusted by the area estimation light source color temperature adjustment means Balance control program.

Images