JP6423669B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly, to an imaging apparatus that controls gradation characteristics and a control method thereof.

  2. Description of the Related Art Conventionally, a technique has been proposed in which gradation control is performed on a scene with a wide dynamic range represented by a backlight scene or the like so as to suppress overexposure and blackout and approach the photographer's appearance. When a person is photographed under backlight conditions, the person is photographed very darkly as shown in FIG. 15A in normal photographing. On the other hand, a process has been proposed for reducing the darkness of a person in an output image by applying a gain to the darkened person region. However, when such processing is performed, the gain applied to the dark person portion increases particularly in a scene where the degree of backlight is strong, so that noise is amplified as shown in FIG. There is a problem of lowering.

  As another means, there has been proposed a process that compensates for brightness by irradiating a person with flash light and reduces the darkness of the person in the output image. However, when such a process is performed, the amount of flash emission increases particularly in a scene where the degree of backlight is strong, and as shown in FIG. There is a problem that the image quality deteriorates.

  To deal with the above-described problems, Patent Documents 1 and 2 propose a gradation control technique for obtaining a natural output image close to a photographer's appearance by controlling a gain amount and a flash emission amount applied to a dark part. .

  In Patent Document 1, the gradation correction characteristic and the exposure reduction amount associated therewith are set from the luminance value of a predetermined area such as the face area, and at the time of flash emission, according to the degree of overexposure of the predetermined area before and after gradation correction. It is disclosed that the amount of light emission is reduced when the overexposure increases.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses that in the dark part correction mode, the sensor is divided into a central part and a peripheral part, and the correction degree of dark part correction is calculated from the obtained luminance information of a plurality of regions. Further, it is disclosed that the flash light emission amount is adjusted so that the light emission amount decreases as the correction amount increases in accordance with the correction amount.

JP 2011-119944 A JP 2009-044221 A

  However, the techniques described in Patent Documents 1 and 2 do not mention measures for reducing the unnaturalness caused by the above-described flash photography (flash photography) when calculating the flash emission amount. Therefore, depending on the scene, there is a possibility that unnaturalness due to flash, such as shine, may occur.

  In the techniques described in Patent Documents 1 and 2, tone characteristics are calculated from signal values of a specific area in an image and areas divided into predetermined areas such as a central part and a peripheral part. For this reason, it is not possible to calculate the optimum gradation characteristics for a meaningful object (for example, a person, background, sky, etc.) present in the shooting scene, and there is a possibility of performing gradation control different from the appearance of the photographer. .

  The present invention has been made in view of the above problems, and an object thereof is to obtain a more natural image in flash photography.

In order to achieve the above object, an imaging apparatus according to the present invention extracts an object region that satisfies a predetermined condition from an image capturing unit that captures an image of a subject and outputs the image, and extracts the subject region that satisfies a predetermined condition from the image. Based on the information on the subject area and the extraction means for acquiring the information on the presence or absence of light emission of the illuminating device and when the illuminating device emits light, the light emission upper limit amount determined in advance based on the image is set as the upper limit. and determining means for determining a light emission amount, the image taken by controlling the lighting device in determined conditions by the determination means, a gradation conversion unit that performs gradation conversion, information of the object area And calculating means for calculating a gain amount used for the gradation conversion based on the condition determined by the determining means.

  According to the present invention, a more natural image can be obtained in flash photography.

1 is a schematic configuration diagram of an imaging apparatus according to an embodiment of the present invention. 5 is a flowchart showing a flow of imaging processing in the embodiment. The figure which shows an example of a backlight scene. The figure for demonstrating the area | region determination process in 1st Embodiment. 5 is a flowchart of imaging condition setting processing in the first embodiment. The conceptual diagram which shows the calculation method of person gain GAIN_HUMAN in 1st Embodiment. 6 is a flowchart showing a flow of flash light emission amount determination processing in the first embodiment. The conceptual diagram which shows the calculation method of the flash light emission amount in 1st Embodiment. The figure which shows the recovery area | region at the time of pre light emission. The graph which shows the relationship between flash light-emission amount and the number of pixels. 6 is a flowchart of processing for calculating a light emission upper limit amount Delta_EF_MAX in the first embodiment. The figure which shows an example of a scene with a dark background area | region. The figure for demonstrating the acquisition method of the light emission amount adjustment coefficient k_EF in 1st Embodiment. The figure which shows an example of the gain calculation method according to brightness | luminance in 2nd Embodiment. The figure for demonstrating the conventional subject.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 shows a schematic configuration of a camera as an example of an imaging apparatus according to an embodiment of the present invention. In FIG. 1, the first lens group 101 is disposed at the tip of the imaging optical system and is held so as to be able to advance and retract in the optical axis direction. The aperture 102 adjusts the aperture diameter to adjust the amount of light during shooting, and also has a function as an exposure time adjustment shutter when shooting a still image. The second lens group 103 moves forward and backward in the optical axis direction integrally with the diaphragm / shutter 102, and a zooming function (zoom function) can be realized in conjunction with the forward / backward movement of the first lens group 101.

  The third lens group 105 (focus lens) performs focus adjustment by advancing and retreating in the optical axis direction. The optical low-pass filter 106 is an optical element for reducing false colors and moire in the captured image. The image sensor 107 is composed of a two-dimensional CMOS photosensor and its peripheral circuit, and is disposed on the imaging surface of the imaging optical system.

  The zoom actuator 111 rotates a cam cylinder (not shown) to drive the first lens group 101 to the third lens group 103 forward and backward in the optical axis direction, and performs a zooming operation. The aperture shutter actuator 112 controls the aperture diameter of the aperture 102 to adjust the amount of photographing light and controls the exposure time during still image photographing. The focus actuator 114 adjusts the focus by driving the third lens group 105 back and forth in the optical axis direction.

  The flash 115 for illuminating the subject at the time of photographing is preferably a flash illumination device using a xenon tube, but an illumination device including an LED capable of continuous light emission (continuous light emission) may be used. The AF auxiliary light emitting unit 116 projects a mask image having a predetermined opening pattern onto the object field via the projection lens, and improves the focus detection capability for a dark subject or a low-contrast subject.

  The in-camera CPU 121 manages various controls of the camera body, and includes a calculation unit, ROM, RAM, A / D converter, D / A converter, communication interface circuit, and the like. The CPU 121 drives various circuits included in the camera based on a predetermined program stored in the ROM, and executes a series of operations such as AF, photographing, image processing, and recording. Further, by executing a predetermined program, it functions as a main subject information calculation unit 203 and an imaging condition setting unit 204. The operations of the main subject information calculation unit 203 and the imaging condition setting unit 204 will be described later in detail.

  The flash control circuit 122 controls lighting of the flash 115 in synchronism with the photographing operation under the condition determined by the CPU 121 by processing to be described later. The auxiliary light driving circuit 123 controls the lighting of the AF auxiliary light emitting unit 116 in synchronization with the focus detection operation. The image sensor driving circuit 124 controls the image capturing operation of the image sensor 107, A / D converts the acquired image signal, and transmits it to the CPU 121. The image processing circuit 125 performs processing such as γ conversion, color interpolation, and JPEG compression of the image acquired by the image sensor 107.

  The focus drive circuit 126 controls the focus actuator 114 based on the focus detection result, and adjusts the focus by driving the third lens group 105 back and forth in the optical axis direction. The aperture shutter drive circuit 128 controls the aperture of the aperture / shutter 102 by drivingly controlling the aperture shutter actuator 112. The zoom drive circuit 129 drives the zoom actuator 111 according to the zoom operation of the photographer.

  A display 131 such as an LCD displays information on the shooting mode of the camera, a preview image before shooting and a confirmation image after shooting, a focus state display image at the time of focus detection, and the like. The operation switch group 132 includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The detachable flash memory 133 records captured images.

  Next, processing in the imaging apparatus having the above configuration will be described with reference to the flowchart of FIG. In addition, in order to assume correction | amendment of a backlight scene in 1st Embodiment, the example of the backlight scene assumed in 1st Embodiment is shown in FIG. As shown in FIG. 3, the backlight scene is a scene where the area (background, sky, etc.) other than the main subject 201 is extremely bright with respect to the brightness of the main subject 201 (person).

  First, pre-photographing is performed in S101. In pre-photographing, prior to main photographing described later, the image sensor 107 and the flash 115 are controlled to acquire a pre-photographed image for determining flash emission conditions and exposure conditions particularly during main photographing. As the pre-photographed image, a non-light-emitting image without flash emission and a light-emission image with flash emission are acquired one by one. Hereinafter, the non-light-emitting image is referred to as a non-light-emitting pre-image, and the light-emitting image is referred to as a light-emitting pre-image. Note that the pre-emission image is captured by causing the flash 115 to emit light by a predetermined emission amount Delta_EF_PRE without changing the exposure conditions with respect to the non-emission pre-image.

  Next, in S102, the image processing circuit 125 performs predetermined signal processing on the non-light emitting pre-image and the light emitting pre-image captured in S101. As an example of the predetermined signal processing, defective pixel correction of the image sensor 107, peripheral light amount drop correction, noise reduction processing, and the like can be given.

  In step S103, the main subject information calculation unit 203 divides the non-light-emitting pre-image that has been subjected to the predetermined signal processing in step S102 into a main subject region and other regions based on predetermined conditions. Get subject information. Hereinafter, in the first embodiment, a main subject is a person, and a region other than the main subject is called a non-main region. The main subject information is a representative luminance value of the main subject area (person area). Here, the process of the main subject information calculation unit 203 performed in S103 will be described in detail.

  First, the non-light emitting pre-image is divided into a person area and a non-main area. In this process, the main subject information calculation unit 203 first divides the block into blocks [i] [j] having a predetermined size as shown in FIG. Then, an area determination process is performed to determine whether each block Block [i] [j] is included in a person area or a non-main area. The method of region determination processing is not limited, but the region determination processing can be performed based on, for example, the representative luminance value or representative color information of each block Block [i] [j]. Further, the size of the block Block [i] [j] is not particularly limited, and for example, the area determination process may be performed on a pixel basis.

  When it is determined whether each block Block [i] [j] is included in the person area or the non-main area, 0-2 are assigned according to the determination result, for example, as shown in FIG. . Here, if the determination result of the block Block [i] [j] is a person, Block_id [i] [j] of the block is 0, and if it is a non-main subject, Block_id [i] [j] is 1 Is not any area (not applicable), 2 is assigned to Block_id [i] [j]. Here, “not applicable” refers to a portion where it is difficult to discriminate the region, such as a boundary portion of each region. Then, the blocks assigned with the same value can be grouped into a person area and a non-main area. That is, the blocks to which 0 is assigned are collectively set as the person area, and the blocks to which 1 is assigned are collectively set as the non-main subject. In the present embodiment, a block to which 2 is assigned is not used.

Next, the representative luminance value of the person area is calculated. The representative luminance value is an average luminance value of a set of divided blocks determined as a person area in the area dividing process. Y_HUMAN_PRE is calculated by the following equation (1), where Y_HUMAN_PRE is the representative luminance value of the person area in the non-light emitting pre-image.
Y_HUMAN_PRE = Σ (W_HUMAN [i] [j] × Block_Y [i] [j]) / ΣW_HUMAN [i] [j]
(Only blocks with Block_id [i] [j] = 0) (1)
Here, Block_Y [i] [j] is an average luminance value of each block, and W_HUMAN [i] [j] is a weighting coefficient for each block, and can be arbitrarily set. However, here, W_HUMAN [i] [j] is set to 1 for all (i, j) in order to calculate the average luminance of all blocks determined to be a human area.

  With the above processing, the representative luminance value Y_HUMAN_PRE of the person area in the non-light emitting pre-image is obtained as the output of the main subject information calculation unit 203.

  In step S <b> 104, the imaging condition setting unit 204 determines the imaging conditions for the main shooting based on the representative luminance value Y_HUMAN_PRE of the person in the non-light emitting pre-image obtained from the main subject information calculation unit 203. Here, the imaging conditions include exposure conditions at the time of actual photographing, a gain amount applied to a person area, presence / absence of flash emission, and flash emission amount in the case of flash emission.

  Here, the operation of the imaging condition setting unit 204 will be described using the flowchart of the imaging condition setting process shown in FIG. Since the backlight scene is assumed in the first embodiment, in the following description, it is assumed that the person area is darker than the non-main area as a typical brightness relationship at the time of backlight.

  First, in S201, a gain (hereinafter referred to as “person gain”) that compensates for the brightness of a person area that has been darkened in a backlight state as shown in FIG. It should be noted that the exposure conditions in the main photographing are set in advance.

FIG. 6 is a conceptual diagram showing a method for calculating the person gain GAIN_HUMAN. The person gain GAIN_HUMAN is a gain for raising the representative brightness value Y_HUMAN_HON of the person area in the actual shooting to a predetermined target brightness value Y_HUMAN_OUT. As can be seen from FIG. 6, assuming that the exposure difference between pre-photographing and main-photographing is dBv, Y_HUMAN_HON and GAIN_HUMAN are calculated by the following equations (2) and (3) from the representative luminance value Y_HUMAN_PRE of the person area in the non-light-emitting preimage. be able to.
Y_HUMAN_HON = Y_HUMAN_PRE × 2 ^ (dBv)… (2)
GAIN_HUMAN = Y_HUMAN_OUT / Y_HUMAN_HON (3)

  As described above, when the person gain GAIN_HUMAN at the time of actual photographing is determined in S201, the flash light emission amount is calculated in S202. Here, the setting of ON / OFF of the flash 115 at the time of actual photographing and the light emission amount Delta_EF when the flash 115 is ON are calculated. Here, the flash light emission amount calculation processing will be described with reference to the flowchart of FIG.

  First, in S301, the person gain GAIN_HUMAN is compared with the maximum gain amount GAIN_MAX. Here, the maximum gain amount GAIN_MAX indicates a gain amount corresponding to a limit value as to whether or not the amount of noise to be amplified is visually acceptable, and is a value set in advance from the performance of the image sensor 107 or the like.

  If the human gain GAIN_HUMAN is less than or equal to the maximum gain amount GAIN_MAX, this indicates that the noise level of the output image will be within the allowable level when the desired tone conversion processing using the gain is performed in actual shooting, and flash emission is not required. is there. Therefore, it is determined that there is no flash emission in S302, and this process ends. On the other hand, when the person gain GAIN_HUMAN is larger than the maximum gain amount GAIN_MAX, the noise amount of the output image exceeds the allowable level as it is. Therefore, the amount of noise is suppressed by supplementing the brightness of the person area with flash emission and reducing the value of the person gain GAIN_HUMAN. That is, only when GAIN_HUMAN> GAIN_MAX in S301, it is determined that the flash 115 is emitted, and only when it is determined that the flash 115 is emitted, the processing after S303 is performed.

  However, when the flash 115 emits light with an amount of light more than necessary, overexposure occurs due to reflection of the flash light from a person, so-called “shininess” occurs, and image quality is deteriorated. Based on the above, it is necessary to control to obtain an output image in which the amount of noise falls within an allowable level while minimizing the amount of gloss.

  In S303, the flash emission amount at the time of actual photographing is calculated. Regarding the calculation of the flash emission amount, a conventional EF process is known in which the reflected light component of the subject is calculated from the emission pre-image and the non-emission pre-image, and the flash emission amount is calculated from the reflected light component and the target luminance value of the subject. ing. Also in the first embodiment, basically the same processing as the conventional EF processing is performed. Here, it is necessary to appropriately set the target luminance value of the subject at the time of calculating the flash emission amount so as to minimize the flash caused by the flash light.

FIG. 8 is a conceptual diagram showing a method for calculating the flash emission amount. As shown in FIG. 8, the brightness of the person is compensated by the flash light, and the representative luminance value Y_HUMAN_HON_FL of the person area at the time of flash emission becomes brighter than that at the time of non-light emission. Therefore, the gain of the person area is recalculated. In the present embodiment, the light emission amount is set so that the recalculated person gain GAIN_HUMAN is equal to the maximum gain amount GAIN_MAX. That is, the light emission amount that can compensate for the difference between the human gain GAIN_HUMAN before recalculation and the maximum gain amount GAIN_MAX is set. In the above case, as can be seen from FIG. 8, Y_HUMAN_HON_FL can be obtained by the following equation (4).
Y_HUMAN_HON_FL = Y_HUMAN_OUT / GAIN_MAX (4)
The flash emission amount Delta_EF is determined by EF processing so that the representative luminance value of the person area in the actual captured image is Y_HUMAN_HON_FL expressed by Expression (4).

  When the flash light emission amount Delta_EF is determined in S303 by the above processing, the light emission upper limit amount Delta_EF_MAX is calculated in S304. Here, the light emission upper limit amount is the maximum flash light emission amount that allows the flash light to fall within an allowable level. In S303, since the flash emission amount is calculated from the maximum gain amount GAIN_MAX, when the subject is photographed with the calculated flash emission amount, the subject may exceed the permissible level and may cause image quality degradation. Therefore, it is necessary to estimate the upper limit value of the flash light emission amount from the viewpoint of subject shine.

  9 to 11 are diagrams showing the processing contents in S304. Here, using the non-emission pre-image shown in FIG. 9 (a) and the emission pre-image shown in FIG. 9 (b), Delta_EF_MAX that is the maximum light emission amount that can be tolerated is calculated. Note that, as described above, the light emission pre-image is a non-light emission pre-image captured by causing the flash 115 to emit light by a predetermined light emission amount Delta_EF_PRE without changing the exposure conditions.

Further, paying attention to an area irradiated with flash light (mainly a person area), this area is called a flash irradiation area (FIG. 9C). The flash irradiation area is an area where the luminance value is greatly changed by the flash irradiation, and the pixel values at the positions (i, j) of the non-light emitting pre-image and the light emitting pre-image are respectively set to N_nfl [i] [j] and N_flpre [ i] [j]
(N_flpre [i] [j]-N_nfl [i] [j]) / N_nfl [i] [j]> TH (5)
This is a set of pixels at a position (i, j) that satisfies. Here, TH in Equation (5) is a predetermined threshold value and is set in advance. The reason why normalization is performed with the pixel value N_nfl [i] [j] of the non-light-emitting pre-image is that the larger the original pixel value, the greater the difference in the pixel value. .

  Next, the number of pixels N_TEKARI_PRE in the light emission pre-image is calculated. Here, the bright pixel is a pixel whose pixel value exceeds a predetermined level TH_TEKARI in the flash irradiation region of the luminescent image. The number of pixels to be taken N_TEKARI_PRE is the number of pixels that are determined to be taken pixels by performing the above threshold processing on all the pixels included in the flash irradiation region.

  Next, the number of pixels N_TEKARI_PRE when the flash emission amount is increased is calculated. When the flash light emission amount is increased, qualitatively, it is considered that the number of leading pixels increases as shown in FIG. Therefore, the flash emission amount Delta_EF_MAX when the number of pixels N_TEKARI_PRE reaches the predetermined maximum allowable number N_TEKARI_MAX is the upper limit of the flash emission amount estimated from the viewpoint of subject lighting (hereinafter referred to as “emission upper limit amount”). It becomes. Note that when the number of pixels N_TEKARI_PRE in the light emission pre-image is larger than the maximum allowable number N_TEKARI_MAX, it means that the light emission amount of the pre-light emission is too large. A luminescent pre-image is acquired. In this manner, the number of pixels N_TEKARI_PRE in the light emission pre-image is made smaller than the maximum allowable number N_TEKARI_MAX.

  FIG. 11 is a flowchart of a process for calculating the light emission upper limit amount Delta_EF_MAX. First, in S401, the initial value of the flash light emission amount Delta_EF_tmp is set to a predetermined light emission amount Delta_EF_PRE used for photographing the light emission pre-image. Next, in S402, a predetermined increment dFL is added to Delta_EF_tmp, and in S403, the number N_TEKARI of estimated pixels when the light emission amount is Delta_EF_tmp is calculated.

  The number of leading pixels N_TEKARI_PRE is calculated under the assumption that the rate of change of the pixel value in the flash irradiation area represented by Expression (5) is proportional to the flash emission amount. In this case, if the pixel value at the coordinates (i, j) at the time of flash emission amount Delta_EF_tmp is N_fl [i] [j], the following equation is established from the proportional relationship with pre-emission.

Delta_EF_tmp: Delta_EF_PRE
= (N_fl [i] [j] -N_nfl [i] [j]) / N_nfl [i] [j]: (N_flpre [i] [j] -N_nfl [i] [j]) / N_nfl [i] [ j] ... (6)
Solving Equation (6) for N_fl [i] [j]
N_fl [i] [j] = N_nfl [i] [j] + (Delta_EF_tmp / Delta_EF_PRE) * (N_flpre [i] [j] -N_nfl [i] [j])… (7)
It becomes. Next, the pixel value N_fl [i] [j] calculated from Expression (7) is compared with a predetermined level TH_TEKARI to determine whether it is a dead pixel.

  The above-described processing is performed on all the pixels included in the flash irradiation area, and the pixel value N_fl [i] [j] exceeds the predetermined level TH_TEKARI, and the number of pixels determined to be a defective pixel is determined as the number of detected pixels N_TEKARI. And Next, in S404, a comparison process between the number of pixels N_TEKARI and a predetermined maximum allowable number N_TEKARI_MAX is performed. When the number of pixels N_TEKARI is less than or equal to the maximum allowable number N_TEKARI_MAX, the processes of S402 to S404 are repeated. On the other hand, when the number of detected pixels N_TEKARI is larger than the maximum allowable number N_TEKARI_MAX, in S405, Delta_EF_tmp-dFL, which is the light emission amount immediately before the number of detected pixels N_TEKARI becomes larger than the maximum allowable number N_TEKARI_MAX, is set as the emission upper limit amount Delta_EF_MAX. To do.

  With the above processing, in S304, the light emission upper limit amount Delta_EF_MAX of the flash 115 that allows the subject to be lit is obtained.

  Next, in S305, the light emission amount adjustment coefficient k_EF is calculated. Here, the reason for adjusting the flash emission amount Delta_EF obtained in S303 is as follows. As shown in FIG. 12 (a), in a scene where there are many dark areas not only in the person area but also in non-main areas, even if the flash 115 is emitted, as shown in FIG. There is a high possibility that the flash will not reach the main area. Therefore, the brightness of the non-main area 1201 cannot be supplemented with flash light, and a desired effect cannot be obtained. For this reason, in a scene where there are many dark areas equivalent to the person area in the non-main area, it is considered necessary to control to reduce the flash emission amount.

  FIG. 13A is a luminance histogram of a non-main area. As shown in FIG. 13A, a luminance area having a luminance value equal to or lower than a threshold TH0 is defined as a dark area, and the ratio of the number of pixels in the dark area to the number of pixels in the entire non-main area is calculated. The calculated ratio is called a dark pixel ratio. FIG. 13B is a graph showing the relationship between the dark portion pixel ratio and the light emission amount adjustment coefficient k_EF. The abscissa represents the dark pixel ratio, and the smaller the dark pixel ratio, the more dark pixels are not present in the non-major region, and the larger the dark pixel ratio is, the dark pixels are present in the non-main region. . As described above, it is considered that it is preferable to decrease the light emission amount as the number of dark portion pixels is larger. Therefore, the light emission amount adjustment coefficient k_EF is set to be smaller as the dark portion pixel ratio is larger.

  Next, in step S306, the flash light emission amount is adjusted using the light emission upper limit amount Delta_EF_MAX and the light emission amount adjustment coefficient k_EF. In the adjustment, when the flash light emission amount Delta_EF obtained in S303 is larger than the light emission upper limit amount Delta_EF_MAX, the flash light emission amount Delta_EF is clipped to the light emission upper limit amount Delta_EF_MAX. Next, the flash emission amount Delta_EF after the clipping process is multiplied by the emission amount adjustment coefficient k_EF. The value k_EF * Delta_EF calculated by the above processing is output as the final flash emission amount Delta_EF_FIX that takes into account the effect of subject lighting and the presence of many dark areas in non-main areas.

  When the process of S302 or S306 is completed, the process in S202 of FIG. 4 ends, and the process proceeds to S203.

Next, in S203, the exposure condition during the main photographing is adjusted. Here, the reason why the exposure conditions need to be adjusted is as follows. Although the exposure conditions at the time of actual photographing are calculated in advance, the flash light emission amount may be adjusted in S202, and the light emission amount of Delta_EF_FIX may be smaller than Delta_EF. Therefore, even if the maximum gain GAIN_MAX is applied to the human area, the target brightness may not be reached.In this case, the exposure condition is set to overexposure from the initial setting, and the brightness is increased to compensate for the insufficient brightness. Take a picture. Therefore, if the exposure condition adjustment amount is Delta_Bv_FL,
Delta_Bv_FL = Delta_EF-Delta_EF_FIX (8)
It becomes. Adjustment is made so that the exposure condition during the actual photographing is over by the amount of Delta_Bv_FL represented by Expression (8).

  With the above-described processing, in S104, the imaging condition setting unit 204 determines the shooting conditions including the exposure conditions at the time of actual shooting, the gain amount applied to the person area, the presence or absence of flash emission, and the flash emission amount in the case of flash emission. Is done.

  Next, in S105, actual shooting is performed according to the imaging conditions determined by the imaging condition setting unit 204 in S104. Hereinafter, an image obtained by actual photographing is referred to as an actual photographed image. Next, in S106, the image processing circuit 125 performs predetermined signal processing on the actual captured image captured in S105. Here, examples of the predetermined signal processing include defective pixel correction, peripheral light amount drop correction, noise reduction processing, and the like of the image sensor 107.

  In step S107, the main subject information calculation unit 203 calculates main subject information of the actual captured image. The main subject information is subject region division information, a representative luminance value of the person region, and a gain value of the person region, similar to the information obtained from the non-light-emitting pre-image in S103. Here, the reason why the main subject information is calculated again is mainly the following two points. The first point is that the image to be captured changes due to the movement of the subject and the camera shake of the photographer because there is a shift in the imaging timing between the pre-photographing and the main photographing. Secondly, in the main subject information calculation process in S103, region division is performed to calculate the gain value of the person region, but in the main subject information calculation process in S107, gradation conversion processing is performed on the main photographed image. It is necessary to divide the region in consideration of the above. Therefore, for example, it is conceivable to improve the accuracy of area division. As an example, the area is divided by a pixel unit or a block smaller than the block obtained by dividing the area in S103. In the main subject information calculation process in S107, the result of the main subject information calculation process in S103 may be used as it is.

  Next, in S108, predetermined gradation conversion processing is performed on the actual captured image. The gradation conversion process is a process for controlling each subject area in the actual captured image to have a predetermined brightness. Here, the predetermined brightness is the target luminance value Y_HUMAN_OUT of the person area as shown in FIG. In the gradation conversion processing in the first embodiment, the human gain amount GAIN_HUMAN (during non-light emission) or GAIN_MAX (during light emission) is uniformly multiplied with respect to the human region in the actual captured image.

  Finally, the output image generated in S108 is sent to the display 131 and / or the flash memory 133 for display and / or recording.

  As described above, according to the first embodiment, adaptive gradation control is performed according to the input image while appropriately controlling the flash light emission amount, and a more natural output image can be obtained.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. In the second embodiment, the processing content of the gradation conversion processing in S108 is different from that in the first embodiment. In the gradation conversion process in the first embodiment, a uniform gain is multiplied to the person area. However, in such processing, it is necessary to accurately extract the person area so as not to multiply the non-main area by a gain. On the other hand, in the second embodiment, gradation conversion processing is performed more simply without requiring precise person region extraction. Specifically, similarly to the conventional digital dodging process, the brightness-specific gain process is performed.

  In other words, in the second embodiment, in the main subject information calculation process of S107 in FIG. 2, the person area extraction similar to S103 is performed on the main captured image, the representative brightness value of the person area is calculated, and the brightness Processing for generating another gain characteristic is performed.

  FIG. 14 is a diagram illustrating an example of calculation of luminance-specific gain characteristics. As shown in FIG. 14, the gain-specific gain characteristic is generated based on the person gain calculated from the input image and the representative brightness value of the person. Also in the second embodiment, a backlight scene as shown in FIG. 3 is assumed. In a typical backlight scene, the brightness of the person region tends to be smaller than the surroundings. On the other hand, it is considered that a uniform gain should be applied to the person area as much as possible in order to maintain the contrast by leaving the shadow of the person's face. Therefore, as shown in FIG. 14, an area having a luminance equal to or lower than Y_HUMAN_HON_FL, which is the representative luminance value of the person area, is uniformly multiplied by GAIN_HUMAN. On the other hand, in a region having a luminance equal to or higher than Y_HUMAN_HON_FL, a luminance-specific gain having characteristics such that the gain gradually approaches 1.0 as the luminance increases is generated. The actual captured image is multiplied by the luminance-specific gain generated by such a method to obtain a desired output image.

  As described above, according to the processing shown in the second embodiment, the desired output image can be more easily compared with the first embodiment by performing the gradation conversion processing using the gain for each luminance. Can be obtained.

  107: Image sensor, 115: Flash, 121: CPU, 122: Flash control circuit, 125: Image processing circuit

Claims (12)

Imaging means for photographing a subject and outputting an image;
Extraction means for extracting a subject area that satisfies a predetermined condition from the image and acquiring information of the extracted subject area;
Based on the information of the object area, the presence or absence of light emission of the lighting device and, in case of emitting the illumination device, the light emission limit amount as an upper limit determined in advance based on the image, determining means for determining the light emission amount ,
Gradation conversion means for performing gradation conversion on an image captured by controlling the illumination device under the conditions determined by the determination means;
An imaging apparatus comprising: a calculation unit that calculates a gain amount used for the gradation conversion based on the information on the subject region and the condition determined by the determination unit.   The calculation unit calculates a gain amount based on the information on the subject area, and the determination unit determines the lighting device when the gain amount obtained by the calculation unit is larger than a predetermined maximum gain amount. The imaging apparatus according to claim 1, wherein the light emission amount is determined by determining that the light emission amount is to be emitted, and the calculation unit recalculates a gain amount according to the light emission amount.   The imaging apparatus according to claim 2, wherein the calculation unit calculates a gain amount for correcting the luminance of the subject region to a predetermined luminance based on the information on the subject region.   The imaging apparatus according to claim 2, wherein the maximum gain amount is a value determined based on a noise amount amplified by the calculated gain amount.   5. The determination unit according to claim 2, wherein, when the calculated gain amount is larger than the maximum gain amount, the determination unit obtains a light emission amount that compensates for a difference between the gain amount and the maximum gain amount. The imaging device according to any one of the above.   6. The determination unit according to claim 2, wherein when the calculated gain amount is equal to or less than the maximum gain amount, the determination unit determines that the lighting device does not emit light. Imaging device.   The light emission upper limit amount is a maximum light emission amount in which the number of pixels having a pixel value exceeding a predetermined value in an image obtained by causing the lighting device to emit light is equal to or less than a predetermined number. The imaging device according to any one of claims 1 to 6.   The said determination means reduces the said light emission amount, so that the ratio of the pixel which has a pixel value below a predetermined value in the area | region except the said object area | region among the said images is large. The imaging device according to any one of the above.   Based on the brightness of the image, the exposure condition used for shooting is determined, and the brightness of the subject area obtained when the subject is shot using the light emission amount and the gain amount is higher than a predetermined brightness. 9. The image pickup apparatus according to claim 1, further comprising means for adjusting the exposure condition so as to compensate for insufficient luminance when low.   The imaging apparatus according to claim 1, wherein the gradation conversion unit applies a gain of the same gain amount to the signal of the subject area.   The imaging apparatus according to claim 1, wherein the calculating unit calculates a gain amount for each luminance. An imaging process in which an imaging unit images a subject and outputs an image;
An extracting step of extracting a subject area that satisfies a predetermined condition from the image and obtaining information of the extracted subject area;
Determining means, based on information of the object area, the presence or absence of light emission of the lighting device and, in case of emitting the illumination device, the light emission limit amount as an upper limit determined in advance based on the image, determining a light emission amount A decision process to
A gradation conversion step in which gradation conversion means performs gradation conversion on an image captured by controlling the illumination device under the conditions determined in the determination step;
And a calculation step of calculating a gain amount used for the gradation conversion based on the information on the subject region and the condition determined by the determination step.