JP4307341B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing method and an image processing apparatus that correct and output an image captured by an image input device.

  An image photographed by an image input device such as a digital camera that has become popular in recent years can be observed and viewed by displaying it on a monitor or printing it on paper or the like. At this time, when the photographed image data is used as it is, it is often expressed in a state where the photographing balance is not necessarily good, such as color covering, inappropriate exposure, or inappropriate contrast. .

  Therefore, conventionally, an application that defines a good image and automatically performs white balance correction and exposure / contrast correction in accordance with the definition has been generally performed.

  However, even in such an environment, there may be a captured image that cannot always be corrected appropriately. For example, when a person is photographed at a ski resort or when a small target is photographed in a landscape where a lot of blue sky is photographed, the region of interest is originally a small region in the image, even though it is a region of interest. May be limited due to the gradation of the areas having different luminance in other areas. In such a case, accurate correction is impossible by conventional image correction using the above-described definition of a good image.

  In order to improve such captured images, the following techniques have been proposed. For example, Japanese Patent Laid-Open No. 2002-135587 (patent document 1) extracts an important region such as a face of a person of interest from an image, and corrects the entire image if the luminance of the region is not preferable. The detected area is adjusted in luminance and synthesized by the principle of dodging.

Japanese Patent No. 3203946 (Patent Document 2) is a television video signal that detects an average skin color luminance level from a color difference signal and a luminance signal, and sends a control signal to a gamma correction circuit in relation to the luminance level of all images. It is something to add.
Japanese Patent Laid-Open No. 2002-135587 Japanese Patent No. 3203946

  However, the conventional example described above has the following problems.

  First, in the technique of Patent Document 1, for example, when a face area of a person is a detection target, if the face area is extremely dark, an unnatural gradation area is generated between the clothes and the clothes. . Further, when executing this image processing, a large amount of computation and memory capacity are required for area correction for the composition processing and region composition. Embedded devices that have an image processing unit in the printer, such as a printer that is directly connected to the digital camera without going through a PC, or a memory card interface that goes without going through a PC However, there is a problem that it is difficult to increase the arithmetic processing capability in terms of the price of the printer and it is difficult to mount a large capacity memory high speed CPU.

  On the other hand, since the technique of Patent Document 2 applies gamma correction based on the luminance level of a signal corresponding to the color range, the skin color in the assumed video is human skin, and the target is only human skin. This is possible if considered. However, when there is an object in the image that cannot originally become the main subject such as a dead forest, the object may be optimized. In addition, since all skin colors in the video are targeted, if there are skin-colored objects that are not human skin in the video, they are also affected, and the accuracy of correction for viewing and printing is high. It may not be satisfactory. Furthermore, since a color range in a luminance range with a poor S / N ratio is also subject to correction, there may be a situation where improvement in image quality cannot be expected even by correction.

  Further, the correction is to develop the main subject itself to a desired correction value by using detection of the main subject. However, in buildings with a lot of bright blue sky and buildings in nature, the building itself is the main subject. That was difficult.

  The present invention has been made in consideration of such circumstances, and by performing individual correction of a specific area extracted from a captured image and reflecting it in the correction of the entire image, it is possible to achieve a good amount of information processing. An object is to provide an image processing method and an image processing apparatus capable of realizing image correction.

In order to solve the above problems, an image processing method according to the present invention includes:
  An acquisition process for acquiring images;
  An extraction step of extracting a human skin region in the image based on chromaticity;
  A first analysis step of analyzing the luminance value of the image and calculating an average luminance value of the entire image;
  A second analysis step of calculating an average luminance value of the skin region based on a luminance value of a pixel included in the skin region extracted by the extraction step;
  A correction coefficient is determined based on the average luminance value of the entire image and the average luminance value of the skin area, and the luminance value after correction of the skin area is obtained by multiplying the average luminance value of the skin area by the correction coefficient. A calculation step of calculating
  Based on the calculation result of the calculation step, a table creation step of creating a gradation correction table in which an average luminance value of the skin region is converted into a luminance value after correction of the skin region;
  A correction step of correcting the brightness of the image using the created gradation correction table;
  It is characterized by having.

In order to solve the above problem, an image processing apparatus according to the present invention provides:
  An acquisition means for acquiring an image;
  Extraction means for extracting a skin area of a person in the image based on chromaticity;
  First analysis means for analyzing a luminance value of the image and calculating an average luminance value of the entire image;
  A second analysis unit that calculates an average luminance value of the skin region based on a luminance value of a pixel included in the skin region extracted by the extraction unit;
  A correction coefficient is determined based on the average luminance value of the entire image and the average luminance value of the skin area, and the luminance value after correction of the skin area is obtained by multiplying the average luminance value of the skin area by the correction coefficient. Calculating means for calculating
  Table creation means for creating a gradation correction table in which an average luminance value of the skin area is converted into a luminance value after correction of the skin area based on a calculation result of the calculation means;
  Correction means for correcting the brightness of the image using the created gradation correction table;
  It is characterized by providing.

  The present invention employs a conversion table method for gradation correction to simplify the processing, and in a region where the main subject is too dark or too bright above a specific level and is likely to be affected by crushing and noise. If it is determined that the improvement effect cannot be expected even after a certain gradation correction, the feature area is not used. A correction target value is set and a gradation correction table is created.

  Further, the present invention detects a human skin or a human face region as a main subject region, and a method for creating a gradation correction table reflecting the feature amount is described in an extreme step of a region other than the detected main subject region. Use the target setting judgment table based on the average brightness of the main subject area and the average brightness of the areas other than the main subject area so that negative effects on other areas such as tone reduction and collapse are not generated. The correction target value is determined. For example, a gradation correction table passing through the correction target value and three points (0, 0) and (255, 255) is created.

  Further, according to the present invention, as means for ensuring the gradation of the main subject area, means for determining the gradation range of the main subject area using the luminance distribution information of the main subject area and setting an upper limit value and a lower limit value thereof In the gradation correction table to be created, an upper limit value and a lower limit value and (0, 0) and a gradation correction table passing through four points (255, 255).

  Furthermore, the present invention detects a non-main subject region such as a blue sky region when a feature region that can be determined as the main subject cannot be detected or when a feature region detection for the main subject is not performed. The degree of influence is detected by the feature amount to affect the quality improvement of the entire image, and a correction target for optimizing a region other than the detection region is set as necessary.

  As specific means, a blue sky region is detected as a feature region, and a luminance average value and detection region information are calculated as a feature amount of the detected region. Furthermore, the average brightness value and detection area information of the image to be detected are calculated, the absolute value of the average brightness value of the blue sky area is compared with the determination value, and the average brightness value and determination value of the detection target image are compared. The degree of influence in the image of the blue sky region is determined by comparing the difference between the average luminance value of the blue sky region and the average luminance value of the detection target image and the determination value.

  If the blue sky area of the feature area affects the image quality of other areas based on the determination result, area information is created by reducing the blue sky area of the feature area from the entire image to be detected, and the value The correction target value is determined from Then, a gradation correction table passing through the correction target value and three points (0, 0) and (255, 255) is created.

  According to the present invention, it is possible to achieve good image correction with a small amount of information processing by performing individual correction of a specific area extracted from a captured image and reflecting it in the correction of the entire image.

  Hereinafter, an image processing apparatus and an image processing method using the same according to an embodiment of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 11 is a block diagram showing a hardware configuration of the image processing apparatus according to the first embodiment of the present invention. As shown in FIG. 11, the image processing apparatus according to the present embodiment is achieved with a so-called personal computer configuration.

  Specifically, an input / output port (I / O port) 51 for inputting / outputting data, a mouse (Mouse) 52 as a pointing device, a keyboard (KeyBoard) 53 for inputting numerical values and characters, and the like. A monitor 54 for displaying, a controller 55 for controlling the operation of the entire apparatus, and image data recorded on a memory card (for example, a photographed image photographed by a digital camera) is read out. A memory card drive 56, a hard disk 57 for storing data and programs, and a bus 58 for connecting the respective units are provided.

  As the monitor 54, for example, a liquid crystal monitor, a CRT monitor, an EL display, a plasma display, a television receiver, or the like can be used. A so-called touch panel having the functions of the mouse 52, the keyboard 53, and the monitor 54 may be used.

  The control unit 55 performs various processes on the image data read by the input / output port 51 or the memory card drive 56 based on the input from the mouse 52 or the keyboard 53.

  In the first embodiment, a personal computer will be described as an example of an apparatus that implements an image processing apparatus. However, any apparatus capable of processing image data may be a multifunction printer (MFP) or photo that handles images. A similar embodiment is conceivable even for a built-in device in a device other than a personal computer such as a direct printer (FDP).

  FIG. 1 is a flowchart for explaining the overall flow of image processing in the image processing apparatus according to the first embodiment. First, the image data read out by the input / output port 51 and the memory card drive 56 of the image processing apparatus shown in FIG. (Step S101).

  Next, the image data acquired in step S101 is developed in a RAM (not shown) or the like, and feature region extraction processing is performed on the acquired image data (step S102). For example, when the definition of the registered feature area is the main person skin area in the image, a processing method for detecting a person's face area in an image including a moving image is known as an entrance / exit management or monitoring system. ing. Then, the human skin or face area is subjected to feature determination to extract the matched area, hold the information, and calculate and hold the feature amount of the extracted area. Here, it is assumed that size information including at least the luminance average value and the number of pixels is included.

  FIG. 10 is a flowchart for explaining details of the feature region extraction processing (step S102) of the flowchart shown in FIG. Here, a case will be described in which the feature region to be extracted is the skin region of the main person in the image.

  First, a group of candidate skin regions of the main person is detected from the image data acquired in step S101 by using a chromaticity ratio and an arrangement combination in units of 8 × 8 pixels that are JPEG image compression units (step S101). S1001). Next, it is determined whether or not the candidate group has been successfully detected by the process of step S1001 (step S1002).

  As a result, if the detection fails (No), the fact that the detection of the skin area of the main person has failed is registered (step S1006). Specifically, the detection result HM is set to “0000”. On the other hand, if the detection is successful (Yes), the detected area is set in advance and stored in the range of the spatial frequency characteristics set for the main person skin area candidate group detected in step S1001. Whether or not to do so is compared (step S1003).

  Then, based on the comparison result in step S1003, it is determined whether or not the detection is successful, that is, whether or not the group of the main person's skin area candidates is determined to be the main person's skin area (step S1004). . As a result of the determination, when it is determined that the detection is unsuccessful, that is, it is determined that the skin is not the main person's skin (No), the process proceeds to the above-described step S1006 and registration of the failure to detect the main person's skin area is registered To do. If the detection is successful, that is, if the skin of the main person is determined (Yes), the fact that the main person's skin area has been detected is registered (step S1005). Specifically, the detection result HM = “1111” is set. Further, the detected feature data is stored.

  In the above-described embodiment, the feature region extraction process in step S102 has been described by detecting the feature region of the main person's skin region. It may be replaced with detection.

  Next, the characteristics of the image data acquired in step S101 are analyzed (step S103). Here, size information including at least the luminance average value and the number of pixels is obtained by feature analysis, and is retained.

  Next, when it is determined that the detection result (HM) in step S102 is “1111” (that is, when the detection is successful), based on the feature amount calculated by the processing in step S102 and step S103, the image It is determined whether or not it is necessary to set a correction target value for an image using the feature region extracted from the data (step S104).

  For example, if a person is present in the captured image acquired in step S101, it is highly likely that the person is the main subject. Therefore, in the present embodiment, the determination is performed based on the comparison relationship between the luminance average value itself of the characteristic area of the person considered as the main subject extracted from the captured image and the average luminance of the captured image.

  On the other hand, when the detection result HM in step S102 is determined to be “0000” (that is, in the case of detection failure), step S105 is skipped and normal image correction is performed in step S106.

  In step S105, when it is determined in step S104 that an image correction target value using the feature area extracted from the photographed image needs to be set, the final brightness level is determined based on the situation of the feature area. Set whether to approach the target value. Also, a correction table is created using the set target value.

  In step S106, it is determined whether or not there is a correction table created using the feature amount of the feature area in step S105. If there is no correction table, an image based on a common definition that does not reflect the detection result of the feature area Perform correction. On the other hand, if a correction table exists, the image correction is performed after replacing the image correction based on the common definition so that the created correction table can be used instead of the image correction based on the common definition. To do.

Here, the image correction based on the common definition will be briefly described. The definition of a good image is generally defined based on the following.
1) The white balance of the light source is good.
2) Good contrast.
3) The gradation of the necessary part is secured.

  Then, prior to the image correction processing in step S106, it is determined by analyzing the captured image whether the image satisfies the above condition. As a result, if it is inappropriate, parameters for necessary correction items are calculated and output in the image correction processing step of step S106.

  First, regarding the white balance of the light source, it is assumed that the vicinity of the highest luminance portion in the image approximates the light source, and the RGB balance of this region is determined. If correction is necessary, a conversion table is created for each RGB component, and the image data is converted and corrected.

  Contrast is corrected by analyzing the distribution of the actual data within the luminance range 0 to 255 (in the case of 8 bits) of the image data, and creating and applying a conversion table so that the distribution is uniform. To do.

  Further, when the gradation of the main part in the image data is not secured, correction is performed by creating and applying a gradation correction table in order to optimize gradation distribution.

  Next, a method for creating an exposure correction table will be specifically described. FIG. 15 is a diagram illustrating an example of an exposure correction amount determination table used in the first embodiment. In the following, in the exposure correction amount determination table shown in FIG. 15, the horizontal axis represents the average luminance (AVE) of the input image (captured image) in the range from 0 to 255, and the vertical axis represents the average luminance after conversion. The target value (NAVE) of the value, and its range is represented by 0 to 255.

  In the target value (NAVE) setting table in FIG. 15, the target value (NAVE) becomes higher than the current state so that an image having a lower luminance than the target luminance level (144 is assumed here as an example) approaches the target level. Set as follows. On the other hand, the target value (NAVE) is set to be lower than that in the current state so that an image having a higher luminance than the target luminance level approaches the target level.

  Then, the value of NAVE is obtained for each luminance average (AVE) of the input image using the exposure correction amount determination table shown in FIG. 15, and the table shown in FIG. 16 is created from the result. FIG. 16 is a diagram illustrating an example of a lookup table for exposure correction used in the first embodiment. The table creation method shown in FIG. 16 first plots the average luminance (AVE = A) value of the input image between 0 and 255 on the horizontal axis. Next, the target value (NAVE = NA) of the luminance average value after conversion is plotted between 0 and 255 on the vertical axis. Then, an intersection point is found vertically from the plot point, and the conversion table is created by connecting the points (0, 0) and (255, 255) as shown in FIG. Alternatively, a lookup table for exposure correction may be created as a conversion table for gamma gradation passing through the target value and three points (0, 0) and (255, 255). By using such a conversion table, the luminance value of the entire image can be easily converted to a desired value.

  Here, correction processing when the image data is a person standing on a white silver slope will be described. Therefore, the average luminance of the entire image is B (= 188), and the average luminance of the human skin detected by the feature region detection is A (= 64). Under such conditions, when the feature region detection process is not performed, the process of reducing the brightness of the entire image from 188 to 170 is performed in the image correction exposure process. However, by performing the correction process, the face of the person who wants to see most as the subject is corrected in a darker direction than 64, and the image correction is rather deteriorated.

  However, by reflecting the output of the feature region detection in the image correction value, the average brightness value 64 of the human skin becomes the target of exposure correction. Therefore, exposure correction is performed to raise the person's face from 64 to 100. Therefore, it is possible to improve the quality of the portion of the image data to be viewed.

  Here, in the exposure correction method described above, although the image is improved to some extent by the correction, the correction amount to be corrected is also small, and the face of a person photographed with a luminance of 100 is generally still dark. It is. Originally, when an image obtained by photographing a human face is viewed on a monitor or printed, it is considered that a range of about 170 to 220 is required as a preferable luminance. However, when a tone correction table is created to increase the average brightness A (= 64) of human skin detected in the above-described feature region detection processing to 170 and exposure correction is performed, the white silver slope that is the background is almost white. It will fly away. Therefore, it is not preferable to simply perform the correction for raising the average luminance A (= 64) of the human skin detected in the characteristic region detection process to 170 as the gradation correction.

  FIG. 2 is a diagram showing a brightness correction target value setting table for the feature region in the image used in the first embodiment. In the luminance correction target setting table shown in FIG. 2, the vertical rows are divided into four classes [0-60], [61-120], [121-190], [191-255] according to the average luminance of the entire image. To classify the target image. In the horizontal column, four classes of [0 to 35], [36 to 80], [81 to 120], and [121 to 255] are used depending on the luminance of the feature area of the final candidate calculated by feature area detection (APD detection). The target area is classified into two. Then, the luminance target value of the final candidate feature region is calculated using the target setting coefficient corresponding to the luminance average of the entire image and the final candidate feature region luminance.

  In other words, the brightness correction target setting process allows the person's skin area detected by the feature area detection process to be set as the main subject, and makes it possible to better correct the feature area without degrading the image quality of all images.

  Therefore, in the luminance correction target value setting table shown in FIG. 2, when the average luminance of the entire image is [191 to 255], the vertical row is too high in the background luminance and cannot reflect the result of the feature region detection. Thus, the gradation correction reflecting the characteristic region detection result is not executed, and execution of only normal image correction is designated.

  In the horizontal row, when the brightness of the final candidate feature area calculated by the feature area detection is [0 to 35], gradation correction reflecting the feature area detection result is not executed, and only normal image correction is performed. Execution is specified. This takes into account the input characteristics of the photographic equipment. That is, it is known that the S / N ratio is low when the input signal to the image sensor such as a CCD is low, and the main subject is considered to be black. Therefore, it is based on the determination that even if the luminance of the main subject in this region is increased, it cannot be corrected to a high quality state.

  Furthermore, when the luminance of the human skin region, which is the final candidate feature region calculated by the feature region detection, is [131 to 255], it is determined that the luminance state is almost favorable or close to whiteout. Therefore, it is determined that the necessity of reflecting the result of the characteristic area detection is low, and the gradation correction reflecting the characteristic area detection result is not executed, and execution of only normal image correction is designated.

  Here, taking as an example an image obtained by photographing a person standing on the white silver slope, the average luminance of the entire image is B (= 188), and the average luminance of the human skin detected by the feature region detection is A. (= 64). Therefore, from the table shown in FIG. 2, “* 1.5” is obtained, and a value obtained by multiplying the average luminance A (= 64) of human skin by 1.5 is the luminance target value. Then, using this value, the NA is set and reflected in the creation of the correction table shown in FIG.

  Next, the correction process will be described in detail using an actual captured image as an example. 21, 25, 29, and 33 are diagrams illustrating sample images of a captured image including a person's skin region set as a main subject as a target for feature region detection.

  First, the processing of the captured image shown in FIG. 21 will be described. FIG. 22 is a diagram illustrating a result of detecting a region as a final candidate of the human skin region in the image of FIG. The human skin portion in the captured image of FIG. 21 includes a human face, both knees, palms, and the like. The region detected as the most weighted final subject candidate is the face portion detected in FIG. is there.

  FIG. 23 is a diagram displaying an example of the feature amount including the average luminance and the luminance histogram of the entire photographed image shown in FIG. In FIG. 23, the luminance is represented by 256 gradations on the horizontal axis of the histogram. That is, the range has a width of 0 to 255, with the left end being 0 and the right end being 255. In FIG. 23, the average luminance is 160, and the luminance distribution is roughly divided into a snow region with the highest luminance distribution, a sky region which is the next high luminance distribution region, and a mountain region with a rather dark background. It can be confirmed.

  FIG. 24 is a diagram displaying an example of the feature amount in the face area of the person detected in the photographed image shown in FIG. In FIG. 24, the average luminance of the detected region is 92, and it can be confirmed that there is a luminance data distribution of the region detected mainly in the upper and lower luminance ranges around this value.

  When setting the brightness correction target value for the feature area using the table shown in FIG. 2, the average brightness of the entire image is B (= 161), and the average brightness of the skin of the person detected by the feature area detection is Since A (= 92), “* 1.3” is obtained, and a value (120) obtained by multiplying the average luminance A (= 92) of human skin by 1.3 is the luminance target value. The NA is set using this value and reflected in the creation of the correction table shown in FIG.

  Next, an example using the captured image shown in FIG. 25 will be described. FIG. 26 is a diagram illustrating a result of detecting a region as a final candidate for a human skin region in the captured image illustrated in FIG. Note that the human skin photographed in the image of FIG. 25 is only the face of the person, and this is the detected portion shown in FIG.

  FIG. 27 is a diagram illustrating an example of the feature amount including the luminance average and luminance histogram of the entire captured image illustrated in FIG. In FIG. 27, the horizontal axis of the histogram represents luminance in 256 gradations and is given by a width of 0 to 255, with the left end being 0 and the right end being 255. Further, the average luminance is 111 in the histogram, and it can be confirmed that the luminance distribution is distributed almost uniformly throughout.

  FIG. 28 is a diagram showing feature amounts in the face area of a person detected as a feature area from the captured image shown in FIG. As shown in FIG. 28, the average luminance of the detected area is 93, and there is a luminance data distribution of the detected area mainly in the luminance range of 30 to 40 above and below centering on this value. I can confirm.

  Here, when setting the luminance correction target value for the feature area detected based on the table shown in FIG. 2, the average brightness of the entire image is B (= 111), and the human skin detected by the feature area detection Since the average luminance of A is A (= 93), “* 1.5” is obtained from the table. A value (140) obtained by multiplying the average luminance A (= 93) of human skin by 1.5 is the luminance target value. The NA is set using this value, and is reflected in the creation of a correction table as shown in FIG.

  Next, an example using the captured image shown in FIG. 29 will be described. FIG. 30 is a diagram illustrating a result of detecting a region as a final candidate of the human skin region with respect to the captured image illustrated in FIG. The human skin photographed in the image of FIG. 29 includes a human face and both arms, but the region detected as the most weighted final subject candidate is the portion detected in FIG.

  FIG. 31 is a diagram illustrating an example of a feature amount including a luminance average and a luminance histogram of the entire captured image illustrated in FIG. In FIG. 31, the horizontal axis of the histogram represents luminance in 256 gradations and is given by a width of 0 to 255, with the left end being 0 and the right end being 255. In the histogram, the average luminance is 84, and it can be confirmed that the luminance distribution includes a sky region of the highest luminance distribution and a forest region that is a relatively dark background.

  FIG. 32 is a diagram showing feature amounts in the face area of a person detected as a feature area in the captured image shown in FIG. As shown in FIG. 32, the average luminance of the detected area is 73, and it can be confirmed that there is a luminance data distribution of the detected area mainly in the upper and lower luminance ranges around this value.

  Here, when setting the luminance correction target value for the feature region detected based on the table shown in FIG. 2, the average brightness of the entire image is B (= 84), and the human skin detected by the feature region detection Since the average luminance is A (= 73), “* 1.8” is obtained from the table. A value (131) obtained by multiplying the average luminance A (= 73) of human skin by 1.8 is the luminance target value. The NA is set using this value, and is reflected in the creation of a correction table as shown in FIG.

  Next, an example using the captured image shown in FIG. 33 will be described. FIG. 34 is a diagram illustrating a result of detecting a region as a final candidate of the human skin region in the captured image illustrated in FIG. The human skin photographed in the image of FIG. 33 includes a human face and both arms, but the region detected as the most weighted final subject candidate is the portion detected in FIG.

  FIG. 35 is a diagram illustrating an example of a feature amount including a luminance average and a luminance histogram of the entire captured image illustrated in FIG. In FIG. 35, the horizontal axis of the histogram represents luminance in 256 gradations and is given by a width of 0 to 255, with the left end being 0 and the right end being 255. In the histogram, the average luminance is 125, and it can be confirmed that the luminance distribution includes a sky region with the highest luminance distribution and a hay region with a dark background.

  FIG. 36 is a diagram showing feature amounts in the face area of a person detected as a feature area in the captured image shown in FIG. As shown in FIG. 36, the average luminance of the detected region is 80, and it can be confirmed that there is a luminance data distribution of the region detected mainly in the upper and lower luminance ranges around this value.

  Next, when setting the luminance correction target value for the feature region detected based on the table shown in FIG. 2, the average brightness of the entire image is B (= 125), and the human skin detected by the feature region detection Since the average luminance is A (= 80), “* 1.5” is obtained from the table. A value (131) obtained by multiplying the average brightness A (= 80) of human skin by 1.5 is the brightness target value. The NA is set using this value, and is reflected in the creation of a correction table as shown in FIG.

  As described above, using the photographed image samples shown in FIG. 21, FIG. 25, FIG. 29, and FIG. Here, the average brightness of the human skin region shown in FIG. 21 is 92, and the brightness increase amount is 28. In addition, the average luminance of the human skin region shown in FIG. 25 is 93, and the luminance increase amount is 47. Further, the average luminance of the human skin region shown in FIG. 29 is 73, and the luminance increase amount is 58. Furthermore, the average brightness of the human skin area shown in FIG. 33 is 80, and the brightness increase amount is 51.

  From these results, it is clear that the luminance correction target for the main subject has an influence on the situation of the entire image including the background due to the luminance relationship of the entire image including the background and the luminance of the main subject extracted as the feature region. Therefore, the tone correction of the main subject region extracted as the feature region can be satisfactorily executed under the condition that the tone of the background portion is not impaired.

  FIG. 7 is a diagram showing another brightness correction target value setting table for the feature region in the image used in the first embodiment. In the brightness correction target value setting table shown in FIG. 2, the target image is classified by classifying the target image according to the brightness average of the entire captured image and the brightness average value of the human skin area of the main subject extracted as the feature area. went. However, in this case, a step where the setting of the target value changes occurs at the class boundary. Accordingly, the target value setting in which this is changed linearly is shown in the table shown in FIG.

  In FIG. 7, when the average luminance in all images is less than 0 to 35 or greater than 180, it is excluded from the target setting target for image correction using the feature region. Similarly, when the average luminance of the main subject whose features have been extracted is less than 0 to less than 35 or greater than 160, it is excluded from the target setting target for image correction using the feature region.

  Furthermore, an image having an average luminance of 35 to 180 in all the images and satisfying the condition that the average luminance of the feature-extracted main subject is 35 to 160 is a target setting target for image correction using the feature region, and each point The target is set by a value that is connected linearly.

  5 and 6 are graphs showing the NAVE setting (correction target value setting) table used in the first embodiment of the present invention. In FIG. 5, the horizontal axis represents the average luminance of the entire image, the back axis represents the average luminance of human skin as the main subject extracted as the feature region, and the vertical axis represents the multiplier to the target setting feature region. As shown in FIG. 5, it can be confirmed that the target setting multiplier changes smoothly.

  FIG. 47 is a diagram illustrating a result of performing the correction process according to the present embodiment on the photographed image sample illustrated in FIG. FIG. 47 shows that the luminance in the human skin of the characteristic area that is the main subject in the photographed image shown in FIG. 29 is increased, and the image quality is improved centering on the area of interest of the observer of the photographed image.

  FIG. 46 is a diagram showing a luminance histogram of the correction result image shown in FIG. In FIG. 46, the horizontal axis of the histogram represents luminance in 256 gradations and has a width of 0 to 255, with the left end being 0 and the right end being 255. Further, the average luminance is 114 in the histogram, and the luminance of the human skin region cannot be optimized from the luminance distribution characteristics of the original image by normal image correction, but is corrected to a good state by the correction processing according to the present embodiment. Can be confirmed.

  As described above, in the present embodiment, the correction process aimed at improving the image quality from the non-main subject region to the main subject region has been described. In the first embodiment, the reflection method to the gradation is specifically shown. However, other correction items such as contrast and white balance can be similarly corrected.

<Second Embodiment>
Next, image processing that uses the same hardware configuration as that of the first embodiment described above to create a tone correction lookup table after setting a correction target value for a feature area and performs correction will be described. . In the first embodiment described above, the area ratio of the feature region and the luminance average are used as the feature amount of the feature region. However, as a result of correcting by this, the luminance is corrected well, but depending on the image. There is a possibility that the gradation of the human skin area as the main subject is reduced. Therefore, in the second embodiment, a correction table creation method for simply performing luminance and gradation correction on a feature region will be described.

  FIG. 4 is a diagram showing an example of conversion of LUT gradation increase according to an embodiment of the present invention. In FIG. 4, the horizontal axis represents the brightness of the feature region, and is represented in 256 levels from 0 to 255. Further, the luminance range that falls within the limited range is set for the distribution of the luminance data of the feature region using a preset threshold M such as 5% above and below the luminance histogram of FIG. The setting range is represented by “SK IN”. The “SK IN” data is converted using the gradation correction table based on the target setting, and the vertical axis indicates the mapping after conversion to the output. That is, in FIG. 4, the vertical axis represents the luminance after conversion, and is represented in 256 levels from 0 to 255. The conversion output to this feature area is “SK OUT”.

  Here, in the image conversion example shown in FIG. 4, since “SK OUT” is larger than “SK IN”, it can be seen that both the target setting of the luminance conversion and the gradation range are well converted. Note that the threshold value M set up and down from the luminance histogram of the feature area described above is set according to the image size, the size of the feature area, and the like.

  FIG. 3 is a diagram illustrating a conversion example of LUT gradation reduction according to an embodiment of the present invention. The configuration of the graph in FIG. 3 is the same as the graph shown in FIG. Here, the “SK IN” data is converted by the tone correction table based on the target setting to “SK OUT” based on the average luminance and luminance distribution of the human skin region final candidate of the main subject that is the feature region. Converted. Here, since “SK IN” becomes larger than “SK OUT”, the target setting of the luminance conversion is converted well, but the gradation after conversion is reduced. Therefore, the correction centering on the data of the characteristic area set as the main subject has a good brightness, but may have a feeling of being reduced as the gradation is reduced.

  Therefore, in order to improve such a phenomenon, if the detection result HM in step S102 is determined to be “1111” in the feature amount determination step (step S104) in the flowchart shown in FIG. The process shown in FIG. 8 and FIG. 9 is executed. FIG. 8 is a flowchart for explaining a reflected value determination processing procedure based on an area ratio in the second embodiment of the present invention. FIG. 9 is a flowchart for explaining the LUT creation processing procedure according to the second embodiment of the present invention.

  The purpose of the processing shown in FIG. 8 and FIG. 9 is to create an object corresponding to securing the gradation of the feature area when creating the brightness correction target table for the main subject area which is the feature area. It is. That is, in the processing shown in FIGS. 8 and 9, the ratio of the area occupied by the main subject area, which is the detected feature area, in the entire image is obtained, and the gradation of the feature area is determined in the brightness correction table creation by reflecting the result. Provided is an instruction method for a luminance correction table to be secured.

  Specifically, first, as shown in FIG. 9, feature area data for which the detection result HM is determined to be “1111” is fetched (step S901). Next, a luminance histogram is created from all the pixel data in the feature area (step S902). Then, as a reference value for securing the gradation, the threshold value M for the ratio set above and below the total number of pixels of the feature area data from the luminance histogram created in step S902 should be used to secure the gradation in the feature area. The upper limit (A point) and lower limit (B point) values are set (step S903). Here, in the present embodiment, the threshold value M = 5%. The setting of the threshold value M may be changed according to the image size, the feature area size, and the ratio of both.

  Next, the number of luminance gradations, which is the difference between the upper limit (point A) and the lower limit (point B) calculated in step S903 from the luminance average of the feature region, and its average value C are calculated (step S904). Then, an amount T1 for securing the gradation of the feature area in the correction table is calculated (step S905). Specific processing in step S905 will be described with reference to the flowchart shown in FIG.

  First, the area ratio between the entire region of the image and the main subject region extracted as the feature region is calculated (step S801). Next, “S1” and “S2”, which are the gradation securing instruction values of the feature region based on the area ratio prepared in advance, are read (step S802). Note that “S1” is a threshold value for determining whether or not the ratio of the feature area in all the images is small. Further, “S2” is a threshold value for determining whether or not the ratio of the feature area in all the images is too large.

  Then, it is determined whether or not the area ratio occupied by the feature region is larger than the threshold value “S1” (step S903). As a result, if it is determined to be small (Yes), 0.8 is set to the gradation securing designated multiplier T1 (step S806). Note that T1 is a multiplier that secures the gradation of the output with respect to the input. On the other hand, if it is determined that the area is larger (No), it is further determined whether or not the area ratio occupied by the feature region is larger than the threshold “S2” (step S804).

  As a result, when it is determined to be small (No), 1.2 is set to the gradation securing designated multiplier T1 (step S805). On the other hand, if it is determined to be large, 1.0 is set to the gradation securing designated multiplier T1 (step S807).

  Based on the processing results up to step S905 described above, a gradation correction conversion table (LUT) is created (step S906). In the present embodiment, as described above, the classification of gradation securing based on the area ratio by the thresholds S1 and S2 is performed. However, the gradation securing amount by function calculation may be calculated from the area ratio.

  And after performing the said process, the following process which is the process performed in 1st Embodiment mentioned above is performed. That is, based on the feature amounts calculated in step S102 and step S103, it is determined whether it is necessary to set an image correction target value using the feature region extracted from the image. As a result of the determination, if there is a person in the captured image, the possibility of being the main subject is high. Therefore, the determination is performed based on the comparison relationship between the average brightness value of the feature region extracted from the image considered as the main subject and the average brightness of the captured image. When the detection result HM in step S102 is determined to be “0000”, step S105 is skipped and normal image correction is executed in step S106.

  Further, in step S105, when it is determined that it is necessary to set an image correction target value using the feature region extracted from the image in step S104, to what luminance level the final target value is set based on the situation of the feature region. Whether to approach or not is set, and a gradation range corresponding to a feature area secured with a target value interposed therebetween is set. Also, a correction table is created using the set target value.

  FIG. 17 is a diagram for explaining a method for creating a correction table in the second embodiment. In FIG. 17, the horizontal axis represents the luminance of the image before correction and is represented by 0 to 255. The vertical axis represents the luminance after correction and is represented by 0 to 255.

  First, the brightness correction target value C using the table shown in FIG. 7 is obtained from the brightness average of the entire image and the brightness average value of the human subject skin area extracted as the feature area, and a gradation is secured at a slope of 1 slope. The upper limit (A point) and lower limit (B point) values to be placed are arranged. Then, the upper limit value (point A) and (255, 255) and the lower limit value (point B) and (0, 0) are connected by straight lines.

  As a result, a conversion table of gamma gradations that passes through the four points of the target value, (0, 0) and (255, 255), the upper limit (point A), and the lower limit (point B) is created. By using this conversion table, the luminance value of the entire image can be converted to a desired value.

<Third Embodiment>
In the third embodiment, in the system having the same configuration as the first and second embodiments described above, the feature region extraction target for an image is not a main skin or a human face as a main subject, but a non-main skin. A case where a blue sky region is set as a subject will be described.

  This will be described in detail using an example of an actual captured image. FIG. 37 and FIG. 40 are diagrams showing photographed image samples including a blue sky region set as a non-main subject as a feature region detection target in the third embodiment.

  First, processing for the captured image shown in FIG. 40 will be described. FIG. 41 is a diagram illustrating an example of a region detected as a final candidate of the blue sky region for the captured image illustrated in FIG.

  FIG. 42 is a diagram displaying an example of the feature amount including the luminance average and luminance histogram of the entire captured image shown in FIG. In FIG. 40, the horizontal axis of the histogram represents luminance in 256 gradations and is given by a width of 0 to 255, with the left end being 0 and the right end being 255. The average luminance of the histogram is 132, and the luminance distribution includes a cloud region which is the highest luminance distribution around luminance 200, a blue sky region where the luminance distribution is approximately 75 to 140, and grass near luminance 100. It can be confirmed that there is a spreading hill area.

  Among these, the blue sky area detected as the characteristic area is a blue sky with a strong bluish color, so that the average brightness is moderately lowered, and as a result, the brightness area is well balanced with other components in the image.

  Next, the captured image shown in FIG. 37 will be described. FIG. 38 is a diagram illustrating a result of detection as the final candidate of the blue sky region for the captured image illustrated in FIG. FIG. 38 shows a region where the blue sky portion in the captured image shown in FIG. 37 is detected as the final subject candidate.

  FIG. 39 is a diagram illustrating an example of a feature amount including a luminance average and a luminance histogram of the entire captured image illustrated in FIG. In the histogram shown in FIG. 39, the horizontal axis represents the brightness in 256 gradations and has a width of 0 to 255, with the left end being 0 and the right end being 255. The average luminance of the histogram is 77, and the luminance distribution includes a blue sky region that is the highest luminance distribution in the vicinity of luminance 210 to 225 and a temple that is a considerably dark main subject region with a luminance distribution in the range of approximately 0 to 40. It can be confirmed that the gate exists.

  The blue sky area detected as the characteristic area as shown in FIG. 37 has a relatively high average luminance because it is a light blue sky. As a result, other components in the image are affected, and the main subject is affected. It is in a luminance range with poor balance in a state close to black crushing.

  In the luminance distribution in the luminance histogram shown in FIG. 39 obtained from the photographed image shown in FIG. 37, there is a lot of data of a region photographed at low luminance or high luminance. Therefore, image correction that assigns gradation cannot be performed.

  Therefore, in the present embodiment, in consideration of the degree of influence on the image of the data of the blue sky region portion that is the non-main subject extracted as the feature region, the data of the extracted blue sky region portion that is the non-main subject is extracted as necessary thereafter. Is removed from the entire image, and evaluation is performed to perform image correction on the captured image data from which the blue sky region is removed. Then, image correction is performed with the value set as a target value in consideration of the degree of influence of the previous blue sky region as necessary. In this way, it is possible to provide appropriate image correction for the captured image shown in FIG.

  Next, the difference in photographing influence on the main subject of the image samples included in the two blue sky regions in FIGS. 37 and 40 is observed more objectively. FIG. 18 is a diagram illustrating a result of searching for a condition under which it is preferable to apply the correction considering the blue sky region by increasing the number of photographed image samples.

  In the graph shown in FIG. 18, the horizontal axis represents the average brightness of the entire image in 256 gradations and is given by a width of 0 to 255, and the actual display range is 60 at the left end and 200 at the right end. The vertical axis represents the average brightness of the final candidate of the blue sky region extracted as the feature region in 256 gradations and given a width of 0 to 255. The actual display range has a lower end of 60 and an upper end of 240. It is.

  FIG. 18 shows the effect of executing image correction reflecting the blue sky region extracted as the feature region in each captured image. That is, in FIG. 18, an object having a particularly remarkable effect is indicated by “◯”, an object whose effect can be clearly confirmed by the normal image correction is indicated by “Δ”, and an effect that is obvious with respect to the normal image correction. Items that cannot be confirmed are indicated by “□”.

  As shown in FIG. 18, it can be confirmed that the effect is large in the region range in which the average luminance of the blue sky region extracted as the feature region is 160 or more and the average luminance of the entire image is 140 or less. Furthermore, even within this range, it can be confirmed that the effect is greater as the difference between the average luminance of the blue sky region extracted as the feature region and the average luminance of the entire image is larger.

  Further, although not clearly shown in the graph shown in FIG. 18, it can be determined that the influence is large when the blue sky region extracted as the feature region exceeds 30% of the entire image region.

  In the following, an optimum correction method based on the correlation between the feature region and the entire image will be described in view of the characteristics as described above. The overall processing flow uses the flowchart shown in FIG. 1 used in the first embodiment described above. However, in the flowchart of FIG. 1, the target feature area in step S102 is not the human skin that is the main subject, but a blue sky area that is set as a non-main subject.

  FIG. 12 is a flowchart for explaining details of a feature extraction processing procedure according to the third embodiment. First, a blue sky region candidate group detection process is performed on the image data acquired in step S101 by a combination of a chromaticity ratio, luminance, and area in units of 8 × 8 pixels, which are JPEG image compression units. (Step S1201). Then, it is determined whether or not a candidate group has been detected in step S1201 (step S1201).

  As a result, if the detection fails (No), it is set that the detection of the blue sky region has failed (step S1206). Specifically, the setting BS = “0000” is performed. On the other hand, if the detection is successful (Yes), it is determined whether or not the group of blue sky region candidates matches the range of the spatial frequency characteristics set and stored in advance (step S1203). Then, it is determined whether or not the group of blue sky region candidates is determined to be a blue sky region by the determination processing in step S1203 (that is, whether or not the detection is successful) (step S1204).

  As a result of the determination, if it is determined that the region is not a blue sky region (No), the process proceeds to step S1206 to set that the detection of the blue sky region has failed (BS = “0000”). On the other hand, if it is determined that the region is a blue sky region (Yes), the fact that the blue sky region has been successfully detected is set (step S1205). Specifically, a set BS = “1111” is performed. Further, the detected feature data is stored.

  FIG. 13 is a flowchart for explaining a procedure of processing performed on a blue sky area (BS = “1111” image as an object. First, feature area data determined by the BS as “1111” is captured. (Step S1301) Then, it is determined whether or not the average brightness of all the image areas, which is a condition that the image correction reflecting the blue sky area data as the characteristic area can be more effective, is less than 140 ( Step S1302).

  As a result, when the average luminance of all the image areas is 140 or more (No), “0000” is set to BSH (step S1303). On the other hand, if the average luminance of all the image areas is less than 140 (Yes), the blue sky area data that is the characteristic area is read (step S1304). Then, it is determined whether or not the average luminance of the blue sky region that is the feature region is greater than 160 (step S1305).

  As a result, when the average brightness of the blue sky area is 160 or less (No), the process proceeds to step S1303, and “0000” is set to BSH. On the other hand, when the average brightness of the blue sky region is larger than 160 (Yes), “1000” is set in BSH (step S1306). It should be noted that an image for which BSH = “0000” is set basically skips step S105 and executes normal image correction in step S106.

  FIG. 14 is a flowchart for explaining a process for comparing and determining the blue sky area and the average luminance of all the images for an image for which BSH = “1000” is set in the process of the flowchart shown in FIG. 13. First, a luminance difference between the average luminance of the blue sky region extracted as the feature region and the average luminance of all images is calculated, and it is determined whether or not the value exceeds 50 (step S1401). As a result, when the calculated luminance difference is 50 or less (No), “0000” is set in BSH (step S1402). On the other hand, if the calculated luminance difference is greater than 50 (Yes), the process proceeds to step S1403.

  In step S1403, a luminance difference between the average luminance of the blue sky region extracted as the feature region and the average luminance of all the images is calculated, and it is determined whether or not the value exceeds 80. As a result, when the calculated luminance difference is 80 or less (No), “1001” is set in the BSH (step S1405). On the other hand, if the calculated luminance difference is greater than 80 (Yes), “1010” is set in BSH (step S1404).

  From the above examination results, it has been confirmed that the image correction reflecting the blue sky region extracted as the feature region is more effective as the difference between the average luminance of the blue sky region extracted as the feature region and the average luminance of all images increases. Therefore, the degree of reflection in the correction is switched based on this luminance difference.

  As a method for reflecting the correction effect, a description will be given by taking as an example the creation of a gradation correction table in the luminance information described in the above embodiment. That is, when “1010” is set in the BSH of the determination result, the luminance average of the entire image is not used as the value of “AVE” when setting the correction target value “NAVE”, but in all the images. Using the luminance data obtained by removing the blue sky region extracted as the feature region from the image, the luminance average here is calculated and input to “AVE”. In this way, the target value “NAVE” is set using the luminance feature amount from which the blue sky region portion is removed. Thereby, the brightness correction target value shown in FIG. 15 can set the correction target in the bright direction.

  In addition, as a reflection to other correction items, in contrast correction, a histogram is created by removing a blue sky region extracted as a feature region from all images, and a contrast correction target value is set based on this.

  That is, when “1001” is set in the BSH of the determination result, the luminance average is calculated using luminance data obtained by removing the blue sky region extracted as a feature region from all the images, and “AVE” is calculated. ”To set the target value“ NAVE ”. Then, a difference between “NAVE” and “AVE” is calculated, and a value obtained by multiplying the difference by a constant less than 1 is subtracted from the calculated “NAVE”, and the target value “NAVE” is used.

  A gradation correction conversion table is created using the brightness correction target value “NAVE” obtained in this way, and image correction is performed on the original image. In addition, a correction target may be set by multiplying the effect of the area ratio of the blue sky region extracted as the feature region in the entire image as a weight.

  FIG. 45 is a diagram illustrating an image obtained as a result of performing the correction process according to the third embodiment on the captured image illustrated in FIG. 37. By comparing FIG. 37 with FIG. 45, the weight of the blue sky in the feature area that is a non-main subject is reduced, and the image quality centered on the area that the observer of the captured image focuses is improved. Recognize.

  FIG. 43 is a diagram showing a luminance histogram after the data of the blue sky region extracted as the feature region from the photographed image shown in FIG. 37 is deleted. Then, a correction target value is set for this data and correction is executed. 44 is a diagram showing a luminance histogram of the image shown in FIG. 45 obtained by correcting the photographed image shown in FIG. As shown in FIG. 44, the horizontal axis of the histogram represents luminance in 256 gradations and is given by a width of 0 to 255, with the left end being 0 and the right end being 255. From the histogram shown in FIG. 44, it can be confirmed that the average brightness is 105, and particularly that the brightness of the main subject region has been greatly improved, so that the correction process according to the present embodiment is in a good state. it can.

  As described above, in the present embodiment, correction is aimed at improving the image quality from the non-main subject region to the main subject region, and the reflection method to the gradation is shown. However, other correction items such as contrast and white The same applies to balance and the like.

<Fourth Embodiment>
In the fourth embodiment, an image correction method based on the correlation between feature areas and all images when a plurality of feature area extraction targets for images are set using the same hardware configuration as that of the above-described embodiment will be described.

  In the extraction of the feature region for the image, the main subject and the non-main subject in the image can be classified according to the target specified in the feature region. Here, when the feature region is the main subject, it is necessary to perform image correction so that the detected feature region can have better image quality. On the other hand, if the feature area is a non-main subject, it is evaluated whether the detected feature area has an adverse effect on the shooting state of the main subject portion in the image. It is necessary to execute correction for improving image quality.

  Therefore, the detected feature area is determined, it is determined whether the subject is a main subject or a non-main subject, and whether or not the correction reflecting the feature area is effective, and correction is performed according to the result. Thereby, it is possible to optimize the individual image required by the user.

  Next, correction processing according to the present embodiment will be described. Since the hardware configuration is the same as that of the above-described embodiment, a duplicate description is omitted. FIG. 19 is a flowchart for explaining a processing procedure for performing exposure correction with a human skin region and a blue sky region as detection targets in the fourth embodiment.

  First, a region detection process is performed on the original image using the feature region as the skin portion of the person who is the main subject (step S1901). Next, it is determined whether or not the detection of the skin of the person who is the main subject, which is the feature area, has succeeded (step S1902). As a result, when a human skin candidate is detected as the feature area (No), the detected feature area is set as the main subject (step S1903). Then, as described in the first and second embodiments, the exposure optimization process is performed on the final human skin candidate extracted from the feature region (step S1904).

  On the other hand, if the candidate for human skin is not detected as a feature area in step S1902 (No), a detection process is performed on the original image with the feature area as a non-main subject blue sky (step S1905). Then, it is determined whether the blue sky that is the non-main subject that is the feature region has been successfully detected (step S1906). As a result, when a blue sky candidate is detected as a feature region (Yes), the detected feature region is set as a non-main subject (step S1907). Next, as described in the third embodiment, the influence degree of the final blue sky candidate extracted from the feature region is determined, and the exposure optimization process is performed on the outside region as necessary (step S1908). .

  On the other hand, if a blue sky candidate is not detected as a feature area in step S1906 (No), image correction processing including normal exposure correction processing that does not reflect feature area data is executed (step S1909).

In the above-described embodiment, the method of reflecting the exposure correction in the image correction has been described. However, this is not limited to the exposure correction. In addition, although there is one feature area setting for each of the main subject and the non-main subject, it is considered that this is achieved by performing a plurality of weights and the like. Furthermore, in the present embodiment, the example in which the effect correction is set separately for the main subject and the non-main subject after detection has been described. However, all the set feature regions may be detected and the values may be reflected. <Fifth Embodiment>
In the fifth embodiment, the same hardware configuration as that of the above-described embodiment is used for the personal face information registered by the extraction target of the feature region for the image, and the image is based on the correlation between the feature region and the entire image. The correction is made.

  Next, correction processing according to the present embodiment will be described. Since the hardware configuration is the same as that of the above-described embodiment, a duplicate description is omitted. FIG. 20 is a flowchart for performing exposure correction using the personal registration information according to the fifth embodiment as a detection target.

  First, it is determined whether or not the final candidate person skin area has been extracted as an extraction result with the feature area in the above-described embodiment as human skin, based on whether or not the HM set value is “1111” (step 11). S2001). As a result, when HM = “1111” is not satisfied (No), an image correction process including a normal exposure correction process that does not reflect the feature area data is executed.

  On the other hand, if the HM is “1111” (Yes), the registration status as to whether or not there is an image setting specifying an individual is confirmed (step S2002). As a result, when there is no personal registration (No), an optimized exposure correction table for a human skin as a feature region implemented in the second and fourth embodiments is created (step S2006). On the other hand, if there is a personal registration (Yes), the registered personal recognition information (setting value) and the corrected target value (target value) set for each registered individual are read (step S2003). .

  Further, image recognition is executed from the individual discrimination registration data read in step S2003, and it is determined whether or not the registered individual's face exists in the original image (step S2004). As a result, when the registered individual cannot be detected (No), the process proceeds to step S2006, and the optimized exposure correction to the human skin area as the characteristic area performed in the second and fourth embodiments. Create a table.

  On the other hand, if a registered individual can be detected in step S2004 (Yes), it is set for each registered individual that has been read first, and the corresponding personal target is read from a certain correction target value, and the exposure correction table is read. Create (step S2005).

  Then, using the exposure correction table created in accordance with the above-described processing procedure, image correction that satisfies the thought is executed for the registered individual according to personal preference.

<Other embodiments>
Although the embodiments have been described in detail above, the present invention can take embodiments as, for example, a system, an apparatus, a method, a program, or a storage medium (recording medium). The present invention may be applied to a system composed of a single device or an apparatus composed of a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the figure) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

3 is a flowchart for explaining the overall flow of image processing in the image processing apparatus according to the first embodiment; It is a figure which shows the brightness correction target value setting table | surface with respect to the characteristic area | region in the image used in 1st Embodiment. It is a figure which shows the conversion example of the LUT gradation reduction which concerns on one Embodiment of this invention. It is a figure which shows the conversion example of the LUT gradation increase which concerns on one Embodiment of this invention. It is the figure which graphed the NAVE setting (correction target value setting) table used by the 1st Embodiment of this invention. It is the figure which graphed the NAVE setting (correction target value setting) table used by the 1st Embodiment of this invention. It is a figure which shows another brightness correction target value setting table | surface with respect to the characteristic area | region in the image used in 1st Embodiment. It is a flowchart for demonstrating the reflected value determination processing procedure by the area ratio in the 2nd Embodiment of this invention. It is a flowchart for demonstrating the LUT creation processing procedure in the 2nd Embodiment of this invention. It is a flowchart for demonstrating the detail of the characteristic area extraction process (step S102) of the flowchart shown in FIG. It is a block diagram which shows the hardware constitutions of the image processing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart for demonstrating the detail of the feature extraction process sequence in 3rd Embodiment. It is a flowchart for demonstrating the procedure of the process performed by having detected the blue sky area | region successfully (BS = "1111" image). 14 is a flowchart for explaining a process of comparing and determining a blue sky region and an average luminance of all images for an image for which BSH = “1000” is set in the process of the flowchart illustrated in FIG. 13. It is a figure which shows an example of the exposure correction amount determination table used in 1st Embodiment. It is a figure which shows an example of the look-up table for exposure correction | amendment used in 1st Embodiment. It is a figure for demonstrating the preparation method of the table for correction | amendment in 2nd Embodiment. It is a figure which shows the result of having searched the conditions where it is preferable to apply the correction | amendment which considered the blue sky area | region by increasing the number of picked-up image samples. It is a flowchart for demonstrating the process sequence which performs exposure correction | amendment which made the human skin area | region and blue sky area | region in 4th Embodiment the detection object. It is a flowchart for performing exposure correction | amendment which made the personal registration information which concerns on 5th Embodiment the detection object. It is a figure which shows the sample example of the picked-up image containing the person's skin area | region set with the main subject as the object of characteristic area detection. It is a figure which shows the result of having detected the area | region as a final candidate of a person's skin area | region with respect to the image of FIG. It is a figure which displays an example of the feature-value containing the brightness | luminance average of the whole picked-up image shown in FIG. 21, and a brightness | luminance histogram. It is a figure which displays an example of the feature-value in the person's face area detected in the picked-up image shown by FIG. It is a figure which shows the sample example of the picked-up image containing the person's skin area | region set with the main subject as the object of characteristic area detection. It is a figure which shows the result of having detected the area | region as a final candidate of a person skin area | region with respect to the picked-up image shown in FIG. It is a figure which shows an example of the feature-value containing the brightness | luminance average and brightness | luminance histogram of the whole picked-up image shown in FIG. It is a figure showing the feature-value in the person's face area detected as a feature area from the picked-up image shown in FIG. It is a figure which shows the sample example of the picked-up image containing the person's skin area | region set with the main subject as the object of characteristic area detection. It is a figure which shows the result of having detected the area | region as a final candidate of a person skin area | region with respect to the picked-up image shown in FIG. It is a figure which shows an example of the feature-value containing the brightness | luminance average and brightness | luminance histogram of the whole picked-up image shown in FIG. It is the figure showing the feature-value in the person's face area detected as a feature area from the picked-up image shown in FIG. It is a figure which shows the sample example of the picked-up image containing the person's skin area | region set with the main subject as the object of characteristic area detection. It is a figure which shows the result of having detected the area | region as a final candidate of a person skin area | region with respect to the picked-up image shown in FIG. It is a figure which shows an example of the feature-value containing the brightness | luminance average of the whole picked-up image shown in FIG. 33, and a brightness | luminance histogram. It is a figure showing the feature-value in the person's face area detected as a feature area from the picked-up image shown in FIG. It is a figure which shows the picked-up image sample containing the blue sky area | region set with a non-main subject as the object of characteristic area detection in 3rd Embodiment. It is a figure which shows the result detected as a final candidate of a blue sky area | region with respect to the picked-up image shown in FIG. It is a figure which shows an example of the feature-value containing the brightness | luminance average and brightness | luminance histogram of the whole picked-up image shown in FIG. It is a figure which shows the picked-up image sample containing the blue sky area | region set with a non-main subject as the object of characteristic area detection in 3rd Embodiment. It is a figure which shows the area example detected as a final candidate of a blue sky area | region with respect to the picked-up image shown in FIG. It is the figure which displayed an example of the feature-value containing the brightness | luminance average of the whole picked-up image shown in FIG. 40, and a brightness | luminance histogram. It is a figure which shows the brightness | luminance histogram after deleting the data of the blue sky area | region extracted as a characteristic area from the picked-up image shown in FIG. It is a figure which shows the brightness | luminance histogram of the image shown in FIG. 45 which correct | amended the picked-up image shown in FIG. It is a figure which shows the image obtained as a result of performing the correction process which concerns on 3rd Embodiment with respect to the picked-up image shown in FIG. It is a figure which shows the luminance histogram of the correction result image shown in FIG. It is a figure which shows the result of having performed the correction process which concerns on this embodiment with respect to the picked-up image sample shown in FIG.

Claims (13)

An acquisition process for acquiring images;
An extraction step of extracting a human skin region in the image based on chromaticity ;
By analyzing the luminance value of the image, a first analysis step of calculating an average luminance value of the entire image,
A second analysis step of calculating an average luminance value of the skin region based on a luminance value of a pixel included in the skin region extracted by the extraction step;
A correction coefficient is determined based on the average luminance value of the entire image and the average luminance value of the skin area, and the luminance value after correction of the skin area is obtained by multiplying the average luminance value of the skin area by the correction coefficient. A calculation step of calculating
Based on the calculation result of the calculation step, a table creation step of creating a gradation correction table in which an average luminance value of the skin region is converted into a luminance value after correction of the skin region ;
And a correction step of correcting the luminance of the image using the created gradation correction table. A determination step of determining whether or not to perform correction using the average luminance value of the skin region as compared with a preset spatial frequency characteristic ;
The calculating step, the determination if it is determined to perform the correction using an average brightness value of the skin area by step, the skin area based on the average luminance value of the entire image and the average luminance value of the skin area The image processing method according to claim 1 , further comprising: calculating a corrected luminance value . The correction step uses, as the gradation correction table, a lookup table that multiplies the average luminance value of the skin area by a correction coefficient based on the average luminance value of the entire image and the average luminance value of the skin area. The image processing method according to claim 2 , wherein the image is corrected. A determination step of determining whether or not to perform correction using the average luminance value of the skin region as compared with the preset spatial frequency characteristics ;
The calculating step, the determination if it is determined to perform the correction using an average brightness value of the skin area by step, the skin area based on the average luminance value of the entire image and the average luminance value of the skin area Calculating a luminance value after correction , and setting a range in which the number of gradations in the skin region should be secured based on luminance distribution information from the average luminance value of the skin region,
The table creation step is a gradation correction table in which an average luminance value of the skin area is converted into a luminance value after correction of the skin area, and the number of gradations is reduced within a range where the gradation should be secured. The image processing method according to claim 1 , wherein a gradation correction table having characteristics that are not generated is created. Wherein the correction step, a look-up table for multiplying the average luminance value and the correction coefficient based on the entire image the average brightness value and the skin area to the average brightness value of the skin area, calculated by the calculating step 5. The image correction is performed using a look-up table that includes a range between an upper limit value and a lower limit value as a range in which the number of gradations is not reduced as the gradation correction table. The image processing method as described. A background region extraction step of extracting a background region in the image ;
A determination step of determining whether or not to perform correction using the average luminance value of the background region as compared with a preset spatial frequency characteristic;
In the calculation step, when it is determined that the correction using the average luminance value of the background region is performed by the determination step, the image is based on the average luminance value of the entire image and the average luminance value of the background region. The image processing method according to claim 1, further comprising: calculating a corrected luminance value of the entire image in consideration of an influence degree of the background region at the time of correction. The calculation step calculates a luminance value after correction of the entire image based on a feature amount obtained by reducing or eliminating a feature amount occupied by the background region among average luminance values of the entire image. Item 7. The image processing method according to Item 6 . An acquisition means for acquiring an image;
Extraction means for extracting a skin area of a person in the image based on chromaticity ;
By analyzing the luminance value of the image, a first analyzing means for calculating an average luminance value of the entire image,
A second analysis unit that calculates an average luminance value of the skin region based on a luminance value of a pixel included in the skin region extracted by the extraction unit;
A correction coefficient is determined based on the average luminance value of the entire image and the average luminance value of the skin area, and the luminance value after correction of the skin area is obtained by multiplying the average luminance value of the skin area by the correction coefficient. Calculating means for calculating
Table creation means for creating a gradation correction table in which an average luminance value of the skin area is converted into a luminance value after correction of the skin area based on a calculation result of the calculation means;
The image processing apparatus characterized by using the tone correction table created in the above and a correcting means for correcting the luminance of the image. A determination means for determining whether or not to perform correction using the average luminance value of the skin region as compared with a preset spatial frequency characteristic ;
The calculating means, the case where it is determined that correction is performed using the average luminance value of the skin area by the determination unit, the skin area based on the average luminance value of the entire image and the average luminance value of the skin area The image processing apparatus according to claim 8, wherein the corrected luminance value is calculated . A determination means for determining whether or not to perform correction using the average luminance value of the skin region as compared with a preset spatial frequency characteristic ;
The calculating means, the case where it is determined that correction is performed using the average luminance value of the skin area by the determination unit, the skin area based on the average luminance value of the entire image and the average luminance value of the skin area Calculating a luminance value after correction , and setting a range in which the number of gradations in the skin region should be secured based on luminance distribution information from the average luminance value of the skin region,
The table creation means is a gradation correction table having different characteristics before and after the correction target value, and has a characteristic that does not reduce the number of gradations within a range where the gradation should be secured. The image processing apparatus according to claim 8, wherein a table is created. Background area extraction means for extracting a background area in the image;
Determination means for determining whether or not to perform correction using the average luminance value of the background region as compared with a preset spatial frequency characteristic ;
When the determination unit determines that correction using the average luminance value of the background region is performed by the determination unit, the calculation unit calculates the image based on the average luminance value of the entire image and the average luminance value of the background region. The image processing apparatus according to claim 8, wherein a luminance value after correction of the entire image is calculated in consideration of an influence degree of the background region at the time of correction. On the computer,
An acquisition means for acquiring an image;
Extraction means for extracting a skin area of a person in the image based on chromaticity ;
By analyzing the luminance value of the image, a first analyzing means for calculating an average luminance value of the entire image,
A second analysis unit that calculates an average luminance value of the skin region based on a luminance value of a pixel included in the skin region extracted by the extraction unit;
A correction coefficient is determined based on the average luminance value of the entire image and the average luminance value of the skin area, and the luminance value after correction of the skin area is obtained by multiplying the average luminance value of the skin area by the correction coefficient. Calculating means for calculating
Table creation means for creating a gradation correction table in which an average luminance value of the skin area is converted into a luminance value after correction of the skin area based on a calculation result of the calculation means;
Program for executing a correcting means for correcting the luminance of the image by using the gradation correction table created in the above. A computer-readable recording medium storing the program according to claim 12 .

Images