JP3709656B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that corrects poor color tone of a pupil in a subject image due to flash emission during color photography.
[0002]
[Prior art]
Conventionally, several techniques for correcting the poor color tone of the pupil, so-called red-eye, have been proposed (see US Pat. No. 5,130,789, JP-A-7-72537, etc.).
The technology disclosed in US Pat. No. 5,130,789 discloses a rectangular mask that surrounds the target eye, designates several points in the red eye area with a pointer, sets a red eye area based on color information, If it is within this region, it is determined as red-eye and corrected. At this time, the correction amount is changed depending on the distance from the boundary of the region.
[0003]
In addition, the technique disclosed in Japanese Patent Laid-Open No. 7-72537 specifies a target eye region, selects candidate pixels based on color information, and after labeling, takes into account information about each label, It will be evaluated whether there is. As for the correction method, the correction amount is changed for the red-eye main body and the peripheral portion.
[0004]
[Problems to be solved by the invention]
However, none of the methods describes a method for selecting a pupil color for correcting red eyes. For example, in commercially available retouching software (PhotoDeluxe, PictureIt), the method of selecting the eye color for correcting the red eye is that a plurality of color palettes are displayed and the operator clicks and selects them.
[0005]
In the above selection method, it is difficult for an operator to select a pupil color for correcting red eyes. That is, lightning damage vary each color in succession a color palette to be displayed, also, because the useless many choices because it includes also the color unrelated to the color of the pupil of the Westerners.
Further, since each color of the displayed color palette does not change continuously, fine adjustment of the color to be corrected cannot be performed.
[0006]
The present invention has been made paying attention to such a conventional problem, and an image processing apparatus that enables an operator to easily and accurately correct a red-eye phenomenon that occurs in color flash photography semi-automatically. The purpose is to provide.
[0007]
[Means for Solving the Problems]
For this reason, as shown in FIG.
A poor color tone pixel detecting means for detecting a pixel of poor color tone for a captured image in which there is one or more poor pupil color tone;
Correction pupil color candidate determination means for determining a pupil color candidate to be corrected using only one color information in the uniform color space as a variable based on the characteristics of the pupil color;
Correction pupil color selection means for inputting a value of the variable and selecting a pupil color to be corrected from among the candidate pupil colors;
Selection pupil color display means for displaying the selected pupil color; pupil color correction means for correcting a poor color tone pixel to the selected pupil color;
It is characterized by including.
[0008]
The color (color) is determined by the values of the two variables of the color information, but as a characteristic of the pupil color, one variable can be approximated as a function of the other variable. In accordance with the color characteristics, a candidate for a pupil color to be corrected using only one color information in the uniform color space as a variable is determined. The operator inputs the value of one color information variable , and selects the pupil color to be corrected while viewing the pupil color displayed by the selected pupil color display means. The pupil color correction unit corrects the poor color tone pixel detected by the poor color tone pixel detection unit to the color of the selected pupil.
[0009]
Thus, by selecting the color of the pupil to be easily corrected from color candidates Eye limited by a simple operation of the operator, Ru can be corrected.
[0010]
Also, the invention according to claim 2,
Said one color information only variables, characterized in that it is a u * or v * in CIEL * u * v * uniform color space.
[0011]
Since u * or v * in the uniform color space of CIE L * u * v * is two variables of the above-described color information, color information for determining candidate pupil colors for correcting one of them. Variable.
The invention according to claim 3
It said one color information only variables, characterized in that it is a a * or b * in CIEL * a * b * uniform color space.
[0012]
Since a * or b * in the uniform color space of CIE L * a * b * is also two variables of the above-described color information, color information for determining candidate pupil colors for correcting one of them. Variable.
The invention according to claim 4
The corrected iris color selection means may comprise means for setting the value of the color information by setting the position of the slide bar operator is displayed on the screen.
[0013]
In this way, it is possible to select the color of the pupil is corrected to input color information by a simple operation.
The invention according to claim 5
The corrected iris color selection means comprising means for inputting operator values directly color information.
[0014]
In this way, Ru can also select the color of the pupil to enter a number of color information directly.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 2 shows a system configuration of one embodiment. Digital image data of color images taken by a still video camera or digital image data obtained by scanning a color image taken by a silver salt film camera and developed on a film is stored in a storage device 1 such as an optical disk. ing.
[0017]
The image data stored in the storage device 1 is read by the control device 2 and displayed on the monitor 3.
While viewing the image displayed on the monitor 3, the operator sets the image area for the subject image in which red eyes are generated by flash photography as follows, whereby the control device 2 sets the set area. A red-eye pixel is extracted from the image and corrected, and the corrected image is displayed on the monitor 3.
[0018]
The red-eye correction routine according to the present invention will be described below with reference to the flowchart of FIG.
FIG. 3 shows a main routine for red-eye correction. Step (denoted as S in the figure. The same applies hereinafter) In 1, in the subject image including the pupil displayed on the monitor 3, a rectangular region surrounding the pair of eyes is set and the middle of the pair of eyes is set. Is designated as a reference point for color information and position information described later (see FIG. 6).
[0019]
As the method for setting the target area and the intermediate point, the operator may specify both the target area and the intermediate point, or specify only the target area, and the intermediate point may be obtained by calculation (for example, the center of the specified rectangular area). May be designated as an intermediate point), and only an intermediate point may be designated, and an area set in advance centering on the intermediate point may be set as the target region. The accuracy of red-eye correction is the highest because the first method manually specifies the target area and the intermediate point.
[0020]
A feature of such region designation is that, for example, a region surrounding a pair of pupils is designated even when there is only one poorly-colored pupil among a pair of pupils. In addition, when both of the pair of pupils have red eyes, it is possible to correct two red eyes at the same time by specifying the region once. For example, in the technique disclosed in the above-mentioned US Pat. No. 5,130,789, since the area is designated for each pupil, the degree of operator intervention increases.
[0021]
In step 2, the color information of the point specified in step 1 above, for example, the total value (R + G + B) of each lightness value of R (red), G (green), and B (blue) is calculated as a feature amount.
In step 3, the skin color pixel region is extracted by using, as a skin color pixel, a pixel whose feature amount difference with the feature amount of the specified point is within a specified threshold value. That is, since the designated point is a point near the middle of the pair of eyes and is located at the base of the nose, the color at that point is estimated to be skin color, so an area having a color close to the point is selected. It can be estimated as a skin color pixel. In addition, hue and saturation, which are general color information, may be used as feature quantities. In this way, pixels within the threshold value are labeled, and a labeling area including the designated point is extracted as a skin color area.
[0022]
In step 4, an average value Rskin of red chromaticity [= R / (R + G + B)] in the flesh color region is obtained.
In Step 5, in order to reduce the following calculation amount and the amount of data necessary for the calculation, a rectangular region circumscribing the skin color region extracted in Step 3 (a part between the eyes so as to circumscribe both eyes simultaneously) A horizontally long area) is set as a new target area. The rectangular area is automatically set. Although it may be performed manually, it is possible to cope with a case where the operator has designated a wide area including an area other than the face by performing it automatically.
[0023]
In step 6, the target area is divided into small areas. This small area division subroutine will be described with reference to the flowchart of FIG.
In step 11, an edge pixel is obtained within the set area. The edge pixel is obtained by, for example, using the lightness (R + G + B) as a feature amount and calculating the sum of absolute values of differences in brightness from the surrounding four pixels of the target pixel. Is determined. In addition, an edge detection filter such as a Sobel operator or a Prewitt operator may be used.
[0024]
In step 12, the edge pixels detected in step 11 are labeled. In step 13, a pixel surrounded by at least three sides by edge pixels having the same label is changed to the same label as the edge pixel. That is, the area surrounded by the edges becomes the same label. In this way, the setting area is divided into small areas having the same label.
[0025]
Returning to the main routine of FIG. 3, in step 7, an area that is likely to include a pupil with poor color tone is selected from the divided small areas as a pupil candidate area.
The pupil candidate area selection subroutine of step 7 will be described with reference to the flowchart of FIG.
[0026]
In the present embodiment, two of the plurality of divided small regions are regarded as one set, and the likelihood of pupils is obtained by calculating the following feature values for all combinations.
In step 21, the red chromaticity [= R / (R + G + B)] of each small area is obtained, the maximum value of red chromaticity (Ri) in the area, and the vicinity of the middle of the eye specified in step 1 above. When the point is the origin, the position (Xi, Yi) on the X-axis and Y-axis of the pixel having the highest value (Ri) is used as a feature amount. Here, i represents a value for identifying a small area.
[0027]
For example, when combining the small area 1 and the small area 2, the pupil candidate area is selected as follows.
In step 22, the feature amount C12 of color information and the feature amount P12 of position information are obtained by the following equations.
Feature value of color information C12 = R1 + R2
Position information feature amount P12 = X1 + X2 + Y1 + Y2
Here, the greater the feature amount of the color information, the smaller the feature amount of the position information, and the higher the pupil likeness. Therefore, in Step 23, the feature amount E12 indicating the pupil likeness is set to E12 = C12-P12, for example. A combination of small regions having the highest value is extracted as a candidate pupil region that has a high possibility of including a pupil having at least one of poor color tone (red-eye).
[0028]
Here, the smaller the feature amount of the position information, the higher the probability of pupils. When both eyes have red eyes, the left and right X1 are + and the other X2 is the point between the eyes. It is because it becomes small when it adds up as-.
Further, when only one pupil has red eyes, the combination of both pupils may be selected, so that the position information P12 is reduced even if there is no red eye in one pupil, and there is only one red eye. However, the feature amount of the color information is larger than that of the combination having no red eye.
[0029]
Returning to the main routine of FIG. 3, in step 8, a poor color tone (red-eye) pixel is extracted from the candidate pupil region extracted in step 7. The extraction method is as follows.
The threshold value TH set based on the feature value (Ri) of the color information of the target area, in which the red chromaticity [= R / (R + G + B)] of the target pixel is larger than the average value Rskin of the skin color area obtained in step 2 If it is larger than the subtracted value, it is determined as a red-eye pixel. That is, a pixel that satisfies the following expression is a red-eye pixel.
[0030]
R / (R + G + B)> Rskin∩R / (R + G + B)> Ri−TH
In step 9, the red-eye pixel extracted in step 8 is corrected to an appropriate color tone.
The correction procedure is shown in FIG. The case where the pupil color is expressed in a uniform color space of CIEL * u * v * is taken as an example. FIG. 8 shows a screen example in which the operator corrects the pupil.
The pupil color adjustment method will be described below with reference to the flowchart of FIG.
[0031]
In step 31, the operator selects a value of u * (or v *, hereinafter represented by u *) while viewing the preview image. In FIG. 8, u * can be selected by a slide bar called [Eye Color]. However, in addition to selecting with the slide bar, a method of directly inputting numerical values may be used.
In step 32, the value of v * is determined according to the following equation.
[0032]
v * = 0.71 × u * −0.24 (1)
This is because the color of the eyes of Westerners can be approximated by the relational expression expressed by equation (1) when expressed in the uniform color space of CIEL * u * v *. Here, the color of the eyes of the Westerners includes the color of the eyes of the entire human race, and includes the colors of black or brown eyes of orientals such as Japanese. FIG. 9 shows the distribution of a sample of Western eyes in the u * v * plane. As is clear from this figure, the relational expression between u * and v * is not limited to the expression (1), and any distribution that approximates these distributions to some extent may be used. Therefore, for example, the u * coefficient 0.71 in the equation (1) may be changed to 0.71 ± Δα, or the constant 0.24 may be changed to 0.24 ± Δβ, or a combination of the slope and the offset amount. (0.71,0.24), (0.71 + Δα, 0.24−Δβ), (0.71−Δα, 0.24 + Δβ) may be configured to be changeable, and fine correction can be performed while maintaining simple operability.
[0033]
In step 33, when the digital value of the red-eye pixel to be corrected is R, G, B, the value of L * (luminance) is determined according to the following equation.
L * = 100 * G / 255 (2)
Here, the same effect can be obtained by substituting B or an average value of G and B instead of G in the equation (2). The reason why the average value of G, B, G and B is used for determining the brightness is that the R component is an abnormally large value due to the occurrence of the red-eye phenomenon, but other components such as G and B are used. Is considered to exist in a size corresponding to the brightness of the original pupil color, so that the brightness of the pupil color can be estimated well using these values.
[0034]
In step 34, L * u * v * prepared in advance is converted to R, G, B using a table for converting R, G, B on the monitor.
In step 35, the color of the pupil is corrected using the converted R, G, and B. The corrected image is displayed as a preview image. The preview image may be a target red-eye image as shown in FIG. 8, or only the pupil color as shown in FIG.
[0035]
Next, a method for adjusting the range for correcting the red eye will be described.
The operator selects a range for correcting the red eye while viewing the preview image. In FIG. 8, a threshold value indicated by a slide bar called [Pupil Range] can be selected to adjust the range for correcting the red eye.
If the operator looks at the preview image and determines that only a part of the red eye is corrected, the threshold indicated by the slide bar is lowered, and conversely, if the part is corrected to the part that is not the red eye, Raise the threshold.
[0036]
In this way, the color of the pupil to be selected is not displayed with a plurality of discontinuous color patches as in the past, but is displayed in a form that changes continuously according to one parameter, so the operator can select Easy to operate and easy to operate.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration / function of the present invention.
FIG. 2 is a diagram showing a system configuration of an embodiment.
FIG. 3 is a flowchart showing a main routine of red-eye correction in the embodiment.
FIG. 4 is a flowchart showing a sub-region division subroutine of the same routine.
FIG. 5 is a flowchart showing a pupil candidate area selection subroutine of the routine described above.
FIG. 6 is a view showing a method for specifying a target area in the embodiment;
FIG. 7 is a flowchart showing a detailed processing subroutine for red-eye correction in the embodiment;
FIG. 8 is a diagram showing an example of a screen on which the operator corrects the pupil color in the embodiment;
FIG. 9 is a diagram showing the distribution of a sample of a Western pupil in the u * v * plane.
FIG. 10 is a diagram showing another example of a screen on which the operator corrects the pupil color in the embodiment;
[Explanation of symbols]
1 Storage device 2 Control device 3 Monitor

Claims (5)

A poor color tone pixel detecting means for detecting a pixel of poor color tone for a captured image in which there is one or more poor pupil color tone;
Correction pupil color candidate determination means for determining a pupil color candidate to be corrected using only one color information in the uniform color space as a variable according to the characteristics of the pupil color;
Correction pupil color selection means for inputting a value of the variable and selecting a pupil color to be corrected from among the candidate pupil colors;
Selected pupil color display means for displaying the selected pupil color;
Pupil color correction means for correcting poor color tone pixels to the selected pupil color;
An image processing apparatus comprising: Said one color information only variables, CIEL * u * v * image processing apparatus according to claim 1, characterized in that the u * or v * in the uniform color space. It said one color information only variables, the image processing apparatus according to claim 1, characterized in that the a * or b * in CIEL * a * b * uniform color space. The corrected iris color selection means, the operator of the claims 1 to 3, characterized in that it comprises means for setting the value of the color information by setting the position of the slide bar displayed on the screen The image processing apparatus according to any one of the above. The corrected iris color selection means, the image processing apparatus according to any one of claims 1 to 3, characterized in that it comprises a means for entering operator values directly color information.

Images