JPH10341352A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve operability in the correction of red-eye image. SOLUTION: The value of u* in the equal color space of CIEL*u*v* is selected (S31), the value of v* is calculated (S32) from the operation expression (such as v*=0.71u*-0.24, for example) for the characteristics of a pupil color, and luminosity L* for the color of a pupil is calculated (S33) from G and B as the colors of the pupil or from the average value of them. The calculated L*, u* and v* are converted (S34) into R, G and B values, and the color tone defective pixel (red-eye part) of pupil is corrected (S35) by these R, G and B values.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for correcting poor color tone of a pupil in a subject image due to flash emission during color photographing.

[0002]

2. Description of the Related Art Conventionally, there have been proposed several techniques for correcting the poor color tone of the pupil, so-called red-eye (US Pat.
130789, JP-A-7-72537, etc.). The technique disclosed in US Pat. No. 5,130,789 encloses a target eye with a rectangular mask, designates several points of a red-eye region with a pointer, sets a red-eye region based on color information, and sets a target pixel. Is within this region, it is determined to be red-eye and corrected. At this time, the correction amount is changed according to the distance from the boundary of the area.

The technique disclosed in Japanese Patent Application Laid-Open No. 7-72537 specifies a target eye area, selects candidate pixels based on color information, and after labeling, also considers information about each label. , And whether or not they have red eyes. As for the correction method, the correction amount is changed for the red-eye main body and the peripheral portion.

[0004]

However, neither method describes a method of selecting a pupil color for correcting red eyes. For example, commercially available retouching software (PhotoDel
In uxe, PictureIt), a method of selecting a pupil color for correcting red eyes is such that a plurality of color palettes are displayed and the operator clicks and selects from among them.

In the above selection method, it is difficult for an operator to select a pupil color for correcting red eyes. That is, the colors of the displayed color palette do not change continuously, and the colors that are not related to the color of the eyes of Westerners are also included.
Further, since the colors of the displayed color palette do not change continuously, fine adjustment of the color to be corrected cannot be performed.

The present invention has been made in view of such a conventional problem, and enables an operator to easily and accurately correct a red-eye effect occurring in a color flash photographing semi-automatically. It is an object to provide an image processing device.

[0007]

Therefore, according to the first aspect of the present invention, as shown in FIG. 1, in a photographed image in which one or more pupil poor color tone exists, the defective color tone pixel is detected. Poor color tone pixel detection means, correction pupil color candidate determination means for determining a pupil color candidate to be corrected based on the characteristics of the pupil color, and inputting a value of a color information parameter to obtain a pupil color candidate Correction pupil color selection means for selecting a pupil color to be corrected from among the pupil colors; selected pupil color display means for displaying the selected pupil color; and a pupil color for correcting poorly tone pixels to the selected pupil color. And a correcting means.

The correction pupil color candidate determining means determines pupil color candidates to be corrected by limiting only to colors having pupil color characteristics. The operator inputs the value of the color information parameter,
The pupil color to be corrected is selected while looking at the pupil color displayed by the selected pupil color display means. The pupil color correction means corrects the poorly-colored pixel detected by the poorly-colored pixel detection means to a selected pupil color.

As described above, it is possible to easily select and correct the pupil color to be corrected from the limited pupil color candidates by a simple operation of the operator. Also, in the invention according to claim 2, the correction pupil color candidate determination means determines a pupil color candidate to be corrected using only one color information as a variable,
The correction pupil color selection means inputs a value of the variable and selects a pupil color to be corrected.

Although the color (color) is determined by the values of two variables of color information, one of the variables can be approximated as a function of the other variable as a characteristic of the pupil color. The candidate of the pupil color to be corrected can be determined, and by inputting the value of the variable, the pupil color to be corrected can be selected. Further, in the invention according to claim 3, the variable having only one piece of color information is CIEL *.
It is u * or v * in a uniform color space of u * v *.

Since u * or v * in the uniform color space of CIEL * u * v * is the two variables of the above color information, a candidate for the pupil color for correcting any one of them is determined. To be used as a color information variable. Further, in the invention according to claim 4, the variable of only one piece of color information is C
It is characterized by being a * or b * in a uniform color space of IEL * a * b *.

Since a * or b * in the uniform color space of CIEL * a * b * is also two variables of the above-mentioned color information, a candidate for a pupil color for correcting one of them is determined. To be used as a color information variable. The invention according to claim 5 is characterized in that the correction pupil color selecting means includes means for setting the value of a color information parameter by setting the position of a slide bar displayed on a screen by an operator. I do.

With this configuration, the color information parameters can be input and the pupil color to be corrected can be selected by a simple operation. The invention according to claim 6 is characterized in that the correction pupil color selecting means includes means for directly inputting a numerical value of a color information parameter by an operator.

As described above, the pupil color can be selected by directly inputting the numerical values of the color information parameters. In the invention according to claim 7, the lightness (L *) of the color of the pupil to be corrected is provided.
Is determined on the basis of a color component other than R of the poorly-colored pixel.

The specific poor color tone of the pupil color is a red-eye phenomenon caused by flash photography, and the R component has an abnormally large value, but the other components such as G and B have the original pupil color. It is thought that it exists in the size according to the brightness of
The luminosity of the color of the pupil can be satisfactorily estimated using these values.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 2 shows a system configuration of an embodiment. Digital image data of a color image taken by a still video camera, or a color image taken by a silver halide film camera and developed on film,
Digital image data read by a scanner is stored in a storage device 1 such as an optical disk.

The image data stored in the storage device 1 is read by the control device 2 and displayed on the monitor 3. The operator sets an image area as follows for the subject image in which red-eye is generated by flash photography while viewing the image displayed on the monitor 3, and the control device 2 sets the set area. Are corrected by extracting a red-eye pixel from among them, and the corrected image is displayed on the monitor 3.

The red-eye correction routine according to the present invention will be described below with reference to the flowchart of FIG. FIG. 3 shows a main routine of red-eye correction. In step (denoted by S in the figure; the same applies hereinafter), in a subject image including a pupil displayed on the monitor 3, a rectangular region surrounding the periphery of a pair of eyes is set, and an area near the middle of the pair of eyes is set. Is designated as a reference point for color information and position information to be described later (see FIG. 6).

As a method for setting the target area and the intermediate point, the operator may specify both the target area and the intermediate point,
Only the target area may be specified, and the midpoint may be obtained by calculation (for example, the center of the specified rectangular area is calculated as the midpoint), or only the midpoint is specified, and the area set in advance with that center as the center is specified. It may be a target area. The accuracy of red-eye correction is the highest because the first method manually specifies the target area and the intermediate point.

A feature of such region designation is that, for example, a region surrounding a pair of pupils is identified even if there is only one pupil having poor color tone among a pair of pupils. Further, when both the pair of pupils have red eyes, two red eyes can be corrected simultaneously by one area designation. For example, US Pat.
In the technique disclosed in Japanese Patent No. 89, since an area is specified for each pupil, the degree of operator intervention increases.

In step 2, the color information of the point designated in step 1 such as R (red), G (green), B (blue)
The total value (R + G + B) of the respective brightness values is calculated as a feature value. In step 3, pixels having a difference between the feature value of the designated point and the feature value within a specified threshold value are extracted as skin color pixels, and a region of the skin color pixel is extracted. That is, since the designated point is located at the base of the nose near the middle of the pair of eyes,
Since the color at that point is estimated to be a skin color, an area having a color close to the point can be estimated to be a skin color pixel. In addition, hue and saturation, which are general color information, may be used as feature amounts. In this way, the pixels within the threshold value are labeled, and the labeling area including the designated point is extracted as the skin color area.

In step 4, an average value Rskin of red chromaticity [= R / (R + G + B)] in the flesh color area is determined. In step 5, in order to reduce the following calculation amount and the amount of data required for the calculation, a rectangular area circumscribing the skin color area extracted in step 3 (a part between eyes so as to circumscribe both eyes simultaneously) (Including one horizontally long area),
Set as a new target area. The setting of the rectangular area is
It is done automatically. Although it may be performed manually, it is possible to cope with a case where the operator specifies a wide area including an area other than the face by performing the operation automatically.

In step 6, the target area is divided into small areas. This small area division subroutine will be described with reference to the flowchart of FIG. In step 11, an edge pixel is obtained in the set area. The edge pixel is determined by, for example, using the brightness (R + G + B) as a feature value and calculating the sum of the absolute values of the differences in brightness from the four surrounding pixels of the target pixel. Is determined. In addition, Sobel operator, P
An edge detection filter such as a rewitt operator may be used.

In step 12, the edge pixels detected in step 11 are labeled. In step 13, pixels surrounded at least three sides by edge pixels of the same label are changed to the same label as the edge pixels. That is, the area surrounded by the edge has the same label. In this way, the setting area is divided into small areas having the same label.

Returning to the main routine of FIG. 3, in step 7, an area having a high possibility of including a pupil having a poor color tone is selected as a pupil candidate area from the divided small areas. The pupil candidate area selection subroutine in step 7 will be described with reference to the flowchart of FIG.

In the present embodiment, the pupil-likeness is obtained by calculating the following characteristic amounts for all combinations of two of the plurality of divided small areas as one set. In step 21, the red chromaticity [= R / (R + G +
B)], and when the origin is the maximum value (Ri) of the red chromaticity in the area and the point near the middle between the eyes specified in the step 1, the pixel of the maximum value (Ri) is determined. The position (Xi, Yi) on the X-axis and the Y-axis is defined as a feature value. Where i
Represents a value for identifying a small area.

For example, when combining small area 1 and small area 2, a pupil candidate area is selected as follows. In step 22, the characteristic amount C12 of the color information and the characteristic amount P12 of the position information are obtained by the following equation. Color information feature amount C12 = R1 + R2 Position information feature amount P12 = X1 + X2 + Y1 + Y2 Here, the larger the color information feature amount and the smaller the position information feature amount, the higher the pupil likeness. The characteristic amount E12 to be represented is, for example, E12 = C12−
P12 is set, and a combination of small areas having the highest value is extracted as a pupil candidate area having a high possibility of including a pupil in which at least one of them has poor color tone (red eye).

Here, the smaller the feature amount of the position information, the higher the pupil likelihood is when both eyes have red eyes.
This is because, for a point between the eyes, when one of the left and right X1 is set to + and the other X2 is set to-, the total becomes smaller. Also, the case where only one pupil has red eye may be that when a combination of both pupils is selected, the value of the position information P12 becomes small even if there is no red eye in one pupil, and only one red eye exists. However, this is because the feature amount of the color information is larger than a combination having no red eyes.

Returning to the main routine of FIG. 3, in step 8, from among the pupil candidate areas extracted in step 7,
Extract poor color (red eye) pixels. The extraction method is as follows. Red chromaticity of the target pixel [= R / (R + G +
B)] is the average value Rsk of the flesh color region obtained in step 2.
If the value is larger than “in” and larger than a value obtained by subtracting the set threshold value TH from the feature value (Ri) of the color information of the target area, the pixel is regarded as a red-eye pixel. That is, a pixel satisfying the following equation is set as a red-eye pixel.

R / (R + G + B)> Rskin @ R /
(R + G + B)> Ri-TH In step 9, the red-eye pixel extracted in step 8 is corrected to an appropriate color tone. FIG. 7 shows the procedure for correction. The case where the pupil color is represented by a uniform color space of CIEL * u * v * is taken as an example. FIG. 8 shows a screen example in which the operator corrects the pupil. Hereinafter, the pupil color adjustment method will be described with reference to the flowchart of FIG.

In step 31, the operator selects a value of u * (or v *, hereinafter represented by u *) while watching the preview image. In FIG. 8, u * can be selected with a slide bar of [pupil color]. However, a method of directly inputting a numerical value may be used instead of the selection using the slide bar. In step 32, the value of v * is determined according to the following equation.

V * = 0.71 × u * −0.24 (1) This is the relation expressed by the expression (1) when the color of the pupil of the Western people is represented by the uniform color space of CIEL * u * v *. This is because the expression can be approximated. Here, the color of the pupil of the Western people includes the color of the pupil of the entire human race, and includes the color of the black or brown-brown pupil of the Japanese oriental such as Japanese. FIG. 9 shows the distribution of the pupil samples of Westerners in the u * v * plane. As is clear from this figure, u * and v
The relational expression with * is not limited to the expression (1) and may be any expression that approximates these distributions to some extent. Therefore, for example, a configuration may be employed in which the coefficient of u * of Equation (1) 0.71 can be changed to 0.71 ± Δα, the constant 0.24 can be changed to 0.24 ± Δβ, or a combination of these inclinations and offset amounts (0.71,0.24), (0.71 + Δα, 0.24−Δβ),
(0.71−Δα, 0.24 + Δβ) may be configured to be changeable, and fine correction can be performed while maintaining simple operability.

In step 33, when the digital values of the red-eye pixel to be corrected are R, G, and B, the value of L * (luminance) is determined according to the following equation. L * = 100 * G / 255 (2) Here, a similar effect can be obtained by substituting B or an average value of G and B in place of G in equation (2). The reason why the average value of G, B, and G and B is used for determining the lightness is that the R component has an abnormally large value due to the occurrence of the red-eye effect, but the other components of G, B, etc. Is considered to exist in a size corresponding to the brightness of the original pupil color, and thus the brightness of the pupil color can be satisfactorily estimated using these values.

In step 34, L * u * prepared in advance
v * is converted to R, G, B using a table on the monitor for converting to R, G, B. In step 35, the pupil color is corrected using the converted R, G, B. The corrected image is displayed as a preview image. The preview image may target the target red-eye image as shown in FIG. 8 or may target only the pupil color as shown in FIG.

Next, a method of adjusting the range for correcting red-eye will be described. The operator selects a red-eye correction range while viewing the preview image. In FIG. 8, [pupil range]
By selecting the threshold value indicated by the slide bar, the range in which red-eye is corrected can be adjusted. When the operator views the preview image and determines that only part of the red eye has been corrected, the threshold indicated by the slide bar is lowered. You just need to raise the threshold.

In this way, the pupil color to be selected is displayed in a form that is continuously changed by one parameter, instead of being displayed by a plurality of discontinuous color patches as in the related art. The operator can easily select and operate.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration and functions of the present invention.

FIG. 2 is a diagram showing a system configuration according to an embodiment.

FIG. 3 is a flowchart showing a main routine of red-eye correction in the embodiment.

FIG. 4 is a flowchart showing a small area division subroutine of the routine.

FIG. 5 is a flowchart showing a subroutine for selecting a pupil candidate area in the same routine.

FIG. 6 is an exemplary view showing a method of specifying a target area in the embodiment.

FIG. 7 is a flowchart showing a detailed subroutine of red eye correction in the embodiment.

FIG. 8 is a view showing an example of a screen on which the operator corrects the pupil color in the embodiment.

FIG. 9 is a diagram showing a distribution of a sample of pupils of Europeans and Americans on a u * v * plane.

FIG. 10 is a diagram showing another example of the screen for correcting the pupil color by the operator in the embodiment.

[Explanation of symbols]

 Reference Signs List 1 storage device 2 control device 3 monitor

Claims (7)

[Claims] 1. A poorly-colored pixel detecting means for detecting a poorly-colored pixel in a captured image in which one or more poorly-colored pupils are present, the pupil color to be corrected according to the characteristics of the pupil color. Correction pupil color candidate determining means for determining a candidate, correction pupil color selecting means for inputting a value of a color information parameter and selecting a pupil color to be corrected from among the pupil color candidates, and the selected pupil An image processing apparatus comprising: a selected pupil color display unit that displays the color of the pupil; and a pupil color correction unit that corrects the poorly-colored pixel to the color of the selected pupil. 2. The correction pupil color candidate determination means determines a pupil color candidate to be corrected using only one color information as a variable, and the correction pupil color selection means inputs a value of the variable to perform correction. The image processing apparatus according to claim 1, wherein a pupil color to be selected is selected. 3. The variable of only one color information is CIEL
3. The image processing apparatus according to claim 2, wherein the image is u * or v * in a uniform color space of * u * v *. 4. The variable of only one color information is CIEL
The image processing apparatus according to claim 2, wherein the image processing apparatus is a * or b * in a uniform color space of * a * b *. 5. The apparatus according to claim 1, wherein said correction pupil color selecting means includes means for setting a value of said color information parameter by setting a position of a slide bar displayed on a screen by an operator. To any one of claims 4 to
The image processing device according to any one of the above. 6. The image processing apparatus according to claim 1, wherein said correction pupil color selecting means includes means for directly inputting a numerical value of a color information parameter by an operator. . 7. The apparatus according to claim 1, further comprising means for determining the lightness (L *) of the color of the pupil to be corrected based on a color component other than R of the poor-tone pixel. The image processing device according to one of the above.