JP2004502212A - Red-eye correction by image processing - Google Patents

Abstract

The present invention includes a computer-implemented method, apparatus and computer-readable medium for recognizing and correcting red-eye in an image. The method includes an act of determining (620) the presence of at least one uniform color region and an act of determining whether the eye image surrounds at least one circular region of uniform color (630, 410, 420, 430, 440, 450) and replacing (640) the color of the at least one circular area with the color of the area directly surrounding the at least one uniform colored circular area.

Description

[0001]
FIELD OF THE INVENTION The present invention relates to the field of image processing, and in particular, to the field of red-eye correction of images.
[0002]
Description of the Background Art In flash photography, the "red-eye effect" is a well-known problem. This “red eye phenomenon” is at least one red dot that appears in the pupil of the eye image. One prior art technique for reducing the red-eye effect uses a preflash light emission technique. In the preflash light emission technology, a preflash is emitted when a shutter release button is pressed. When the preflash collides with the subject's eye, the pupil of the subject's eye contracts in response to bright light. Then, within a short time, the original flash is emitted before the pupil returns to normal size. Therefore, since the size of the pupil is reduced, the red point of the pupil is reduced. See, for example, U.S. Pat. No. 4,285,588 for suitable preflash light emission techniques.
[0003]
One of the disadvantages of the preflash light emission technique is that the captured image is less desirable because of the time delay between the desired and actual shutter release times. Another disadvantage of the preflash light emission technique is that preflash distorts the intended image, for example, by shrinking the pupil of the subject. Yet another disadvantage of the preflash light emission technique is that if the subject is not looking at the camera, the subject's eyes will not respond to the preflash and the prior art cannot prevent red eye.
[0004]
Accordingly, it is an object of the present invention to provide an apparatus and method for avoiding the red-eye effect without distorting an image of a subject.
[0005]
DISCLOSURE OF THE INVENTION The present invention includes a computer-implemented method for recognizing and correcting red eyes. The method includes determining (620) the presence of a red region, determining whether the eye image surrounds the red region (410, 420, 430, 440, 450, and 630), and directly mapping the red region. (510, 520, 530, and 640) for replacing the color of the red area with the color of the area surrounded by the circle.
[0006]
The invention further includes a computer device (100) suitable for searching for and correcting red eyes in the first image, the computer device comprising a first image (100) including a red dot and at least one eye image. 110) and a red point detector (104) coupled to the storage medium for receiving a first image from the storage medium and determining the presence of a red dot in the first image. A red point detector (104) for detecting and confirming the red point; and an eye detector (106) coupled to the red point detector, wherein the identified red point is detected by the red point detector (104). ) To determine if this red dot is located in the eye image, and to output the position of the red dot located in the eye image, coupled to the eye detector (106). Red eye remover (108) Receiving a position of the red dot disposed in the image, the color of the red dot disposed in the eye image, and a red-eye remover (108) for replacing the color of the area surrounding the red dot directly.
[0007]
The above objects and features of the present invention, as well as other more detailed and specific objects and features, will become more fully apparent from the following detailed description, taken in conjunction with the accompanying drawings.
[0008]
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention can be used with any image storage device, such as a camera (e.g., digital or non-digital), or for processing still images. It can be used in an environment (eg, a photo development minilab or the Internet based photo processing service). Furthermore, since the video can be a series of still images, embodiments of the present invention can be used with any video recorder.
[0009]
According to an embodiment of the present invention, there is disclosed at least an apparatus and process for searching for a red dot in an image of an eye and replacing the red dot with the color of the surrounding pupil or iris. Hereinafter, “red eye” means a red dot located in the eye image. With the apparatus and process described above, the red-eye effect is corrected without the need for a preflash that distort the image.
[0010]
FIG. 1A is a block diagram illustrating an exemplary embodiment of a red-eye correction device 100. The red-eye correction device (computer device) 100 can be implemented by any one or any combination of hardware, software, and firmware. The red-eye correction device 100 includes a storage medium 102, a red-spot detector 104, an eye detector 106, and a red-eye remover 108.
[0011]
The storage medium 102 in a suitable embodiment is a random access memory (RAM), a magnetic or optical storage medium, or an allocated storage area. For example, if the red-eye correction device 100 is used with a camera, an exemplary storage medium 102 is a flash memory such as an Iomega Click or an IBM Microdisk. The storage medium 102 stores an image 110, which in this embodiment is represented in a conventional red, green and blue (RGB) format. In this example, image 110 represents a still image that includes at least one red eye. Here, the image 110 is also the image represented by the image 110, unless otherwise specified. FIG. 2 shows an example image 200 represented by image 110, which includes a plurality of red dots 202-1 through 202-5.
[0012]
Storage medium 102 is coupled to provide image 110 to red point detector 104. The red point detection device 104 detects all red substantially circular regions ("red points") in the image 110. An exemplary solution for recognizing red dots in image 110 is provided, as in process 300 of FIG. 3A.
[0013]
In operation 310 of process 300, region A having N red pixels is identified by using a classical component algorithm such that each pixel (xi, yi) is "at least another pixel (xj, yj) of region A. 8 neighbors. Here, all the pixels in the region A are (x1, y1), (x2, y2),. . . , (Xn, yn). If | xi−xj | ≦ 1 and | yi−yj | ≦ 1, the two pixels (xi, yi) and (xj, yj) are “near eight”. Starting from the red pixel (x1, y1), region A is established by using the classic connected component algorithm to establish region A containing (x1, y1).
[0014]
In operation 320, the red point detector 104 determines whether the shape of the red pixel area A is circular. One suitable technique for determining whether region A is circular is shown in FIG. 3B.
In operation 320-1 of FIG. 3B, the red point detection device 104
xg = 1 / N (x1 + x2 + ... + xN) and yg = 1 / N (y1 + y2 + ... + yN)
The center of gravity (xg, yg) of the area A is determined using a conventional calculation such as
[0015]
In act 320-2, the red detector 104 encloses region A with a bounding box B to determine the compatibility C and the excentricity E of region A. The capacity C of the area A is defined as a ratio of the number N of pixels to the area of the bounding box B, that is, the area B, and is further defined by the following equation.
C = N / area B
Here, the area B = H × L
here,
H = yr-yl + 1
L = xr-xl + 1
here,
(Xl, yl) is the upper left coordinate of the bounding box B, obtained by xl = min (x1, x2 ... xN) and yl = min (yl, y2 ... yN),
(Xr, yr) is the lower right coordinate of the bounding box B, obtained by xr = max (x1, x2 ... xN) and yr = max (y1, y2 ... yN).
[0016]
The excentricity E of region A is defined as the ratio of the short sides of the bounding box B together with the ratio of the long sides of the bounding box B. The following equation defines the excentricity E: That is,
E = min (H, L) / max (H, L)
[0017]
In operation 320-3, the red detection device 104 determines whether the area A is circular, using the capacity C and the excentility E. In this embodiment, area A is circular if any of the following conditions are true: That is,
i. (N <minimum pixel) or ii. (N> minimum pixel) and (N <maximum pixel) and (C> minimum compactness) and (E> minimum excentricity)
here,
The smallest pixel is the threshold below which the area A is very small and is therefore perceived as a red dot. A typical value for the smallest pixel is 20,
The maximum pixel is a threshold above which region A is rejected as a red dot because it is too large. For example, the largest pixel can be fixed to a quarter or full size image,
The minimum compactness is a threshold below which region A is rejected because it is too compact. If the minimum compact is set to 0, this check is canceled. With the minimum compact set to 1.0, the red point detector 104 is highly selective with respect to capacity,
The minimum excentricity is a threshold below which region A is rejected because it is too elongated. If the minimum excentricity is set to 0, this check is cancelled. If the minimum excentricity is set to 1.0, the red point detector 104 will be very selective with respect to excentricity.
For a 10 cm × 14.9 cm photograph containing one or several people, typically the minimum pixel = 40, the maximum pixel = 1000, the minimum compact = 0.3 and the minimum excentricity = 0.3.
[0018]
Next, red point detector 104 informs eye detector 106 of all red points in pixel 110. A typical way to indicate each red point is to use the coordinates of each pixel within the red point, and the following properties of the red point: 1) Compatibility C, 2) Excentricity E, 3) Center of gravity (xg, yg) And 4) the radius rd of the smallest circle surrounding the red point. The radius rd is the maximum distance connecting the center of gravity (xg, yg) to each point (xi, yi) within the red point, and is further defined by the following equation. That is,
rd = max (d ((xg, yg), (xi, yi))), I = 1 to N
Here, the function d () is a Euclidean distance between two points.
Accordingly, the red point detection device 104 notifies the red point 202-1 to 202-5 to the eye detector 106 with respect to the example of FIG.
[0019]
The eye detector 106 detects whether each red dot recognized by the red point detection device 104 is spatially enclosed in the eye image. One suitable solution implemented by the eye detector 106 to determine whether a red dot is spatially enclosed in an eye image is described with respect to the process 400 of FIG. Process 400 is repeated for each of the red dots.
[0020]
In an operation 410 of the process 400, the eye detector 106 extends a ray from the center of gravity (xg, yg) to determine whether a first uniform color circular area surrounds a red dot. In one embodiment, the circular region of the first uniform color corresponds to the pupil. For example, referring to image 200 of FIG. 2, region 220 corresponds to a circular region of a first uniform color. If the first uniform color circular area surrounds the red dot, then in operation 420, eye detector 106 causes center of the first uniform color circular area (x ^*). , Y ^* Find out). For example, referring to image 200 of FIG. 2, point 225 coincides with the center of the first uniform color circular area. If the first uniform color circular area does not surround the red dot, the eye detector 106 determines that the red dot is not "red eye".
[0021]
In operation 430, eye detector 106 determines whether a circular region of the second uniform color exists. In this embodiment, eye detector 106 finds a second uniform color circular area by examining the size of the red dot. If the size of the red dot is small enough, the eye detector 106 estimates that the circular region of the first uniform color does not match the pupil image. The reason is that the pupil is not resolved. Eye detector 106 recognizes the first uniform color circular area as an iris and proceeds to operation 450. If the size of the red dot is large enough, as in the example of FIG. 2, the eye detector 106 recognizes the circular region of the first uniform color as a pupil and proceeds to operation 440.
[0022]
In operation 440, eye detector 106 determines the center (x ^{*) of} the first uniform color circular area ^. , Y ^* 2.) Find a second uniform colored circular area by extending another ray from The circular region of the second uniform color corresponds to the iris. For example, referring to image 200 of FIG. 2, region 230 corresponds to a circular region of a second uniform color. If such a circular region of the second uniform color is found, operation 450 continues.
[0023]
In operation 450, eye detector 106 determines the center (x ^{*) of} the first uniform color circular area ^. , Y ^* ) Or by extending another ray from the edge of the first uniform color circular area (operation 440) to provide a uniform color area that is outermost from the red point (eg, a first uniform color circle). An area or a lighter area than the second uniform color circular area ("middle area"). This bright area corresponds to the sclera. For example, referring to image 200 in FIG. 2, region 240 corresponds to this bright region. If the region surrounding the intermediate region is lighter than the intermediate region, then in operation 460, the eye detector 106 recognizes the red dot as "red eye".
[0024]
If the area surrounding the intermediate area is not lighter than the intermediate area, eye detector 106 determines that the red dot is not "red eye".
[0025]
FIG. 1B is a block diagram illustrating an exemplary embodiment of the eye detector 106 of the red-eye correction device 100. The eye detector 106 includes a first uniform region detector 152 that performs the operations 410, 420, and 430 of the process 400, a second uniform region detector 154 that performs the operation 440 of the process 400, and an operation 450 of the process 400. And a luminance detector 156 that performs the following.
[0026]
Many alternatives to process 400 can be used. For example, one alternative to the process 400 is described in U.S. Pat. No. 5,164,992. This specification is hereby incorporated by reference.
[0027]
After repeating the process 400 for all red points in the image 110, the eye detector 106 notifies the red eye remover 108 of all red eyes in the image 110. A suitable way for the eye detector 106 to signal the red eye to the red eye remover 108 is 1) the coordinates of each pixel in the red eye, 2) the center coordinates (x) of a circular region of a first uniform color (eg, pupil or iris). ^* , Y ^* ), 3) a ray r ^* defining the radius of the circular region of the first uniform color ^. This is done by using Red eye remover 108 also receives image 110 from storage medium 102.
[0028]
FIG. 5 illustrates a suitable process 500 performed by the red-eye remover 108 to correct each red-eye color. In operation 510 of the process 500, the red-eye remover 108 determines the color of the first uniform color circular area (ie, pupil or iris). One suitable technique for determining the color of the first uniform color circular area surrounding the red eye is as follows. Red eye remover 108 determines the average color of all pixels within the first uniform color circular area. In this embodiment, the average color is determined for red, green and blue (RGB). In operation 520, red eye remover 108 replaces the red, green and blue components of red eye in image 110 with the average red, green and blue of each of the first uniform color circular regions. Hereinafter, the “corrected image” refers to the image 110 whose red eye has been corrected by the operation 520. In operation 530, the red-eye remover 108 stores the corrected image 110 on the storage medium 102.
[0029]
FIG. 1C is a block diagram illustrating an exemplary embodiment of the red-eye remover 108 of the red-eye correction device 100. Red eye remover 108 includes a color detector 170 that performs operation 510 of process 500 and a color replacer 172 that performs operations 520 and 530 of process 500.
[0030]
Appropriate processing Appropriate processing according to one embodiment of the present invention is shown in FIG. In operation 610 of the process 600, the digital image 110 can be used in RGB format. In operation 620, a red dot in the digital image is detected and recognized. In operation 630, each red dot is examined to determine if the red dot is a red eye. Exemplary operation 630 is process 400 or the alternative described above. In operation 640, each red-eye color in the image 110 is replaced with an average color of a first uniform color circular area (ie, iris or pupil). An exemplary operation 640 is process 500 described above.
[0031]
Modifications The above description includes clarifying the operation of the preferred embodiment and does not limit the scope of the invention. For example, distortion of a non-red-eye image can be corrected. The scope of the present invention should be limited only by the appended claims. Obviously, many modifications will be possible to practitioners within the spirit and scope of the invention.
[Brief description of the drawings]
FIG. 1A shows a representative embodiment of a red-eye correction device 100 in a block diagram.
FIG. 1B illustrates, in a block diagram, an exemplary embodiment of an eye detector 106 of a red-eye correction device 100.
FIG. 1C illustrates, in a block diagram, an exemplary embodiment of a red-eye remover 108 of the red-eye correction device 100.
FIG. 2 shows an exemplary image 110 having a plurality of red dots 202-1 to 202-5.
FIG. 3A shows a suitable process 300 for recognizing a red dot.
FIG. 3B illustrates a suitable process for determining whether a red area is circular.
FIG. 4 illustrates a suitable process 400 for determining whether a red dot is spatially enclosed in an eye image.
FIG. 5 illustrates a suitable process 500 according to one embodiment of the present invention.
FIG. 6 illustrates a suitable process 600 according to one embodiment of the present invention.

Claims (19)

A computer-implemented method for recognizing and correcting red eyes, the computer-implemented method comprising:
Determining the presence of a red region;
Determining whether an image of the eye surrounds the red area;
Replacing the color of the red area with the color of the area immediately surrounding the red area. 2. The computer-implemented method of claim 1, wherein determining whether an image of the eye surrounds the red area comprises:
Sub-step of determining whether a first uniform color area surrounds the red area;
Determining whether a second uniform color area surrounds the first uniform color area;
Determining whether a region lighter than said second uniform color region surrounds said second uniform color region. 3. The computer-implemented method of claim 2, wherein said replacing step comprises:
A sub-step of determining a color of the first uniform color area;
Substituting the color of the red area with the color of the first uniform color area. 3. The computer-implemented method of claim 2, wherein the sub-step of determining whether a first uniform color area surrounds the red area comprises:
Sub-step of determining the center of the red area;
Sub-step of extending at least one ray from the center to determine whether the first uniform color region surrounds the red region. 3. The computer-implemented method of claim 2, wherein the sub-step of determining whether a second uniform color region surrounds the first uniform color region comprises:
Sub-step of determining the center of said first uniform color area;
Sub-step of extending at least one ray from the center of said first uniform color area to determine whether said second uniform color area surrounds said first uniform color area. The computer-implemented method of claim 5, wherein the computer-implemented method further comprises:
Computer-implemented extending at least one ray from the center of the first uniform color area to determine if an area lighter than the second uniform color area surrounds the second uniform color area. Method. The computer-implemented method of claim 1, wherein the red area is circular. 2. The computer-implemented method of claim 1, wherein determining whether an image of the eye surrounds the red area comprises:
Sub-step of determining whether a first uniform color area surrounds the red area;
Determining whether an area lighter than the first uniform color area surrounds the first uniform color area. 9. The computer-implemented method of claim 8, wherein the sub-step of determining whether a first uniform color area surrounds the red area comprises:
Sub-step of determining the center of the red area;
Sub-step of extending at least one ray from said center to determine whether said first uniform color region surrounds said red region. A computer-implemented method for diagnosing and correcting red-eye in an image, the computer-implemented method comprising:
Determining the presence of a red region;
Determining whether a first uniform color region surrounds the red region;
Determining whether an area brighter than the first uniform color area surrounds the first uniform color area;
Determining a color of the first uniform color area;
Replacing the color of the red area with the color of the first uniform color area. The computer-implemented method of claim 10, wherein the red area is circular. 11. The computer-implemented method of claim 10, wherein the first uniform color region is an iris image. 11. The computer-implemented method of claim 10, wherein areas brighter than the first uniform color area are scleral images. A computer device suitable for finding and correcting red eyes in a first image, the computer device comprising:
A storage medium for storing the first image including a red dot and at least one eye image;
A red point detector coupled to the storage medium, the red point detector receiving the first image from the storage medium, detecting the presence of the red point in the first image, and recognizing the red point. And
An eye detector coupled to the red point detector, receiving a recognized red point from the red point detector, determining whether the red point is located in an image of the eye, An eye detector that outputs the position of the red dot located in the image of
A red-eye remover coupled to the eye detector, the red-eye remover receiving the position of the red dot located in the eye image and changing the color of the red dot located in the eye image to the red dot; And a red-eye remover that replaces with the color of the area directly surrounding it. The computer device according to claim 14, wherein the eye detector comprises:
A module for determining whether a first uniform color area surrounds the red dot;
A module for determining whether an area brighter than the first uniform color area surrounds the first uniform color area. The computer device according to claim 15, wherein the red eye remover comprises:
A module for determining a color of the first uniform color area;
A module that replaces the color of the red dot with the color of the first uniform color area. The computer device according to claim 14, wherein the eye detector comprises:
A module for determining whether a first uniform color area surrounds the red dot;
A module for determining whether a second uniform color area surrounds the first uniform color area;
A module for determining whether an area lighter than the second uniform color area surrounds the second uniform color area. The computer device according to claim 17, wherein the red eye remover comprises:
A module for determining a color of the first uniform color area;
A module that replaces the color of the red dot with the color of the first uniform color area. A computer readable medium for finding and correcting red eyes in an image, the computer readable medium comprising:
An image with a red dot,
An instruction segment for determining whether a first uniform color area surrounds the red dot;
An instruction segment for determining whether an area lighter than the first uniform color area surrounds the first uniform color area;
An instruction segment for determining a color of the first uniform color area;
Instructions for replacing the color of at least one red dot with the color of the first uniform color region.