JP2006344166A - Image processing device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To optimally correct a failure eye image. <P>SOLUTION: A face and an eye are detected from an image. Based on the detection result, whether an eye image is the failure eye image such as an image of a closed eye, a half closed eye, a red eye, an eye not directed to a camera or not is determined by plate pattern matching. When the failure eye image is determined, a correction candidate image for an area determined as the failure eye image is retrieved from a previously created image database, and the size, angle, gradation, brightness, and hue of the retrieved candidate image are corrected for composing a corrected candidate image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method for correcting a digital photographic image.

  In recent years, with the spread of digital cameras, digitization of photographic images has become easier. In addition, the quality of printing apparatuses, particularly inkjet systems, has been improved, and photographic print results can be obtained from digital images taken by users. Furthermore, it is also easy to edit and process captured image data using various application software on a personal computer.

  From such a background, development of an environment that can print a photographed image more easily is proceeding.

  In particular, when a person is photographed, the eye of the person who is the subject is often closed or half-eye, or the photographed person's eyes often have red eyes. This is because the subject is not paying attention to the moment when the shutter is released, or the eyes are closed due to the flash pre-flash, or the flash light is reflected by the open pupil and the eyes are red. It happens by being photographed.

  In addition, when taking a group photo, it is not possible to synchronize the timing of the blinks of all the people to be photographed, so a plurality of images are taken to obtain an image with all eyes open as much as possible. .

  A photograph with closed eyes or half-eye is often taken as a bad impression of a person and is generally judged as a failed photograph.

  As a method for correcting such a failed photograph with closed eyes, methods disclosed in Patent Documents 1 and 2 are known.

JP 2001-197296 A JP 2002-199202 A JP 2002-183731 A JP 2003-30667 A JP-A-8-63597 JP 2000-105829 A JP-A-8-77334 JP 2001-216515 A JP-A-5-197793 JP-A-11-53525 JP 2000-132688 A JP 2000-235648 A JP-A-11-250267 Japanese Patent Registration No. 2541688

  Although there is a method of combining other eyes with closed eyes, half-eye, or red-eye areas by using manual image processing software, the user can easily obtain satisfactory results. It was difficult to get.

  The invention of Patent Document 1 discloses a method for correcting an image with closed eyes, but does not describe correction for an image when an eye image such as a half-eye or a red eye is not perfect.

  Further, according to the invention described in Patent Document 2, since two continuous images are required, there is a problem that there are restrictions on photographing conditions and it is not easy to use.

  Accordingly, a first object of the present invention is to provide an image processing apparatus and method capable of solving such problems and performing the failure correction optimum for the failed photographed image.

  The second aspect of the present invention provides an image processing apparatus and method capable of solving such problems and changing the search condition and the quality of correction according to the size of the failure area and the printing condition. The purpose.

  The invention of claim 1 is a face organ detection means for detecting a face and eyes from an image, a failure eye determination means for determining whether or not the eye is a failure based on a result of eye detection by the face organ detection means, A correction candidate image search unit that searches for a correction candidate image for an area determined to be a failure by the failure determination unit; a candidate image correction unit that corrects the candidate image searched by the correction candidate image search unit; And combining means for combining the candidate images corrected by the candidate image correcting means.

  In the first aspect of the invention, the failure eye determination means can determine whether or not it is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern.

  In the invention of claim 1, the failure eye determination means can determine that the eyes are closed.

  In the first aspect of the invention, the failure eye determination means can determine that the eyes are half closed.

  In the first aspect of the invention, the failure eye determination means can determine that the eyes are red eyes.

  According to a sixth aspect of the present invention, there is provided a facial organ detection means for detecting a face and eyes from an image, a failure eye determination means for determining whether or not a failure has occurred based on a result of eye detection by the face detection means, Evaluation means for evaluating whether or not the quality required for the failure area determined to be a failure by the failure determination means is greater than or equal to a predetermined standard, and the evaluation means evaluates that the quality is not greater than or equal to the predetermined standard. A correction candidate image search unit that searches for a correction candidate image for an area determined to be a failure by the failure determination unit, and a candidate image that corrects the candidate image searched by the correction candidate image search unit The image processing apparatus includes a correcting unit and a combining unit that combines the candidate images corrected by the candidate image correcting unit.

  In the invention of claim 6, the failure eye determination means can determine whether or not the eye is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern.

  In the invention of claim 6, the evaluation means can evaluate the ratio between the area determined to be a failure and the size of the image.

  In the invention of claim 6, the evaluation means can perform evaluation based on printing conditions.

  In the invention of claim 9, the printing condition may be a printing medium.

  In the invention of claim 9, the printing condition can be a print quality.

  In the invention of claim 9, the printing condition can be evaluated from the printing layout.

  The invention of claim 13 includes a face organ detection step for detecting a face and eyes from an image, a failure eye determination step for determining whether or not the eye is a failure based on a detection result of eyes by the face detection step, A correction candidate image search step for searching for a correction candidate image for the area determined to be a failure by the failure determination step, a candidate image correction step for correcting the candidate image searched by the correction candidate image search step, A combining step of combining the candidate images corrected by the candidate image correcting step.

  In the invention of claim 13, the failure eye determination step can determine whether or not the eye is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern.

  In the invention of claim 13, the failure eye determination step can determine that the eyes are closed.

  In the invention of claim 13, the failure eye determination step can determine that the eyes are half closed.

  In the invention of claim 13, the failure eye determination step can determine that the eyes are red eyes.

  The invention of claim 18 includes a face organ detection step for detecting a face and eyes from an image, a failure eye determination step for determining whether or not the eye is a failure based on an eye detection result by the face detection step, An evaluation step for evaluating whether or not the quality required for the failure area determined to be a failure by the failure determination step is greater than or equal to a predetermined criterion, and the evaluation step evaluates that the quality is not greater than or equal to the predetermined criterion. A correction candidate image search step for searching for a correction candidate image for the region determined to be a failure by the failure determination step, and a candidate image for correcting the candidate image searched by the correction candidate image search step It comprises a correction step and a synthesis step for synthesizing the candidate images corrected by the candidate image correction step.

  In the invention of claim 18, the failure eye determination step can determine whether or not it is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern.

  In the invention of claim 18, the evaluation step can evaluate a ratio of the size of the area determined to be a failure and the size of the image.

  In the invention of claim 18, the evaluation means can perform evaluation based on printing conditions.

  In the twenty-first aspect, the printing condition may be a print medium.

  In the invention of claim 21, the printing condition can be a print quality.

  In the invention of claim 21, the printing conditions can be evaluated from a printing layout.

  26. The program according to claim 25, wherein the computer detects a facial organ detection unit that detects a face and an eye from an image, and determines whether or not a failure has occurred based on a detection result of the eye by the facial organ detection unit. Means, a correction candidate image search means for searching for a correction candidate image for the area determined to be a failure by the failure determination means, and a candidate image correction for correcting the candidate image searched by the correction candidate image search means And a combining means for combining the candidate images corrected by the candidate image correcting means.

  27. The program according to claim 26, wherein the computer detects facial organ detection means for detecting a face and eyes from an image, and failure eye determination means for determining whether or not a failure has occurred based on a result of eye detection by the face detection means. And an evaluation means for evaluating whether or not the quality required for the failure area determined to be a failure by the failure determination means is not less than a predetermined standard, and the evaluation means does not exceed the predetermined standard. The correction candidate image search means for searching for a correction candidate image for the area determined to be a failure by the failure determination means, and the candidate image searched by the correction candidate image search means is corrected. And a candidate image correcting means for combining the candidate image corrected by the candidate image correcting means.

  According to the present invention, because it is configured as described above, it is possible to perform failure eye correction that is optimal for a captured image of a failed eye.

  Further, according to the present invention, since it is configured as described above, the search condition and the quality of correction can be changed according to the size of the failure area and the printing condition.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Although the present embodiment describes an example in which the present invention is applied to a printing system, the present invention is not limited to this example.

<First Embodiment>
FIG. 1 shows a first embodiment of the present invention. This is an example of an image processing apparatus.

(Image processing device)
In FIG. 1, an image processing apparatus 101 includes a central processing unit (CPU) 102, a read only memory (ROM) 103, a random access memory (RAM) 104, a secondary storage device 105, a display device 106, an input device 107, an I / I An F (interface) control unit 108 is included.

  The CPU 102 controls peripheral devices by operating the programs stored in the ROM 103, RAM 104, and secondary storage device 105 in a timely manner.

  The ROM 103 stores a program called BIOS (basic input / output system) for controlling input / output of a peripheral device, which is executed when the power is turned on, and initializes the RAM 104 and other input / output devices. The RAM 104 provides a working memory space. The secondary storage device 105 includes a large-capacity magnetic storage device represented by an HDD (hard disk drive) or a CD-ROM (compact disk-ROM), and stores an OS (operating system), applications, and various data. Has been.

  The display device 106 is a display device such as a CRT (cathode ray tube) display or a liquid crystal display. The display device 106 displays application processing results, messages, and the like. The input device 107 is a device for inputting user instructions such as a keyboard and a mouse, and provides a function for inputting user instructions to the image processing apparatus 101.

  The I / F control unit 108 provides an interface with an external peripheral device. As the interface, IEEE1284, which is also called a parallel port, SCSI (small computer system interface), USB (universal serial bus), IEEE1394, or a serial bus can be used. Which interface is used depends on the configuration of the image processing apparatus 101 and the correspondence of external peripheral devices.

  External peripheral devices include a digital still camera 109, a memory card reader / writer 110, and a printer 111.

  The digital still camera 109 is a photographing device and stores photographed image data photographed by a user. For captured image data, the digital still camera 109 is connected to the image processing apparatus 101 via the I / F control unit 108, or a recording medium built in the digital still camera is taken out and inserted into the memory card reader / writer 110. Thus, the captured image data can be transferred to the secondary storage device 105 and stored. Further, the stored photographed image data can be printed by the printer 111.

(Image processing application)
FIG. 2 shows an example of a user interface (hereinafter referred to as “UI”). Reference numerals 202 to 205 denote an image selection tab, a correction selection tab, a paper selection tab, and a layout / printing tab, respectively.

  When the image selection tab 202 is selected, although not shown, an image for correction or printing can be selected. An input device 107 (FIG. 1) is used to select an image. When the paper selection tab 204 is selected, although not shown, it is possible to select the type and size of paper to be used for printing and the print quality. When layout / print 205 is selected, although not shown, a print layout, for example, one image is printed on the entire sheet, or four images are arranged on the entire sheet when a plurality of images are selected. You can select a layout such as printing.

  When the correction selection tab 203 is selected, a UI as shown in FIG. 2 is displayed on the display device 106, and a button 206 for selecting failure correction processing and a button for selecting another correction can be selected on the UI. ing. Further, an image 207 to be corrected is displayed. The user can correct the correction intensity and change the correction pattern while viewing the image 207.

(Image analysis)
FIG. 3 is a flowchart illustrating an example of an image processing procedure. Here, the user executes the image processing application, selects the correction target image 207 in the UI of FIG. 2, selects the failure correction processing 206 for the correction target image 207, and presses the execution button 210 to perform correction. Image processing will be described under the assumption that a start instruction has been issued.

  First, the user selects the automatic tab 208 or the manual tab 209 as to whether to execute the failure correction process automatically or manually. When the manual tab 209 is selected, a failure area in the correction target image 207 is designated by the input device 107. When the automatic tab 208 is selected and the execution button 210 is pressed, face detection is performed on the correction target image 207 in S301.

  As this face detection method, various methods have already been proposed. According to the technique described in Patent Literature 3, an eye area is detected from an input image, and the periphery of the eye area is set as a face candidate area. When the brightness gradient for each pixel and the brightness gradient weight are calculated for this face candidate area, and these values are compared with the preset ideal face reference image gradient and the gradient weight. In addition, when the average angle between the gradients is equal to or smaller than a predetermined threshold, it is determined that the input image has a face area.

  Further, according to the technique described in Patent Document 4, first, a skin color area is detected from an image, and an eye position is detected by detecting a human iris color pixel in the area.

  Further, according to the method described in Patent Document 5, the degree of matching between a template having a plurality of face shapes and an image is calculated, the template with the highest matching degree is selected, and the highest degree of matching is determined in advance. If it is equal to or greater than the threshold value, the region in the selected template is set as a face candidate region, and the eye position is detected by using this template.

  Furthermore, according to the method described in Patent Document 6, first, the nose image pattern is used as a template, and the entire image or a specified region in the image is scanned, and the best matching position is output as the nose position. Next, the region above the nose position of the image is considered as the region where the eyes exist, and the eye presence region is scanned using the eye image pattern as a template for matching, and a set of pixels having a degree of matching greater than a certain threshold value. A certain eye presence candidate position set is obtained. Further, continuous regions included in the eye presence candidate position set are divided as clusters, and the distance between each cluster and the nose position is calculated. By determining the cluster having the shortest distance as the cluster in which the eye exists, the organ position is detected.

  In addition, as a method for detecting the face and organ positions, many methods such as Patent Documents 7 to 14 have been proposed. In this embodiment, the method is not particularly limited.

  Here, as a result of the face detection, for example, an image 401 as shown in FIG. 4 is assumed as follows. That is, the face area 402 is a rectangular area defined by two points, LT (upper left) and RB (lower right). The right eye region 403 is detected at two points defined by REyeT (right eye head) and REyeB (right eye corner), and the left eye region 404 is detected by two points defined by LEyeT (left eye head) and LEyeB (left eye corner). Shall be detected.

  If a face and eyes are detected in S301, it is determined whether or not the eye area detected in S302 is a failure. Here, for example, the eyes of a person are half-opened (half-eye) or closed, eyes whose eyes are not facing the camera, red eyes due to flash light, eye color Is a discolored eye. To determine whether or not this is a failure, the eye area is divided into white eyes, irises, and pupil portions of the eye regions, and an evaluation is made as to whether the iris portion has a nearly circular shape. Furthermore, pattern matching with an eye template (image with open eyes) is performed, and evaluation is made with a predetermined accuracy that the eyes are open, closed, or in a state of change. In addition, if it is determined whether the pupil portion is red, it is possible to evaluate whether the pupil is red. However, the evaluation item for the failure is not limited to this, and it is possible to determine the failure by defining various evaluation amounts.

  In S303, determination of failure is performed for all detected eye regions based on the plurality of evaluation amounts calculated in S302.

  If no failure is detected in S303, the process proceeds to S304 to notify the user that no failure has been detected and wait for the next instruction. It is impossible to set the detection rate of automatic failure detection to 100%. For this reason, even if a failure is photographed in the image 401, it may not be detected. If it cannot be detected, in S305, the user can designate the eye area that is considered to be a failure and instruct the execution of the failure correction. If there is no area designation from the user and the cancel button 211 is pressed, the process proceeds to S306 to display a message asking the user whether to continue or end the failure correction. If the user wishes to continue failure correction in S306, the process proceeds to S304 again. On the other hand, if it is desired to end the failure correction, the process ends.

  If a failure is detected in S303, or if the user designates a failure region in S306, the process proceeds to S307. In S307, a correction candidate image is searched. The correction candidate image database 310 is an image database created in advance. The image database stores eye images to which information for classifying eye image features such as aspect ratio, brightness, iris color, race, age, and the like is added. Note that a search condition may be given by designating a check box on the automatic tab 208. Furthermore, you may enable it to add to the correction candidate image DB the image of the eyes which succeeded from the user's own or family's photograph.

  In S307, the feature amount of the detected face area 402 is determined, and an optimal correction candidate image is selected from the correction candidate image DB. The search result may be shown as a candidate image to the user. If the user does not like the selected candidate image, the satisfaction of the correction result may be increased by showing the next candidate image.

  The correction candidate images for the plurality of failed areas detected in S307 are selected, and the process proceeds to S308. In S308, the correction candidate image is converted so as to match the coordinates REyeT and REyeB of the detected eye region or the values of LEyeT and LEyeB. By this conversion, the color adjustment of the correction candidate image is performed in consideration of the size adjustment, the angle adjustment, the blurring of the peripheral area, the brightness of the color near the eye area of the original image, the hue, and the like.

  In step S309, the correction candidate image created in step S308 can be combined with the original image to obtain a failure correction image.

<Second Embodiment>
In the first embodiment, no consideration is given to the size of the failure for the entire image, the size of the failure for actual printing, and the like. However, if the size of the failure in the captured image is small, it may not be necessary to search for a correct correction candidate image or to correct the correction candidate image. Further, in the method of using the failure correction image, when printing is performed with a printer, it may not be necessary to strictly perform correction depending on the print size and print quality.

  Therefore, in the second embodiment, the size of the failure and the size of the failure during use such as printing are considered. Hereinafter, the failure correction considering these points will be described. FIG. 5 shows a flowchart of image processing. In FIG. 5, the steps from S301 to S310 are the same as those in FIG.

  If a failure is detected in step S303, the process proceeds to step S511. In step S511, the size of the failure and how it is printed are checked.

  In S512, the quality required for the failed image area is determined from the result obtained in S511.

  FIG. 6 shows an example of an image in which a failure is taken. In FIG. 6, reference numeral 601 denotes the entire image, and the vertical and horizontal directions can be represented by the number of pixels. For example, an image taken with a digital camera having 2 million pixels is about 1600 pixels × 1200 pixels in the vertical and horizontal directions. The eye size 602 indicates the size of the failed eye. The eye size 602 can be obtained by a distance between XEyeT and XEyeB (X is R or L).

  The ratio between the image 601 and the size of the failed eye is defined by the ratio between the smaller number of pixels of the image 601 in the vertical and horizontal directions and the eye size 602. When this ratio is equal to or less than a predetermined value, it is evaluated that the quality of the failed image is low. For example, if the ratio of the size of failure to the image is 10% or less, it is determined that the quality is low.

  Next, the quality of the failure is determined from the printing conditions. For example, when the image 601 is printed on A4 size paper, and when four images are simultaneously printed on A4 size paper as indicated by 605 in FIG. 2 Therefore, it is possible to calculate the size of the failed eye on the medium to be printed on the basis of the A4 size and use it for evaluating the quality of the failed eye area.

  Also, the print quality can be specified during printing. As the print quality parameters, media type and print image quality designation can be used. In general, it can be determined that high quality media such as glossy paper and photographic paper have high quality, and plain paper and the like have low quality. Therefore, when plain paper is selected as the print medium, the evaluation amount of the quality of the failed area is reduced by a predetermined ratio. For example, for plain paper, the quality of the failed area is halved. Further, the quality of the print image is often defined by the image quality of printing such as draft printing, standard printing, and high-quality printing. Therefore, in the draft printing which does not cause any problem even if the quality of the print image quality is low, the evaluation amount of the quality of the failed area is reduced by a predetermined ratio such as 1/2.

  It is preferable to check whether the print media conditions are the same in the paper selection tab 204 and the layout / print tab 205 of the application 201.

  As a result of evaluating the quality in S512, if the quality of the failure area is larger than a predetermined value, the processing from S307 to S309 is performed as in the first embodiment. On the other hand, if the quality is smaller than the predetermined value, the process proceeds to S513, where it is checked whether there is an image that has not been determined to be a failure in the same image, and whether it can be used as a candidate area in S514. Make a decision. In the determination of S514, when the predetermined ratio of the size of the failure and the non-failure area is, for example, not less than 0.5 and not more than 1.5, it can be adopted as a candidate area. If there are a plurality of non-failed areas, the most similar one is selected from a plurality of evaluation amounts such as the size of the eye area and the color matching degree, and the process proceeds to S308.

  On the other hand, if there is no candidate eye region in S514, a candidate image is selected from the standard eye images stored in advance in the standard candidate image database (S516), and the process proceeds to S308. The steps after S308 are the same as those in the first embodiment.

(Other)
Even if the present invention is applied to a system constituted by a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), it is applied to an apparatus (for example, a copying machine, a facsimile machine, etc.) comprising a single device. May be.

  Also, an object of the present invention is to supply a storage medium (or recording medium) that records a program code of software that realizes the functions of the above-described embodiments to a system or apparatus, and to perform computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS running on the computer performs actual processing based on an instruction of the program code. Needless to say, a case where the function of the above-described embodiment is realized by performing part or all of the processing is also included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows the 1st Embodiment of this invention. It is a figure which shows an example of a user interface. It is a flowchart which shows an example of the image processing procedure in 1st Embodiment. It is explanatory drawing for demonstrating the example of a face and eyes detection. It is a flowchart which shows an example of the image processing procedure which can be applied to 2nd Embodiment. It is a figure which shows the example of a failure image, and the example of printing.

Explanation of symbols

101 Image processing apparatus 102 CPU
103 ROM
104 RAM
105 Secondary Storage Device 106 Display Device 107 Input Device 108 I / F Control Unit

Claims (26)

Facial organ detection means for detecting a face and eyes from an image;
A failure eye determination means for determining whether or not the eye is a failure based on the detection result of the eyes by the facial organ detection means;
Correction candidate image search means for searching for a correction candidate image for an area determined to be a failure by the failure eye determination means;
Candidate image correcting means for correcting the candidate image searched by the correction candidate image searching means;
An image processing apparatus comprising: a combining unit that combines the candidate images corrected by the candidate image correcting unit.   The image according to claim 1, wherein the failure eye determination unit determines whether or not the eye is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern. Processing equipment.   The image processing apparatus according to claim 1, wherein the failure eye determination unit determines that the eyes are closed.   The image processing apparatus according to claim 1, wherein the failure eye determination unit determines that the eyes are half closed.   The image processing apparatus according to claim 1, wherein the failure eye determination unit determines that the eyes are red eyes. Facial organ detection means for detecting a face and eyes from an image;
Failure eye determination means for determining whether or not the eye is a failure based on the detection result of the eyes by the face detection means;
Evaluation means for evaluating whether or not the quality required for the failure area determined to be a failure by the failure determination means is greater than or equal to a predetermined reference;
A correction candidate image search means for searching for a correction candidate image for an area determined to be a failure by the failure eye determination means when the evaluation means evaluates not to be above the predetermined reference;
Candidate image correcting means for correcting the candidate image searched by the correction candidate image searching means;
An image processing apparatus comprising: a combining unit that combines the candidate images corrected by the candidate image correcting unit.   7. The image according to claim 6, wherein the failure eye determination unit determines whether or not the eye is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern. Processing equipment.   The image processing apparatus according to claim 6, wherein the evaluation unit evaluates a ratio between the area determined to be a failure and the size of the image.   7. The image processing apparatus according to claim 6, wherein the evaluation unit performs evaluation based on printing conditions.   The image processing apparatus according to claim 9, wherein the print condition is a print medium.   The image processing apparatus according to claim 9, wherein the printing condition is print quality.   The image processing apparatus according to claim 9, wherein the print condition is evaluated from a print layout. A facial organ detection step for detecting a face and eyes from an image;
A failure eye determination step for determining whether or not the eye is a failure based on the detection result of the eyes in the face detection step;
A correction candidate image search step for searching for a correction candidate image for the region determined to be a failure by the failure determination step;
A candidate image correcting step of correcting the candidate image searched by the correction candidate image searching step;
An image processing method comprising: a combining step of combining the candidate images corrected by the candidate image correcting step.   14. The image according to claim 13, wherein the failure eye determination step determines whether or not the eye is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern. Processing method.   14. The image processing method according to claim 13, wherein the failure eye determination step determines that the eyes are closed.   14. The image processing method according to claim 13, wherein the failure eye determination step determines that the eyes are half closed.   14. The image processing method according to claim 13, wherein the failure eye determination step determines that the eyes are red eyes. A facial organ detection step for detecting a face and eyes from an image;
A failure eye determination step for determining whether or not the eye is a failure based on the detection result of the eyes in the face detection step;
An evaluation step for evaluating whether or not the quality required for the failure area determined to be a failure by the failure determination step is equal to or higher than a predetermined reference;
A correction candidate image search step for searching for a correction candidate image for a region determined to be a failure by the failure eye determination step when it is evaluated that the evaluation step is not equal to or greater than the predetermined reference;
A candidate image correcting step of correcting the candidate image searched by the correction candidate image searching step;
An image processing method comprising: a combining step of combining the candidate images corrected by the candidate image correcting step.   19. The image according to claim 18, wherein the failure eye determination step determines whether or not the eye is a failure based on whether or not the eye area pattern matches a predetermined open eye pattern. Processing method.   19. The image processing method according to claim 18, wherein the evaluation step evaluates a ratio between the size of the area determined to be a failure and the size of the image.   The image processing method according to claim 18, wherein the evaluation unit performs evaluation based on printing conditions.   The image processing method according to claim 21, wherein the printing condition is a printing medium.   The image processing method according to claim 21, wherein the print condition is print quality.   The image processing method according to claim 21, wherein the printing condition is evaluated from a print layout. Computer
Facial organ detection means for detecting a face and eyes from an image;
A failure eye determination means for determining whether or not the eye is a failure based on the detection result of the eyes by the facial organ detection means;
Correction candidate image search means for searching for a correction candidate image for an area determined to be a failure by the failure eye determination means;
Candidate image correcting means for correcting the candidate image searched by the correction candidate image searching means;
A program for causing a candidate image corrected by the candidate image correcting unit to function as a combining unit that combines the candidate images. Computer
Facial organ detection means for detecting a face and eyes from an image;
Failure eye determination means for determining whether or not the eye is a failure based on the detection result of the eyes by the face detection means;
Evaluation means for evaluating whether or not the quality required for the failure area determined to be a failure by the failure determination means is greater than or equal to a predetermined reference;
A correction candidate image search means for searching for a correction candidate image for an area determined to be a failure by the failure eye determination means when the evaluation means evaluates not to be above the predetermined reference;
Candidate image correcting means for correcting the candidate image searched by the correction candidate image searching means;
A program for causing a candidate image corrected by the candidate image correcting unit to function as a combining unit that combines the candidate images.

Images