JP4148903B2 - Image processing apparatus, image processing method, and digital camera - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a digital camera that detect red eyes generated when a person or an animal is photographed using a flash and correct the detected red eyes.

  Digital cameras are rapidly becoming widespread and are replacing silver salt cameras because of advances in technology such as miniaturization and high image quality, and ease of use compared to silver salt cameras.

  When shooting in dark places, digital cameras require lighting such as a flash as well as silver halide cameras. When a person or an animal is photographed using a flash, a red-eye phenomenon may occur in which the light of the flash is reflected by the retina and the eyes of the subject person or animal become red. This red-eye phenomenon is likely to occur when the optical axis of the lens and the optical axis of the flash are parallel and the distance between the two optical axes is short. In particular, in digital cameras that are becoming smaller in size, the distance between the lens and the flash is shortened, so that a red-eye phenomenon often occurs. Therefore, not only digital cameras but also small cameras are often equipped with a red-eye reduction function for reducing red eyes.

  One of the red-eye reduction functions is to irradiate light twice from a flash. First, the pupil of a person or animal is narrowed by irradiating a dummy light with a flash just before shooting to reduce the reflected light on the retina due to the flash light during shooting, thereby preventing the occurrence of red-eye phenomenon To do.

  However, in the method of irradiating light twice from the flash, the dummy light has to be irradiated, which causes a problem that the battery is consumed very much.

  Further, even if the light is irradiated twice from the flash so that the light reflected from the retina by the flash light is reduced, the red-eye phenomenon cannot be completely eliminated. Therefore, it was necessary to correct red eyes after shooting.

  In Patent Document 1 (first prior art), face region detection for detecting a face region is performed as pre-processing for detecting a red eye region, and the detection of the face region is used for red eye processing such as detection / removal of the red eye region. Techniques to do this are disclosed. Specifically, first, when a point or a part of a face including a red-eye area is specified from an image displayed on the display by the operator, an image area obtained by enlarging the specified area by a certain range Initialize search range for. The skin color reference color, which is the basic color of the face image area, is determined by referring to the pixel color of the area specified by the operator or the color of all or part of the pixels in the initial search range. The face image area is searched based on the color difference between the determined reference color and the pixel color of the image area.

  Next, feature points such as eyes, mouth, and eyebrows are detected by clustering processing using the color difference of each pixel in the searched face image region. Then, the color of the detected feature point is determined, black areas such as hair and black eye shadows and white areas such as white eye shadows are detected based on luminance information, and spots, red eyes, lips are detected based on saturation information. The color of the feature point such as the discrimination of the red region is discriminated.

  Finally, in addition to the discrimination results of each color, comprehensive judgment is also made considering the shape of the feature points, etc. to detect the red-eye area and reduce the saturation and lightness of the detected red-eye area to correct red eyes To do.

  Further, in Patent Document 2 (second prior art), when a face area of a person is extracted from an input image of the person to be recognized, the face of the face is extracted from the face area using a separability filter. Each feature point candidate is extracted, a feature point set candidate is selected from the feature point candidates based on the structural constraint information of the face, and the feature point neighborhood cut out based on each feature point set candidate Each matching degree is calculated by matching the region with a pre-registered template pattern to find its relaxation, and the feature point set with the highest relaxation degree is selected as the correct feature point set. A technique for stably extracting feature points such as the nose is disclosed.

  Furthermore, Non-Patent Document 1 (third prior art) obtains a similarity degree at any time while moving a face dictionary pattern including a pupil and a nostril over the entire image, and determines a local maximum point of the similarity degree. Extracted as a face area, extracted each feature point candidate of the face from the extracted face area using a separability filter, and the extracted feature point candidates are correct by pattern matching using the subspace method. And a technique for extracting feature points such as eyes and nostrils from the entire image is disclosed.

JP-A-10-233929 Japanese Patent No. 3279913 IEICE Transactions D-II Vol.J80-D-II No.8 pp.2170-2177 August 1997 "Facial feature point extraction by combination of shape extraction and pattern matching"

  However, in the first prior art, for example, when only the hue is used, the detection accuracy of the red-eye region is low, and erroneous detection may occur. For this reason, comprehensive determination is made in consideration of a plurality of conditions such as area, saturation, brightness, and shape, but there is a problem that it takes time to detect red-eye when the number of conditions to be considered increases.

  In the first prior art, in order to correct the red eye, the operator designates one point or a partial range of the face including the red eye region from the image displayed on the display. Depending on this specification, it takes time to detect red-eye. In order to shorten the red-eye detection time, an appropriate area must be specified. However, there is a problem that the designation of the area is very inconvenient because it is manually performed by the operator.

  Furthermore, in the second and third conventional techniques, the pupil can be accurately detected, but since the pupil which is a feature point is detected after the face area is detected, the processing time is long, There is a problem that it is difficult to directly apply to red-eye detection of a digital camera or the like.

  Digital cameras also have an automatic focus adjustment function that adjusts the focus of the lens by automatically measuring the distance to the subject. In a digital camera, it is often predetermined which part of an image is focused in the automatic focus adjustment function. However, when a person or animal is photographed against the background of a landscape, the person or animal is not necessarily located at a predetermined portion. In such a case, there is a problem that an image out of focus on the person may be taken.

  The present invention has been made in view of the above, and an image processing apparatus and an image processing method capable of performing red-eye correction by reducing the processing time by detecting red eyes without detecting a face area. The first object is to obtain a digital camera.

  The second object is to obtain a digital camera that can accurately focus on a person or animal when photographing including a person or animal.

  In order to solve the above-described problems and achieve the object, the present invention uses the separability filter having an inner area and an outer area having a predetermined shape to separate the inner area and the outer area from the input image. And detecting a degree of separation for each pixel of the input image that indicates a degree of separation between the image feature amount of the input image included in the internal region and the image feature amount of the input image included in the external region. A separability map creating means for creating a separability map in which each calculated separability is associated with each pixel; each separability of the separability map created by the separability map creating means and a predetermined threshold value And a feature point candidate detecting means for detecting a pixel whose degree of separation is greater than the threshold, and detecting a pixel whose degree of separation of the detected pixel indicates a local maximum value as a red-eye candidate, and Feature point Feature inspection for determining whether or not the red-eye candidate detected by the complementary detection means satisfies a predetermined condition, and detecting a predetermined area including the red-eye candidate satisfying the predetermined condition as a red-eye area And an exit means.

  According to another aspect of the present invention, there is provided an image processing method for detecting a red-eye area from an input image and correcting the detected red-eye area, by using a separability filter having an inner area and an outer area having a predetermined shape. The internal region and the external region are detected from among the images, and the degree of separation between the image feature amount of the input image included in the internal region and the image feature amount of the input image included in the external region is indicated. A separation degree map creating step for calculating a separation degree for each pixel of the input image and creating a separation degree map in which each calculated separation degree is associated with each pixel, and the separation degree map creating step. Each separation degree of the obtained separation degree map is compared with a predetermined threshold value to detect a pixel having the separation degree larger than the threshold value, and the separation degree of the detected pixel indicates a local maximum value A feature point candidate detection step for detecting as a red eye candidate, and whether or not the red eye candidate detected by the feature point candidate detection step satisfies a predetermined condition, and satisfies the predetermined condition And a feature point detecting step of detecting a predetermined area including a red eye candidate as a red eye area.

  According to the present invention, the separability map creating means detects an inner area and an outer area from an input image by a separability filter having an inner area and an outer area having a predetermined shape, and includes an input included in the inner area. The degree of separation indicating the degree of separation between the image feature quantity of the image and the image feature quantity of the input image included in the external area is calculated for each pixel of the input image, and each calculated separation degree is associated with each pixel. The feature point candidate detection means compares each separation degree of the separation degree map with a predetermined threshold value to detect pixels having a separation degree greater than the threshold value, and separates the detected pixels. A pixel whose degree is a local maximum value is detected as a candidate for red eye, and the feature point detection unit determines whether the candidate for red eye satisfies a predetermined condition, and a candidate for red eye that satisfies the predetermined condition A predetermined area including Since it is, and it is possible to accurately detect the red eye without detecting redeye from the input image after detecting the face area, it is possible to shorten the time required for the detection of red-eye.

  Further, according to the present invention, the pupil position detection unit detects a human or animal pupil from the input image, the reference region determination unit determines a reference region including the detected pupil, and the parameter adjustment unit includes: Since the parameters for shooting the subject are adjusted based on the reference area, if the subject contains a person or an animal, the shooting parameters are accurately matched to the person or animal. Can be taken.

  Hereinafter, embodiments of an image processing apparatus, an image processing method, and a digital camera according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

(First embodiment)
A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to the first embodiment of the present invention. The image processing apparatus according to the first embodiment of the present invention includes an image input unit 101, a separability map generation unit 102, a feature point candidate detection unit 103, a feature point detection unit 104, an image correction unit 105, and an image output unit 106. ing.

  The image input unit 101 is an input unit that captures an image captured by a digital camera. There are various image formats photographed with a digital camera, such as an RBG format and a YUV format. When the captured image format is an image format different from the RBG format, the image input unit 101 converts the input image format into the RGB format, and functions as an input image conversion unit in the scope of claims. I have.

  The separability map generation unit 102 generates a separability map of an image to be subjected to red-eye correction using a separability filter shown in FIG. 2 described later. The feature point candidate detection unit 103 compares the value of the degree of separation in the degree of separation map with a predetermined threshold value, detects a point where the value of the degree of separation is larger than the predetermined threshold value, and among the detected points A pixel indicating the local maximum value is detected as a feature point candidate (red-eye candidate).

  The feature point detection unit 104 determines whether or not a plurality of feature point candidates detected by the feature point candidate detection unit 103 satisfy a predetermined condition, and determines the feature point candidates satisfying the predetermined condition as red eyes. It is detected as a feature point.

  The image correction unit 105 performs red-eye correction processing on a region near the feature point detected by the feature point detection unit 104, that is, a red-eye region. The image output unit 106 outputs the image subjected to the red-eye correction process by the image correction unit 105 to a general display device such as a CRT (Cathode-Ray Tube) or a liquid crystal display. The display device may be provided in the image correction processing device, or the image correction processing device may be connected to the display device for use.

  Next, the operation of the image processing apparatus according to the first embodiment of the present invention will be described. The image input unit 101 captures an image captured by a digital camera, and determines whether the captured image is in RGB format. If the captured image is different from the RGB format, the captured image format is converted to the RGB format. The image input unit 101 outputs an RGB format image to the separability map generation unit 102.

によって算出される。なお、分離度フィルタを用いて特徴点を検出する方法は、特許文献２によって公知である。 The separability map generation unit 102 calculates an output value (separation degree) of the separability filter for each pixel of the image and generates a separability map. As shown in FIG. 2, the separability filter includes an outer region 1 outside the circle with the radius r and an inner region 2 inside the circle with the radius r. The separability map generation unit 102 calculates a separability indicating how much image feature quantity such as luminance is separated between the outer region 1 and the inner region 2 of the separability filter. Specifically, the degree of separation η is

Is calculated by A method of detecting feature points using a separability filter is known from Patent Document 2.

  In normal feature point candidate detection, the brightness in the black and white image is used as the brightness Pi that is the image feature amount of (Expression 2) and (Expression 3). However, the image processing apparatus of the present invention detects a red-eye area, that is, a feature point where a red area exists as a circular area as compared with the surrounding area. Accordingly, the separability map generation unit 102 uses the luminance of RGB G value (green) as the luminance Pi of (Expression 2) and (Expression 3). Here, with reference to FIGS. 3-1 to 3-5, it will be described that the G value is suitable for the luminance Pi in (Expression 2) and (Expression 3) in the detection of the red-eye region.

  FIG. 3A illustrates an image area of the eye including red eyes. In FIG. 3A, a region A shows red eyes, a region B shows white eyes, and a region C shows skin around the eyes. Here, attention is focused on individual images of R value (red), G value (green), and B value (blue) in each of the areas A to C in FIG. As shown in FIG. 3B, the R-value image generally has a high luminance in all areas of the red-eye area A, the white-eye area B, and the skin area C. As shown in FIG. 3C, the G-value image generally has a lower red-eye area A brightness and a white-eye area B and skin area C brightness. In the B-value image, generally, as shown in FIG. 3-4, the luminance of the red-eye region A and the skin region C decreases, and the luminance of the white-eye region B increases. That is, as shown in FIG. 3-5, in the red-eye region A, the luminance of the R value image is “high”, the luminance of the G value image is “small”, and the luminance of the B value image is “small”. In the white-eye region B, the luminance of the R value image is “high”, the luminance of the G value image is “high”, and the luminance of the B value image is “high”. The brightness of the image is “high”, the brightness of the G value image is “high”, and the brightness of the B value image is “small”. In order to detect the red-eye region A, it is desirable that the difference between the luminance of the white-eye region B and the luminance of the skin region C and the luminance of the red-eye region A is large. The difference between the luminance of the white-eye region B and the skin region C and the red-eye region A is the largest in the G-value image. Suitable for.

  The separability map generation unit 102 creates a separability map in which each pixel of the image is associated with the separability from the separability calculated by (Expression 1) to (Expression 3) using the luminance of the G value. . The separability map may be created by calculating the separability for a plurality of assumed radii by changing the radius r of the separability filter according to the size of the face image or the red-eye area. Then, a plurality of separability maps may be integrated according to the value of the separability map for each radius. When integrating a plurality of separability maps, the separability filter radius that is the maximum value is associated with the separability value at a certain position by using the maximum separability value of each radius. .

  The feature point candidate detecting unit 103 compares the value of the degree of separation in the degree of separation map with a predetermined threshold value, and detects a pixel having a value of the degree of separation larger than the predetermined threshold value. In the circular separability filter shown in FIG. 2, the separability becomes a value closer to “1.0” as a circular feature point such as a pupil or nostril is fitted, and the shape of the target region for which the separability is calculated is As the distance from the circle increases, the degree of separation decreases to a value close to “0”. For example, as shown in FIG. 4, the degree of separation of the portion indicated by the dotted line in the image increases at the portion related to the region 3 indicating the pupil. The feature point candidate detection unit 103 detects, as a feature point candidate (red-eye candidate), a pixel that indicates a local maximum value among points where the detected degree of separation is greater than a predetermined threshold.

  In addition, if the smoothing filter process is performed on the value of the separation degree, for example, using a Gaussian filter before the local maximum value is obtained, resistance to noise is increased, and more accurate feature point candidates can be detected. .

  The feature point detection unit 104 determines whether or not a plurality of feature point candidates detected by the feature point candidate detection unit 103 satisfy a predetermined condition, and selects a feature point candidate satisfying the predetermined condition as a red-eye. Are detected as feature points.

The predetermined conditions are
(1) The image feature amount (in this case, luminance) of the red eye candidate that is a feature point is smaller than a predetermined threshold value.
(2) The average value of the image feature amount (in this case, luminance) of the G value of each pixel in the inner region of the separability filter centering on the red eye candidate that is the feature point candidate is smaller than a predetermined threshold value.
(3) The redness indicated by the hue value of the candidate red eye that is the feature point is greater than a predetermined threshold.
(4) The average value of redness indicated by the hue value of the inner region of the separability filter centering on the red eye candidate that is the feature point candidate is larger than a predetermined threshold.
(5) There are feature point candidates arranged as a pair indicating the right eye and the left eye from among a plurality of candidate red eye candidates.
And

  The feature point detection unit 104 detects, as feature points, red-eye candidates that satisfy at least one of the above five conditions. The above five conditions are determined based on whether each value is larger or smaller than a predetermined threshold, but may be determined based on a relative value in a predetermined image. You may make it judge in combination with a relative value. Of course, the determination may be made based on the maximum value or the minimum value.

  The image correcting unit 105 subtracts a predetermined value from the R value of the red eye region, for example, to reduce the redness of the red eye region, for example, near the feature point detected by the feature point detecting unit 104, that is, the red eye region. The red eye correction process is performed, and the image after the red eye correction process is output to the image output unit 106.

  The image output unit 106 displays the image that has been subjected to the red-eye correction process by the image correction unit 105 on a display device.

  As described above, in the first embodiment, in the image in the RGB format, the luminance of the R value is large and the luminance of the B value and the G value is small in the red eye portion, and in the white eye portion near the red eye portion. Pay attention to the fact that the brightness of all of the R value, the G value, and the B value is large, the brightness of the R value and the G value is large, and the brightness of the B value is small in the skin part near the red-eye part. The separation degree is calculated from the input image using the luminance of the G value having the largest luminance difference between the red-eye portion and the white-eye portion and the skin portion, and the red-eye is detected based on the calculated separation degree. On the other hand, red-eye correction processing is performed. As a result, it is possible to accurately detect the red eye without detecting the red eye after detecting the face area from the input image, and the detected red eye is corrected by detecting the red eye from the target image as compared with the prior art. The time required for processing can be shortened.

  Further, since it is not necessary to input information necessary for the red-eye correction process from the outside, the red-eye correction process can be performed without bothering the operator, and convenience can be improved.

  In the first embodiment, the separability map generation unit 102 uses the luminance of the G value as the luminance Pi of (Equation 2) and (Equation 3) for calculating the separability. However, the brightness Pi is not limited to the brightness of the G value, and for example, all of the brightness of the R value, the brightness of the G value, and the brightness of the B value may be used, or a value obtained by calculation by an arbitrary combination May be used.

  When the separability map generation unit 102 calculates the separability, the outer area 1 of the separability filter shown in FIG. 2 has R-value luminance, and the inner area 2 has G-value luminance and B-value luminance. Different types of values may be used, such as using the luminance sum.

  In the first embodiment, when an image different from the RGB format such as YUV is input, the image input unit 101 converts the input image format into the RGB format. The degree of separation may be calculated from the value of the input image format without converting the format to the RGB format. For example, when the input image format is the YUV format, the degree of separation may be calculated using the YUV value as it is. When the input image is in JPEG format, it is desirable to perform processing after decoding the input image.

  In addition, the image processing apparatus according to the first embodiment may be provided with a communication interface function for connecting to a computer or the like so as to be connected to a computer or the like. In this case, the image input unit 101 and the image output unit 106 may perform image input / output with a computer connected via the communication interface function. The image output unit 106 not only outputs an image subjected to red-eye correction processing to a display device, but also, for example, a recording medium such as a CD-ROM, a floppy (registered trademark) disk (FD), a DVD, An image that has been subjected to the red-eye correction process may be output to the memory or the like. Of course, the image processing apparatus according to the first embodiment may be provided in the digital camera.

  Further, the separability map generated by the separability map generation unit 102 may target the entire image input from the image input unit 101, or a general input such as a keyboard or a mouse in the image processing apparatus. A device may be provided, and an operator may select a region including red eyes using the input device, and the separability map may be generated using only the selected region as a processing target image. Of course, when an image processing apparatus is connected to a computer or the like, the processing target image may be selected using an input device of the connected computer.

  Further, the image input unit 101, the separability map generation unit 102, the feature point candidate detection unit 103, the feature point detection unit 104, the image correction unit 105, and the image output unit 106 of the image processing apparatus according to the first embodiment of the present invention. The functions described above may be provided as an image processing program for causing a general-purpose computer to execute the functions. In this case, the image processing program is recorded in a computer-readable recording medium such as a CD-ROM, a floppy (registered trademark) disk, and a DVD in a format that can be installed in a computer or a computer-executable format. provide. The image processing program recorded on the recording medium is read from the recording medium and realized on the computer.

  A computer that executes an image processing program according to the present invention includes at least a control device and a storage device. The control device includes an arithmetic processing device such as a CPU that controls the entire computer, and a ROM. The storage device includes a general RAM, and stores an input image to be processed, a separability map, and an image subjected to red-eye correction processing. The control device reads out and executes the image processing program from the recording medium on which the image processing program is recorded, whereby the separability map generation unit 102, the feature point candidate detection unit 103, the feature point detection unit 104, and the image correction unit 105. The computer is operated as each of the functions described above realized by the image output unit 106.

  Alternatively, the image processing program may be stored on a computer connected to a network and provided by being downloaded via the network. Further, the image processing program may be provided or distributed via a network.

(Second Embodiment)
A second embodiment of the present invention will be described with reference to FIGS. FIG. 5 is a block diagram showing an example of the configuration of a digital camera according to the second embodiment of the present invention. A digital camera according to the second embodiment of the present invention includes a shutter 509, a lens 501 (an optical system in the claims), a CCD 510, an image input unit 508, a storage unit 505, a display unit 504, and a pupil position determination unit. 507, an AF area determination unit 506 (reference area determination unit in the claims), a focus control unit 503 (parameter adjustment unit in the claims), and a motor 502.

  The shutter 509 is used as a means for causing the image input unit 508 to capture an image of a subject obtained by converting the light input from the lens 501 into digital data by the CCD 510. The shutter 509 is composed of, for example, a two-stage push button, and when the operator presses the focus adjustment instruction, which is an instruction to adjust the focus of the lens 501, when the button is pressed to the first stage, A shooting instruction that is an instruction to store an image is output to the image input unit 508.

  When the focus adjustment instruction is input, the image input unit 508 captures the image input from the CCD 510 and outputs the captured image to the display unit 504 and the separation degree map generation unit 517 in the pupil position determination unit 507. . When a shooting instruction is input, the image input unit 508 captures an image input from the CCD 510 and stores it in the storage unit 505.

  The display unit 504 is configured with a liquid crystal display, for example, and displays an image input from the image input unit 508 or an image stored in the storage unit 505.

  The pupil position determination unit 507 detects whether or not a human or animal pupil is present in the image input from the image input unit 508. If a pupil is present, the pupil position determination unit 507 outputs the detected pupil position information as AF. The data is output to the area determination unit 506. When there is no pupil in the image, the pupil position determination unit 507 notifies the AF area determination unit 506 that there is no pupil, for example, by using a value that is impossible as position information in the image as position information. .

  The pupil position determination unit 507 includes a separability map generation unit 517, a feature point candidate detection unit 527, and a focus control unit 503.

  The separability map generation unit 517 calculates an output value (separability) of the separability filter for the image, and generates a separability map. Since the separability filter for generating the separability map is the same as that described in the first embodiment, the description thereof is omitted here. The separability map generation unit 517 calculates the separability by (Equation 1) to (Equation 3), but here, since the separability map is generated in order to detect the pupil, (Equation 2) and (Equation 3) The brightness Pi of the monochrome image is used as the brightness Pi.

  The feature point candidate detection unit 527 compares the value of the degree of separation in the degree of separation map with a predetermined threshold value, detects a pixel having a value of the degree of separation larger than the predetermined threshold value, and detects among the detected pixels. A pixel indicating the local maximum value is detected as a feature point candidate.

  The feature point detection unit 537 performs pattern matching processing on the plurality of feature point candidates detected by the feature point candidate detection unit 527 to calculate the similarity, and calculates the similarity value and each feature point candidate. The position of the pupil and the radius filter radius are integrated to detect the pupil position. Then, the detected pupil position information is output to the AF area determination unit 506.

  The AF area determination unit 506 determines an AF area for adjusting the focus of the lens 501 based on the position information input from the pupil position determination unit 507. The focus control unit 503 controls the lens 501 by driving a motor 502 that adjusts the focus of the lens 501 in the AF area determined by the AF area determination unit 506.

  Next, the operation of the digital camera according to the second embodiment of the present invention will be described with reference to the flowchart of FIG. The operation in which the separability map generation unit 517 generates a separability map and the feature point candidate detection unit 527 detects feature point candidates (here, pupil area candidates) using the separability map is the first operation. Therefore, detailed description thereof is omitted here.

  When a focus adjustment instruction is input from the shutter 509, the image input unit 508 captures an image input from the CCD 510, and outputs the captured image to the display unit 504 and the separation degree map generation unit 517 (step S100, S110). The display unit 504 displays the image input from the image input unit 508.

  The separability map generation unit 517 calculates the separability by (Equation 1) to (Equation 3) using the luminance of the black and white image of the image input from the image input unit 508, and generates the separability map ( Step S120).

  The feature point candidate detection unit 527 compares the value of the degree of separation in the degree-of-separation map with a predetermined threshold value, and detects a pixel having a value of the degree of separation larger than the predetermined threshold value. The feature point candidate detection unit 527 detects, as a feature point candidate, a pixel indicating a local maximum value among the detected values of the degree of separation larger than a predetermined threshold value (step S130).

  The feature point detection unit 537 performs pattern matching processing on a plurality of feature point candidates detected by the feature point candidate detection unit 527 to detect a pupil that is a feature point, and uses the detected pupil position information as AF. It outputs to the area | region determination part 506 (step S140). Specifically, the feature point detection unit 537 acquires a local normalized image according to the radius of the separability filter in the vicinity of the plurality of feature point candidates detected by the feature point candidate detection unit 527. Then, the similarity between the acquired local normalized image and the dictionary is calculated by the subspace method.

  The dictionary for calculating the similarity is the first dictionary created by collecting learning patterns of left and right pupils in various persons, lighting conditions, face orientations, etc., and the first among the feature point candidates. It is assumed that a second dictionary created by selecting and collecting error patterns such as the corners of the eyes, the head of the eyes, and the eyebrows whose similarity to the first dictionary is higher than a predetermined threshold is prepared in advance.

  The feature point detection unit 537 calculates the similarity by subtracting the similarity between the local normalized image and the second dictionary from the similarity between the local normalized image and the first dictionary. The feature point detection unit 537 aligns the feature point candidates according to the similarity values of the left pupil and the right pupil, and selects feature point candidates whose similarity values are equal to or greater than a predetermined number or a predetermined threshold value, respectively. . Then, a set of left and right pupils is created using the selected feature point candidates.

  The feature point detection unit 537 determines whether or not the set satisfies a predetermined condition determined from a face size and an angle assumed in advance. It is determined whether or not the face area estimated from the left and right pupils of a set that satisfies a predetermined condition holds between different sets. That is, it is determined whether or not there are a plurality of sets that hold as estimated pupil positions of the face area. When there are a plurality of sets that are established as the pupil positions of the estimated face area, the feature point detection unit 537 further detects one set that is established as the pupil position using a predetermined evaluation value. As the evaluation value, for example, a set having a higher similarity value is detected as the pupil position by using the sum of the similarities between the left pupil and the right pupil of the set. The feature point detection unit 537 outputs the detected pupil position information to the AF area determination unit 506. A method for detecting feature points using the subspace method is known from Non-Patent Document 1.

  For example, if the feature point detection unit 537 cannot detect the position of the pupil, which is a feature point, by the pattern processing described above (the subject does not include a person or an animal), for example, a value that is not possible as position information in the image The position information is used to notify the AF area determination unit 506 that there is no pupil.

  The AF area determination unit 506 determines whether or not a pupil is detected by the feature point detection unit 537 (step S160). When pupil position information is input from the feature point detection unit 537, the AF region determination unit 506 determines an AF region for adjusting the focal point of the lens 501 based on the pupil position information. (Step S170). For example, if an image 701 as shown in FIG. 7 is captured by the image input unit 508, position information indicating the left and right pupils 703 of the person 702 is input from the feature point detection unit 537 to the AF area determination unit 506. . The AF area determination unit 506 determines a rectangular area including the left and right pupils 703 as the AF area 704. Then, the determined AF area position information is output to the focus control unit 503.

  When no pupil is detected, the AF area determination unit 506 determines a predetermined area as an AF area (step S180). Then, the determined AF area is output to the focus control unit 503.

  The focus control unit 503 performs the focus control process of the lens 501 by driving the motor 502 to adjust the focus of the lens 501 to the AF area determined by the AF area determination unit 506 (step S190). For focus control, for example, a TTL contrast detection method is used in which the lens 501 is moved stepwise to focus the image at the highest contrast in the AF area. Of course, a method different from the TTL contrast detection method may be used as long as focus control is performed on the AF area.

  The image input unit 508 captures the image after the focus control process is completed, and displays the captured image on the display unit 504 (step S200). When a shooting instruction is input from the shutter 509, the image input unit 508 captures an image input from the CCD 510, stores it in the storage unit 505, and saves the image (step S210).

  As described above, in the second embodiment, since the pupil of a person or animal is detected and focus control is performed using the area including the detected pupil as the AF area, the person or animal is included in the subject to be photographed. Can be taken with accurate focus control.

  In the second embodiment, the focus adjustment instruction is output to the image input unit 508 when the shutter 509 is pressed in one step, and the shooting instruction is output to the image input unit 508 when the shutter 509 is pressed in two steps. The timing for outputting the instruction is not limited to this. For example, in a state where the digital camera can shoot, that is, when the focus control process is repeatedly performed while the input image is displayed on the display unit 504, the time counting function in the digital camera is used at predetermined intervals. A focus adjustment instruction may be generated.

  In the second embodiment, the example in which the CCD 510 is used as an image element for converting the light from the lens 501 into digital data has been described. However, for example, an image element such as a CMOS sensor may be used. .

  In the second embodiment, the AF area determination unit 506 determines the rectangular area including the left and right pupils as the AF area. However, the present invention is not limited to this, and the area includes only one of the pupils. May be used as the AF region, and the shape of the AF region may not be rectangular.

  Further, when two or more human eyes are detected in the image, the feature point detection unit 537 compares the estimated face sizes of the persons, and as a result of the comparison, the estimated face is detected. It is possible to determine that the larger the size is, the closer the distance from the digital camera is, and to output the position information of the pupil whose estimated face size is larger to the AF area determination unit 506, The detected pupil position may be superimposed on the display unit 504 and displayed by the operator.

  In general, a digital camera often has a CPU for controlling the digital camera. The above-described functions realized by the separability map generation unit 517, the feature point candidate detection unit 527, the feature point detection unit 537, and the AF area determination unit 506 according to the second embodiment are realized by software, and the digital camera The CPU may be executed by the CPU.

  As described above, the image processing apparatus, the image processing method, and the digital camera according to the present invention are useful for image processing of a digital camera with a built-in flash, and in particular, a red-eye phenomenon that occurs when shooting using a flash. Suitable for red-eye correction processing.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. It is a figure for demonstrating a separability filter. It is a figure for demonstrating the suitable brightness | luminance used when calculating a separation degree. It is a figure for demonstrating the suitable brightness | luminance used when calculating a separation degree. It is a figure for demonstrating the suitable brightness | luminance used when calculating a separation degree. It is a figure for demonstrating the suitable brightness | luminance used when calculating a separation degree. It is a figure for demonstrating the suitable brightness | luminance used when calculating a separation degree. It is a figure for demonstrating the determination method of a feature point candidate. It is a block diagram which shows an example of a structure of the digital camera of 2nd Embodiment in this invention. It is a flowchart for demonstrating operation | movement of the digital camera of 2nd Embodiment in this invention. It is a figure for demonstrating the determination method of AF area | region.

Explanation of symbols

101,508 Image input unit 102,517 Separation map generation unit 103,527 Feature point candidate detection unit 104,537 Feature point detection unit 105 Image correction unit 106 Image output unit 501 Lens 502 Motor 503 Focus control unit 504 Display unit 505 Storage 506 AF area determination unit 507 Pupil position determination unit 509 Shutter 510 CCD

Claims (10)

In an image processing apparatus that detects a red-eye area from an input image and corrects the detected red-eye area,
For each pixel of the input image, RGB of the region included in the internal region and the region included in the external region of the input image is determined by a separability filter having an internal region and an external region having a predetermined shape. A degree-of- separation map creating means for calculating a degree of separation indicating the degree of separation of the luminance of the G value in the color space, and creating a degree-of-separation map in which each calculated degree of separation is associated with each pixel;
Compare each separation degree of the separation degree map created by this separation degree map creating means with a predetermined threshold value,
Detecting pixels with a degree of separation greater than the threshold;
A feature point candidate detecting means for detecting, as a red-eye candidate, a pixel whose separation degree of the detected pixel indicates a local maximum value;
It is determined whether or not the red eye candidate detected by the feature point candidate detecting means satisfies a predetermined condition,
Feature point detecting means for detecting a predetermined region including a candidate for red eye that satisfies the predetermined condition as a red eye region;
An image processing apparatus comprising:
Image correction means for performing red-eye correction processing on the area detected as the red-eye area by the feature point detection means;
The image processing apparatus according to claim 1, further comprising:
The separability filter has a circular outer region and an inner region,
A predetermined condition for the feature point detection means to detect the red-eye region,
The image processing apparatus according to claim 1 , wherein the red-eye candidate G value has a luminance smaller than a predetermined threshold.
The separability filter has a circular outer region and an inner region,
A predetermined condition for the feature point detection means to detect the red-eye region,
The image processing apparatus according to claim 1 , wherein an average value of luminances of G values of pixels included in an inner region of the separability filter centering on the red eye candidate is smaller than a predetermined threshold value. .
The separability filter has a circular outer region and an inner region, and the feature point detection unit detects a predetermined condition for detecting the red eye region, and the redness indicated by the hue value of the red eye candidate is greater than a predetermined threshold value. The image processing apparatus according to claim 1 , wherein the image processing apparatus is large.
The separability filter has a circular outer region and an inner region, and a predetermined condition for the feature point detection means to detect the red eye region is set to the inner region of the separability filter centering on the red eye candidate. The image processing apparatus according to claim 1 , wherein an average value of redness indicated by a hue value of each pixel included is greater than a predetermined threshold value.
The separability filter has a circular outer region and an inner region,
A predetermined condition for the feature point detection means to detect the red-eye region,
The image processing apparatus according to claim 1 , wherein a candidate for red eye arranged in a pair indicating a right eye and a left eye is detected from the red eye candidates detected by the feature point candidate detection unit. .
An input image conversion means for converting the input image into the RGB format,
The image processing apparatus according to claim 1, further comprising:
In an image processing method for detecting a red-eye area from an input image and correcting the detected red-eye area,
For each pixel of the input image, RGB of the region included in the internal region and the region included in the external region of the input image is determined by a separability filter having an internal region and an external region having a predetermined shape. A separation degree map creating step for calculating a separation degree indicating a degree of separation of the luminance of the G value in the color space, and creating a separation degree map in which the calculated separation degrees and the respective pixels are associated;
Each separation degree of the separation degree map created by the separation degree map creation step is compared with a predetermined threshold value, and a pixel having the separation degree larger than the threshold value is detected, and the separation degree of the detected pixel is A feature point candidate detection step of detecting a pixel indicating a local maximum value as a candidate for red eye;
It is determined whether or not the red eye candidate detected by the feature point candidate detection step satisfies a predetermined condition, and a predetermined area including the red eye candidate satisfying the predetermined condition is detected as a red eye area. A feature point detecting step,
An image processing method comprising:
A digital camera comprising the image processing apparatus according to claim 1.

Images