JP2005332382A - Image processing method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable blur information in an image to be properly acquired. <P>SOLUTION: A blur analysis means 200 calculates a direction and a degree of blurring using a pupil image D5 of an image D0 captured by a pupil detection means 100, and determines whether the image D0 is a blurred image or a normal image. To the image D0 which has been determined as a blurred image, the analysis means calculates the degree of blurring and the blurring width from the pupil image D5, and outputs the degree of blurring, the blurring width, the blurring direction, and the degree of blurring determined from the pupil image D5 to the blur correction means 230 as blur information Q of the image D0. The blur correction means 230 executes correction to the blurred image D0 from the blur information Q, and obtains a corrected image D'. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method and apparatus for acquiring blur information of a digital photographic image, and a program therefor.

  Digital photographic images obtained by photoelectrically reading photographic images recorded on photographic films such as negative films and reversal films with a reading device such as a scanner, and digital photographic images obtained by taking images with a digital still camera (DSC), etc. On the other hand, printing is performed by performing various image processing. As one of these image processes, there is a blurred image restoration process that removes a blur from a blurred image (blurred image).

  Reasons for blurring a photographic image obtained by capturing an image of a subject include out-of-focus due to the focal length being out of focus, and out-of-focus blur due to camera shake (hereinafter referred to as blurring). . In the case of out-of-focus, the point image spreads two-dimensionally, that is, the spread on the photographic image exhibits non-directionality, whereas in the case of blur, a locus with a point image is drawn on the image one-dimensionally. Spreads, that is, has a direction with a spread on a photographic image.

  In the field of digital photographic images, various methods have been conventionally proposed for restoring blurred images. If you know information such as blur direction and blur width when capturing a photographic image, you can restore it by applying a restoration filter such as a Wiener filter or inverse filter to the photographic image. There is widely known a method of providing an apparatus (for example, an acceleration sensor) that can be acquired in an imaging apparatus, acquiring information such as a blur direction and a blur width together with imaging, and performing repair based on the acquired information (for example, , See Patent Document 1).

  Also, a degradation function is set for the blurred image (image with blur), the blurred image is repaired with a restoration filter corresponding to the set degradation function, the restored image is evaluated, and the evaluation result is evaluated. A method is also known in which the deterioration function is reset, and repair is performed by repeating the repair, evaluation, and resetting of the deterioration function until a desired image quality is obtained. This method has a problem that it takes processing time because it is necessary to repeat the process of setting, repairing, evaluating the deterioration function, resetting the deterioration function, and so on. In Patent Document 2, the user specifies a small area including an edge in a blurred image, and instead of the entire blurred image, the above-described degradation function is set, repaired, evaluated, and the specified small area. Repeat the process of resetting the degradation function to find the optimal degradation function, apply the restoration filter corresponding to this degradation function to the entire blurred image, and use the above-mentioned small area for the image used to obtain the degradation function A method has been proposed in which the amount of calculation is reduced and the efficiency is improved by using this image.

  On the other hand, with the rapid spread of mobile phones, the functions of mobile phones have improved, and among them, the improvement of the functions of digital cameras attached to mobile phones (hereinafter referred to as mobile cameras) has been attracting attention. In recent years, the number of pixels of a portable camera has increased to one million, and the portable camera is used in the same way as a normal digital camera. Of course, commemorative photos when traveling with friends, as well as scenes of picking up favorite talents and athletes with a portable camera, are becoming commonplace. In such a background, a photographic image obtained by capturing with a mobile camera is not limited to being viewed on a monitor of a mobile phone, and for example, is often printed in the same manner as a photographic image acquired with a normal digital camera. It has become.

  On the other hand, since the main body (mobile phone) is not manufactured exclusively for imaging, the portable camera has a problem of poor holdability during imaging. Moreover, since a portable camera does not have a flash, the shutter speed is slower than that of a normal digital camera. For these reasons, camera shake is more likely to occur when shooting a subject with a portable camera than with a normal camera. Extreme camera shake can be confirmed on the monitor of the portable camera, but small camera shake cannot be confirmed on the monitor, and often you will notice image blur for the first time after printing. There is a high need to perform blur correction on the obtained photographic image.

Further, as described above, the blur causes the spread of the point image in the image, and therefore, the blur image has an edge spread corresponding to the spread of the point image. That is, the manner of edge spreading in the image is directly related to the blur in the image. By paying attention to this point, a method of obtaining information relating to blur in the image, for example, blur direction, blur width, etc., by analyzing the aspect of the edge in the image using image data can be considered.
Japanese Patent Laid-Open No. 2002-112099 JP-A-7-121703

  However, downsizing of mobile phones is one of the focus of competition among mobile phone manufacturers, along with their performance and cost, and a camera attached to a mobile phone is provided with a device that acquires the direction and width of blur. Since it is not realistic, the method proposed in Patent Document 1 cannot be applied to a portable camera.

  In addition, the method proposed in Patent Document 2 and the method proposed in Patent Document 2 need to repeat the process of setting, repairing, evaluating, and resetting the deterioration function. Therefore, there is a problem that processing time is required and efficiency is not good.

  In addition, the method for obtaining information about blur in an image by analyzing the state of the edge in the image cannot obtain a correct analysis result when there is a blurred edge such as a gradation in a part of the image. There is a fear.

  In view of the above circumstances, the present invention does not require a special device to be provided in the image pickup device, and can obtain correct blur information even with respect to a digital photographic image having a gradation portion. An object of the present invention is to provide an image processing method and apparatus capable of obtaining an effect, and a program therefor.

The image processing method of the present invention is an image processing method for obtaining blur information indicating a blur mode in a digital photographic image.
From the digital photographic image, a point-like portion is detected,
The blur information of the digital photographic image is obtained using data of the image of the dot-like portion.

  Here, obtaining the blur information using the image data of the point-like portion is, for example, analyzing the state of the edge in the image of the point-like portion using the data of the image of the point-like portion. Can do.

  Further, as the point-like portion, it is preferable to use a pupil of the person when the digital photograph image is a photograph image of a person. Moreover, even if it is not a pupil, a clear face outline can also be made into a dotted | punctate part. Although the face outline is not a point, it is considered as a kind of point-like part in this specification.

  The “blurring information” means information that can represent a blur mode in a digital photographic image, and can be, for example, blur direction information and a blur width regarding a blur direction. “Bokeh” includes non-directional blur, that is, out-of-focus blur and directional blur, ie, blur. In the case of blur, the blur direction corresponds to the blur direction. In the case of blur, the “blurring direction” is “ It can be “no direction”. The “blurring width” means a blur width in the blur direction, and can be, for example, an average value of the edge widths of the edges in the blur direction. Further, in the case where the blur is non-directional out-of-focus, the edge width of the edge in any one direction may be the blur width, or may be the average value of the edge width of the edge in the entire image.

  Furthermore, the digital photographic image in the present invention is not limited to a blurred image, and may be a normal image that is not out of focus or blurred. Such a normal image has no blur, for example, a blur having a blur width that is “a predetermined threshold value or less”. It can have information.

  In the image processing method of the present invention, all elements of the blur information may be obtained using the detected image data of the dot-like portion, but the blur in the digital photograph image is in this case (in this case) The blur direction information includes blur direction information indicating that the blur is a blur of a non-directional blur and a directional blur, and the blur direction). While the blur direction information is acquired using image data, blur information other than the blur direction information (for example, blur width) is obtained using data of the entire digital photographic image based on the blur direction information. It is preferable to obtain.

Detecting an edge for each of a plurality of different directions with respect to the image of the dotted portion;
Obtaining feature values of the edge in each of the directions;
The blur direction information can be acquired based on the feature amount in each direction.

  Here, the “edge feature amount” means a feature amount related to an aspect of the edge spread in the image, and may include, for example, the edge sharpness and the edge sharpness distribution.

  As the “edge sharpness”, any parameter can be used as long as it can express the sharpness of the edge. For example, in the case of the edge shown by the edge profile in FIG. The edge profile is used so that the sharpness of the edge becomes higher as the sharpness of the brightness change of the edge (the gradient of the profile curve in FIG. 22) is higher, as well as the edge width is used as the sharpness of the edge so that the sharpness is low. The slope of the curve may be used as the edge sharpness.

  Further, the “plurality of different directions” means directions for specifying the direction of the blur in the target image, and it is necessary to include directions close to the direction of the blur. Although the specific accuracy is high, it is preferable to use an appropriate number according to the balance with the processing speed, for example, eight directions as shown in FIG.

An image processing apparatus of the present invention is an image processing apparatus that obtains blur information indicating a blur mode in a digital photographic image.
From the digital photographic image, point-like part detection means for detecting a point-like part,
And analyzing means for obtaining the blur information of the digital photographic image using data of the image of the dot-like portion.

  Moreover, it is preferable that the said dotted | punctate part detection means detects the pupil or face outline of the said person as the said dotted | punctate part with respect to the said digital photograph image which is a person's photograph image. As a method for performing such detection, in addition to the face detection technique described later, a morphological filter that is used for breast cancer detection or the like can be applied.

  The blur information includes blur direction information indicating the blur direction in the case of blur when the blur is non-directional out-of-focus blur or directional blur, and the analysis The means acquires the blur direction information using the image data of the dot-like portion, and uses the data of the entire digital photographic image based on the blur direction information indicating blurring, the blur direction information It is preferable to obtain the blur information excluding.

The analysis means detects an edge for each of a plurality of different directions with respect to the image of the dotted portion,
Obtaining feature values of the edge in each of the directions;
The blur direction information is preferably acquired based on the feature amount in each direction.

  The image processing apparatus according to the present invention may further include a correcting unit that corrects the digital image after obtaining the blur information by the analyzing unit. And the correction | amendment means shall enlarge the degree to correct | amend, so that the said point-like part is large. Increasing the degree of correction as the point-like portion is larger is not limited to changing the degree of correction according to the size of the point-like portion, i.e., blur or blur, and the blur width is not less than a predetermined value. It also includes correction only for the size. Specifically, for example, the correction may be performed only when a blur width of 1/10 or more of the size such as the face width or a blur of the pupil size or more is detected by the blur analysis.

  You may make it provide as a program which makes a computer perform the image processing by the image processing method of this invention.

  According to the image processing method, apparatus, and program of the present invention, a point-like portion is detected from a digital photograph image, and blur information of the digital photograph image is obtained using data of the detected point-like portion image. Therefore, it is possible to obtain blur information without the necessity of attaching a special device to the imaging device, and to obtain correct blur information even if a gradation is applied to a part of a digital photo image. it can.

  Also, in the case of blurring in a digital photographic image, the blur direction information is obtained using the image data of the dotted portion, while blur information other than the blur direction information, for example, blur width (here, blur width) is obtained. For example, for the entire digital photo image, the blur direction obtained using the image data of the dotted portions so that the average width of the edges in the blur direction indicated by the blur direction information is the blur width. If it is obtained from the data of the entire digital photographic image based on the information, correct blur direction information can be obtained, and the amount of data when obtaining other blur information such as a blur width is large. It can be obtained more accurately.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing system A which is a first embodiment of an image processing method and apparatus and a program therefor according to the present invention. The image processing system A according to the present embodiment prints an input digital photographic image (hereinafter referred to as an abbreviated image) by performing blur correction processing, and the blur correction processing is read into an auxiliary storage device. The blur correction processing program is executed on a computer (for example, a personal computer). The blur correction processing program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer.

  Further, since the image data represents an image, the following description will be given without particularly distinguishing the image from the image data.

  As shown in FIG. 1, the image processing system A according to the present embodiment detects a pupil from an image D0 and obtains an image (hereinafter referred to as “pupil image”) D5 of a pupil portion, and a pupil image D5 or image. D0 is used to analyze the blur in the image D0 to determine whether the image D0 is a blurred image, and for the image D0 that is not a blurred image, information P indicating that it is not a blurred image. Is output to the output unit 270 described later, and the blur analysis unit 200 that transmits the blur information Q to the blur correction unit 230 described below for the image D0 that is a blurred image, and the blur obtained by the blur analysis unit 200 A blur correction unit 230 that obtains a corrected image D ′ by performing blur correction on the image D0 that is a blur image based on the information Q, and a corrected image D ′ or a blur image obtained by the blur correction unit 230 Print out the image D0 not image becomes an output unit 270 to obtain a print. Hereinafter, each unit of the image processing system A will be described.

  FIG. 2 is a block diagram showing the configuration of the pupil detection means 100 in the image processing system A shown in FIG. As shown in the figure, the pupil detection means 100 identifies whether or not a face portion is included in the image D0, and outputs the photographic image D0 as it is to the output unit 50 described later when the face portion is not included. On the other hand, when the face portion is included, the left eye and the right eye are further detected, and the detection unit 1 that outputs information S including the positions of both eyes and the distance d between both eyes to the trimming unit 10 and the collation unit 40 described later. And trimmed images D1a and D1b that include the left eye and the right eye by trimming the photographic image D0 based on the information S from the detection unit 1 (hereinafter, meaning to indicate both when there is no need to distinguish between them) And a gray converting unit 1 that performs gray conversion on the trimmed image D1 to obtain a grayscale image D2 (D2a, D2b) of the trimmed image D1. A pre-processing unit 14 that performs pre-processing on the grayscale image D2 to obtain a pre-processed image D3 (D3a, D3b), and calculates a threshold T for binarizing the pre-processed image D3 2 A binarization threshold calculation unit 18 is provided, and the preprocessed image D3 is binarized using the threshold T obtained by the binarization threshold calculation unit 18 to obtain a binary image D4 (D4a, D4b). The binarization unit 20 and the coordinates of each pixel of the binary image D4 are voted to the Hough space of the ring to obtain the vote value of each voted position, and the integrated vote of the vote positions having the same circle center coordinates A voting unit 35 that calculates a value W (Wa, Wb), and a center position candidate G (Ga, Gb) corresponding to the center coordinates corresponding to the largest integrated voting value among the integrated voting values obtained by the voting unit 35 And searching for the next center position candidate from the collation unit 40 described later. The center position candidate acquisition unit 35 for obtaining the next center position candidate, and whether or not the center position candidate acquired by the center position candidate acquisition unit 35 satisfies the verification criterion. If the condition is satisfied, the center position candidate is output to the fine adjustment unit 45 described later as the center position of the pupil. On the other hand, if the matching reference is not satisfied, the center position candidate acquisition unit 35 is made to acquire the center position candidate again. The collation unit 40 that causes the center position candidate acquisition unit 35 to repeat acquisition of the center position candidate until the center position candidate acquired by the center position candidate acquisition unit 35 satisfies the collation criteria, and the output from the collation unit 40 Fine adjustment unit 45 that performs fine adjustment on the center position G (Ga, Gb) of the pupil and outputs the final center position G ′ (G′a, G′b) to the output unit 50; Based on the heart position G ′, a predetermined range surrounding the central position G′a and a predetermined range surrounding G′b are cut out to obtain a pupil image D5 (D5a, D5b), and this pupil image D5 is subjected to blur analysis. And an output unit 50 for outputting to the means 200. When the image D0 is an image that does not include a face portion, the output unit 50 outputs the image D0 to the blur analysis unit 200 as it is.

  FIG. 3 is a block diagram showing a detailed configuration of the detection unit 1 in the pupil detection unit 100 shown in FIG. As illustrated, the detection unit 1 includes a feature amount calculation unit 2 that calculates a feature amount C0 from a photographic image D0, a storage unit 4 that stores first and second reference data E1 and E2 described later, A first identification unit for identifying whether or not a photographic image D0 includes a human face based on the feature amount C0 calculated by the feature amount calculation unit 2 and the first reference data E1 in the storage unit 4. 5 and when the first identification unit 5 identifies that the face is included in the photographic image D0, the feature amount C0 in the face image calculated by the feature amount calculation unit 2 and the first in the storage unit 4 On the basis of the second reference data E2, a second identification unit 6 for identifying the position of an eye included in the face and a first output unit 7 are provided.

  The eye position identified by the detection unit 1 is the center position (indicated by x in FIG. 4) between the corner of the face and the eye of the face, and the eye facing directly in front as shown in FIG. 4 is the same as the center position of the pupil. However, as shown in FIG. 4B, in the case of the eye facing right, it is not the center position of the pupil, but the position away from the center of the pupil or the white eye portion. To do.

  The feature amount calculation unit 2 calculates a feature amount C0 used for face identification from the photograph image D0. When it is identified that the photograph image D0 includes a face, a similar feature amount C0 is calculated from the extracted face image as described later. Specifically, the gradient vector (that is, the direction in which the density of each pixel on the photographic image D0 and the face image changes and the magnitude of the change) is calculated as the feature amount C0. Hereinafter, calculation of the gradient vector will be described. First, the feature amount calculation unit 2 performs filtering processing on the photographic image D0 by the horizontal edge detection filter shown in FIG. 5A to detect horizontal edges in the photographic image D0. Further, the feature amount calculation unit 2 performs filtering processing by the vertical edge detection filter shown in FIG. 5B on the photographic image D0 to detect the vertical edge in the photographic image D0. Then, a gradient vector K at each pixel is calculated from the horizontal edge size H and the vertical edge size V at each pixel on the photographic image D0, as shown in FIG. Similarly, the gradient vector K is calculated for the face image. Note that the feature amount calculation unit 2 calculates a feature amount C0 at each stage of deformation of the photographic image D0 and the face image, as will be described later.

  It should be noted that the gradient vector K calculated in this way is an eye in a dark part such as the eyes and mouth as shown in FIG. 7B in the case of a human face as shown in FIG. It faces the center of the mouth and faces outward from the position of the nose in a bright part like the nose. Further, since the change in density is larger in the eyes than in the mouth, the gradient vector K is larger in the eyes than in the mouth.

  The direction and magnitude of the gradient vector K are defined as a feature amount C0. The direction of the gradient vector K is a value from 0 to 359 degrees with reference to a predetermined direction of the gradient vector K (for example, the x direction in FIG. 6).

  Here, the magnitude of the gradient vector K is normalized. This normalization obtains a histogram of the magnitude of the gradient vector K in all pixels of the photographic image D0, and the distribution of the magnitudes is a value that each pixel of the photographic image D0 can take (0 to 255 if it is 8 bits). The histogram is smoothed so as to be uniformly distributed, and the magnitude of the gradient vector K is corrected. For example, when the gradient vector K is small and the histogram is distributed with the gradient vector K biased toward the small side as shown in FIG. The magnitude of the gradient vector K is normalized so that it extends over the region so that the histogram is distributed as shown in FIG. In order to reduce the calculation amount, as shown in FIG. 8C, the distribution range in the histogram of the gradient vector K is divided into, for example, five, and the frequency distribution divided into five is shown in FIG. 8D. It is preferable to normalize so that the value of 0 to 255 is in a range divided into five.

  The first and second reference data E1 and E2 stored in the storage unit 4 constitute each pixel group for each of a plurality of types of pixel groups composed of a combination of a plurality of pixels selected from a sample image to be described later. The identification condition for the combination of the feature amount C0 in each pixel is defined.

  In the first and second reference data E1 and E2, the combination and identification condition of the feature amount C0 in each pixel constituting each pixel group may not be a plurality of sample images and faces that are known to be faces. This is determined in advance by learning a sample image group including a plurality of known sample images.

  In the present embodiment, when generating the first reference data E1, the sample image known to be a face has a 30 × 30 pixel size, and as shown in FIG. The distance between the centers of both eyes in the image of one face is 10 pixels, 9 pixels, and 11 pixels, and a face standing vertically at the distance between the centers of both eyes is stepped in units of 3 degrees within a range of ± 15 degrees on the plane. Sample images that have been rotated (that is, the rotation angles are −15 degrees, −12 degrees, −9 degrees, −6 degrees, −3 degrees, 0 degrees, 3 degrees, 6 degrees, 9 degrees, 12 degrees, and 15 degrees). Shall be used. Therefore, 3 × 11 = 33 sample images are prepared for one face image. In FIG. 9, only sample images rotated at −15 degrees, 0 degrees, and +15 degrees are shown. The center of rotation is the intersection of the diagonal lines of the sample image. Here, if the distance between the centers of both eyes is a 10-pixel sample image, the center positions of the eyes are all the same. The center position of this eye is set to (x1, y1) and (x2, y2) on the coordinates with the upper left corner of the sample image as the origin. In addition, the eye positions in the vertical direction in the drawing (ie, y1, y2) are the same in all sample images.

  Further, when the second reference data E2 is generated, the sample image known to be a face has a 30 × 30 pixel size, and as shown in FIG. The distance between the centers is 10 pixels, 9.7 pixels, and 10.3 pixels, and the face standing vertically at the distance between the centers of each eye is rotated step by step in a range of ± 3 degrees on the plane. It is assumed that the sample image (that is, the rotation angle is −3 degrees, −2 degrees, −1 degrees, 0 degrees, 1 degree, 2 degrees, 3 degrees) is used. Therefore, 3 × 7 = 21 sample images are prepared for one face image. Note that FIG. 10 shows only sample images rotated to −3 degrees, 0 degrees, and +3 degrees. The center of rotation is the intersection of the diagonal lines of the sample image. Here, the positions of the eyes in the vertical direction in the drawing are the same in all the sample images. In order to set the distance between the centers of both eyes to 9.7 pixels and 10.3 pixels, the sample image whose distance between the centers of both eyes is 10 pixels is enlarged or reduced to 9.7 times or 10.3 times. Thus, the size of the sample image after enlargement / reduction may be set to 30 × 30 pixels.

  Then, the center position of the eye in the sample image used for learning the second reference data E2 is set as the eye position to be identified in the present embodiment.

  As a sample image that is known not to be a face, an arbitrary image having a 30 × 30 pixel size is used.

  Here, as a sample image that is known to be a face, learning is performed using only a center image whose distance between the centers of both eyes is 10 pixels and the rotation angle on the plane is 0 degree (that is, the face is vertical). When performed, the face or eye position is identified with reference to the first and second reference data E1 and E2 only for a face that is not rotated at all with a center-to-center distance of both eyes of 10 pixels. is there. Since the size of the face that may be included in the photographic image D0 is not constant, when identifying whether or not the face is included or the position of the eyes, the photographic image D0 is enlarged or reduced as described later. The position of the face and eyes that match the size of the sample image can be identified. However, in order to accurately set the distance between the centers of both eyes to 10 pixels, the size of the photographic image D0 needs to be identified while being enlarged or reduced in steps of, for example, 1.1 units as an enlargement ratio. Will be enormous.

  Further, the face that may be included in the photographic image D0 is not only rotated at 0 degree on the plane as shown in FIG. 11A, but also rotated as shown in FIGS. 11B and 11C. Sometimes it is. However, when learning is performed using only a sample image in which the distance between the centers of both eyes is 10 pixels and the rotation angle of the face is 0 degrees, FIGS. As shown in (), the rotated face cannot be identified.

  Therefore, in this embodiment, as a sample image known to be a face, the distance between the centers of both eyes is 9, 10, 11 pixels as shown in FIG. 9, and ± 15 degrees on the plane at each distance. In this range, a sample image obtained by rotating the face step by step in increments of 3 degrees is used to allow the learning of the first reference data E1. As a result, when the identification is performed in the first identification unit 5 to be described later, the photographic image D0 can be enlarged or reduced stepwise in increments of 11/9 with the photographic image D0 as the enlargement rate. For example, the calculation time can be reduced as compared with a case where the enlargement / reduction is performed in units of 1.1. In addition, as shown in FIGS. 11B and 11C, a rotating face can be identified.

  On the other hand, in learning of the second reference data E2, the distance between the centers of both eyes is 9.7, 10, 10.3 pixels as shown in FIG. 10, and each distance is within a range of ± 3 degrees on the plane. Since the sample image obtained by rotating the face step by step in units of 1 degree is used, the learning tolerance is smaller than that of the first reference data E1. Further, when the identification is performed by the second identification unit 6 to be described later, the photographic image D0 needs to be enlarged / reduced in units of 10.3 / 9.7 as an enlargement ratio. It takes a longer time to calculate than the identification. However, since only the image in the face identified by the first identification unit 5 is identified by the second identification unit 6, the eye position is identified as compared with the case where the entire photographic image D0 is used. The amount of calculation for performing can be reduced.

  Hereinafter, an example of a learning method for the sample image group will be described with reference to the flowchart of FIG. Here, learning of the first reference data E1 will be described.

  The group of sample images to be learned includes a plurality of sample images that are known to be faces and a plurality of sample images that are known not to be faces. As described above, the sample image that is known to be a face has 9, 10, 11 pixels in the center position of both eyes for one sample image, and is 3 in a range of ± 15 degrees on the plane at each distance. Use a face rotated stepwise in degrees. Each sample image is assigned a weight or importance. First, the initial value of the weight of all sample images is set equal to 1 (S1).

  Next, a discriminator is created for each of a plurality of types of pixel groups in the sample image (S2). Here, each discriminator provides a reference for discriminating between a face image and a non-face image by using a combination of feature amounts C0 in each pixel constituting one pixel group. In the present embodiment, a histogram for a combination of feature amounts C0 in each pixel constituting one pixel group is used as a discriminator.

The creation of a classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 13, each pixel constituting the pixel group for creating the discriminator is a pixel at the center of the right eye on a plurality of sample images that are known to be faces. P1, a pixel P2 on the right cheek, a pixel P3 on the forehead, and a pixel P4 on the left cheek. Then, combinations of feature amounts C0 in all the pixels P1 to P4 are obtained for all sample images that are known to be faces, and a histogram thereof is created. Here, the feature amount C0 represents the direction and magnitude of the gradient vector K. Since the gradient vector K has 360 directions from 0 to 359 and the gradient vector K has 256 sizes from 0 to 255, If used as they are, the number of combinations is 360 × 256 four pixels per pixel, that is, (360 × 256) ^four , and the number of samples, time and memory for learning and detection are large. Will be required. For this reason, in this embodiment, the gradient vector directions are 0 to 359, 0 to 44, 315 to 359 (right direction, value: 0), 45 to 134 (upward value: 1), and 135 to 224 (left). Direction, value: 2), 225-314 (downward, value 3), and quaternarization, and the gradient vector magnitude is ternarized (value: 0-2). And the value of a combination is computed using the following formula | equation.

Combination value = 0 (when gradient vector size = 0)
Combination value = ((gradient vector direction + 1) × gradient vector magnitude (gradient vector magnitude> 0)
Thus, since the number of combinations is nine patterns ^4, can reduce the number of data of the characteristic amounts C0.

  Similarly, histograms are created for a plurality of sample images that are known not to be faces. For the sample image that is known not to be a face, pixels corresponding to the positions of the pixels P1 to P4 on the sample image that is known to be a face are used. A histogram used as a discriminator shown on the right side of FIG. 13 is a histogram obtained by taking logarithmic values of ratios of frequency values indicated by these two histograms. The value of each vertical axis indicated by the histogram of the discriminator is hereinafter referred to as an identification point. According to this classifier, an image showing the distribution of the feature quantity C0 corresponding to the positive identification point is highly likely to be a face, and it can be said that the possibility increases as the absolute value of the identification point increases. Conversely, an image showing the distribution of the feature quantity C0 corresponding to the negative identification point is highly likely not to be a face, and the possibility increases as the absolute value of the identification point increases. In step S <b> 2, a plurality of classifiers in the above-described histogram format are created for combinations of feature amounts C <b> 0 in the respective pixels constituting a plurality of types of pixel groups that can be used for identification.

  Subsequently, the most effective classifier for identifying whether or not the image is a face is selected from the plurality of classifiers created in step S2. The most effective classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rate of each classifier is compared, and the classifier showing the highest weighted correct answer rate is selected (S3). That is, in the first step S3, since the weight of each sample image is equal to 1, the number of sample images in which the image is correctly identified by the classifier is simply the largest. Selected as a valid discriminator. On the other hand, in the second step S3 after the weight of each sample image is updated in step S5, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step S3 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the classifiers selected so far, that is, the result of identifying whether each sample image is a face image using a combination of the classifiers selected so far, is actually It is ascertained whether or not the rate that matches the answer indicating whether the image is a face image exceeds a predetermined threshold (S4). Here, the sample image group to which the current weight is applied or the sample image group to which the weight is equal may be used for evaluating the correct answer rate of the combination. When the predetermined threshold value is exceeded, learning can be completed because it is possible to identify whether the image is a face with a sufficiently high probability by using the classifier selected so far. If it is equal to or less than the predetermined threshold value, the process proceeds to step S6 in order to select an additional classifier to be used in combination with the classifier selected so far.

  In step S6, the discriminator selected in the most recent step S3 is excluded so as not to be selected again.

  Next, the weight of the sample image that could not be correctly identified as a face by the classifier selected in the most recent step S3 is increased, and the sample image that can be correctly identified as whether or not the image is a face is increased. The weight is reduced (S5). The reason for increasing or decreasing the weight in this way is that in selecting the next discriminator, an image that cannot be discriminated correctly by the already selected discriminator is regarded as important, and whether or not those images are faces can be discriminated correctly. This is to increase the effect of the combination of the discriminators by selecting the discriminators.

  Subsequently, the process returns to step S3, and the next valid classifier is selected based on the weighted correct answer rate as described above.

  By repeating the above steps S3 to S6, the classifier corresponding to the combination of the feature amount C0 in each pixel constituting the specific pixel group is selected as a classifier suitable for identifying whether or not a face is included. If the correct answer rate confirmed in step S4 exceeds the threshold value, the type of the discriminator used for discriminating whether or not a face is included and the discriminating condition are determined (S7). The learning of the reference data E1 is finished.

  Then, the second reference data E2 is learned by obtaining the classifier type and identification conditions in the same manner as described above.

  In the case of adopting the above learning method, the discriminator provides a reference for discriminating between a face image and a non-face image using a combination of feature amounts C0 in each pixel constituting a specific pixel group. As long as it is not limited to the above histogram format, it may be anything, for example, binary data, a threshold value, a function, or the like. Further, even with the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 13 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  The first identification unit 5 configures each pixel group with reference to the identification conditions learned by the first reference data E1 for all combinations of the feature amounts C0 in the respective pixels constituting the plurality of types of pixel groups. An identification point for the combination of the feature amount C0 in each pixel is obtained, and all the identification points are combined to identify whether or not a face is included in the photographic image D0. At this time, the direction of the gradient vector K, which is the feature amount C0, is quaternized and the magnitude is quinary. In the present embodiment, all the identification points are added, and identification is performed based on the positive / negative of the added value. For example, when the sum of the identification points is a positive value, it is determined that the photograph image D0 includes a face, and when the sum is negative, it is determined that no face is included. The identification performed by the first identification unit 5 as to whether or not a face is included in the photographic image D0 is referred to as a first identification.

  Here, the size of the photographic image D0 is different from the sample image of 30 × 30 pixels, and has various sizes. When a face is included, the rotation angle of the face on the plane is not always 0 degrees. Therefore, as shown in FIG. 14, the first identification unit 5 scales the photographic image D0 stepwise until the vertical or horizontal size becomes 30 pixels and rotates it 360 degrees stepwise on the plane. However, a mask M having a size of 30 × 30 pixels is set on the photographic image D0 enlarged / reduced at each stage, and the mask M is set to 1 on the enlarged photographic image D0. While moving pixel by pixel, it is identified whether or not the image in the mask is a face image, thereby identifying whether or not a face is included in the photographic image D0.

  Note that since the sample image learned at the time of generating the first reference data E1 has 9, 10, and 11 pixels at the center position of both eyes, the enlargement ratio at the time of enlargement / reduction of the photographic image D0 is 11 / 9. Since the sample image learned at the time of generating the first and second reference data E1 and E2 is a sample image whose face is rotated in a range of ± 15 degrees on a plane, the photographic image D0 is 30 degrees. What is necessary is just to rotate 360 degree | times per unit.

  Note that the feature amount calculation unit 2 calculates the feature amount C0 at each stage of deformation such as enlargement / reduction and rotation of the photographic image D0.

  Then, whether or not a face is included in the photographic image D0 is identified for the photographic image D0 at all stages of enlargement / reduction and rotation. And a 30 × 30 pixel region corresponding to the position of the identified mask M is extracted as a face image from the photographic image D0 of the size and rotation angle at the stage where it is identified that the face is included. .

  The second identification unit 6 learns the second reference data E2 for all the combinations of the feature amounts C0 in the respective pixels constituting the plurality of types of pixel groups on the face image extracted by the first identification unit 5. With reference to the identification conditions, the identification points for the combination of the feature amounts C0 in the respective pixels constituting each pixel group are obtained, and the positions of the eyes included in the face are identified by combining all the identification points. At this time, the direction of the gradient vector K, which is the feature amount C0, is quaternized and the magnitude is quinary.

  Here, the second discriminating unit 6 enlarges / reduces the size of the face image extracted by the first discriminating unit 5 stepwise and is enlarged / reduced at each step while rotating stepwise 360 degrees on the plane. A mask M having a 30 × 30 pixel size is set on the face image, and the eye position in the image in the mask is identified while moving the mask M pixel by pixel on the enlarged / reduced face.

  Since the sample image learned at the time of generating the second reference data E2 has the number of pixels at the center position of both eyes of 9.07, 10, and 10.3 pixels, enlargement when the face image is enlarged or reduced The rate may be 10.3 / 9.7. Further, as the sample image learned at the time of generating the second reference data E2, a face image rotated in a range of ± 3 degrees on the plane is used, so the face image is rotated 360 degrees in units of 6 degrees. Just do it.

  Note that the feature amount calculation unit 2 calculates the feature amount C0 at each stage of deformation such as enlargement / reduction and rotation of the face image.

  In this embodiment, all the identification points are added at all stages of deformation of the extracted face image, and the upper left corner of the face image in the mask M of 30 × 30 pixels at the stage of deformation having the largest added value is obtained. The coordinates corresponding to the coordinates (x1, y1) and (x2, y2) of the eye position in the sample image are obtained, and the position corresponding to this position in the photographic image D0 before deformation is set as the coordinate. Identify the location.

  When the first identification unit 5 identifies that the face is not included in the photographic image D0, the first output unit 7 outputs the photographic image D0 as it is to the output unit 50, while the first identification unit 5 Recognizes that a face is included in the photographic image D0, the distance d between both eyes is obtained from the position of both eyes identified by the second identification unit 6, and the position d of both eyes and the distance d between both eyes is used as information S. The data is output to the trimming unit 10 and the collation unit 40.

  FIG. 15 is a flowchart showing the operation of the detection unit 1 in the pupil detection unit 100. For the photographic image D0, first, the feature amount calculation unit 2 calculates the direction and size of the gradient vector K of the photographic image D0 as the feature amount C0 at each stage of enlargement / reduction and rotation of the photographic image D0 (S12). . Then, the first identification unit 5 reads the first reference data E1 from the storage unit 4 (S13), and performs first identification as to whether or not a face is included in the photographic image D0 (S14).

  When determining that the face is included in the photographic image D0 (S14: Yes), the first identification unit 5 extracts the face from the photographic image D0 (S15). Here, not only one face but a plurality of faces may be extracted. Next, the feature amount calculation unit 2 calculates the direction and size of the gradient vector K of the face image as the feature amount C0 at each stage of enlargement / reduction and rotation of the face image (S16). Then, the second identification unit 6 reads the second reference data E2 from the storage unit 4 (S17), and performs second identification for identifying the position of the eyes included in the face (S18).

  Subsequently, the first output unit 7 outputs the eye position identified from the photographic image D0 and the distance d between both eyes obtained based on the eye position as information S to the trimming unit 10 and the collation unit 40. (S19).

  On the other hand, if it is determined in step S14 that no face is included in the photographic image D0 (S14: No), the first output unit 7 outputs the photographic image D0 as it is to the output unit 50 (S19).

  The trimming unit 10 cuts out a predetermined range including only the left eye and only the right eye based on the information S output from the detection unit 1 to obtain trimmed images D1a and D1b. Here, the predetermined range at the time of trimming is a range in which the vicinity of each eye is an outer frame. For example, as shown in the hatched range in FIG. Centered around the center point) of the figure, the lengths in the X direction and Y direction in the figure are d and 0.5d, respectively. The hatched area shown in the figure is the trimming range of the left eye in the figure, but the same applies to the right eye.

  The gray conversion unit 12 performs a gray conversion process on the trimmed image D1 obtained by the trimming unit 10 according to the following equation (1) to obtain a grayscale image D2.

Y = 0.299 × R + 0.587 × G + 0.114 × B (1)
Y: Luminance value
R, G, B: R, G, B values

The preprocessing unit 14 performs preprocessing on the grayscale image D2, and here, smoothing processing and hole filling processing are performed as preprocessing. The smoothing process is performed by applying, for example, a Kaussian filter, and the hole filling process can be an interpolation process.

  As shown in FIG. 4, in the pupil portion of the photographic image, there is a tendency that the portion above the center is partially brightened. Therefore, the detection accuracy of the center position of the pupil is obtained by performing hole filling processing and interpolating the data of this portion. Can be improved.

  The binarization unit 20 includes a binarization threshold value calculation unit 18, and uses the threshold value T calculated by the binarization threshold value calculation unit 18 to store the preprocessed image D <b> 3 obtained by the preprocessing unit 14. The binary image D4 is obtained by digitization. Specifically, the binarization threshold value calculation unit 18 creates a luminance histogram shown in FIG. 17 for the preprocessed image D3, and is a fraction of the total number of pixels of the preprocessed image D3 (in the drawing, The luminance value corresponding to the appearance frequency corresponding to 1/5 (20%) is obtained as the threshold T for binarization. The binarization unit 20 binarizes the preprocessed image D3 using the threshold T to obtain a binary image D4.

  The voting unit 30 first votes the coordinates of each pixel (pixel having a pixel value of 1) in the binarized image D4 to the annular Hough space (circle center point X coordinate, circle center point Y coordinate, radius r). Then, the voting value at each voting position is calculated. Normally, when one vote position is voted by a pixel, the vote value at each vote position is obtained by adding 1 to the vote value as if it was voted once. When one vote is voted for a certain pixel, instead of adding 1 to the vote value, the brightness value of the voted pixel is referred to, and the smaller the brightness value, the higher the weight is added. The voting value at each voting position is obtained. FIG. 18 shows a table of weighting coefficients used in the voting unit 30 in the pupil detection means 100 shown in FIG. Note that T in the figure is a binarization threshold T calculated by the binarization threshold calculation unit 18.

  After the voting unit 30 obtains the voting value of each voting position in this way, among these voting positions, the coordinate value of the center point of the ring, that is, (X, Y, r) in the ring Hough space (X, Y, r). Y) The vote values of the vote positions having the same coordinate value are added to obtain an integrated vote value W corresponding to each (X, Y) coordinate value, and corresponding to the corresponding (X, Y) coordinate value Then, the data is output to the center position candidate acquisition unit 35.

  The center position candidate acquisition unit 35 first acquires (X, Y) coordinate values corresponding to the largest integrated vote value as the center position candidate G of the pupil from each integrated vote value from the voting unit 30. Output to the verification unit 40. Here, the center position candidate G acquired by the center position candidate acquisition unit 35 is two, that is, the center position Ga of the left pupil and the center position Gb of the right pupil, and the collation unit 40 is output by the detection unit 1. Based on the distance d between the eyes, the two center positions Ga and Gb are collated.

  Specifically, the collation unit 40 performs collation based on the following two collation criteria.

1. The difference in Y coordinate value between the center position of the left pupil and the center position of the right pupil is (d / 50) or less.

2. The X coordinate value difference between the center position of the left pupil and the center position of the right pupil is within the range of (0.8 × d to 1.2 × d).

The collation unit 40 determines whether or not the two pupil center position candidates Ga and Gb from the center position candidate acquisition unit 35 satisfy the above two collation criteria. The pupil center position candidates Ga and Gb are output to the fine adjustment unit 45 as the pupil center position. On the other hand, if one of the two criteria or one of the two criteria is not satisfied (hereinafter referred to as not satisfying the collation criteria), the center position candidate acquisition unit 35 is instructed to acquire the next center position candidate. , For the next center position candidate acquired by the center position candidate acquisition unit 35, the above-described collation, the center position output when the collation criteria are satisfied, and the center position candidate when the collation criteria are not met are reacquired. Processing such as instructions is repeated until the verification criteria are satisfied.

  When the acquisition of the next center position candidate is instructed from the collation unit 40, one of the center position candidate acquisition units 35 first fixes the center position of one side (here, the left pupil) and the other side (here, the left center position). The (X, Y) coordinate value of the voting position satisfying the following three conditions is acquired as the next center position candidate from each integrated voting value Wb of (right pupil).

1. Finally, it is separated from the position indicated by the (X, Y) coordinate value of the center position candidate output to the collation unit 40 by d / 30 or more (D: distance between both eyes).

2. The corresponding integrated voting value corresponds to the (X, Y) coordinate value of the center position candidate that is finally output to the collation unit 40 among the integrated voting values corresponding to the (X, Y) coordinate value satisfying the condition 1. Next to the integrated vote value.

3. The corresponding integrated voting value is 10% or more of the integrated voting value (the largest integrated voting value) corresponding to the (X, Y) coordinate value of the center position candidate output to the collation unit 40 for the first time.

The center position candidate acquisition unit 35 first fixes the center position of the left pupil and searches for a center position candidate of the right pupil that satisfies the above three conditions based on the integrated vote value Wb obtained for the right pupil. If no candidate satisfying the above three conditions is found, the center position of the right pupil is fixed, and the left pupil satisfying the above three conditions is determined based on the integrated vote value Wa obtained for the left pupil. Find the center position.

  The fine adjustment unit 45 performs fine adjustment on the pupil center position G output from the collation unit 40 (center position candidate satisfying the collation criteria). First, fine adjustment of the center position of the left pupil will be described. The fine adjustment unit 45 repeats the mask operation three times using the all-one mask having a size of 9 × 9 on the binary image D4a of the left-eye trimmed image D1a obtained by the binarization unit 20, Based on the position (Gm) of the pixel having the maximum result value obtained as a result of this mask calculation, fine adjustment is performed on the center position Ga of the left pupil output from the matching unit 40. Specifically, for example, the average position obtained by taking the average of the position Gm and the center position Ga may be set as the final center position G′a of the pupil, or the center position Ga is weighted. The average position obtained by the average calculation may be used as the final center position G′a of the pupil. Here, an average calculation is performed with weights applied to the center position Ga.

  Further, the fine adjustment of the center position of the right pupil is performed in the same manner as described above using the binary image D4b of the trimmed image D1b of the right eye.

  In this way, the fine adjustment unit 45 provides the output unit 50 with final center positions G′a and G′b obtained by performing fine adjustment on the pupil center positions Ga and Gb output from the collation unit 40. Output.

  The output unit 50 outputs the image D0 that does not include the face as it is to the blur analysis unit 200, but for the image D0 that includes the face, the center position G′a is based on the final center position G ′. And a predetermined range surrounding G′b is cut out to obtain a pupil image D5 (D5a, D5b), and this pupil image D5 is output to the blur analysis means 200.

  FIG. 19 is a flowchart showing the processing of the pupil detection means 100 shown in FIG. As shown in the figure, the photographic image D0 is first discriminated whether or not a face is included in the detection unit 1 (S110). As a result of the determination, if the face is not included in the photographic image D0 (S115: No), the photographic image D0 is output from the detection unit 1 to the output unit 50, while if the photographic image D0 includes a face ( Further, the position of the eyes in the photographic image D0 is detected by the detection unit 1, and the position of both eyes and the distance d between the eyes are output as information S to the trimming unit 10 (S120). In the trimming unit 10, the photographic image D0 is trimmed to obtain a trimmed image D1a including only the left eye and a trimmed image D1b including only the right eye (S125). The trimmed image D1 is gray-converted by the gray converter 12 to become a grayscale image D2 (S130). The grayscale image D2 is subjected to smoothing processing and hole filling processing by the preprocessing unit 14, and is further binarized by the binarizing unit 20 to become a binary image D4 (S135, S140). In the voting unit 30, the coordinates of each pixel of the binary image D4 are voted to the annular Hough space, and as a result, an integrated vote value W corresponding to the (X, Y) coordinate value indicating each circle center point is obtained. (S145). The center position candidate acquisition unit 35 first outputs the (X, Y) coordinate value corresponding to the largest integrated vote value to the collation unit 40 as the pupil center position candidate G (S150). The collation unit 40 collates the two center position candidates Ga and Gb from the center position candidate acquisition unit 35 based on the collation reference described above (S115), and the two center position candidates Ga and Gb use the collation reference. If the two are satisfied (S160: Yes), the two center position candidates Ga and Gb are output to the fine adjustment unit 45 as the center position, while the two center position candidates Ga and Gb do not satisfy the collation criteria (S160). : No), the center position candidate acquisition unit 35 is instructed to search for the next center position candidate (S150). The processing from step S150 to step S160 is repeated until the collation unit 40 determines that the center position candidate G from the center position candidate acquisition unit 35 satisfies the collation criteria.

  The fine adjustment unit 45 performs fine adjustment on the center position G output from the collation unit 40, obtains the final center position G ', and outputs it to the output unit 50 (S165).

  The output unit 50 outputs the image D0 (S115: No) that does not include the face to the blur analysis unit 200 as it is, but for the image D0 that includes the face, based on the final center position G ′, A predetermined range surrounding the center position G′a and a predetermined range surrounding G′b are cut out to obtain a pupil image D5 (D5a, D5b), and this pupil image D5 is output to the blur analysis means 200 (S170). .

  As described above, the blur analysis unit 200 of the image processing system A illustrated in FIG. 1 receives the image D0 that does not include the face or the pupil image D5 of the image D0 that includes the face.

  FIG. 20 is a block diagram illustrating a configuration of the blur analysis unit 200. As illustrated, the blur analysis unit 200 includes an edge detection unit 212, an edge profile creation unit 213, an edge narrowing unit 214, an edge feature amount acquisition unit 216, an analysis execution unit 220, and a storage unit 225. It has.

  The edge detection unit 212 uses the image D0 or the pupil image D5 (hereinafter referred to as a target image) to detect edges having a predetermined intensity or more in every eight directions as shown in FIG. Obtained and output to the edge profile creation means 213. Based on the coordinate position of each edge in each direction detected by the edge detection unit 212, the edge profile creation unit 213 uses the corresponding target image to perform an edge as shown in FIG. A profile is created and output to the edge narrowing means 214.

  The edge narrowing means 214 is based on the edge profile output from the edge profile creating means 213, and has an edge having a complex profile shape or an edge including a light source (specifically, an edge having a certain lightness or higher, for example). And the like, and the remaining edge profile is output to the edge feature quantity acquisition means 216.

  The edge feature quantity acquisition unit 216 obtains an edge width as shown in FIG. 22 for each edge based on the edge profile output from the edge narrowing unit 214, and obtains the edge width as shown in FIG. A histogram is created for each of the eight directions shown in FIG. 21, and is output to the analysis execution means 220 as an edge feature amount S together with the edge width.

  The analysis execution unit 220 mainly performs the following two processes.

  1. The blur direction and the blur degree N in the target image are obtained to determine whether the target image is a blur image or a normal image.

  2. When it is determined that the target image is a blurred image, a blur width L and a blur degree K are calculated.

  Here, the first process will be described.

  In order to obtain the blur direction in the target image, the analysis execution unit 220 first sets two directions orthogonal to each other to a histogram of edge widths in eight directions shown in FIG. As a set, the correlation value of the histogram of each direction set (1-5, 2-6, 3-7, 4-8) is obtained. Note that there are various types of correlation values, depending on how they are obtained.If the correlation value is large, the correlation type is small, and if the correlation value is the same as the correlation level, that is, if the correlation value is small, the correlation type is small. It can be roughly divided into two types. In this embodiment, as an example, a correlation value of a type in which the magnitude of the correlation value matches the magnitude of the correlation is used. As shown in FIG. 24, when there is blur in the image, the correlation between the blur direction histogram and the histogram in the direction orthogonal to the blur direction is small (see FIG. 24A). In the orthogonal direction set that is not related to or in the case where there is no blur in the image (no blur or out of focus), the correlation of the histogram is large (see FIG. 24B). The analysis execution means 220 in the image processing system A of the present embodiment pays attention to such a tendency, obtains the correlation value of the histogram of each group for the four direction groups, and obtains the two direction groups having the smallest correlation. Find directions. If there is a blur in the image D, one of these two directions can be considered as a direction closest to the blur direction among the eight directions shown in FIG.

  FIG. 24 (c) shows a histogram of edge widths in the direction of blurring obtained for each image obtained by imaging the same subject under imaging conditions without blurring, blurring and blurring (blurring and blurring). Show. As can be seen from FIG. 24 (c), the normal image without blur has the smallest average edge width, that is, of the two directions found above, the one with the largest average edge width is the most blurred. The direction should be close to.

  In this way, the analysis execution unit 220 finds the direction set having the smallest correlation, and sets the direction with the larger average edge width of the two directions of the direction set as the blur direction.

  Next, the analysis execution unit 220 calculates the degree of blur N of the target image. The degree of blur of the image indicates the magnitude of the degree of blur in the image. For example, the average edge width in the direction most blurred in the image (here, the blur direction obtained above) may be used. However, in this case, the edge width of each edge in the blur direction is used to obtain more accurately using the database based on FIG. FIG. 25 is based on a normal image database for learning and a blurred (blurred and blurred) image database, and the direction in which the image is most blurred (in the case of a normal image, a direction corresponding to this direction is desirable, but arbitrary The edge width distribution histogram (which may be the direction of the image) is created, and the ratio between the frequency in the blurred image and the frequency in the normal image (the vertical axis in the drawing) is obtained for each edge width as an evaluation value (the score in the drawing). Is. Based on FIG. 25, a database (hereinafter referred to as a score database) in which the edge width and the score are associated with each other is created and stored in the storage unit 225.

  The analysis execution unit 220 refers to the score database created based on FIG. 25 and stored in the storage unit 225, acquires a score from the edge width of each edge in the blur direction of the target image, and blur direction Is obtained as the degree of blur N of the target image. If the obtained blur degree N of the target image is smaller than a predetermined threshold (T1), the analysis execution unit 220 displays the image D0 when the target image is the image D0, and the target image is the pupil image D5. In this case, the image D0 corresponding to the pupil image D5 is determined as a normal image, and information P indicating that the image D0 is a normal image is output to the output means 60, and the processing is terminated.

  On the other hand, if the degree of blur N of the target image is greater than or equal to the threshold T1, the analysis execution unit 220 determines that the target image is a blur image and enters the second process.

  As the second process, the analysis execution unit 220 first obtains the degree of blur K of the target image.

  The degree of blur K indicating the degree of blur in the blur image can be obtained based on the following factors.

1. Correlation value of the direction group with the smallest correlation (hereinafter referred to as the minimum correlation group): The smaller the correlation value, the greater the degree of blurring. The analysis execution means 220 pays attention to this point and is based on the curve shown in FIG. To obtain the first degree of blur K1. Note that an LUT (look-up table) created according to the curve shown in FIG. 26A is stored in the storage unit 225, and the analysis execution unit 220 sets the first correlation value corresponding to the correlation value of the minimum correlation set. The first blur degree K1 is obtained by reading the blur degree K1 from the storage means 225.

2. Of the two directions of the minimum correlation set, the average edge width in the direction where the average edge width is large: The larger the average edge width, the greater the degree of blurring. The analysis execution means 220 pays attention to this point, and FIG. ) To determine the second degree of blur K2. Note that an LUT (look-up table) created according to the curve shown in FIG. 26B is stored in the storage unit 225, and the analysis executing unit 220 has a direction in which the average edge width of the minimum correlation set is large. The second blurring degree K2 corresponding to the average edge width is read from the storage means 225 to obtain the second blurring degree K2.

3. Difference in average edge width between the two directions of the minimum correlation pair: The greater this difference, the greater the degree of blurring. The analysis execution means 220 pays attention to this point and based on the curve shown in FIG. A third blurring degree K3 is obtained. Note that an LUT (look-up table) created according to the curve shown in FIG. 26C is stored in the storage unit 225, and the analysis execution unit 220 calculates the average of each of the two directions of the minimum correlation set. The third blur degree K3 corresponding to the edge width difference is read from the storage means 225 to obtain the third blur degree K3.

  The analysis execution means 220 obtains the first blur degree K1, the second blur degree K2, and the third blur degree K3 in this way, and uses K1, K2, and K3 according to the following equation (2). The degree of blur K of the target image that becomes a blurred image is obtained.

K = K1 × K2 × K3 (2)
Where K: degree of blurring K1: first degree of blurring K2: second degree of blurring K3: third degree of blurring

Next, the analysis execution unit 220 obtains the blur width L of the target image. Here, regardless of the blurring degree K, the average width of edges in the blur direction may be obtained as the blur width L, or the average edge width of edges in all eight directions shown in FIG. L may be used.

  When the target image is the image D0, the analysis execution unit 220 outputs the obtained blur degree K and blur width L to the blur correction unit 230 as blur information Q of the image D0 together with the blur degree N and the blur direction. Even when the target image is the pupil image D5, the blur correction means uses the blur degree K and the blur width L obtained from the pupil image D5 as blur information Q of the image D0 corresponding to the pupil image D5 together with the blur degree N and the blur direction. 230.

  FIG. 27 is a flowchart showing the processing of the blur analysis unit 200 shown in 20. As illustrated, the blur analysis unit 200 first applies an image D0 in the case of an image D0 that does not include a face to the target image that is the pupil image D5 of the image D0 in the case of an image D0 that includes a face. The edge detection means 212 detects edges having a predetermined intensity or more in each of the eight different directions shown in FIG. 21 to obtain the coordinate position of each edge, and the edge profile creation means 213 determines the object based on these coordinate positions. An edge profile as shown in FIG. 22 is created for each edge using the image and output to the edge narrowing means 214 (S212). The edge narrowing unit 214 removes invalid edges based on the edge profile transmitted from the edge profile creating unit 213, and outputs the remaining edge profile to the edge feature quantity acquisition unit 216 (S214). The edge feature quantity acquisition means 216 obtains the width of each edge based on the profile of each edge transmitted from the edge narrowing means 214 and creates a histogram of edge width for each direction shown in FIG. The histogram of the edge width and the edge width in each direction is output to the analysis execution unit 220 as the edge feature amount S of the target image (S216). The analysis execution unit 220 first calculates the blur direction and the blur degree N of the target image using the edge feature amount S, and determines whether the image D0 is a blur image or a normal image (S220, S225). If the image D0 is a normal image (S225: Yes), the analysis execution unit 220 outputs information P indicating that the image D0 is a normal image to the output unit 270 (S230). On the other hand, when the image D0 is determined to be a blurred image (S225: No), the analysis execution unit 220 further calculates the degree of blur K and the blur width L with respect to the target image, and the degree of blur N obtained in step S220. Then, together with the blur direction, the blur information Q of the image D0 is output to the blur correction unit 230 (S240, S245).

  Note that the blur analysis unit 200 in this embodiment performs analysis using two pupil images (D5a, D5b), but only one pupil image may be used.

  The blur correction unit 230 performs blur correction on the image D0 determined to be a blur image based on the blur information Q of the image D0 obtained by the blur analysis unit 200. FIG. It is a block diagram which shows a structure.

  As shown in FIG. 28, the blur correction unit 230 includes a parameter setting unit 235 for setting a parameter E for correcting the image D0 based on the blur information Q, and various databases for the parameter setting unit 235. The stored storage means 240, the high frequency component extraction means 245 for extracting the high frequency component Dh from the image D0, and the correction execution means 250 for executing the blur correction for the image D0 using the parameter E and the high frequency component Dh are provided. Do it.

  The blur correction unit 230 in the image processing system A of the present embodiment corrects the image D0 that becomes a blurred image by an unsharpness masking (USM) correction method, and the parameter setting unit 235 includes blur information. In accordance with the blur width L and the blur direction included in Q, a one-dimensional correction mask M1 for directivity correction that acts in the blur direction is set so that the larger the blur width L, the larger the correction mask size. In accordance with the blur width L, the two-dimensional correction mask M2 for isotropic correction is set so that the larger the blur width L, the larger the correction mask size. The two-dimensional correction mask corresponding to each blur width and the one-dimensional correction mask corresponding to each blur width and blur direction are stored in the storage unit 240 as a database (referred to as a mask database), and the parameter setting unit 235 includes: From the mask database stored in the storage unit 240, a one-dimensional correction mask M1 is acquired based on the blur width L and the blur direction, and a two-dimensional correction mask M2 is acquired based on the blur width L.

  Next, the parameter setting means 235 sets the one-dimensional correction parameter W1 for directionality correction and the two-dimensional correction parameter W2 for isotropic correction according to the following equation (3).

W1 = N × K × M1
W2 = N * (1-K) * M2 (3)
However, W1: One-dimensional correction parameter
W2: Two-dimensional correction parameter
N: Defocus degree
K: Degree of blur
M1: One-dimensional correction mask
M2: Two-dimensional correction mask

That is, the parameter setting means 235 increases the correction parameters W1 and W2 so that the greater the degree of blur N, the stronger the isotropic correction and the higher the directionality correction, and the greater the blur degree K, the greater the weight for the directionality correction. (Also referred to as parameter E) is set.

  The correction execution unit 250 executes blur correction of the image D0 by emphasizing the high frequency component Dh obtained by the high frequency component extraction unit 245 using the parameter E set by the parameter setting unit 235, The blur correction is performed according to the following equation (4).

D ′ = D0 + E × Dh (4)
Where D ': corrected image
D0: Image before correction
E: Correction parameter
Dh: High frequency component

The output unit 270 outputs the image D0 when receiving the information P indicating that the image D0 is a normal image from the blur analysis unit 200, while receiving the corrected image D ′ from the blur correction unit 230. Outputs a corrected image D ′. In the image processing system A of the present embodiment, “output” by the output unit 270 is printing, and the output unit 270 corrects the image D ′ that has been obtained by correcting the image D0 of the normal image and the image D0 of the blurred image. Is printed to obtain a print, but it is stored in a recording medium, sent to an image storage server on the network, an address on the network specified by the client who requested the image correction, etc. It may be.

  FIG. 29 is a flowchart showing the operation of the image processing system A according to the embodiment shown in FIG. As shown in the figure, the face detection is first performed on the image D0 by the pupil detection unit 100 (S250). If no face is detected (S255: No), the blur analysis unit 200 analyzes blur using the data of the entire image D0 (S260). On the other hand, if a face is detected (S255: Yes), the pupil detection unit 100 further detects the pupil to obtain a pupil image D5 (S270), and the blur analysis unit 200 uses the pupil image data. The blur is analyzed (S275).

  When the blur analysis unit 200 determines that the image D0 is a normal image as a result of analyzing the image D0 or the pupil image D5, the blur analysis unit 200 outputs information P indicating that the image D0 is a normal image to the output unit 270. Then, the output unit 270 prints out the image D0 (S280: Yes, S290). On the other hand, if it is determined that the image D0 is a blurred image, the blur correction unit 230 uses the blur information Q obtained for the image D0. Then, the blur correction unit 230 performs blur correction on the image D0 based on the blur information Q (S280: No, S285). The corrected image D ′ obtained by the blur correction unit 230 is also printed out by the output unit 270 (S290).

  FIG. 30 is a block diagram showing a configuration of an image processing system B according to the second embodiment of the present invention. As shown in the figure, the image processing system B of this embodiment includes a pupil detection unit 100, a blur analysis unit 300, a blur correction unit 350, and an output unit 270. Among the units of the image processing system B of the present embodiment, the blur analysis unit 300 and the blur correction unit 350 are partially different from the corresponding units of the image processing system A of the embodiment shown in FIG. The other means are the same as the corresponding means of the image processing system A of the embodiment shown in FIG. 1, and therefore, other means other than the blur analysis means 300 and the blur correction means 350 will be described in the implementation shown in FIG. The same reference numerals are assigned to the corresponding means of the image processing system A of the embodiment, and detailed descriptions thereof are omitted.

  FIG. 31 is a block diagram showing a configuration of the blur analysis means 300 in the image processing system B shown in FIG. As illustrated, the blur analysis unit 300 includes an edge detection unit 312, an edge profile creation unit 313, an edge narrowing unit 314, an edge feature quantity acquisition unit 316, an analysis unit 320, and an analysis unit 320. It has a storage means 330 for storing various databases and a control means 305 for controlling each of the above means. The analysis unit 320 includes a first analysis unit 322, a second analysis unit 324, and a third analysis unit 326.

  The control unit 305 of the blur analysis unit 300 performs control based on whether or not a face is detected by the pupil detection unit 100. When the face is not detected from the image D0 by the pupil detection unit 100, the control unit 305 causes the edge detection unit 312 to perform edge detection on the image D0. The specific operations of the edge detection unit 312, the edge profile creation unit 313, the edge narrowing unit 314, and the edge feature amount acquisition unit 316 are the same as those of the blur analysis unit 200 in the image processing system A shown in FIG. Since the operations of the corresponding means are the same, detailed description is omitted here. With respect to the edges detected by the edge detection means 312, the edge profile creation means 313, the edge narrowing means 314, and the edge feature quantity acquisition means 316 are respectively processed, and the edge feature quantity Sz in the image D0. Is acquired. Note that the feature value Sz of the edge and the feature value Se described later are obtained from the edge width and edge width histogram in each direction in the same manner as the feature value S in the image processing system A of the embodiment shown in FIG. Become.

  The control unit 305 causes the first analysis unit 322 to analyze the edge feature quantity Sz. The first analysis unit 322 determines whether or not the image D0 is a blurred image based on the edge feature amount Sz, and transmits information P to the output unit 270 when the image D0 is a normal image. If it is a blurred image, the blur information Q is transmitted to the blur correction unit 350. The specific processing of the first analysis unit 322 is the same as the processing of the analysis execution unit 220 of the blur analysis unit 200 in the image processing system A of the embodiment shown in FIG.

  On the other hand, when a face or a pupil is detected by the pupil detection unit 100 and a pupil image D5 is obtained, the control unit 305 causes the edge detection unit 312 to perform edge detection on the pupil image D5. Further, the edge profile generation unit 313, the edge narrowing unit 314, and the edge feature amount acquisition unit 316 are subjected to the processing of the edge detected by the edge detection unit 312 and the feature of the edge in the pupil image D5. A quantity Se is obtained.

  Here, the control unit 305 first causes the second analysis unit 324 to analyze whether or not the pupil image D5 is a blurred image, and if the pupil image D5 is a blurred image, further analyze whether the blur is out of focus. First, the second analysis unit 324, based on the feature amount Se of the pupil image D5, similarly to the analysis unit 220 of the blur analysis unit 200 in the image processing system A of the embodiment shown in FIG. H), and the degree of blur N is obtained. When the obtained blur degree N is equal to or less than the threshold value T1, the image D0 corresponding to the pupil image D0 is determined as a normal image, and information P indicating that the image D0 is a normal image is transmitted to the output unit 270. To do. On the other hand, when the obtained blur degree N is larger than the threshold value T1, the image D0 corresponding to the pupil image D0 is determined as a blur image, and the blur degree K is further obtained. The method of calculating the degree of blur K by the second analysis unit 324 is the same as the calculation method of the analysis unit 220 of the blur analysis unit 200 in the image processing system A of the embodiment shown in FIG. Based on the obtained degree of blur K, the second analysis unit 324 determines whether the image D0 corresponding to the pupil image D0 is an out-of-focus image or a blurred image. Specifically, if the degree of blur K is equal to or less than a predetermined threshold T2, the image D0 is determined as a blurred image, and if the degree of blur K is greater than the threshold T2, the image D0 is determined as a blurred image.

  For the image D0 determined as the out-of-focus image, the second analysis unit 324 further obtains the blur width L from the edge feature amount Se of the pupil image D5, and information indicating that the image D0 is a out-of-focus image. Then, the blur width L is transmitted to the blur correction unit 350 as blur information Q, and the process ends.

  On the other hand, the second analysis unit 324 transmits the blur direction, that is, the blur direction h to the third analysis unit 326 with respect to the image D0 determined as the blur image, and ends the process. If it is determined that the image D0 is a blurred image, the control unit 305 causes the edge detection unit 312 to detect an edge in the blur direction h of the image D0 for the entire image D0. The edge profile creation means 313 and the edge narrowing means 314 are respectively processed for the edge detected in the blur direction h, and the profile of each edge in the blur direction h is the feature quantity Sz1 in the image D0. Get as.

  The third analysis means 326 calculates the average width of the edges in the blur direction h as the blur width from the profile of each edge of the feature amount Sz1, information indicating that the image D0 is a blur image, the blur width and The blur direction h is transmitted to the blur correction unit 350 as blur information Q1.

  FIG. 32 is a flowchart showing processing of the blur analysis unit 300 shown in FIG. As shown in the figure, the control means 305 of the blur analysis means 300 performs an edge detection from the entire image D0 to the edge detection means 312 for every eight directions shown in FIG. 21 with respect to the image D0 whose face is not detected by the pupil detection means 100. Is detected. The detected edge is processed by the edge profile creation means 313, the edge narrowing means 314, and the edge feature quantity acquisition means 316, and the edge feature quantity Sz of the image D0 is obtained. Then, the first analysis unit 322 uses the feature amount Sz to determine the blur direction and the blur degree N in the image D0 to determine whether the image D0 is a normal image, and is also determined as a normal image. Information P indicating that the image D0 is not a blurred image is output to the output unit 270. On the other hand, for the image D0 determined as a blurred image, a blur width L and a blur degree K are further obtained to determine the blur direction. Then, it transmits to the blur correction means 350 together with the blur degree N as blur information Q (S300: No, S305, S310).

  On the other hand, for the image D0 in which a face or pupil is detected by the pupil detection unit 100 (S300: Yes), the control unit 305 causes the edge detection unit 312 to detect the pupil image D5 of the image D0 every eight directions shown in FIG. Let the edge be detected. The edge profile creation unit 313, the edge narrowing unit 314, and the edge feature quantity acquisition unit 316 are each processed on the extracted edge, and the edge feature quantity Se in the pupil image D5 is obtained. The second analyzing means 324 uses the feature amount Se to determine the blur direction and the blur degree N in the pupil image D5 and determine whether the corresponding image D0 of the pupil image D5 is a normal image or a blur image. Information P indicating that the image D0 determined as an image is not a blurred image is output to the output means 270 (S320, S325: Yes, S330). For the image D0 determined as a blurred image in step S325 (S320, S325: No), the second analysis unit 324 further determines whether the image is a blurred image or a blurred image. The blur width is obtained from the feature amount Se of the pupil image D5 of D0 to obtain the blur width of the image D0, and is transmitted to the blur correction unit 350 as blur information Q of the blur image D0 together with information indicating that the image D0 is a blur image. S340: Yes, S345) On the other hand, in the case of a blurred image, the blur direction that is the blur direction h is transmitted to the third analysis unit 326 (S340: No, S350). The third analysis unit 326 is the blur direction h obtained from the entire image D0 corresponding to the pupil image D5 by the edge detection unit 312, the edge profile creation unit 313, the edge narrowing unit 314, and the edge feature amount acquisition unit 316. Using the edge feature quantity Sz1 at, the average edge width in the blur direction h is calculated as the blur width, and the blur width, the blur direction h, and information indicating that the image D0 is the blurred image is the blurred image D0. The blur information Q1 is transmitted to the blur correction unit 350 (S355, S360).

  As described above, three types of blur information Q are transmitted to the blur correction unit 350. The first is composed of the blur degree N, the blur width L, the blur direction, and the blur degree K in the image D0 obtained by the first analysis unit 322 using the entire image D0 in which no face is detected. The second is blur information, and the second is information indicating that the image D0 is a defocused image obtained by the second analysis unit 324 using the pupil image D5 of the image D0 from which the face or pupil is detected. The third is blur information including the width of the blur, and the third is the blur direction h of the image D0 obtained by the second analysis unit 324 using the pupil image D5 of the image D0 and the entire image D0. The blur information Q1 includes the blur width in the blur direction h obtained by the third analysis unit 326 and information indicating that the image D0 is a blur image.

  FIG. 33 is a block diagram showing a configuration of the blur correction unit 350. As illustrated, the blur correction unit 350 includes a parameter setting unit 352 that sets a correction parameter E based on blur information from the blur analysis unit 300, and a storage unit 354 that stores various databases for the parameter setting unit 352. A high frequency component extraction unit 356 that extracts a high frequency component from the image D0, and a blur execution unit 360 that corrects the blur of the image D0 by emphasizing the high frequency component Dh using the parameter E and adding it to the image D0. It has.

  When the parameter setting unit 352 receives the first blur information Q, it is included in the blur information Q in the same manner as the parameter setting unit 235 of the blur correction unit 230 in the image processing system A of the embodiment shown in FIG. In accordance with the blur width L and the blur direction, a one-dimensional correction mask M1 for directivity correction acting in the blur direction is set so that the larger the blur width L, the larger the correction mask size. Accordingly, the two-dimensional correction mask M2 for isotropic correction is set so that the size of the correction mask increases as the blur width L increases. The two-dimensional correction mask corresponding to each blur width and the one-dimensional correction mask corresponding to each blur width and blur direction are stored in the storage unit 354 as a database (referred to as a mask database), and the parameter setting unit 352 includes: From the mask database stored in the storage unit 354, the one-dimensional correction mask M1 is acquired based on the blur width L and the blur direction, and the two-dimensional correction mask M2 is acquired based on the blur width L.

  Next, the parameter setting unit 352 sets the one-dimensional correction parameter W1 for directionality correction and the two-dimensional correction parameter W2 for isotropic correction according to the following equation (3).

W1 = N × K × M1
W2 = N * (1-K) * M2 (3)
However, W1: One-dimensional correction parameter
W2: Two-dimensional correction parameter
N: Defocus degree
K: Degree of blur
M1: One-dimensional correction mask
M2: Two-dimensional correction mask

That is, the parameter setting means 352 has the correction parameters W1 and W2 such that the greater the degree of blur N, the stronger the isotropic correction strength and the directionality correction strength, and the greater the blur degree K, the greater the weight of the directionality correction. (Also referred to as parameter E) is set.

  On the other hand, when the parameter setting unit 352 receives the second blur information Q, the parameter setting unit 352 stores a two-dimensional correction mask M2 for correcting the blurring that is isotropic according to the blur width included in the blur information Q. It is read from the means 354 and set as the correction parameter E of the out-of-focus image D0.

  Further, when the parameter setting means 352 receives the third blur information Q1, the one-dimensional correction mask for correcting the blur in the direction corresponding to the blur width and the blur direction h included in the blur information Q. M1 is read from the storage unit 354 and used as the correction parameter E of the blurred image D0.

  Similar to the correction execution unit 250 of the blur correction unit 230 in the image processing system A of the embodiment shown in FIG. 1, the correction execution unit 360 emphasizes the high frequency component Dh using the parameter E, thereby correcting the image D0. The blur correction is executed. Specifically, the blur correction is performed according to the following equation (4).

D ′ = D0 + E × Dh (4)
Where D ': corrected image
D0: Image before correction
E: Correction parameter
Dh: High frequency component

The corrected image D ′ obtained by the blur correction unit 350 and the image D0 that is a normal image are output by being printed out by the output unit 270.

1 is a block diagram showing a configuration of an image processing system A according to a first embodiment of the present invention. 1 is a block diagram showing the configuration of the pupil detection means 100 in the image processing system A shown in FIG. The block diagram which shows the structure of the detection means 1 of the pupil detection means 100 Illustration showing eye position (A) is a diagram showing a horizontal edge detection filter, (b) is a diagram showing a vertical edge detection filter Diagram for explaining calculation of gradient vector (A) is a figure which shows a person's face, (b) is a figure which shows the gradient vector of eyes and mouth vicinity of the person's face shown to (a). (A) is a diagram showing a histogram of the magnitude of a gradient vector before normalization, (b) is a diagram showing a histogram of the magnitude of a gradient vector after normalization, and (c) is a magnitude of a gradient vector obtained by quinarization. The figure which shows the histogram of the length, (d) is a figure which shows the histogram of the magnitude | size of the quinary gradient vector after normalization The figure which shows the example of the sample image known to be the face used for learning of reference data The figure which shows the example of the sample image known to be the face used for learning of reference data Illustration for explaining face rotation Flow chart showing learning method of reference data Diagram showing how to derive a classifier The figure for demonstrating the stepwise deformation | transformation of an identification object image The flowchart which shows the process of the detection means 1 shown in FIG. The figure for demonstrating the pupil detection means 100 Luminance histogram Example of table of weighting coefficients used in voting unit 30 in pupil detecting means 100 The flowchart which shows the process of the pupil detection means 100 1 is a block diagram showing a configuration of a blur analysis unit 200 in the image processing system A shown in FIG. The figure which shows the example of the direction used when detecting an edge Diagram showing edge profile Diagram showing edge width histogram The figure for demonstrating operation | movement of the analysis execution means 220 Diagram for explaining the calculation of the degree of blur Diagram for explaining the calculation of blurring degree 20 is a flowchart showing processing of the blur analysis unit 200 shown in FIG. The block diagram which shows the structure of the blur correction | amendment means 230 1 is a flowchart showing processing of the image processing system A shown in FIG. The block diagram which shows the structure of the image processing system B used as the 2nd Embodiment of this invention. The block diagram which shows the structure of the blur analysis means 300 in the image processing system B The flowchart which shows the process of the blur analysis means 300 The block diagram which shows the structure of the blur correction | amendment means 350 in the image processing system B

Explanation of symbols

100 pupil detection means 200,300 blur analysis means 230,350 blur correction means 270 output means C0 feature quantity D0 digital photograph image D ′ corrected image H0 reference data E correction parameter K blurring degree L blur width M1 One-dimensional correction mask M2 Two-dimensional correction mask N Degree of blur Q, Q1 Blur information S Edge feature

Claims (18)

In an image processing method for obtaining blur information indicating a blur mode in a digital photographic image,
From the digital photographic image, a point-like portion is detected,
An image processing method characterized in that the blur information of the digital photographic image is obtained using image data of the dot-like portion. The digital photographic image is a photographic image of a person;
The image processing method according to claim 1, wherein the dotted portion is a pupil of the person. The digital photographic image is a photographic image of a person;
The image processing method according to claim 1, wherein the dotted portion is a face outline portion of the person. The blur information includes blur direction information indicating whether the blur is non-directional out-of-focus blur or directional blur, and the direction of blur in the case of blur,
The blur direction information is obtained using image data of the dot-like part,
The blur information excluding the blur direction information is obtained using data of the entire digital photographic image based on the blur direction information indicating blurring. Image processing method. Detecting an edge for each of a plurality of different directions with respect to the image of the dotted portion;
Obtaining feature values of the edge in each of the directions;
The image processing method according to claim 4, wherein the blur direction information is acquired based on the feature amount in each direction. Detecting an edge for each of a plurality of different directions with respect to the image of the dotted portion;
Obtaining feature values of the edge in each of the directions;
The image processing method according to claim 4, wherein the blur direction information is acquired based on the feature amount in each direction.   The image processing method according to claim 1, wherein after obtaining the blur information, the digital photographic image is corrected so as to eliminate the blur. In an image processing apparatus for obtaining blur information indicating a blur mode in a digital photographic image,
From the digital photographic image, point-like part detection means for detecting a point-like part,
An image processing apparatus comprising: analysis means for obtaining the blur information of the digital photographic image using data of the image of the dot-like portion. The digital photographic image is a photographic image of a person;
The image processing apparatus according to claim 8, wherein the point-like portion detection unit detects a pupil or a face outline of the person as the point-like portion. The blur information includes blur direction information indicating whether the blur is non-directional out-of-focus blur or directional blur, and the direction of blur in the case of blur,
The analysis means acquires the blur direction information using the image data of the dot-like portion, and uses the data of the entire digital photographic image based on the blur direction information indicating blurring. The image processing apparatus according to claim 8, wherein the blur information is obtained by removing direction information. The analysis means detects an edge for each of a plurality of different directions with respect to the image of the dotted portion,
Obtaining feature values of the edge in each of the directions;
The image processing apparatus according to claim 10, wherein the blur direction information is acquired based on the feature amount in each direction.   The image processing apparatus according to claim 8, further comprising a correcting unit that corrects the digital image after obtaining the blur information by the analyzing unit.   The image processing apparatus according to claim 12, wherein the correction unit increases the degree of correction as the point-like portion increases. A program for causing a computer to execute processing for obtaining blur information indicating a blur mode in a digital photographic image,
The processing is a punctiform part detection process for detecting a punctiform part from the digital photographic image;
A program comprising: analysis processing for obtaining the blur information of the digital photographic image using data of the image of the dot-like portion. The digital photographic image is a photographic image of a person;
15. The program according to claim 14, wherein the point-like portion detection processing is processing for detecting the human pupil as the point-like portion. The blur information includes blur direction information indicating whether the blur is non-directional out-of-focus blur or directional blur, and the direction of blur in the case of blur,
The analysis processing acquires the blur direction information using data of the image of the dotted portion, and uses the data of the entire digital photographic image based on the blur direction information indicating blurring. 16. The program according to claim 9 or 15, which is a process for obtaining the blur information excluding direction information. The analysis process detects an edge for each of a plurality of different directions with respect to the image of the dotted portion,
Obtaining feature values of the edge in each of the directions;
The program according to claim 16, wherein the blur direction information is obtained based on the feature amount in each direction. The analysis process detects an edge for each of a plurality of different directions with respect to the image of the dotted portion,
Obtaining feature values of the edge in each of the directions;
The program according to claim 15, wherein the blur direction information is obtained based on the feature amount in each direction.

Images