JP2010244579A - Image processing method, apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently apply blurring correction with respect to an image having shallow depth of field. <P>SOLUTION: A face extraction means 100 extracts a facial image d0 from a reduced image d acquired by reducing an image D. An analysis means 20 calculates a blurring direction and a blurring level of the image D, by using the face image d0 and an image of a part corresponding to a part of the face image d0 in the image D, and distinguishes whether the image D is a defocused image or a normal image. With respect to the image D being distinguished as being a blurred image, the level of blurring and the width of blurring are further calculated. A parameter-setting means 30 sets a one-dimensional correction mask and a two-dimensional correction mask, according to the blurring width and furthermore sets correction intensity, according to the level of defocusing and adjusts the ratio between the one-dimensional correction mask and the two-dimensional correction mask, according to the level of blurring. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing method and apparatus for correcting blur on a digital photographic image, and a program therefor.

  Digital photographic images obtained by photoelectrically reading photographic images recorded on photographic films such as negative films and color reversal films with a reading device such as a scanner, and digital photographic images obtained by taking images with a digital still camera (DSC) For example, printing with various image processing is performed. As one of these image processes, there is a blurred image restoration process that removes a blur from a blurred image (blurred image).

  Reasons for blurring a photographic image obtained by capturing an image of a subject include out-of-focus due to the focal length being out of focus, and out-of-focus blur due to camera shake (hereinafter referred to as blurring). . In the case of out-of-focus, the point image spreads two-dimensionally, that is, the spread on the photographic image exhibits non-directionality, whereas in the case of blur, a locus with a point image is drawn on the image one-dimensionally. Spreads, that is, has a direction with a spread on a photographic image.

  In the field of digital photographic images, various methods have been conventionally proposed for restoring blurred images. If you know information such as blur direction and blur width when taking a photographic image, you can restore it by applying a restoration filter such as a Wiener filter or inverse filter to the photographic image. There is widely known a method of providing an apparatus (for example, an acceleration sensor) that can be acquired in an imaging apparatus, acquiring information such as a blur direction and a blur width together with imaging, and performing repair based on the acquired information (for example, , See Patent Document 1).

  Also, a degradation function is set for the blurred image (image with blur), the blurred image is repaired with a restoration filter corresponding to the set degradation function, the restored image is evaluated, and the evaluation result is evaluated. A method is also known in which the deterioration function is reset, and repair is performed by repeating the repair, evaluation, and resetting of the deterioration function until a desired image quality is obtained. This method has a problem that it takes processing time because it is necessary to repeat the process of setting, repairing, evaluating the deterioration function, resetting the deterioration function, and so on. In Patent Document 2, the user specifies a small area including an edge in a blurred image, and instead of the entire blurred image, the above-described degradation function is set, repaired, evaluated, and the specified small area. Repeat the process of resetting the degradation function to find the optimal degradation function, apply the restoration filter corresponding to this degradation function to the entire blurred image, and use the above-mentioned small area for the image used to obtain the degradation function A method has been proposed in which the amount of calculation is reduced and the efficiency is improved by using this image.

  On the other hand, with the rapid spread of mobile phones, the functions of mobile phones have improved, and among them, the improvement of the functions of digital cameras attached to mobile phones (hereinafter referred to as mobile cameras) has been attracting attention. In recent years, the number of pixels of a portable camera has increased to one million, and the portable camera is used in the same way as a normal digital camera. Of course, commemorative photos when traveling with friends, as well as scenes of picking up favorite talents and athletes with a portable camera, are becoming commonplace. In such a background, a photographic image obtained by capturing with a mobile camera is not limited to being viewed on a monitor of a mobile phone, and for example, is often printed in the same manner as a photographic image acquired with a normal digital camera. It has become.

  On the other hand, since the main body (mobile phone) is not manufactured exclusively for imaging, the portable camera has a problem of poor holdability during imaging. Moreover, since a portable camera does not have a flash, the shutter speed is slower than that of a normal digital camera. For these reasons, camera shake is more likely to occur when shooting a subject with a portable camera than with a normal camera. Extreme camera shake can be confirmed on the monitor of the portable camera, but small camera shake cannot be confirmed on the monitor, and often you will notice image blur for the first time after printing. There is a high need to perform blur correction on the obtained photographic image.

  In recent years, image recognition technology has advanced, and it has become possible to accurately recognize and extract main subjects such as faces from photographic images.

Japanese Patent Laid-Open No. 2002-112099 JP-A-7-121703

  However, downsizing of mobile phones is one of the focus of competition among mobile phone manufacturers, along with their performance and cost, and a camera attached to a mobile phone is provided with a device that acquires the direction and width of blur. Since it is not realistic, the method proposed in Patent Document 1 cannot be applied to a portable camera.

  Further, the method proposed in Patent Document 2 is troublesome because it requires a user to specify a small area. Furthermore, photographic images include images with a shallow depth of field that are captured with the main subject such as the face up. In the case of images with such a shallow depth of field, even if there is no blur, The other part is blurred. If the small area specified by the user is located in a portion other than the main subject, an appropriate deterioration function cannot be obtained, and the image quality of the corrected image is deteriorated.

  In view of the above circumstances, the present invention makes it possible to efficiently perform blur correction in a digital photographic image without requiring a special device to be provided in the imaging device, and for an image with a shallow depth of field. It is an object of the present invention to provide an image processing method and apparatus capable of performing appropriate correction, and a program therefor.

A first image processing method of the present invention is an image processing method for correcting blur in a digital photographic image.
Extracting the main subject area from the digital photographic image,
Set parameters for correcting the blur using the image of the main subject area,
The digital photographic image is corrected using the set parameter.

A second image processing method of the present invention is an image processing method for correcting blur in a digital photographic image.
Determining whether the digital photographic image is a shallow depth of field image;
For the digital photographic image determined to be an image with a shallow depth of field, a main subject area is extracted from the digital photographic image, and parameters for correcting the blur using the image of the main subject area are set. On the other hand, for the digital photographic image other than the image determined to be an image having a shallow depth of field, a parameter for correcting the blur using the entire digital photographic image is set,
The digital photographic image is corrected using the set parameter.

A third image processing method of the present invention is an image processing method for correcting blur in a digital photographic image.
Extracting the main subject area from the digital photographic image,
A parameter for correcting the blur using the image of the main subject area is set as a first parameter, and the blur is reduced using the entire digital photograph image or an image of an area other than the main subject area. Set the parameter for correction as the second parameter,
Determining the certainty that the digital photographic image is an image with a shallow depth of field;
Obtaining a parameter for correcting the blur by weighting and adding the first parameter and the second parameter so that the weight of the first parameter increases as the certainty factor increases.
The digital photographic image is corrected using the parameters.

  When the digital photographic image is attached with attached information including the F value of the lens of the imaging device that captured the digital photographic image, the image processing method of the present invention reads the attached information and acquires the F value. Then, based on the F value, it is possible to determine whether or not the digital photographic image is an image having a shallow depth of field or to obtain the certainty factor.

  Here, when determining whether the digital photograph image is an image with a shallow depth of field based on the F value, for example, the F value included in the attached information of the digital photograph image is a predetermined threshold value. (For example, 5.6) or less, it can be determined that the digital photographic image is an image having a shallow depth of field, and if the F value is larger than the predetermined threshold, the digital photographic image is not covered. It can be determined that the image is not a shallow depth of field.

  Further, when determining the certainty that the digital photo image is an image having a shallow depth of field based on the F value, for example, the maximum certainty factor and the minimum certainty factor are set to 100% and 0%, respectively. The certainty factor of a digital photographic image having an F value equal to or smaller than a threshold value (for example, 2.8) is 100%, and the certainty factor of a digital photographic image having an F value equal to or larger than a predetermined threshold value (for example, 11) greater than the predetermined threshold value 100% excluding 100% and 0% of the certainty of the digital photograph image of F value within the range between 0% and the two thresholds (for example, the range of 2.8 to 11 excluding 2.8 and 11) The value may be between 0% and 0% and increase as the F value increases.

  In the image processing method of the present invention, an edge is detected in each of a plurality of different directions with respect to an image of a main subject area extracted from the digital photographic image and an image of an area other than the main subject area, and each of the directions The blur feature acquisition processing is performed to acquire the blur information acquisition processing for acquiring the blur feature acquisition process for acquiring the blur feature information based on the edge feature amount, and acquiring the blur information acquisition processing based on the edge feature amount. A process of determining whether the digital photo image is an image with a shallow depth of field or determining the certainty factor by comparing the information and the blur information acquired from an image of an area other than the main subject area. It can be carried out.

  Since the blur causes the spread of the point image in the image, the spread of the edge corresponding to the spread of the point image occurs in the blurred image (hereinafter referred to as the blurred image). That is, the manner of edge spreading in the image is directly related to the blur in the image. The blur information acquisition process in the present invention focuses on this point, detects an edge for each of a plurality of different directions, and obtains the blur information based on the feature amount of the edge in each of the directions.

  Here, the “edge feature amount” is the spread of the edge in the target image (hereinafter referred to as “target image”, here, the image of the main subject region or the image of the region other than the main subject region in the digital photographic image). It means a feature quantity related to an aspect, and may include, for example, edge sharpness and the distribution of the edge sharpness.

  As the “edge sharpness”, any parameter can be used as long as it can express the sharpness of the edge. For example, in the case of the edge shown by the edge profile in FIG. The edge profile is used so that the sharpness of the edge is higher as the sharpness of the brightness change of the edge (the gradient of the profile curve in FIG. 5) is higher as well as the edge width is used as the sharpness of the edge so that the sharpness is low. The slope of the curve may be used as the edge sharpness.

  The “blurring information” means information that can represent a blur mode in the target image, and can be blur direction information and a blur width regarding the blur direction. “Bokeh” includes non-directional blur, that is, out-of-focus blur and directional blur, ie, blur. In the case of blur, the blur direction corresponds to the blur direction. In the case of blur, the “blurring direction” is “ "No direction". The “blurring width” means a blur width in the blur direction, and can be, for example, an average value of the edge widths of the edges in the blur direction. Further, in the case where the blur is non-directional out-of-focus, the edge width of the edge in any one direction may be the blur width, but the average value of the edge widths of the edges in all the directions included in the plurality of different directions It is preferable that

  Furthermore, the target image in the present invention is not limited to a blurred image, but is a normal image with no blur or blur (the image of the main subject area in a digital photo image with a shallow depth of field without blur or blur is a normal image, and the main subject An image of an area other than the area is a blurred image), and such a normal image has no blur, that is, blur information including blur direction information of “no direction” and a blur width of “below a predetermined threshold”. I will have it.

  Further, the “plurality of different directions” means directions for specifying the direction of the blur in the target image, and it is necessary to include directions close to the direction of the blur. Although the specific accuracy is high, it is preferable to use an appropriate number according to the balance with the processing speed, for example, eight directions as shown in FIG.

  In the image processing method of the present invention, whether the digital photographic image is an image with a shallow depth of field by comparing the blur information of the image of the main subject region with the blur information of the image of the region other than the main subject region. For example, the blur information of the image of the main subject area is “no direction” and “the blur width equal to or less than a predetermined first threshold”, and the blur information of the area other than the main subject area is “ If it is “no direction” and “a blur width equal to or greater than a predetermined first threshold value”, it is possible to determine that this digital photograph image is an image with a shallow depth of field. In this case, the digital photographic image can be a photographic image with no blur or blur. However, in the case of a digital photographic image with a small depth of field that is out of focus, the blur information of the image of the main subject region is “None”. Direction "and" blur width greater than or equal to a predetermined first threshold value ", and blur information of an image in a region other than the main subject region is" no direction "and" a blur width larger than the blur width in the image of the main subject region ". . In the case of digital photographic images with blurring and shallow depth of field, the blur direction information of the image of the main subject area and the image of the area other than the main subject area will be “directional”, but there will still be a difference in the blur width . Conversely, in the case of a digital photographic image that is not an image with a shallow depth of field, the blur information of the image in the main subject area and the blur information of the image in the area other than the main subject area should not be so different. The image processing method of the present invention pays attention to such a tendency, and compares the blur information of the image of the main subject area in the digital photographic image with the blur information of the image other than the main subject area, thereby comparing the digital photographic image. Can be discriminated whether or not the image has a shallow depth of field.

  In the image processing method of the present invention, the digital photographic image has a depth of field by comparing the blur information of the image of the main subject area in the digital photographic image with the blur information of the image of the area other than the main subject area. The calculation of the certainty level of a shallow image is not just a determination of whether the image has a shallow depth of field due to the difference in blur information between the two regions, but the greater the difference in the digital photographic image The certainty factor may be calculated so that the certainty factor is a shallow image.

Furthermore, the image processing method of the present invention divides the digital photographic image into a plurality of blocks,
A blur information acquisition process for detecting an edge in each of a plurality of different directions for each image of the block, acquiring a feature amount of the edge in each of the directions, and acquiring blur information based on the edge feature amount. Apply each to get the blur information of each,
By comparing the blur information acquired from the images of the respective blocks, it is possible to determine whether or not the digital photo image is an image with a shallow depth of field or to obtain the certainty factor.

  If the image has a shallow depth of field, or if the certainty that the image is a shallow depth of field is high, each block has a difference in blur information between the groups that is equal to or greater than a predetermined threshold value. It can be divided into two or more groups having features with little difference in blur information of each block. These groups can be considered to correspond to the image of the main subject region and the image other than the main subject region, and the image processing method of the present invention compares the blur information of the images of the respective blocks by paying attention to this point. The determination or the certainty factor may be calculated as follows.

  In the image processing method of the present invention, as a method of setting a parameter for correcting the blur, an image used for setting the parameter (in the first image processing method, an image of a main subject region, In the second image processing method, the image of the main subject area and the entire digital photographic image are used, and in the third image processing method, the image of the main subject area and the entire digital photographic image or an image of an area other than the main subject area are used. Any method can be used as long as the parameter can be set (for example, the method described in Patent Document 2). However, the image processing method of the present invention is different from the image used for setting the parameter. An edge is detected for each direction, the feature amount of the edge in each direction is acquired, and blur information is obtained based on the edge feature amount. Subjected to blur information acquisition process Tokusuru, it is preferable to set the parameters on the basis of the blur information.

  As described above, “blur” includes non-directional blur, that is, out-of-focus blur, and directional blur, that is, blur. In the case of blur, the blur direction corresponds to the blur direction. “Bokeh direction” is “no direction”. In the image processing method of the present invention, the blur direction information includes information indicating whether the blur is non-directional defocus (blur is non-directional) or directional blur (blur is directional), and blur When acquiring information indicating the direction of blur and setting the parameters, an isotropic correction parameter for isotropic correction in the case of defocusing, and a directionality correction for directionality correction in the case of blur A parameter can be set.

  Here, “isotropic correction” means correction that works equally in each direction, and “directional correction” means correction that works only in a predetermined direction or correction that works differently depending on the direction. means. On the other hand, since blur appears in the image as the spread of the edge, a conventionally known correction method for improving the sharpness of the image, for example, USM (Unsharpness Masking) is used as a method for correcting the blur. Can be used. When setting the correction parameters, the parameters are set according to the required correction method. For example, when USM is used as a correction method, an isotropic correction mask is set as an isotropic correction parameter, and a directionality correction mask that operates in the blur direction is set as a directionality correction parameter. That's fine. Furthermore, the size of the correction mask may be set according to the blur width included in the blur information.

  In addition, blur may not be clearly divided into out-of-focus and blur due to the blur information, and there may be both out-of-focus and blur in a blur image. In order to prepare for such a case, the image processing method of the present invention does not determine whether the blur is out of focus or blur, and also acquires the blur degree indicating the degree of blur blur in the blur direction as the blur information. It is preferable to set the parameter by weighting and summing the isotropic correction parameter for isotropic correction and the directionality correction parameter for directionality correction so that the weight of the directionality correction becomes higher as the value becomes higher. .

  Further, the image processing method of the present invention may acquire a degree of blur indicating the degree of blur as the blur information, and set the parameter so that the correction intensity increases as the blur degree increases. Good.

  Here, an image whose degree of blur is smaller than a predetermined threshold value can be set as a normal image without blur. Therefore, such an image may be corrected by setting a parameter that “does not correct”. (As a result, the corrected image and the original image are the same), in order to increase the efficiency of processing, the parameter setting and only for the digital photographic image whose degree of blur is equal to or greater than a predetermined threshold It is preferable that the correction is performed and the parameter is not set and of course not corrected for an image having a degree of blur smaller than the threshold.

  In addition, the reduced image obtained by reducing the digital photograph image is easier to detect the edge than the original image, and the amount of calculation required for edge detection is small. Therefore, in the present invention, the target image is reduced. It is preferable to detect the edge from the obtained reduced image.

  Furthermore, an edge having low strength is highly likely to be noise, and humans recognize blur by the state of the edge having high strength. Therefore, in the present invention, an edge having a strength equal to or higher than a predetermined threshold with respect to the target image. It is preferable to detect only.

  Furthermore, it is preferable to remove invalid edges from the edges detected from the target image. Note that `` invalid edge '' means an edge that is not related to acquisition of blur information, an edge that makes it difficult to acquire blur information, or that may cause incorrect blur information to be acquired. Can include complex edges and edges containing light sources.

A first image processing apparatus of the present invention is an image processing apparatus for correcting blur in a digital photographic image,
Main subject extraction means for extracting a main subject region from the digital photographic image;
Parameter setting means for setting parameters for correcting the blur using the image of the main subject area;
And correction means for correcting the digital photographic image using the parameter set by the parameter setting means.

A second image processing apparatus of the present invention is an image processing apparatus for correcting blur in a digital photographic image,
Main subject extraction means for extracting a main subject region from the digital photographic image;
Discriminating means for discriminating whether the digital photographic image is an image having a shallow depth of field;
The blur is corrected using the image of the main subject area of the digital photographic image extracted by the main subject extracting unit with respect to the digital photographic image determined as an image having a shallow depth of field by the determining unit. On the other hand, for the digital photographic image other than the image determined to be an image with a shallow depth of field, a parameter for correcting the blur using the entire digital photographic image is set. Parameter setting means;
And correction means for correcting the digital photographic image using the parameter set by the parameter setting means.

A third image processing apparatus of the present invention is an image processing apparatus for correcting blur in a digital photographic image,
Main subject extraction means for extracting a main subject region from the digital photographic image;
A certainty factor calculating means for determining a certainty factor that the digital photographic image is an image having a shallow depth of field;
First parameter setting means for setting a parameter for correcting the blur using the image of the main subject area extracted by the main subject extracting means;
Second parameter setting means for setting a parameter for correcting the blur using the entire digital photographic image or an image of an area other than the main subject area;
The parameter set by the first parameter and the second parameter are set such that the higher the certainty factor obtained by the certainty factor calculating unit, the greater the weight of the parameter set by the first parameter setting unit. Integrated parameter acquisition means for acquiring an integrated parameter by adding together the parameters set by the parameter setting means;
And a correction unit configured to correct the digital photographic image using the integrated parameter acquired by the integrated parameter acquisition unit.

The image processing apparatus of the present invention reads the attached information of the digital photographic image to which the attached information including the attached F information including the F value of the lens of the imaging device that picked up the digital photograph image is acquired, and acquires the F value. With means,
The determination unit or the certainty factor calculation unit may perform determination of whether or not the digital photographic image is an image having a shallow depth of field based on the F value, or processing for obtaining the certainty factor.

The image processing apparatus of the present invention detects edges in a plurality of different directions with respect to the target image, acquires feature amounts of the edges in the respective directions, and acquires blur information based on the edge feature amounts. Bokeh information acquisition means,
The determination unit or the certainty factor calculation unit compares the blur information acquired by the blur information acquisition unit using the image of the main subject region and the image of the region other than the main subject region, respectively, as the target image. By doing so, it is possible to determine whether or not the digital photographic image is an image with a shallow depth of field or to obtain the certainty factor.

The image processing apparatus of the present invention comprises a block dividing means for dividing the digital photographic image into a plurality of blocks,
A blur information acquisition unit that detects an edge for each of a plurality of different directions with respect to the target image, acquires a feature amount of the edge in each of the directions, and acquires blur information based on the edge feature amount;
The determination unit or the certainty factor calculation unit compares the blur information acquired by using the image of each block as the target image by the blur information acquisition unit, so that the digital photographic image is captured. It is possible to determine whether or not the image has a shallow depth of field or to obtain the certainty factor.

  The parameter setting means in the image processing apparatus of the present invention detects an edge for each of a plurality of different directions in the image with respect to the image used for setting the parameter for correcting the blur, and each of the directions It is preferable that the image processing apparatus includes a blur information acquisition unit that acquires the feature amount of the edge in the image, acquires blur information based on the edge feature amount, and sets the parameter based on the acquired blur information.

  In addition, there may be two or more main subjects in a digital photographic image. For example, in one image showing a plurality of humans, there are face portions for the number of persons as long as the faces do not overlap. In the case of such a digital photographic image, a plurality of main subjects are extracted. Parameter setting means for setting parameters using the image of the main subject area (parameter setting means of the first image processing apparatus, parameter setting means of the second image processing apparatus, and first parameter setting of the third image processing apparatus) The means) may set the parameters (first parameter in the case of the third image processing apparatus) using the extracted images of all the main subject areas, but the image processing apparatus of the present invention It is desirable to select one main subject from the plurality of extracted main subjects and set parameters using the selected main subject. That is, the main subject extraction unit in the image processing apparatus of the present invention includes a main subject candidate extraction unit that extracts a main subject region candidate from the digital photographic image, and a plurality of the main subject candidates extracted by the main subject candidate extraction unit. It is preferable that the main subject selection unit selects a region of the main subject from the subject candidates.

  Here, the main subject selection means, specifically, the main subject candidate that seems to be the most important in setting correction parameters such as analysis of blur information, or the main subject candidate that the photographer thinks is most important May be selected as the main subject. For example, the main subject candidate having the largest size may be selected as the main subject from the plurality of main subject candidates, or the main subject candidate closest to the central portion of the digital photo image may be selected as the main subject. It may be.

  Further, for example, a main subject candidate instructed by a user among a plurality of main subject candidates may be selected as a main subject.

  Also, the sizes of images used for the parameter setting means (digital photographic images themselves, images of main subject regions, images of regions other than the main subject regions) vary. Setting parameters using these images as they are takes time and is not efficient, so it is desirable to use them after performing reduction processing. However, if the image is reduced at the same reduction ratio for a large image and a small image, for example, in the process of extracting edges to set parameters, the number of edges to be extracted is large, so parameters are set later. It takes a long time for analysis, etc., and is not efficient. On the other hand, in order to shorten the processing time, if a small size image is reduced with an appropriate reduction ratio with respect to a large size image, the image has been reduced but is too small to be able to be analyzed properly. There is a problem that cannot be set. The image processing apparatus of the present invention includes a reduction unit that performs a reduction process on an image used by the parameter setting unit to obtain a reduced image, and the parameter setting unit sets the parameter using the reduced image. In accordance with the size of the image of the main subject area used by the parameter setting unit, the reduction means increases the reduction strength for the image as the size increases (conversely, the smaller the size, the more It is preferable that the reduction processing is performed so that the reduction strength of the image is weakened (including that the reduction strength is zero, that is, the image is not reduced).

  Also, here, focusing on the fact that the size of the image of the main subject region extracted from the digital photographic image varies greatly, the reduction means adjusts the size of the image of the main subject region when performing reduction processing. Although the reduction intensity is adjusted accordingly, the reduction is performed, but the same adjustment is performed on the other images used in the parameter setting means (the entire digital photographic image, the image of the region other than the main subject region) to reduce the image. Of course, you may do.

  The size of the image can be, for example, the number of pixels of image data representing the image.

A first program of the present invention is a program for causing a computer to execute image processing for correcting blur in a digital photographic image,
The image processing is a main subject extraction process for extracting a main subject region from the digital photographic image;
A parameter setting process for setting parameters for correcting the blur using the image of the main subject area;
And correction processing for correcting the digital photographic image using the parameters set by the parameter setting processing.

A second program of the present invention is a program for causing a computer to execute image processing for correcting blur in a digital photographic image,
The image processing is a main subject extraction process for extracting a main subject region from the digital photographic image;
A determination process for determining whether the digital photographic image is an image with a shallow depth of field;
The blur is corrected by using the image of the main subject area of the digital photographic image extracted by the main subject extraction process for the digital photographic image determined as an image having a shallow depth of field by the discrimination process. On the other hand, for the digital photographic image other than the image determined to be an image with a shallow depth of field, a parameter for correcting the blur using the entire digital photographic image is set. Parameter setting process,
And correction processing for correcting the digital photographic image using the parameters set by the parameter setting processing.

A third program of the present invention is a program for causing a computer to execute image processing for correcting blur in a digital photographic image,
A main subject extraction process for extracting a main subject region from the digital photographic image;
A certainty factor calculation process for determining the certainty factor that the digital photo image is an image having a shallow depth of field;
A first parameter setting process for setting a parameter for correcting the blur using an image of the main subject area extracted by the main subject extraction process;
A second parameter setting process for setting a parameter for correcting the blur using the entire digital photograph image or an image of an area other than the main subject area;
The parameter set by the first parameter and the second parameter are set such that the higher the certainty factor obtained by the certainty factor calculation process, the greater the weight of the parameter set by the first parameter setting process. An integrated parameter acquisition process for acquiring an integrated parameter by adding together the parameters set by the parameter setting process;
And a correction process for correcting the digital photographic image using the integrated parameter acquired by the integrated parameter acquisition process.

  The image processing includes attached information reading processing for obtaining the F value by reading the attached information of the digital photographic image to which attached information including an F value of a lens of an imaging apparatus that has captured the digital photograph image is attached. Then, the determination process or the certainty factor calculation process may determine whether the digital photograph image is an image with a shallow depth of field or obtain the certainty factor based on the F value.

  The image processing detects an edge for each of a plurality of different directions with respect to the target image, acquires a feature amount of the edge in each of the directions, and acquires blur information based on the edge feature amount And each of the determination processing or the certainty factor calculation processing acquired by the blur information acquisition processing using the image of the main subject region and the image of the region other than the main subject region as the target images, respectively. By comparing the blur information, it may be determined whether or not the digital photographic image is an image with a shallow depth of field or the process of obtaining the certainty factor.

  The image processing includes a block dividing process for dividing the digital photographic image into a plurality of blocks, an edge is detected for each of a plurality of different directions with respect to the target image, and a feature amount of the edge in each of the directions is acquired, A blur information acquisition process for acquiring blur information based on the edge feature amount, and the determination process or the certainty factor calculation process sets the image of each block as the target image by the blur information acquisition process. By comparing each of the acquired blur information, it may be determined whether or not the digital photographic image is an image with a shallow depth of field or the process of obtaining the certainty factor.

  The parameter setting process detects an edge for each of a plurality of different directions in the image with respect to an image used for setting a parameter for correcting the blur, and calculates a feature amount of the edge in each of the directions. It is preferable to have a blur information acquisition process that acquires and acquires blur information based on the edge feature amount, and sets the parameter based on the acquired blur information.

  According to the first image processing method, apparatus, and program of the present invention, a digital photographic image is extracted by extracting a main subject region from a digital photographic image and setting parameters for correcting blur using the image of the main photographic subject region. Therefore, it is not necessary for the user to specify an area for setting parameters. In particular, in the case of a digital photographic image with a shallow depth of field, the correction accuracy can be improved by setting the correction parameters using only the image of the main subject area that requires the most correction. In addition, by setting correction parameters using the image of the main subject area, it is not necessary to provide a device for acquiring information regarding blurring during imaging, and thus it is possible to avoid increasing the size of the imaging device. In particular, in the case of a digital camera attached to a mobile phone, which requires miniaturization as a prerequisite, there are significant advantages.

  According to the second image processing method, apparatus, and program of the present invention, it is determined whether or not the digital photographic image is an image having a shallow depth of field, and the main subject region is detected with respect to the image having a shallow depth of field. Since correction is performed by setting a correction parameter using an image, the same effect as the first image processing method, apparatus and program of the present invention can be obtained, and the depth of field is shallow. For digital photographic images that are not images, the entire digital photographic image is used for correction by setting correction parameters, so the correction accuracy for digital photographic images that are not images with a shallow depth of field is correct. Can be increased.

  According to the third image processing method, apparatus and program of the present invention, the certainty factor that the digital photographic image is an image with a shallow depth of field is calculated, and the higher the certainty factor, the more the image of the main subject region is used. The first parameter and the parameter (second parameter) set using the entire digital photographic image or an image of an area other than the main subject area so that the weight of the parameter (first parameter) set in the Since the digital photographic image is corrected using the parameters obtained by weighting and summing the values, it is possible to obtain the same effect as the second image processing method, apparatus and program, and to determine the depth of field. Even if there is a slight deviation, there can be obtained a good correction effect.

  The image processing method, apparatus, and program according to the present invention acquire blur information based on edge feature values detected in different directions for a target image, set parameters based on the blur information, and perform correction. If this is done, the amount of calculation is smaller than in the method of repeating the correction and evaluation processing without blur information, so the processing is fast and efficient.

1 is a block diagram showing a configuration of an image processing system A according to a first embodiment of the present invention. 1 is a block diagram showing the configuration of edge detection means 12 in the image processing system A shown in FIG. FIG. 2 is a block diagram showing the configuration of the second reduction means 13 in the edge detection means 12 shown in FIG. Diagram showing the direction used when detecting edges Diagram showing edge profile Diagram showing edge width histogram The figure for demonstrating operation | movement of the analysis means 20 Diagram for explaining the calculation of the degree of blur Diagram for explaining the calculation of blurring degree The block diagram which shows the structure of the face extraction means 100 (A) is a diagram showing a horizontal edge detection filter, (b) is a diagram showing a vertical edge detection filter Diagram for explaining calculation of gradient vector (A) is a figure which shows a person's face, (b) is a figure which shows the gradient vector of eyes and mouth vicinity of the person's face shown to (a). (A) is a diagram showing a histogram of the magnitude of a gradient vector before normalization, (b) is a diagram showing a histogram of the magnitude of a gradient vector after normalization, and (c) is a magnitude of a gradient vector obtained by quinarization. The figure which shows the histogram of the length, (d) is a figure which shows the histogram of the magnitude | size of the quinary gradient vector after normalization The figure which shows the example of the sample image known to be the face used for learning of reference data Illustration for explaining face rotation Flow chart showing learning method of reference data Diagram showing how to derive a classifier The figure for demonstrating the stepwise deformation | transformation of an identification object image The flowchart which shows operation | movement of the image processing system A of embodiment shown in FIG. The block diagram which shows the structure of the image processing system B used as the 2nd Embodiment of this invention. 21 is a flowchart showing the operation of the image processing system B shown in FIG. 21 is a flowchart showing the operation of the image processing system B shown in FIG. The block diagram which shows the structure of the image processing system C used as the 3rd Embodiment of this invention. Flowchart showing the operation of the image processing system C shown in FIG. Flowchart showing the operation of the image processing system C shown in FIG. The block diagram which shows the structure of the image processing system D used as the 4th Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing system A which is a first embodiment of an image processing method and apparatus and a program therefor according to the present invention. An image processing system A shown in FIG. 1 performs image processing by reading an image from a recording medium 1 on which a photographic image (hereinafter simply referred to as an image) acquired by a digital camera attached to a mobile phone (hereinafter referred to as a portable camera) is recorded. However, an image group consisting of a large number of images is recorded on the recording medium 1. Here, description will be given by taking one image D of the image group as an example.

  As shown in the figure, the image processing system A according to the present embodiment performs reduction processing on an image D read from the recording medium 1 at a reduction ratio of, for example, 1/8 to obtain a reduced image d of the image D. The main object (here, a face as an example) is detected from the means 10a and the reduced image d. If a face is detected, an image of the face portion (here, a reduced image of the face portion image, In the following description, the face image and the reduced image of the face image are distinguished by a code, and both are referred to as face images) and extracted d0 and output to the edge detection means 12 described later. On the other hand, if no face is detected, the reduced image Using the face extraction unit 100 that outputs d to the edge detection unit 12, and the face image d0 and the image D from the face extraction unit 100, the portion of the image D0 corresponding to the face image d0, or the reduced image d and the image D, FIG. The edge detection means 12 for detecting edges in each of the eight different directions shown, the edge narrowing means 14 for removing invalid edges among the edges detected by the edge detection means 12, and the edge narrowing means 14 The edge feature amount acquisition means 16 for acquiring the feature value S of the edge and the edge feature amount S are used to calculate the blur direction and the blur degree N to determine whether the image D is a blur image or a normal image. In the case of an image, information P indicating that the image D is a normal image is transmitted to the output means 60 to be described later, and the processing is terminated. On the other hand, in the case of a blurred image, the blur degree K and the blur width L are further set. The calculation unit 20 calculates and transmits to the parameter setting unit 30 (to be described later) as the blur information Q together with the blur degree N and the blur direction, and the image D that becomes a blur image is compensated based on the blur information Q from the analysis unit 20. The parameter D is corrected using the parameter setting means 30 for setting a plurality of parameters E (E0, E1, E2,...) And the parameters E0, E1, E2,. From the correction means 40 for obtaining corrected images D′ 0, D′ 1, D′ 2,... Corresponding to the parameters, and the corrected images D′ 0, D′ 1, D′ 2,. A determination unit 45 that determines the target image D ′ and outputs it to the output unit 60, a storage unit 50 that stores various databases for the analysis unit 20 and the parameter setting unit 30, and an image D from the analysis unit 20 is a normal image. When receiving the information P indicating that, the output unit 60 outputs the image D, and outputs the target image D ′ when receiving the target image D ′ from the determining unit 45. .

  Hereinafter, each configuration of the image processing system A illustrated in FIG. 1 will be described.

  First, the face extraction unit 100 will be described.

  FIG. 10 is a block diagram showing a detailed configuration of the face extraction unit 100. As illustrated, the face extraction unit 100 includes a feature amount calculation unit 110 that calculates a feature amount C0 from an image d, a database 115 that includes reference data H0 described later, and a feature amount that is calculated by the feature amount calculation unit 110. Based on C0 and the reference data H0 in the database 115, it is identified whether or not a face of a person is included in the image D. If a face is included, the face image d0 is cut out to the edge detection means 12. On the other hand, if the face is not included, the face detecting means 120 for outputting the reduced image d to the edge detecting means is provided.

  The feature amount calculation unit 110 of the face extraction unit 100 calculates a feature amount C0 used for face identification from the reduced image d. Specifically, the gradient vector (that is, the direction in which the density of each pixel on the reduced image d changes and the magnitude of the change) is calculated as the feature amount C0. Hereinafter, calculation of the gradient vector will be described. First, the feature quantity calculating unit 110 performs filtering processing by the horizontal edge detection filter shown in FIG. 11A on the reduced image d to detect horizontal edges in the reduced image d. Further, the feature amount calculating unit 110 performs a filtering process on the reduced image d by a vertical edge detection filter shown in FIG. 11B to detect vertical edges in the reduced image d. Then, a gradient vector K for each pixel is calculated from the horizontal edge size H and the vertical edge size V of each pixel on the reduced image d, as shown in FIG.

  It should be noted that the gradient vector K calculated in this way is an eye in a dark part such as the eyes and mouth as shown in FIG. 13B in the case of a human face as shown in FIG. It faces the center of the mouth and faces outward from the position of the nose in a bright part like the nose. Further, since the change in density is larger in the eyes than in the mouth, the gradient vector K is larger in the eyes than in the mouth.

  The direction and magnitude of the gradient vector K are defined as a feature amount C0. The direction of the gradient vector K is a value from 0 to 359 degrees with reference to a predetermined direction of the gradient vector K (for example, the x direction in FIG. 12).

  Here, the magnitude of the gradient vector K is normalized. This normalization obtains a histogram of the magnitude of the gradient vector K in all the pixels of the reduced image d, and the distribution of the magnitudes is a value that each pixel of the reduced image d can take (0 to 255 if 8 bits). The histogram is smoothed so as to be uniformly distributed, and the magnitude of the gradient vector K is corrected. For example, when the gradient vector K is small and the histogram is distributed with the gradient vector K biased toward the small side as shown in FIG. The magnitude of the gradient vector K is normalized so that it extends over the region so that the histogram is distributed as shown in FIG. In order to reduce the calculation amount, as shown in FIG. 14C, the distribution range in the histogram of the gradient vector K is divided into, for example, five, and the frequency distribution divided into five is shown in FIG. 14D. It is preferable to normalize so that the value of 0 to 255 is in a range divided into five.

  The reference data H0 constituting the database 115 is an identification for the combination of the feature amount C0 in each pixel constituting each pixel group, for each of a plurality of types of pixel groups composed of combinations of a plurality of pixels selected from a sample image to be described later. The conditions are specified.

  In the reference data H0, the combination of the feature amount C0 in each pixel constituting each pixel group and the identification condition are a plurality of sample images that are known to be faces and a plurality of sample images that are known not to be faces. It is predetermined by learning a sample image group consisting of

  In the present embodiment, when generating the reference data H0, the sample image that is known to be a face has a 30 × 30 pixel size, and as shown in FIG. The distance between the centers of both eyes of the image is 10 pixels, 9 pixels, and 11 pixels, and the face standing vertically at the distance between the centers of both eyes is rotated stepwise by 3 degrees within a range of ± 15 degrees on the plane. (That is, the rotation angle is -15 degrees, -12 degrees, -9 degrees, -6 degrees, -3 degrees, 0 degrees, 3 degrees, 6 degrees, 9 degrees, 12 degrees, 15 degrees) To do. Therefore, 3 × 11 = 33 sample images are prepared for one face image. In FIG. 15, only sample images rotated at −15 degrees, 0 degrees, and +15 degrees are shown. The center of rotation is the intersection of the diagonal lines of the sample image. Here, if the distance between the centers of both eyes is a 10-pixel sample image, the center positions of the eyes are all the same. The center position of this eye is set to (x1, y1) and (x2, y2) on the coordinates with the upper left corner of the sample image as the origin. In addition, the eye positions in the vertical direction in the drawing (ie, y1, y2) are the same in all sample images.

  As a sample image that is known not to be a face, an arbitrary image having a 30 × 30 pixel size is used.

  Here, as a sample image that is known to be a face, learning is performed using only a center image whose distance between the centers of both eyes is 10 pixels and the rotation angle on the plane is 0 degree (that is, the face is vertical). When performed, only the face that is identified as a face with reference to the reference data H0 is not rotated at all because the distance between the centers of both eyes is 10 pixels. Since the size of a face that may be included in the reduced image d is not constant, when identifying whether or not a face is included, the size of the sample image is reduced by enlarging and reducing the reduced image d as described later. It is possible to identify the position of a face of a size that fits. However, in order to accurately set the distance between the centers of both eyes to 10 pixels, the size of the reduced image d needs to be identified while being enlarged or reduced in steps of, for example, 1.1 units as an enlargement ratio. Will be enormous.

  In addition, the face that may be included in the reduced image d is not only rotated at 0 degrees on the plane as shown in FIG. 16A, but also rotated as shown in FIGS. 16B and 16C. Sometimes it is. However, when learning is performed using only a sample image in which the center-to-center distance between both eyes is 10 pixels and the face rotation angle is 0 degrees, FIG. 16B and FIG. As shown in (), the rotated face cannot be identified.

  Therefore, in this embodiment, as a sample image known to be a face, the distance between the centers of both eyes is 9, 10, and 11 pixels as shown in FIG. 15, and ± 15 degrees on the plane at each distance. In this range, a sample image obtained by rotating the face stepwise in units of 3 degrees is allowed to learn the reference data H0. As a result, when the face detection unit 120, which will be described later, performs identification, the reduced image d may be enlarged or reduced in steps of 11/9 as an enlargement rate. The calculation time can be reduced as compared with the case of enlarging / reducing in steps of 1.1 units. In addition, as shown in FIGS. 16B and 16C, a rotating face can also be identified.

  Hereinafter, an example of a learning method for the sample image group will be described with reference to the flowchart of FIG.

  The group of sample images to be learned includes a plurality of sample images that are known to be faces and a plurality of sample images that are known not to be faces. As described above, the sample image that is known to be a face has 9, 10, 11 pixels in the center position of both eyes for one sample image, and is 3 in a range of ± 15 degrees on the plane at each distance. Use a face rotated stepwise in degrees. Each sample image is assigned a weight or importance. First, the initial value of the weight of all sample images is set equal to 1 (S1).

  Next, a discriminator is created for each of a plurality of types of pixel groups in the sample image (S2). Here, each discriminator provides a reference for discriminating between a face image and a non-face image by using a combination of feature amounts C0 in each pixel constituting one pixel group. In the present embodiment, a histogram for a combination of feature amounts C0 in each pixel constituting one pixel group is used as a discriminator.

The creation of a classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 18, each pixel constituting the pixel group for creating the discriminator is a pixel at the center of the right eye on a plurality of sample images that are known to be faces. P1, a pixel P2 on the right cheek, a pixel P3 on the forehead, and a pixel P4 on the left cheek. Then, combinations of feature amounts C0 in all the pixels P1 to P4 are obtained for all sample images that are known to be faces, and a histogram thereof is created. Here, the feature amount C0 represents the direction and magnitude of the gradient vector K. Since the gradient vector K has 360 directions from 0 to 359 and the gradient vector K has 256 sizes from 0 to 255, If used as they are, the number of combinations is 360 × 256 four pixels per pixel, that is, (360 × 256) ^four , and the number of samples, time and memory for learning and detection are large. Will be required. For this reason, in this embodiment, the gradient vector directions are 0 to 359, 0 to 44, 315 to 359 (right direction, value: 0), 45 to 134 (upward value: 1), and 135 to 224 (left). Direction, value: 2), 225-314 (downward, value 3), and quaternarization, and the gradient vector magnitude is ternarized (value: 0-2). And the value of a combination is computed using the following formula | equation.

Combination value = 0 (when gradient vector size = 0)
Combination value = ((gradient vector direction + 1) × gradient vector magnitude (gradient vector magnitude> 0)
Thus, since the number of combinations is nine patterns ^4, it can reduce the number of data of the characteristic amounts C0.

Similarly, histograms are created for a plurality of sample images that are known not to be faces. For the sample image that is known not to be a face, pixels corresponding to the positions of the pixels P1 to P4 on the sample image that is known to be a face are used. The histogram used as a discriminator shown on the right side of FIG. 18 is a histogram obtained by taking logarithmic values of the ratios of the frequency values indicated by these two histograms. The value of each vertical axis indicated by the histogram of the discriminator is hereinafter referred to as an identification point. According to this classifier, an image showing the distribution of the feature quantity C0 corresponding to the positive identification point is highly likely to be a face, and it can be said that the possibility increases as the absolute value of the identification point increases. Conversely, an image showing the distribution of the feature quantity C0 corresponding to the negative identification point is highly likely not to be a face, and the possibility increases as the absolute value of the identification point increases. In step S <b> 2, a plurality of classifiers in the above-described histogram format are created for combinations of feature amounts C <b> 0 in the respective pixels constituting a plurality of types of pixel groups that can be used for identification.

  Subsequently, the most effective classifier for identifying whether or not the image is a face is selected from the plurality of classifiers created in step S2. The most effective classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rate of each classifier is compared, and the classifier showing the highest weighted correct answer rate is selected (S3). That is, in the first step S3, since the weight of each sample image is equal to 1, the number of sample images in which the image is correctly identified by the classifier is simply the largest. Selected as a valid discriminator. On the other hand, in the second step S3 after the weight of each sample image is updated in step S5, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step S3 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the classifiers selected so far, that is, the result of identifying whether each sample image is a face image using a combination of the classifiers selected so far, is actually It is ascertained whether or not the rate that matches the answer indicating whether the image is a face image exceeds a predetermined threshold (S4). Here, the sample image group to which the current weight is applied or the sample image group to which the weight is equal may be used for evaluating the correct answer rate of the combination. When the predetermined threshold value is exceeded, learning can be completed because it is possible to identify whether the image is a face with a sufficiently high probability by using the classifier selected so far. If it is less than or equal to the predetermined threshold, the process advances to step S6 to select an additional classifier to be used in combination with the classifier selected so far.

  In step S6, the discriminator selected in the most recent step S3 is excluded so as not to be selected again.

  Next, the weight of the sample image that could not be correctly identified as a face by the classifier selected in the most recent step S3 is increased, and the sample image that can be correctly identified as whether or not the image is a face is increased. The weight is reduced (S5). The reason for increasing or decreasing the weight in this way is that in selecting the next discriminator, an image that cannot be discriminated correctly by the already selected discriminator is regarded as important, and whether or not those images are faces can be discriminated correctly. This is to increase the effect of the combination of the discriminators by selecting the discriminators.

  Subsequently, the process returns to step S3, and the next valid classifier is selected based on the weighted correct answer rate as described above.

  By repeating the above steps S3 to S6, the classifier corresponding to the combination of the feature amount C0 in each pixel constituting the specific pixel group is selected as a classifier suitable for identifying whether or not a face is included. If the correct answer rate confirmed in step S4 exceeds the threshold value, the type of the discriminator used for discriminating whether or not the face is included and the discriminating condition are determined (S7), thereby the reference data H0. Finish learning.

  In the case of adopting the above learning method, the discriminator provides a reference for discriminating between a face image and a non-face image using a combination of feature amounts C0 in each pixel constituting a specific pixel group. As long as it is not limited to the above histogram format, it may be anything, for example, binary data, a threshold value, a function, or the like. Moreover, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 18 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  The face detection unit 120 refers to the identification conditions learned by the reference data H0 for all the combinations of the feature amounts C0 in the respective pixels constituting a plurality of types of pixel groups, and the feature amounts in the respective pixels constituting each pixel group. An identification point for the combination of C0 is obtained, and all the identification points are combined to identify whether or not a face is included in the reduced image d. At this time, the direction of the gradient vector K that is the feature amount C0 is quaternized and the magnitude is ternary. In the present embodiment, all the identification points are added, and identification is performed based on the positive / negative of the added value. For example, when the sum of the identification points is a positive value, it is determined that the reduced image d includes a face, and when it is a negative value, it is determined that no face is included.

  Here, the size of the reduced image d is different from the sample image of 30 × 30 pixels, and may have various sizes. When a face is included, the rotation angle of the face on the plane is not always 0 degrees. For this reason, as shown in FIG. 19, the face detection unit 120 enlarges and reduces the reduced image d stepwise until the vertical or horizontal size becomes 30 pixels and rotates it 360 degrees stepwise on the plane ( FIG. 19 shows a state of reduction), a mask M having a size of 30 × 30 pixels is set on the reduced image d enlarged / reduced at each stage, and the mask M is set on the reduced image d enlarged / reduced one pixel at a time. It is identified whether or not the reduced image d includes a face by identifying whether or not the image in the mask is a face image while moving. Whether or not a face is included in the reduced image d is determined with respect to the reduced image d at all stages of enlargement / reduction and rotation. Is extracted, and a 30 × 30 pixel area corresponding to the position of the identified mask M is extracted as a face image d0 from the reduced image d of the size and rotation angle at the stage where it is identified that the face is included. If the face is not identified as being included even once, the reduced image d is identified as not containing a face, and the reduced image d is output to the edge detecting means 12 as it is. To do.

  Note that since the sample image learned at the time of generating the reference data H0 has 9, 10, and 11 pixels at the center position of both eyes, the enlargement ratio when the reduced image d is enlarged / reduced is 11 / 9 is enough. Further, as the sample image learned at the time of generating the reference data H0, a sample image whose face is rotated in a range of ± 15 degrees on the plane is used. Therefore, if the reduced image d is rotated 360 degrees in units of 30 degrees. Good.

  Note that the feature amount calculation unit 110 calculates the feature amount C0 at each stage of deformation, that is, enlargement / reduction and rotation of the reduced image d.

  In this way, the face extraction unit 100 identifies whether or not a face is included in the reduced image d (that is, whether or not a face is included in the image D). The extracted face image d0 is output to the edge detection unit 12, while if the face is not included, the reduced image d is output to the edge detection unit 12 as it is.

  When a plurality of face images d0 are extracted, the face extraction unit 100 outputs all the extracted face images d0 to the edge detection unit 12.

  FIG. 2 is a block diagram showing the configuration of the edge detection means 12. As illustrated, the edge detection unit 12 includes a selection unit 11a, a second reduction unit 13, and a detection execution unit 11b.

  When there is one image (only one face image d0 or reduced image d) output from the face extracting unit 100, the selecting unit 11a sends this one image to the second reducing unit 13 as it is. On the other hand, when a plurality of images (face images d0) are output from the face extraction means 100, the face image used for extracting an edge from the plurality of face images d0 is selected and the second reduction means is selected. 13 is output. In the present embodiment, the selection unit 11a of the edge detection unit 12 confirms the number of pixels of each of the plurality of face images d0 and selects the face image d0 having the largest number of pixels. The image output from the selection unit 11a to the second reduction unit 13 is hereinafter referred to as an image d2.

  FIG. 3 is a block diagram showing the configuration of the second reduction means 13 in the edge detection means 12. As illustrated, the second reduction unit 13 includes a reduction rate determination unit 13a that determines a reduction rate β, and a reduction execution unit 13b that executes a reduction process using the reduction rate β.

  There are three types of images d2 output to the second reduction unit 13, each of which is a reduced image d output from the face extraction unit 100, and only one face image d0 output from the face extraction unit 100. The face image d0 is selected from the plurality of face images d0 by the selection unit 11a. The reduction rate determining means 13a corresponds to the image d2 and is an unreduced image (the image D in the case of the reduced image d and the area corresponding to the face image d0 in the image D in the case of the face image d0). The reduction ratio β is determined as follows according to the number of pixels of the image.

1. When the number of pixels is 1 million pixels or less, the reduction ratio β is determined to be 1, that is, not reduced.

2. When the number of pixels is larger than 1 million pixels and 2 million pixels or less, the reduction ratio β is determined to be ½.

3. When the number of pixels is larger than 2 million pixels and 3 million pixels or less, the reduction ratio β is determined to be ¼.

4). When the number of pixels is larger than 3 million pixels and 4 million pixels or less, the reduction ratio β is determined to be 1/8.

5. When the number of pixels is larger than 4 million pixels and 6 million pixels or less, the reduction ratio β is determined to be 1/16.

6). When the number of pixels is larger than 6 million pixels, the reduction ratio β is determined to be 1/32.

The reduction rate determination unit 13a outputs the reduction rate β thus determined to the reduction execution unit 13b.

  The reduction execution unit 13b uses the reduction rate β determined by the reduction rate determination unit 13a to use an unreduced image of the corresponding image d2 (as described above, in the case of the reduced image d, the image D and the face image). In the case of d0, the image D in the region corresponding to the face image d0) is reduced to obtain a reduced image d3 for detecting an edge and output to the detection execution unit 11b.

  First, the detection execution unit 11b uses the reduced image d3 to detect edges having a predetermined intensity or more in every eight directions as shown in FIG. 4 and obtain the coordinate positions of these edges. Next, based on the detected coordinate position of each edge in each direction, the detection execution unit 11b uses the corresponding non-reduced image (hereinafter generally referred to as a target image) of the reduced image d3 to detect these edges. In contrast, an edge profile as shown in FIG. 5 is created and output to the edge narrowing means 14.

  The edge narrowing means 14 is based on the edge profile output from the edge detection means 12 (specifically, the detection execution means 11b), or an edge having a complex profile shape or an edge (specifically, a light source). For example, an invalid edge such as an edge having a certain brightness or higher is removed, and the profile of the remaining edge is output to the edge feature quantity acquisition unit 16.

  The edge feature quantity acquisition unit 16 obtains the edge width as shown in FIG. 5 for each edge based on the edge profile output from the edge narrowing unit 14, and obtains the edge width as shown in FIG. A histogram is created for each of the eight directions shown in FIG. 4 and is output to the analysis means 20 as an edge feature amount S together with the edge width.

  The analysis means 20 mainly performs the following two processes.

  1. The blur direction and the blur degree N in the target image are obtained to determine whether the target image is a blur image or a normal image.

  2. When it is determined that the target image is a blurred image, a blur width L and a blur degree K are calculated.

  Here, the first process will be described.

  In order to obtain the blur direction in the target image, the analysis unit 20 first sets two directions orthogonal to each other to a histogram of edge widths in eight directions shown in FIG. As a result, the correlation value of the histogram of each direction set (1-5, 2-6, 3-7, 4-8) is obtained. Note that there are various types of correlation values, depending on how they are obtained.If the correlation value is large, the correlation type is small, and the correlation value is the same as the correlation level, that is, if the correlation value is small, the correlation type is small. It can be roughly divided into two types. In this embodiment, as an example, a correlation value of a type in which the magnitude of the correlation value matches the magnitude of the correlation is used. As shown in FIG. 7, when there is a blur in the image, the correlation between the blur direction histogram and the histogram in the direction orthogonal to the blur direction is small (see FIG. 7A). The correlation between the histograms is large in the orthogonal direction set not related to or in the orthogonal direction set when there is no blur in the image (no blur or out of focus) (see FIG. 7B). The analysis unit 20 in the image processing system A of the present embodiment pays attention to such a tendency, obtains a correlation value of the histogram of each group with respect to the four direction groups, and obtains two directions of the direction group having the smallest correlation. Find out. If there is a blur in the image D, one of the two directions can be considered as a direction closest to the blur direction among the eight directions shown in FIG.

  FIG. 7C shows a histogram of edge widths in the blur direction obtained for each image obtained by imaging the same subject under imaging conditions without blur, defocus, and blur (out of focus and blur). Show. As can be seen from FIG. 7 (c), the normal image without blur has the smallest average edge width, that is, of the two directions found above, the one with the largest average edge width is the most blurred. The direction should be close to.

  In this way, the analysis unit 20 finds the direction set having the smallest correlation, and sets the direction having the larger average edge width as the blur direction among the two directions of the direction set.

  Next, the analysis unit 20 obtains the degree of blur N of the target image. The degree of blur of the image indicates the magnitude of the degree of blur in the image. For example, the average edge width in the direction most blurred in the image (here, the blur direction obtained above) may be used. However, here, the edge width of each edge in the blur direction is used to obtain more accurately using the database based on FIG. FIG. 8 is based on a normal image database for learning and a blurred (blurred and blurred) image database, and the direction in which the image is most blurred (in the case of a normal image, a direction corresponding to this direction is desirable, but arbitrary The edge width distribution histogram (which may be the direction of the image) is created, and the ratio between the frequency in the blurred image and the frequency in the normal image (the vertical axis in the drawing) is obtained for each edge width as an evaluation value (the score in the drawing). Is. Based on FIG. 8, a database (hereinafter referred to as a score database) in which the edge width and the score are associated with each other is created and stored in the storage unit 50.

  The analysis unit 20 refers to the score database created based on FIG. 8 and stored in the storage unit 50, acquires a score from the edge width of each edge in the blur direction of the target image, The average value of the scores of all edges is obtained as the degree of blur N of the target image. If the obtained blur degree N of the target image is smaller than a predetermined threshold value T, the analysis unit 20 determines that the target image is a normal image and that the image D that is the original image of the target image is a normal image. When the information P indicated is output to the output means 60, the processing is terminated.

  On the other hand, if the degree of blur N of the target image is greater than or equal to the threshold T, the analysis unit 20 determines that the target image is a blur image and enters the second process.

  As the second process, the analysis unit 20 first obtains the degree of blur K of the target image.

  The degree of blur K indicating the degree of blur in the blur image can be obtained based on the following factors.

1. Correlation value of the direction set with the smallest correlation (hereinafter referred to as the minimum correlation set): The smaller the correlation value, the greater the degree of blurring. The analysis means 20 pays attention to this point and based on the curve shown in FIG. First blur degree K1 is obtained. Note that the LUT (look-up table) created according to the curve shown in FIG. 9A is stored in the storage unit 50, and the analysis unit 20 uses the first correlation value corresponding to the correlation value of the minimum correlation set. The first blur degree K1 is obtained by reading the blur degree K1 from the storage means 50.

2. Of the two directions of the minimum correlation set, the average edge width in the direction where the average edge width is large: The larger the average edge width, the greater the degree of blurring. The analysis means 20 pays attention to this point, and FIG. A second blurring degree K2 is obtained based on the curve shown in FIG. Note that the LUT (look-up table) created according to the curve shown in FIG. 9B is stored in the storage unit 50, and the analysis unit 20 calculates the average in the direction where the average edge width of the smallest correlation set is large. The second blur degree K2 corresponding to the edge width is read from the storage means 50 to obtain the second blur degree K2.

3. Difference between the average edge widths in the two directions of the minimum correlation set: The greater the difference, the greater the degree of blurring. The analysis means 20 pays attention to this point, based on the curve shown in FIG. 3 is calculated. Note that the LUT (look-up table) created according to the curve shown in FIG. 9C is stored in the storage means 50, and the analysis means 20 calculates the average edge in each of the two directions of the minimum correlation set. The third blur degree K3 corresponding to the width difference is read from the storage means 50 to obtain the third blur degree K3.

  In this way, the analysis means 20 obtains the first blur degree K1, the second blur degree K2, and the third blur degree K3, and blurs using K1, K2, and K3 according to the following equation (1). The blurring degree K of the target image to be an image is obtained.

K = K1 × K2 × K3 (1)
Where K: degree of blurring K1: first degree of blurring K2: second degree of blurring K3: third degree of blurring

Next, the analysis unit 20 obtains the blur width L of the target image. Here, regardless of the blurring degree K, the average width of edges in the blur direction may be obtained as the blur width L, or the average edge width of edges in all eight directions shown in FIG. L may be used.

  The analysis unit 20 outputs the blur degree K and the blur width L obtained for the target image, which is a blur image, to the parameter setting unit 30 as blur information Q together with the blur degree N and the blur direction.

  The image processing system A according to the present embodiment performs correction on the image D by an unsharpness masking (USM) correction method, and the parameter setting unit 30 performs the blur width L according to the blur width L and the blur direction. A one-dimensional correction mask M1 for directivity correction that acts in the blur direction is set such that the larger the blur width L is, the larger the blur width L is. Is set so as to increase the isotropic correction mask M2. The two-dimensional correction mask corresponding to each blur width and the one-dimensional correction mask corresponding to each blur width and blur direction are stored in the storage means 50 as a database (referred to as a mask database). From the mask database stored in the storage unit 50, a one-dimensional correction mask M1 is acquired based on the blur width L and the blur direction, and a two-dimensional correction mask M2 is acquired based on the blur width L.

  Next, the parameter setting unit 30 sets the one-dimensional correction parameter W1 for directionality correction and the two-dimensional correction parameter W2 for isotropic correction according to the following equation (2).

W1 = N × K × M1
W2 = N * (1-K) * M2 (2)
However, W1: One-dimensional correction parameter
W2: Two-dimensional correction parameter
N: Defocus degree
K: Degree of blur
M1: One-dimensional correction mask
M2: Two-dimensional correction mask In other words, the parameter setting means 30 increases the isotropic correction strength and the directionality correction strength as the degree of blur N increases, and the directionality correction weight increases as the blur degree K increases. Are set correction parameters W1 and W2 (collectively, parameter E0).

  Here, the parameter setting means 30 further adjusts the degree of blur N, the degree of blur K, and the correction masks M1, M2 with respect to the set parameter E0, and adjusts the degree of blur N, the degree of blur K, and the correction. A plurality of correction parameters E1, E2,... Different from the correction parameter E0 are obtained according to the equation (2) using the masks M1, M2.

  The parameter setting means 30 outputs the correction parameter E (E0, E1, E2,...) Thus obtained to the correction means 40.

  The correction means 40 performs correction by applying the parameters E0, E1, E2,... To the image D, respectively, and corrected images D′ 0, D′ 1, D′ 2,.・ ・ Get.

  The determination unit 45 includes a display unit (not shown) that displays each corrected image obtained by the correction unit 40, and an input unit (not shown) that allows the user to select a corrected image desired by the user from the displayed corrected images. The image selected by the user from the corrected images D′ 0, D′ 1, D′ 2,... Is determined as the target image D ′ and output to the output means 60.

  The output unit 60 outputs the image D when the information P indicating that the image D is a normal image is received from the analysis unit 20, while the target image when the target image D ′ is received from the determination unit 45. D 'is output. In the image processing system A of the present embodiment, “output” by the output unit 60 is printing, and the output unit 60 prints a target image D ′ obtained by correcting the image D of the normal image and the image D of the blurred image. To obtain a print, but store it in a recording medium, send it to an image storage server on the network, an address on the network specified by the client who requested the image correction, etc. It may be.

  FIG. 20 is a flowchart showing the operation of the image processing system A of the present embodiment. As shown in the figure, the image D read from the recording medium 1 is first reduced by the reduction means 10a at a reduction ratio of 1/8 to become a reduced image d (S10). The face extraction unit 100 identifies whether or not there is a face in the reduced image d (S11), and if there is a face (S11: Yes), extracts the face image d0 (a plurality if there are a plurality) from the reduced image d. On the other hand, when there is no face (S11: No), the reduced image d is output to the edge detection unit 12 as it is. When performing edge detection, the edge detection means 12 first detects an image d2 for detecting an edge (if there is one image output from the face extraction means 100, this one image, face extraction). When a plurality of face images d0 are output from the means 100, a face image d0 having the largest number of pixels among the plurality of face images d0 is selected and a reduction corresponding to the target image (image d2) is selected. The reduced image d3 is obtained by reducing the target image at a reduction rate corresponding to the number of pixels). Then, the edge detection unit 12 detects edges having a predetermined intensity or more in every eight directions as shown in FIG. 4 using the reduced image d3, and obtains coordinate positions of these edges. Next, the edge detection means 12 uses the target image (non-reduced image) corresponding to the reduced image d3 based on the detected coordinate position of each edge in each direction, An edge profile as shown in FIG. 5 is created and output to the edge narrowing means 14 (S13). The edge narrowing means 14 removes invalid edges based on the edge profile transmitted from the edge detection means 12 and outputs the remaining edge profiles to the edge feature quantity acquisition means 16 (S14). The edge feature quantity acquisition means 16 obtains the width of each edge based on the profile of each edge transmitted from the edge narrowing means 14 and creates a histogram of edge width for each direction shown in FIG. The histogram of the edge width and the edge width in each direction is output to the analyzing means 20 as the edge feature amount S (S16). The analysis unit 20 first calculates the blur direction and the blur degree N using the edge feature amount S, and determines whether the target image is a blur image or a normal image (S20, S25). If the target image is a normal image (S25: Yes), the analysis unit 20 outputs information P indicating that the image D that is the original image of the target image is a normal image to the output unit 60 (S30). Upon receiving the information P, the means 60 prints the image D to obtain a print (S35).

  On the other hand, when the target image is determined to be a blurred image in step S25 (S25: No), the analysis unit 20 further calculates a blur degree K and a blur width L with respect to the target image, and is obtained in step S20. It outputs to the parameter setting means 30 as blur information Q with blur degree N and a blur direction (S40, S45). The parameter setting unit 30 obtains the one-dimensional correction parameter W1 and the two-dimensional correction parameter W2 based on the blur information Q from the analysis unit 20. The obtained pair of correction parameters W1 and W2 is set as a correction parameter E0, and a plurality of correction parameters E1, E2,... Are further adjusted by adjusting the degree of blur N, the blurring degree K, the mask size of the correction masks M1, M2, etc. Is output to the correction means 40 (S50).

  The correction means 40 applies correction parameters E0, E1, E2,... To the image D to perform correction, and corrects images D′ 0, D′ 1, D′ 2, respectively corresponding to the respective correction parameters. Is obtained (S55).

  The determining unit 45 displays each corrected image obtained by the correcting unit 40 on a display unit (not shown), and determines the corrected image selected by the user as the target image D ′ via the input unit (not shown). To the output means 60 (S60).

  The output unit 60 obtains a print by printing the target image D ′ from the determination unit 45 (S70).

  As described above, according to the image processing system A of the present embodiment, blur information is acquired from a digital photographic image and the blur is corrected. Therefore, it is possible to reduce the blur of an image without requiring a special device at the time of shooting. It can be corrected.

  Further, since correction parameters are set and corrected based on the obtained blur information, it is not necessary to repeat trial and error of parameter setting, correction, evaluation, resetting,.

  In addition, since the blur information is acquired using only the image of the main subject (here, the face) in the digital photograph image, it is necessary to allow the user to specify the area for acquiring the blur information. It is convenient because it does not.

  Furthermore, since blur information is acquired using the image of the main subject area and parameters are set, accurate correction can be performed even in the case of an image with a shallow depth of field.

  FIG. 21 is a block diagram showing a configuration of an image processing system B according to the second embodiment of the present invention. The image processing system B according to the present embodiment is different from the image processing system A according to the embodiment shown in FIG. 1 except that the reduction means 10b and 10a are different and a determination means 200 is added. Therefore, only the reduction unit 10b and the added determination unit 200 will be described here, and the same reference numerals are given to the same configuration as the image processing system A shown in FIG.

  The discriminating means 200 in the image processing system of the embodiment shown in FIG. 21 reads the attached information of the image D and determines whether the image D is an image with a shallow depth of field based on the attached information. It is. Specifically, the discriminating means 200 reads the attached information of the image D by the provided reading means 210 to acquire the F value of the lens of the camera that has captured the image D, and this F value is a predetermined threshold value (here. As an example, if 5.6) or less, it is determined that the image D is an image with a shallow depth of field, while if the F value is larger than the predetermined threshold, the image D is an image with a shallow depth of field. And information indicating the result of the determination is output to the reduction means 10b. If the image D has no attached information or the attached information of the image D does not include an F value, the determining unit 200 outputs a determination result indicating that there is no F value to the reducing unit 10b. .

  The reduction unit 10b performs a reduction process on the image D at a reduction ratio of, for example, 1/8 to obtain a reduced image d, and the reduced image d is converted into the face extraction unit 100 or the edge according to the determination result of the determination unit 200. Output to the detection means 12. Specifically, if the determination result from the determination unit 200 is information indicating that the image D is an image having a shallow depth of field or information indicating that there is no F value of the image D, the image D If the result of determination is information indicating that the image D is not an image with a shallow depth of field, the reduced image d of the image D is detected as an edge detection unit. 12 is output.

  22 and 23 are flowcharts showing the operation of the image processing system B of the embodiment shown in FIG. As shown in FIG. 22, the image D read from the recording medium 1 is first determined by the determination unit 200 as to whether the image has a shallow depth of field (S100 to S115). Specifically, the reading unit 210 in the determination unit 200 checks whether or not attached information is attached to the image D, and if there is attached information, there is an F value of the lens of the camera that captured the image D. Whether or not (S102). When the attached information is not attached to the image D, or when the F information is not included in the attached information of the image D (S102: No), the determination unit 200 reduces the determination result of “no F value”. 10b (S115). On the other hand, when the attached information is attached to the image D in step S102 and the attached information includes the F value (S102: Yes), the determining unit 200 uses the F obtained by the reading unit 210. The value is compared with the aforementioned threshold value (S104), and if the F value is less than or equal to this threshold value (S104: No), it is determined that the image D is an image with a shallow depth of field (S106), while the F value is If it is larger than this threshold (S104: Yes), it is determined that the image D is not an image with a shallow depth of field (S108), and information indicating the result of the determination is output to the reduction means 10b (S115).

  The reduction unit 10b performs a reduction process on the image D to obtain a reduced image d, and the image D is an image with a shallow depth of field or the image D based on the determination result by the determination unit 200. When there is no F value (S122: Yes), the reduced image d of the image D is output to the face extraction means 100, while when the image D is not an image with a shallow depth of field (S122: No). The reduced image d of the image D is output to the edge detecting means 12 as it is. The face extraction unit 100 identifies whether or not there is a face with respect to the reduced image d from the reduction unit 10b (S124), and when there is a face (S124: Yes), the face image d0 from the reduced image d. (If there are a plurality of images), they are extracted and output to the edge detection means 12 (S126). On the other hand, if there is no face (S124: No), the reduced image d is output to the edge detection means 12 as it is. . The processing after step S126 is shown as processing A in the flowchart of FIG.

  As shown in FIG. 23, when performing edge detection, the edge detection unit 12 first detects an image d2 for detecting an edge (when there is only one image output from the face extraction unit 100). When a plurality of face images d0 are output from this one image / face extraction means 100, a face image d0 having the largest number of pixels among the plurality of face images d0) is selected and the target image ( The target image is reduced at a reduction rate corresponding to the number of pixels of the image d2 corresponding to the non-reduced image) to obtain a reduced image d3. Then, the edge detection unit 12 detects edges having a predetermined intensity or more in every eight directions as shown in FIG. 4 using the reduced image d3, and obtains coordinate positions of these edges. Next, the edge detection means 12 uses the target image (non-reduced image) corresponding to the reduced image d3 based on the detected coordinate position of each edge in each direction, An edge profile as shown in FIG. 5 is created and output to the edge narrowing means 14 (S130). The edge narrowing means 14 removes invalid edges based on the edge profile transmitted from the edge detection means 12, and outputs the remaining edge profiles to the edge feature quantity acquisition means 16 (S134). The edge feature quantity acquisition means 16 obtains the width of each edge based on the profile of each edge transmitted from the edge narrowing means 14 and creates a histogram of edge width for each direction shown in FIG. The histogram of the edge width and the edge width in each direction is output to the analyzing means 20 as the edge feature amount S (S136). The analysis unit 20 first calculates the blur direction and the blur degree N using the edge feature amount S, and determines whether the target image is a blur image or a normal image (S138, S140). If the target image is a normal image (S140: Yes), the analysis unit 20 outputs information P indicating that the image D that is the original image of the target image is a normal image to the output unit 60 (S142). Upon receipt of the information P, the means 60 prints the image D to obtain a print (S146).

  On the other hand, when the target image is determined to be a blurred image in step S140 (S140: No), the analysis unit 20 further calculates a blur degree K and a blur width L with respect to the target image, and is obtained in step S138. It outputs to the parameter setting means 30 with the blur degree N and the blur direction as blur information Q (S150, S155). The parameter setting unit 30 obtains the one-dimensional correction parameter W1 and the two-dimensional correction parameter W2 based on the blur information Q from the analysis unit 20. The obtained pair of correction parameters W1 and W2 is set as a correction parameter E0, and a plurality of correction parameters E1, E2,... Are further adjusted by adjusting the degree of blur N, the blurring degree K, the mask size of the correction masks M1, M2, etc. Is output to the correction means 40 (S160).

  The correction means 40 applies correction parameters E0, E1, E2,... To the image D to perform correction, and corrects images D′ 0, D′ 1, D′ 2, respectively corresponding to the respective correction parameters. Are obtained (S170).

  The determining unit 45 displays each corrected image obtained by the correcting unit 40 on a display unit (not shown), and determines the corrected image selected by the user as the target image D ′ via the input unit (not shown). To the output means 60 (S175).

  The output unit 60 obtains a print by printing the target image D ′ from the determination unit 45 (S180).

  As described above, according to the image processing system B of the present embodiment, the same effect as the image processing system A of the embodiment shown in FIG. 1 can be obtained, and whether the digital photographic image has a shallow depth of field. If the image is not a shallow depth-of-field image, only the digital photo image is determined using the entire digital photo image to set the correction parameters and perform correction. Since digital photographic images are corrected by setting correction parameters using images of the main subject area, it is possible to improve the correction accuracy of digital photographic images that are not images with a shallow depth of field. it can.

  FIG. 24 is a block diagram showing a configuration of an image processing system C according to the third embodiment of the present invention. As shown in the figure, the image processing system C of this embodiment performs a reduction process on the image D read from the recording medium 1 to obtain a reduced image d of the image D, and a reduced image d. On the other hand, a main subject (here, a face as an example) is detected, and if a face is detected, an image d0 of a face part (hereinafter referred to as a face image) d0 and an image d1 other than the face part are extracted and described later. While outputting to the edge detection means 12c, if the face is not detected, the face extraction means 100c that outputs the reduced image d to the edge detection means 12c, and the edge obtained based on the image from the face extraction means 100c are detected. Edge detecting means 12 for detecting an edge using a reduced image (details will be described later) for this purpose and an unreduced image (hereinafter referred to as a target image) corresponding to the reduced image. Among the edges detected by the edge detection means 12c, the edge narrowing means 14c for removing invalid edges, and the edge feature values (S0, S1, S ′ 3) obtained by the edge narrowing means 14c. The image feature D is obtained by calculating the blur direction and the blur degree N of the target image using the edge feature value acquisition unit 16c that acquires the edge feature value S of the target image. It is determined whether the image is a blurred image or a normal image. If the image is a normal image, information P indicating that the image D is a normal image is transmitted to the output unit 60 described later, and the process ends. In this case, the blur degree K and the blur width L are further calculated with respect to the target image, and the parameter setting unit 30c and the certainty factor calculation unit 250 described later as blur information together with the blur degree N and the blur direction. An analyzing unit 20c for transmitting, and a parameter setting unit 30c for setting a parameter for correcting the image D to be a blurred image based on the blur information from the analyzing unit 20c and outputting the parameter to the parameter integrating unit 260 or the correcting unit 40c described later. And a certainty factor calculating unit 250 that calculates a certainty factor α based on the blur information from the analyzing unit 20c and an image D having a shallow depth of field, and a parameter output from the parameter setting unit 30c. Based on the certainty factor α calculated by the certainty factor calculation unit 250, the parameter integration unit 260 that obtains an integrated parameter and outputs it to the correction unit 40c, and the parameter output from the parameter setting unit 30C or the parameter integration unit 260 The image D is corrected using the parameters output from the above to obtain a corrected image A correction unit 40c that outputs the corrected image as the target image D ′ to the output unit 60; a storage unit 50c that stores various databases for the analysis unit 20c, the parameter setting unit 30c, and the certainty factor calculation unit 250; When the information P indicating that the image D is a normal image is received from the analyzing unit 20c, the image D is output. On the other hand, when the target image D ′ is received from the correcting unit 40c, the target image D ′ is output. Output means 60.

  The reduction unit 10c is the same as the reduction unit 10a in the image processing system A of the embodiment shown in FIG. 1, and performs a reduction process on the image D at a reduction rate of, for example, 1/8 to obtain a reduced image d. Is.

  The face extraction unit 100c identifies whether or not there is a face in the reduced image d. If there is a face, the face extraction unit 100c extracts the face image d0 (if there are a plurality of them) and outputs it to the edge detection unit 12c. On the other hand, when there is no face, outputting the reduced image d to the edge detection unit 12c is the same as the face extraction unit 100 in the image processing system A of the embodiment shown in FIG. Extracts not only the face image d0 but also the image d1 of the part other than the face and outputs it to the edge detecting means 12c. The identification of whether or not there is a face in the reduced image d and the extraction of the face image when there is a face are the same as those of the face extraction unit 100 of the image processing system A, and thus the description thereof is omitted here.

  That is, the image output to the edge detection unit 12c is a face image d0 (there may be only one or a plurality of cases) and a reduced image other than the face image d0 when a face is detected in the reduced image d. If the face is not detected in the reduced image d, it is the reduced image d. When a plurality of face images d0 are output from the face extraction unit 100c (in this case, the reduced image d1 is also output), the edge detection unit 12c has the largest number of pixels among the plurality of face images d0. A large number of face images d0 are selected, and another face image d0 that has not been selected is combined with the above-described reduced image d1 to obtain a new reduced image (hereinafter, the same reference numeral d1 is used for convenience of explanation). Both face image d0 and reduced image d1 (if there is only one face image d0, this one face image d0 and reduced image d1, and if there are a plurality of face images d0, the selected face image d0 and a new reduced image Based on the image d1) or the reduced image d, the reduction ratio corresponding to the number of pixels of the corresponding unreduced image (the face image D0 and the image D1 other than the face or the image D; hereinafter referred to as the target image) The target image is reduced to obtain a reduced image for detecting an edge. Note that since the reduction rate determination method uses the same method as the reduction rate determination unit 13a of the edge detection unit 12 in the image processing system A, description thereof is omitted here. The edge detection means 12 performs processing for extracting an edge using the reduced image thus obtained and processing for creating a profile using the target image. If a face is detected in the reduced image d, the edge detection unit 12 detects the edge. Except for the fact that there are two images (a face image d0 (D0) and a non-face image d1 (D1)) for the image processing system A. Description of the operation is omitted.

  The same applies to the edge narrowing means 14c, the edge feature quantity acquisition means 16c, and the analysis means 20c.

  In this way, when a face is detected in the reduced image d, the edge feature amounts S0 and S1 of the face image D0 and the image D1 of the portion other than the face are acquired and output to the analysis unit 20c, and the face is added to the reduced image d. Is not detected, the edge feature amount S ′ of the image D is acquired and output to the analyzing means 20c.

  When the edge feature amounts S0 and S1 are output from the edge feature amount acquisition unit 16c, the analysis unit 20c firstly, based on the edge feature amount S0, the blur direction and the degree of blur in the face image D0 corresponding to the edge feature amount S0. N is calculated. If the obtained blur degree N of the face image D0 is smaller than the predetermined threshold T, the analysis unit 20c determines that the image D corresponding to the face image D0 is a normal image and indicates that the image D is a normal image. By outputting the information P to the output means 60, the process is terminated. On the other hand, if the degree of blur N of the face image D0 is equal to or greater than the threshold T, the analysis unit 20c determines that the image D corresponding to the face image D0 is a blur image, and further increases the degree of blur K, The blur width L is calculated and used as blur information Q0 of the face image D0, and the blur direction, blur degree N, blur degree K, and blur width L in the image D1 using the edge feature amount S1 of the image D1 other than the face. Is calculated as blur information Q1 of the image D1 of the portion other than the face.

On the other hand, when the edge feature amount S ′ is output from the feature amount acquisition unit 16c, the analysis unit 20c first calculates a blur direction and a blur degree N in the image D based on the edge feature amount S ′. If the obtained blur degree N of the image D is smaller than the predetermined threshold T, the analysis unit 20c determines that the image D is a normal image, and also outputs information P indicating that the image D is a normal image to the output unit 60. The process is terminated with the output. If the degree of blur N of the image D is greater than or equal to the threshold T, the analysis unit 20c determines that the image D is a blur image, and further calculates a blur degree K and a blur width L with respect to the image D. The analysis means 20c, which uses the blur information Q ′ of D, the blur information Q1 calculated from the blur information Q0 calculated from the edge feature value S0 of the face image D0 and the edge feature value S1 of the image D1 of the portion other than the face in this way. Is output to the certainty calculation means 250 and the parameter setting means 30c, while the blur information Q ′ calculated using the image D is output only to the parameter setting means 30c.

  The image processing system C according to the present embodiment corrects the image D by an unsharpness masking (USM) correction method, like the above-described image processing system A and image processing system B, and includes parameter setting means. 30c refers to the storage means 50c based on the blur information (Q0 and Q1, or Q ′) from the analysis means 20c, as with the parameter setting means 30 in the image processing system A and the image processing system B. A parameter for correcting D is set. When the blur information Q0 and Q1 is output from the analysis unit 20c, the parameter setting unit 30c sets the parameters Ec0 and Ec1 using the two blur information Q0 and Q1, respectively, and integrates the parameters. On the other hand, when only the blur information Q ′ has been output, the parameter Ec is set using the blur information Q ′ and directly output to the correction unit 40 c.

  The certainty factor calculation means 250 obtains a certainty factor α that the image D is an image with a shallow depth of field based on the blur information Q0 and the blur information Q1. The certainty factor calculating unit 250 may calculate the certainty factor α using all the information included in the blur information, but may calculate the certainty factor α using only a part of the blur information. It may be. Here, as an example, the certainty factor α is calculated using only the blur degree N included in the blur information. Specifically, the blur degree N included in the blur information Q0 is equal to or greater than the blur degree N included in the blur information Q1, and the blur degree N included in the blur information Q0 is greater than the blur degree N included in the blur information Q1. Certainly, if the difference between the two blur degrees N (that is, the blur degree N included in the blur information Q0 is smaller than the blur degree N included in the blur information Q1, and so on) is smaller than the predetermined first threshold value. When the degree α is 0% and the difference between the two blur degrees N is equal to or greater than a predetermined second threshold value that is greater than the first threshold value, the certainty factor α is 100% and the difference between the two blur degrees N is When it is between the first threshold value and the second threshold value, the certainty factor α is calculated such that the greater the difference, the greater the certainty factor α is between 0% and 100%. Note that a table indicating the correspondence between the difference in blur degree and the certainty factor α is stored in the storage unit 50c, and the certainty factor calculating unit 250 calculates the certainty factor α with reference to this table.

  The parameter integration unit 260 calculates the integrated parameter E′c according to the following equation (3) using the parameters Ec0 and Ec1 set by the parameter setting unit 30c and the certainty factor α calculated by the certainty factor calculation unit 250. It outputs to the correction means 40c.

E′c = α × Ec0 + (1−α) × Ec1 (3)
E'c: Integration parameter
Ec0: parameter set using the face image d0
Ec1: Parameters set using the image d1 of the part other than the face
α: Certainty that image D is an image with a shallow depth of field

As can be seen from Equation (3), the parameter integration unit 260 increases the weight of the parameter Ec0 set using the face portion image D0 as the certainty factor α in which the image D is an image with a shallow depth of field is higher. In this way, the parameter Ec0 and the parameter Ec1 are weighted and added to obtain the integrated parameter E′c.

  The correcting unit 40c applies the parameter Ec output from the parameter setting unit 30c using the entire image D or the integrated parameter E′c output from the parameter integrating unit 260 to be applied to the image D. Correction is performed to obtain a corrected image D ′.

  Since the output unit 60 is the same as the output unit 60 in the image system A and the image system B, description thereof is omitted here.

  25 and 26 are flowcharts showing the operation of the image processing system C shown in FIG. As shown in FIG. 25, in the image processing system C of the present embodiment, the image D read from the recording medium 1 is first subjected to reduction processing by the reduction means 10c to become a reduced image d (S200). The face extraction unit 100c identifies whether or not there is a face with respect to the reduced image d obtained by the reduction unit 10c (S202), and when there is a face (S202: Yes), While extracting the image d0 and extracting the image d1 of the part other than the face and outputting it to the edge detecting means 12c, if there is no face (S202: No), the reduced image d is outputted as it is to the edge detecting means 12c. .

  Processing B shown in FIG. 26 includes processing by the edge detection unit 12c, edge narrowing unit 14c, edge feature amount acquisition unit 16c (S210 to S214) and partial processing of the analysis unit 20c (S216). As shown in FIG. 26, the process B starts from edge detection by the edge detection unit 12c, and the edge detection unit 12c first creates a new reduced image created based on the reduced image output from the face extraction unit 100c. Are used to detect edges having a predetermined intensity or more in every eight directions as shown in FIG. 4 and obtain the coordinate positions of these edges. Next, the edge detection unit 12c uses the target image (unreduced image) corresponding to the above-described new reduced image based on the detected coordinate position of each edge in each direction to detect these edges. On the other hand, an edge profile as shown in FIG. 5 is created and output to the edge narrowing means 14c (S210). The edge narrowing means 14c removes invalid edges based on the edge profile transmitted from the edge detection means 12c, and outputs the remaining edge profiles to the edge feature quantity acquisition means 16c (S212). The edge feature quantity acquisition means 16c obtains the width of each edge based on the profile of each edge transmitted from the edge narrowing means 14c, and creates an edge width histogram for each direction shown in FIG. The histogram of the edge width and the edge width in each direction is output to the analyzing means 20c as an edge feature amount (S214). The analysis unit 20c first calculates the blur direction and the blur degree N of the target image using the edge feature amount (S216).

  Here, when the face image d0 and the image d1 of the part other than the face are output from the face extraction unit 100c (S202: Yes, S204 in the flowchart of FIG. 25), the target image of the process B is the face image D0 and the face The edge feature amount acquisition unit 16c calculates the respective edge feature amounts S0 and S1, and the analysis unit 20c calculates the respective blur direction and blur degree N. When the reduced image d is output from the face extraction unit 100c (S202: No in the flowchart of FIG. 25), the target image of the process B is only the image D, and the edge feature amount acquisition unit 16c The edge feature amount S ′ is calculated, and the blur direction and the blur degree N of the image D are calculated by the analysis unit 20c.

Returning to the flowchart of FIG. 25, when the analysis unit 20c calculates the blur direction and the blur degree N of the face image D0 and the non-face image D1 in step S216 of the process B, based on the blur degree N of the face image D0. It is determined whether the image D is a blurred image or a normal image (S220),
When it is determined as a normal image (S220: Yes), the process is terminated by outputting information P indicating that the image D is a normal image to the output means 60 (S222), while it is determined as a blurred image. (S220: No), the degree of blur K and the blur width L are further calculated for the face image D0 and the image D1 of the part other than the face (S230), and each blur direction, blur degree N, blur degree K is calculated. The blur information Q0, Q1 obtained by combining the blur width L is output to the certainty factor calculation means 250 and the parameter setting means 30c (S232). The parameter setting means 30c sets the corresponding parameters Ec0 and Ec1 using the two blur information Q0 and Q1, respectively, and outputs them to the parameter integration means 260 (S234). In parallel with this, the certainty factor calculation means 250 is set. Uses the blur information Q0 and Q1 to calculate the certainty factor α that the image D is an image with a shallow depth of field, and outputs it to the parameter integration unit 260 (S236). The parameter integration unit 260 integrates the parameters Ec0 and Ec1 using the certainty factor α, and outputs the integrated parameter E′c to the correction unit 40c (S238).

  On the other hand, when the analysis unit 20c calculates the blur direction and the blur degree N of the image D (in this case, no face is detected by the face extraction unit 100c) in step S216 of the process B, the analysis unit 20c is based on the blur degree N of the image D. Whether the image D is a blurred image or a normal image is determined (S240), and when it is determined as a normal image (S240: Yes), information P indicating that the image D is a normal image is output to the output means 60. In step S222, the process ends. On the other hand, if it is determined as a blurred image (S240: No), a blur degree K and a blur width L are calculated for the image D (S242), and the blur direction, Together with the degree of blur N, it is output to the parameter setting means 30c as blur information Q '(S244). The parameter setting means 30c sets the parameter Ec using the blur information Q 'of the image D and outputs it to the correction means 40c (S246).

  The correction unit 40c performs correction by applying the integrated parameter E′c output from the parameter integration unit 260 or the parameter E′c output from the parameter setting unit 30c to the image D, thereby correcting the corrected image. Is output to the output means 60 as the target image D ′ (S250).

  When receiving the information P from the analyzing unit 20c (S222), the output unit 60 prints the image D to obtain a print (S224), and receives the target image D ′ from the correcting unit 40c. Printing is performed to obtain a print (S260).

  As described above, according to the image processing system C of the present embodiment, the certainty factor α is calculated in which the image D is an image with a shallow depth of field, and the higher the certainty factor α, the more the image of the main subject region (here Then, the parameter Ec0 and the parameter Ec1 set using the image D1 of the region other than the face are weighted and added so that the weight of the parameter Ec0 set using the face image d0) is increased. Since correction is performed, the same effects as those of the image processing system A and the image processing system B according to the above-described embodiment can be obtained, and the determination with respect to the depth of field may be slightly different. A correction effect can be obtained.

  In the image processing system C of the present embodiment, the parameter set using the image of the main subject area is weighted and summed with the parameter set using the image of the area other than the main subject area. Instead of the image of the area other than the main subject area, the entire digital photograph image (the entire reduced image d in the image processing system C) may be weighted and added.

  In the image processing system B, whether or not the image D is an image with a shallow depth of field is determined based on the auxiliary information of the image D. However, the blur information of the image in the main subject area and the main subject This determination may be performed by comparing blur information of an image in a region other than the region. Similarly, in the image processing system C, the certainty factor that the image D is an image with a shallow depth of field may be calculated based on the additional information of the image D.

  The image processing systems B and C used to determine whether or not a digital photographic image is an image with a shallow depth of field or to calculate a certainty factor that a digital photographic image is an image with a shallow depth of field. Not limited to the method, the digital photographic image may be divided into a plurality of blocks, and the blur information of the images of the respective blocks may be compared. FIG. 27 shows a configuration of an image processing system D according to the embodiment using this method. In the image processing system D shown in FIG. 27, for example, a digital photographic image is divided into a plurality of blocks, and the digital photographic image is an image with a shallow depth of field by comparing the blur information of the image of each block. Image processing is performed by determining whether or not a digital photo image is an image with a shallow depth of field by comparing the blur information of the image of each block. May be performed.

  The image processing system D of the embodiment shown in FIG. 27 is different from the image processing system B of the embodiment shown in FIG. 21 in the processing for determining whether or not the image D is an image with a shallow depth of field. Since the other processes are substantially the same except for, only the process for determining whether the image D is an image with a shallow depth of field in the image processing system D will be described in detail here. .

    In FIG. 27, processing for determining whether or not the image D is an image having a shallow depth of field is indicated by a dotted line. As shown in the figure, first, a reduced image d of the image D obtained by the reduction means 10d is also output to the determination means 200d. The determination unit 200d includes a block dividing unit 210d that divides the reduced image d into a plurality of blocks. Here, as an example, the reduction unit 10d reduces the image D to a reduced image d of 200 pixels × 150 pixels, and the block dividing unit 210d provided in the determination unit 200d converts the reduced image d into 20 pixels × 150 pixels. It is assumed that the data is divided into a total of 100 blocks of 15 pixels. The discriminating means 200d outputs the 100 block images obtained by the block dividing means 210d to the edge detecting means 12d. The edge detection unit 12d first detects edges having a predetermined intensity or more in each of the eight directions as shown in FIG. 4 with respect to the image (reduced image) of these blocks, and the coordinate positions of these edges. Get. Next, the edge detection unit 12d uses the image of the part corresponding to each block part in the image D on the basis of the detected coordinate position of each edge in each direction. An edge profile as shown in FIG. 5 is created and output to the edge narrowing means 14d. Each edge profile created by the edge detection means 12d is processed in the order of the edge narrowing means 14d, the edge feature quantity acquisition means 16d, and the analysis means 20d, and finally, each block from the analysis means 20d. The blur information of the image is output to the determination unit 200d. Here, when the blur information of the image of each block is calculated by the analysis unit 20d and output to the determination unit 200d, the blur direction, the blur degree N, the blur degree K, and the blur width L are all calculated and output. However, here, in order to reduce the amount of calculation and speed up the processing, the degree of blur of the image of each block is used as blur information for determining whether or not the image D has a shallow depth of field. Assume that only N is calculated and output to the discriminating means 200d.

  The determination unit 200d compares the blur information (here, only the blur degree N) of the image of each block, and determines whether the image D is an image with a shallow depth of field. Specifically, the determination is performed by the following method.

1. Group blocks so that blocks with a difference in blur N within a predetermined threshold value (threshold value t1) are in the same group. If there is only one group obtained in 2.1, image D is captured It is determined that the image is not a shallow depth of field. If there are two or more groups, only a group including four or more adjacent blocks is extracted, and the process proceeds to the following process 3.

  If the number of groups extracted in 3.2 is one or less, it is determined that the image D is not an image with a shallow depth of field. If there are two or more groups extracted in 3, the average degree of blur N ′ 0 for each block is calculated, and the process proceeds to processing 5.

  The average blur degree N ′ of all blocks of other groups excluding the group having the smallest average blur degree N′0 (the blur degree is N′min) from the two or more groups extracted in 5.3 1 is compared with N′min, and if the absolute value of the difference between N′1 and N′min is equal to or smaller than a predetermined threshold (threshold t2), the image D has a shallow depth of field. On the other hand, if the absolute value of the difference between N′1 and N′min is larger than the threshold value t2, it is determined that the image D is an image with a shallow depth of field.

  The discriminating means 200d thus discriminates whether or not the image D is an image having a shallow depth of field, and outputs the discrimination result to the reducing means 10d.

  The processing after the reduction unit 10d receives the determination result from the determination unit 200d is the same as the processing after the reduction unit 10b receives the determination result from the determination unit 200 in the image processing system B described above. Detailed description is omitted here.

  As described above, the image processing system D according to the present embodiment determines whether the image D is an image with a shallow depth of field by a different determination method from the image system B, and has the same effect as the image system B. In addition, it is possible to make a determination even when there is no attached information in the image D or when the F information is not included in the attached information of the image D.

  The preferred embodiments of the present invention have been described above. However, the image processing method and apparatus of the present invention and the program therefor are not limited to the above-described embodiments, and each of the above-described embodiments is not departed from the gist of the present invention. The configuration of the embodiment can be combined, increased, decreased, or changed.

  For example, identification of whether or not a face is included in a photographic image and extraction of a face image are not limited to the methods used in the above-described embodiments, and various conventionally known techniques can be used. it can.

  Further, in the image processing system A and the image processing system B of the embodiment shown in FIG. 1, the parameter setting unit 30 sets a plurality of correction parameters, and the correction unit 40 performs correction using the plurality of correction parameters. To obtain a plurality of corrected images, the determining means 45 determines the corrected image selected by the user from the plurality of corrected images as a target image, provided with a correction control means and a confirmation means, First, only one correction parameter is set in the parameter setting means, the correction means also obtains only one corrected image at a time, the confirmation means allows the user to confirm the corrected image, and the correction control means is determined by the user. Processing for resetting correction parameters by the parameter setting means until the result of confirmation is that the corrected image is the target image; According, the process for obtaining a corrected image by performing the correction using the correction parameter re-set, may be determined a target image so as to repeat the confirmation by confirming means. Alternatively, the user can confirm only one corrected image without repeatedly setting parameters, performing correction, and checking, and if the user does not like the original image before correction corresponding to the corrected image. You may make it determine as a target image.

  In the image processing system of the above-described embodiment, the direction with the larger average edge width is set as the blur direction among the two directions of the minimum correlation set. For example, the minimum correlation set (the direction set with the smallest correlation value) is used. ) And the direction set with the second smallest correlation value, the degree of blurring is calculated for each direction set, and the direction with the larger average edge width of the two directions in the direction set is set as the blurred direction. Candidate directions are acquired, and the obtained blur candidate directions are weighted according to the two calculated blur degrees, the greater the blur group, the greater the weight of the blur candidate directions included in the direction group. The blur direction may be obtained by weighting as described above. In this case, the blur width is also weighted so that the average edge width in each of the two blur candidate directions has a greater weight for the blur candidate direction included in the direction set in a direction set with a higher degree of blurring. The bokeh width can be obtained.

  In the image processing system of the above-described embodiment, the direction with the largest average edge width is set as the blur direction among the two directions of the minimum correlation set regardless of the presence / absence of blur in the image, and further blur is performed based on the blur direction. While calculating the degree, the normal image and the blurred image are discriminated based on the degree of blur, and the correction parameter is set so as to further obtain the degree of blur for the image discriminated as the blur image. For example, if the correlation value of the minimum correlation set is equal to or greater than the predetermined threshold T1, the blur in the image is “no direction” (that is, the image is a normal image or a defocused image), and the blur direction is “no direction”. For the image, the degree of blur is obtained and an image with the degree of blur smaller than the predetermined threshold T2 is determined as a normal image so as not to be corrected. And determining only the parameters for isotropic correction to perform correction, and using the image with the correlation value of the minimum correlation set smaller than T1 as the blur image, the direction with the larger average edge width of the minimum correlation set is defined as the blur width. You may make it do. In addition, when setting the correction parameter for the blurred image, only the directionality correction parameter acting in the blur direction may be set as the correction parameter, or the degree of blur is further obtained, according to the degree of blur, etc. The correction parameter may be set by weighting and adding the directionality correction parameter and the directionality correction parameter.

  In addition, the image processing system of the present embodiment described above determines whether a digital photo image is a normal image or a blurred image in order to improve processing efficiency, and the degree of blurring only with respect to an image determined as a blurred image, The blur width is calculated and the parameters are set and corrected, but the normal image and the blurred image are not distinguished, and the blur degree, blur width, and blur degree are set for all the processing target images. The correction may be performed by calculating as blur information and setting a parameter based on the blur information. Since the normal image has a low degree of blurring, the normal image correction parameters set according to the degree of blurring are correction parameters that perform fine correction or parameters that are not corrected, and the blurring degree of the out-of-focus image is small. When the isotropic correction parameter and the directionality parameter are weighted and summed to obtain a correction parameter, the weight of the directionality correction parameter is small or zero.

  Further, in the above-described embodiment, the analysis means 20, 20c, and 20d obtain the degree of shake based on the above-described three elements based on FIG. There may be. For example, only two of the three elements described above may be used, and the number of elements is increased, for example, the largest correlation between the minimum correlation set and the minimum correlation set (in the case of the direction set shown in FIG. 4, 45 degrees). The degree of blur may be obtained by taking into account the difference in the correlation value with the misaligned direction set.

  Further, in the above-described embodiment, the analysis units 20, 20c, and 20d do not distinguish whether the blur is out of focus with respect to the image D that is a blur image, and if the image is determined as a blur image, The correction parameter is obtained by calculating the blur degree and weighting and adding the isotropic correction parameter and the directionality correction parameter with a weighting coefficient corresponding to the blur degree (in the image processing system of the present embodiment, the blur degree itself is used as the weighting coefficient). For example, a blur image with a degree of blur smaller than a predetermined threshold is set as a blur image, and only the isotropic correction parameter is set for the blur image to perform correction. Also good.

  In addition, the image processing system of the above-described embodiment assumes that it is not known whether the digital photographic image to be processed is a blurred image or a normal image. However, in a processing system that targets only a blurred image, for example, a processing system that corrects an image designated as a blurred image by a customer or an operator, a normal image and a blurred image are used. It is not always necessary to make this determination.

  In the image processing system of the above-described embodiment, processing from image reduction to output of a target image is performed in one apparatus. However, the present invention is not limited to such an aspect. The image processing system is divided into an analysis device and a processing device, and the analysis device performs processing up to the setting of the correction parameter and records the set correction parameter on the recording medium as an attached information of the image. Then, the processing apparatus may read out the image from the recording medium and perform correction using the correction parameter that is the information attached to the image.

  Further, in such an aspect, the analysis apparatus performs only the processing until obtaining the blur information, attaches the blur information to the image as the auxiliary information of the image, and records the blur information on the recording medium. The image may be read out, and correction may be performed by setting a correction parameter based on blur information that is attached information of the image.

  Furthermore, a correction execution instruction unit is provided to perform processing up to the acquisition of blur information or the setting of the correction parameter, but the correction is not executed until the correction execution instruction unit instructs to execute the correction. Correction may be executed after an instruction to do so.

  In addition, the image processing system of the above-described embodiment is intended for a digital photographic image acquired by a portable camera. However, the image processing method and apparatus of the present invention and the program therefor are digital photographic images acquired by a portable camera. The digital photograph obtained by reading any digital photograph image, for example, a digital photograph image obtained by a normal digital camera, or a photograph image printed on a printing medium such as photographic film or paper by a reading device such as a scanner. It can also be applied to images.

  In addition, in the image processing system according to the above-described embodiment, when a plurality of face regions are detected from the image, the edge detection unit selects an image having the largest number of pixels from the images of the plurality of face regions and performs edge processing. However, it is also possible to select an image of the face region located at the most central portion. Furthermore, an image instructed by the user may be selected from a plurality of detected face area images. In this case, the detected plurality of face area images are displayed as candidate images, for example, on a monitor or the like. A candidate image specified by an input means such as a mouse may be selected by the user for a plurality of displayed candidate images displayed on the apparatus, or a desired position may be set to the user before or after face detection. You may make it select the image of the face area nearest to the position pointed by the user among the several face area images detected by pointing.

  Of course, instead of selecting only one image from a plurality of face area images, it is also possible to select all or not all of the plurality of face areas, but one or more area images.

1 Recording medium 10a, 10b, 10c, 10d Reduction means 12, 12c, 12d Edge detection means 14, 14c, 14d Edge narrowing means 16, 16c, 16d Edge feature quantity acquisition means 20, 20c, 20d Analysis means 30, 30c, 30d Parameter setting means 40, 40c, 40d Correction means 45 Determination means 50, 50c Storage means 60 Output means 100, 100c, 100d Face extraction means 200, 200d Discriminating means 210 Reading means 210d Block dividing means C0 Features for identifying a face Amount D Digital photograph image D 'Target image H0 Reference data E Correction parameter K Blur degree L Blur width M1 One-dimensional correction mask M2 Two-dimensional correction mask N Blur degree Q Blur information S Edge feature amount α Certainty factor β Reduction rate

Claims (20)

In an image processing method for correcting blur in a digital photographic image,
Extracting the main subject area from the digital photographic image,
A parameter for correcting the blur using the image of the main subject area is set as a first parameter, and the blur is reduced using the entire digital photograph image or an image of an area other than the main subject area. Set the parameter for correction as the second parameter,
Determining the certainty that the digital photographic image is an image with a shallow depth of field;
Obtaining a parameter for correcting the blur by weighting and adding the first parameter and the second parameter so that the weight of the first parameter increases as the certainty factor increases.
An image processing method, wherein the digital photographic image is corrected using the parameter. Attached information including the F value of the lens of the imaging device that captured the digital photograph image is attached to the digital photograph image,
Read the attached information to obtain the F value,
The image processing method according to claim 1, wherein a process of determining whether the digital photographic image is an image having a shallow depth of field or determining the certainty factor is performed based on the F value. For the image of the main subject region extracted from the digital photographic image and the image of the region other than the main subject region, an edge is detected for each of a plurality of different directions, and the feature amount of the edge in each of the directions is acquired, Each of the blur information acquisition processing for acquiring blur information based on the edge feature amount is performed to obtain the blur information.
By comparing the blur information acquired from the image of the main subject area and the blur information acquired from the image of the area other than the main subject area, the digital photographic image is an image having a shallow depth of field. The image processing method according to claim 1, further comprising: determining whether or not or determining the certainty factor. Dividing the digital photographic image into a plurality of blocks;
A blur information acquisition process for detecting an edge in each of a plurality of different directions for each image of the block, acquiring a feature amount of the edge in each of the directions, and acquiring blur information based on the edge feature amount. Apply each to get the blur information of each,
By comparing the blur information acquired from the images of the respective blocks, it is possible to determine whether or not the digital photographic image is an image with a shallow depth of field or to obtain the certainty factor. The image processing method according to claim 1. An edge is detected for each of a plurality of different directions with respect to an image used when setting a parameter for correcting the blur, and the feature amount of the edge in each of the directions is acquired, and based on the edge feature amount Bokeh information acquisition processing to obtain the blur information
The image processing method according to claim 1, wherein the parameter is set based on the blur information. An image processing apparatus for correcting blur in a digital photographic image,
Main subject extraction means for extracting a main subject region from the digital photographic image;
A certainty factor calculating means for determining a certainty factor that the digital photographic image is an image having a shallow depth of field;
First parameter setting means for setting a parameter for correcting the blur using the image of the main subject area extracted by the main subject extracting means;
Second parameter setting means for setting a parameter for correcting the blur using the entire digital photographic image or an image of an area other than the main subject area;
The parameter set by the first parameter and the second parameter are set such that the higher the certainty factor obtained by the certainty factor calculating unit, the greater the weight of the parameter set by the first parameter setting unit. Integrated parameter acquisition means for acquiring an integrated parameter by adding together the parameters set by the parameter setting means;
An image processing apparatus comprising: a correcting unit that corrects the digital photographic image using the integrated parameter acquired by the integrated parameter acquiring unit. Attached information reading means for acquiring the F value by reading the attached information of the digital photographic image to which attached information including the F value of the lens of the imaging device that captured the digital photographic image is attached,
The discriminating means or the certainty factor calculating means performs processing for discriminating whether or not the digital photographic image is an image having a shallow depth of field based on the F value or obtaining the certainty factor. The image processing apparatus according to claim 6. For a target image, it comprises an edge information for each of a plurality of different directions, obtains the feature quantity of the edge in each of the directions, and comprises blur information acquisition means for acquiring blur information based on the edge feature quantity,
The determination unit or the certainty factor calculation unit compares the blur information acquired by the blur information acquisition unit using the image of the main subject region and the image of the region other than the main subject region, respectively, as the target image. The image processing apparatus according to claim 6, wherein the image processing apparatus determines whether the digital photographic image is an image having a shallow depth of field or obtains the certainty factor. Block dividing means for dividing the digital photographic image into a plurality of blocks;
A blur information acquisition unit that detects an edge for each of a plurality of different directions with respect to the target image, acquires a feature amount of the edge in each of the directions, and acquires blur information based on the edge feature amount;
The determination unit or the certainty factor calculation unit compares the blur information acquired by using the image of each block as the target image by the blur information acquisition unit, so that the digital photographic image is captured. The image processing apparatus according to claim 6, wherein the image processing apparatus determines whether or not the image has a shallow depth of field or obtains the certainty factor. The main subject extraction means is
Main subject candidate extraction means for extracting main subject region candidates from the digital photographic image;
The image according to any one of claims 6 to 9, further comprising main subject selection means for selecting a region of the main subject from the plurality of main subject candidates extracted by the main subject candidate extraction means. Processing equipment.   The image processing apparatus according to claim 10, wherein the main subject selection unit is configured to select a main subject candidate having the largest size from the plurality of main subject candidates as the main subject.   The image processing apparatus according to claim 10, wherein the main subject selecting unit is configured to select a main subject candidate closest to a central portion from the plurality of main subject candidates as the main subject.   The image processing apparatus according to claim 10, wherein the main subject selection unit selects a main subject candidate instructed by a user from the plurality of main subject candidates as the main subject. A reduction unit that obtains a reduced image by performing a reduction process on the image used by the parameter setting unit;
The parameter setting means sets the parameter using the reduced image;
In accordance with the size of the image of the main subject area used by the parameter setting unit, the reduction unit performs the reduction process so that the larger the size is, the higher the reduction strength for the image is. The image processing apparatus according to any one of claims 6 to 13. The parameter setting means detects an edge for each of a plurality of different directions in the image for an image used for setting a parameter for correcting the blur, and calculates a feature amount of the edge in each of the directions. A blur information acquisition means for acquiring and acquiring blur information based on the edge feature amount;
The image processing apparatus according to claim 6, wherein the parameter is set based on the blur information. A program for causing a computer to execute image processing for correcting blur in a digital photographic image,
A main subject extraction process for extracting a main subject region from the digital photographic image;
A certainty factor calculation process for determining the certainty factor that the digital photo image is an image having a shallow depth of field;
A first parameter setting process for setting a parameter for correcting the blur using an image of the main subject area extracted by the main subject extraction process;
A second parameter setting process for setting a parameter for correcting the blur using the entire digital photograph image or an image of an area other than the main subject area;
The parameter set by the first parameter and the second parameter are set such that the higher the certainty factor obtained by the certainty factor calculation process, the greater the weight of the parameter set by the first parameter setting process. An integrated parameter acquisition process for acquiring an integrated parameter by adding together the parameters set by the parameter setting process;
And a correction process for correcting the digital photographic image using the integrated parameter acquired by the integrated parameter acquisition process. The image processing includes an attached information reading process for obtaining the F value by reading the attached information of the digital photographic image to which attached information including an F value of a lens of an imaging device that has captured the digital photograph image is attached. And
The determination process or the certainty factor calculation process performs a determination as to whether or not the digital photographic image is an image with a shallow depth of field based on the F value, or a process for obtaining the certainty factor. 16. The program according to 16. The image processing detects an edge in a plurality of different directions with respect to the target image, acquires a feature amount of the edge in each of the directions, and acquires blur information based on the edge feature amount Have processing,
The discrimination process or the certainty factor calculation process compares the blur information acquired by using the blur information acquisition process as an image of the main subject region and an image of a region other than the main subject region, respectively. The program according to claim 16, wherein the process of determining whether the digital photographic image is an image with a shallow depth of field or determining the certainty factor is performed. The image processing is a block dividing process for dividing the digital photographic image into a plurality of blocks;
A blur information acquisition process for detecting an edge for each of a plurality of different directions with respect to the target image, acquiring a feature amount of the edge in each of the directions, and acquiring blur information based on the edge feature amount; ,
The determination process or the certainty factor calculation process compares the blur information acquired by using the image of each block as the target image by the blur information acquisition process, so that the digital photographic image is captured. The program according to claim 16, wherein the program determines whether or not the image has a shallow depth of field or obtains the certainty factor.   The parameter setting process detects an edge for each of a plurality of different directions in the image with respect to an image used for setting a parameter for correcting the blur, and calculates a feature amount of the edge in each of the directions. 20. The method according to claim 16, further comprising: a blur information acquisition process that acquires and acquires blur information based on the edge feature amount, and sets the parameter based on the acquired blur information. Program described in the section.

Images