JP2000149018A - Image processing method, and device and recording medium thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce influences that are caused by the erroneous extraction of an area which is estimated to be equivalent to the primary part of an image. SOLUTION: Plural face candidate areas, (e.g. areas A-D), which are estimated to be equivalent to the face of a person are extracted based on the data on an image to be processed, and overlap degree is calculated for a pair of face candidate areas which overlap each other. Then a weight mark is set to every face candidate area so that the weight mark of a face candidate area, which is overlapping another face candidate area is higher than a face candidate area which is not overlapping another face candidate area, and also the weight mark of a face candidate area becomes higher as its overlapping degree is increased with another face candidate area. An area equivalent to the face of a person is extracted from every face candidate area, by comparing the weight mark of every face candidate area with its threshold, and the weight mean value obtained by weighting the density value of every face candidate are according to its weight mark is calculated as the face area density.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing method, an image processing apparatus, and a recording medium, and more particularly to an image processing method for extracting an area estimated to be an area corresponding to a main part in an image, and the image processing method. The present invention relates to an image processing apparatus to which the present invention can be applied and a recording medium on which a program for causing a computer to execute the image processing method is recorded.

[0002]

2. Description of the Related Art The most noticeable part when viewing a photograph of a person is the face of the person. In this case, it is desirable to control the exposure so that the color and density of the person's face are appropriate. However, in order to realize this exposure control, an area corresponding to the person's face in the original image It is necessary to accurately detect the color and density of the image. In addition, for image data obtained by reading an image, various image processing developed for the purpose of improving the image quality of the image represented by the image data includes an area corresponding to the face of a person in the image or Some of them perform specific image processing (for example, local density correction and red-eye correction, etc.) on only a part of them. It is necessary to accurately detect the size.

[0003] For this reason, conventionally, various methods have been proposed for extracting a region presumed to correspond to a main part such as a face of a person in an image. For example, Japanese Patent Application Laid-Open No. 8-184925 discloses that a shape pattern specific to each part of a person present in an image (for example, a contour of a head, a contour of a face, an internal structure of a face, a contour of a body, etc.) (A shape pattern representing a human face) in accordance with the size and orientation of the detected shape pattern and the positional relationship between a predetermined portion of the person represented by the detected shape pattern and the person's face. In addition to setting an area that is estimated to be equivalent, another shape pattern different from the detected shape pattern is searched for, and the consistency of the previously set area as a person's face is determined. A method for extracting a face region for extracting a region (face region) is disclosed.

[0004]

However, none of the various methods proposed at present can always always extract a region corresponding to a main part such as a face of a person in an image with high accuracy. May be erroneously extracted. Then, when a region corresponding to the main part is erroneously extracted, there is a problem that a processing result by exposure control or image processing performed using the extraction result becomes inappropriate.

For example, when an image of a person photographed with a strobe light under backlighting conditions is erroneously extracted as a region corresponding to a person's face, a high-luminance background portion in the image is extracted. Assuming that the exposure amount when exposing and recording an image on a recording material is controlled based on the color and density, the recorded image has an inappropriate image quality in which an area corresponding to a person's face is blackened. Also, in the case where the specific image processing is performed only on the extracted region or a part thereof, the specific processing is performed only on the region different from the region corresponding to the face of the person in the image. However, the image quality of the image represented by the image data may be rather deteriorated.

The present invention has been made in view of the above facts, and has an image processing method, an image processing apparatus, and a recording method capable of reducing the influence of erroneous extraction of a region estimated to correspond to a main part in an image. The purpose is to get a medium.

[0007]

The inventor of the present application selects an arbitrary extraction method from various extraction methods for extracting a region presumed to correspond to a main part such as a human face in an image,
Applying the selected extraction method to extract a region corresponding to a main part from a large number of images may be performed by changing a processing condition of the extraction method (for example, a threshold value for determining whether or not the main part is used), or An experiment was repeated while changing the method itself. Then, as a result of comparing the regions extracted in each extraction, the extracted region that matches or substantially matches the main part in the image overlaps with another extracted region, and the degree of overlap with other extracted regions is small. It was found that it was almost large, and it was thought that the accuracy of the extracted region as a region corresponding to the main part in the image had a strong correlation with the degree of overlap with other extracted regions.

[0008] Further, among various extraction methods, due to differences in algorithms, even when a plurality of regions are extracted in one extraction process, each extraction region does not overlap. When a plurality of regions are extracted, there is a possibility that each of the extracted regions may overlap. (For example, in an algorithm for tracking an edge in an image and extracting a region corresponding to a main part, a plurality of regions different from each other may be used. When the edge tracking is performed with the starting point as a starting point, different areas that are partially overlapped but different areas may be extracted as an area corresponding to a main part.) However, even in the latter extraction method, a plurality of areas are extracted by one extraction process. The extracted region where the region is extracted and overlaps with other extracted regions often coincides or substantially coincides with the main part in the image, and as an area corresponding to the main part in the image of the extracted region. Accuracy, as described above, it was confirmed that there is a degree of overlapping strongly correlated with other extraction region.

[0009] Based on the above, the image processing method according to the first aspect of the present invention extracts, based on the image data, a candidate area estimated to correspond to a main part in an image represented by the image data, and extracts the extracted area. Among the candidate areas, for a candidate area overlapping with another candidate area on the image, the degree of overlap with the other candidate area is determined, and a main area in the image corresponding to the candidate area overlapping with the other candidate area is determined. The evaluation of the accuracy as a region corresponding to the portion is higher than the evaluation for a candidate region that does not overlap with other candidate regions, and further higher as the degree of overlap with other candidate regions increases,
The accuracy of the extracted candidate region as a region corresponding to the main part is evaluated.

According to the first aspect of the present invention, first, a candidate area estimated to correspond to a main part in an image represented by the image data is extracted based on the image data. The number of candidate regions actually extracted by the above extraction also depends on the content of the image represented by the image data, but in the present invention, it is desirable to extract a plurality of candidate regions. Improving the probability that a plurality of candidate regions are extracted is, for example, applying a different extraction method each time to extract a candidate region multiple times,
This can be realized by performing the extraction of the candidate region a plurality of times while changing the processing conditions by a single extraction method. Also,
When a plurality of candidate regions are extracted in one extraction process, and an extraction method that may cause overlap in each candidate region is applied, the candidate region may be extracted only once.

Next, among the extracted candidate areas, for a candidate area overlapping with another candidate area on the image, the degree of overlap with the other candidate area is determined. The calculation of the degree of overlap can be performed, for example, when a plurality of candidate regions are extracted and there is a candidate region overlapping with another candidate region.

The degree of overlap may be, for example, a distance between the pair of candidate regions based on the size of each of the pair of candidate regions overlapping with each other (each of the pair of candidate regions). May be the distance of the center of gravity position, or may be the distance between representative points representing the positions of the pair of candidate regions), and the magnitude of each of the pair of candidate regions. The value obtained by normalizing the area of the overlapping region based on the reference, and the size along the predetermined direction of the overlapping region based on the size along the predetermined direction of each of the pair of candidate regions. At least one of the normalized values can be used. Alternatively, an average value or a weighted average value of two or more values among the above values may be used. By using the values normalized as described above, it is possible to accurately digitize the degree of overlap of the candidate regions regardless of the size of the candidate regions. In addition,
When the size of a pair of candidate regions overlapping each other is different, a value representing a smaller degree of overlap among a pair of values obtained by using the sizes of both candidate regions as a reference is duplicated. It is preferable to use it as a degree.

According to the first aspect of the present invention, the evaluation of the accuracy of a candidate area overlapping with another candidate area as an area corresponding to a main part in an image is performed on a candidate area not overlapping with another candidate area. The accuracy of the extracted candidate region as a region corresponding to the main part is evaluated so that the extracted candidate region becomes higher than the above-described evaluation and further increases as the degree of overlap with another candidate region increases. As a result, of the extracted candidate regions, the candidate region confirmed to have a high probability of being a region corresponding to the main part in the image by the above-described experiment, that is, the candidate region overlapping with another candidate region and having another candidate region For a candidate region having a relatively high degree of overlap with the region, the evaluation of the certainty as a region corresponding to the main part in the image is made higher than for other candidate regions.

The evaluation of the extracted candidate area can be performed, for example, by setting an evaluation value indicating the level of the evaluation by a numerical value. For the setting of the evaluation value, for example, the evaluation value is set to 0 or a low value for a candidate area not overlapping with another candidate area, and for a pair of candidate areas overlapping each other,
The evaluation value is set so that the value increases as the degree of overlap increases. For candidate areas overlapping with a plurality of candidate areas, the evaluation set according to the degree of overlap for each overlapping candidate area. This can be done by integrating and setting the values.

As described above, the post-processing using the result of evaluating the accuracy of the extracted candidate region as a region corresponding to the main part in the image includes, for example, A process for selecting a candidate region having a high degree of certainty as a region corresponding to a main part from the candidate regions, and weighting an extracted candidate region (or a candidate region selected from the extracted candidate regions) as described in claim 4 However, even if the extracted candidate area includes a candidate area that is not a main part in the image due to erroneous extraction of the area estimated to be the main part in the image, Item 1
By performing the evaluation according to the invention of the above, in the post-processing,
Based on the result of the evaluation, it is possible to reduce the influence of the candidate area that is not the main part.

As an example, when selecting a candidate area having a high degree of certainty as an area corresponding to a main part, only candidate areas having a high degree of certainty evaluation as an area corresponding to a main part are selected from the extracted candidate areas. By selecting, it is possible to exclude a candidate area that is not actually a main part. In addition, for example, when weighting is performed on an extracted candidate region, a candidate region having a high degree of accuracy evaluation as a region corresponding to a main part is weighted with a high weight, and evaluation of accuracy as a region corresponding to a main part is performed. By reducing the weight of a candidate region having a low, it is possible to reduce the influence of a candidate region that is not actually a main part.

As described above, according to the first aspect of the present invention, based on the presence / absence of overlap with other candidate areas and the degree of overlap, the accuracy as an area corresponding to the main part is evaluated for the extracted candidate area. So, based on the results of this assessment,
The effect of erroneous extraction of a region estimated to correspond to a main part in an image can be reduced.

As a post-process using the evaluation result of the extracted candidate region, when a process of selecting a candidate region having high accuracy as a region corresponding to a main part from the extracted candidate region is performed, As described in, by comparing the evaluation value representing the result of evaluating the accuracy as a region corresponding to the main portion with respect to the extracted candidate region to a threshold, the extracted candidate region is determined as a region corresponding to the main portion from the extracted candidate region. It is preferable to select a candidate area with high accuracy. With this, even if the extracted candidate area includes a candidate area that is not a main part in practice, the candidate area can be excluded based on the evaluation value, and the candidate area is determined as an area corresponding to the main part. A candidate area with high accuracy can be selected with high accuracy by simple processing, and the criterion for selecting a candidate area can be corrected by changing the threshold.

Further, as a post-process utilizing the evaluation result for the extracted candidate area,
An extracted candidate area or a candidate area selected from the extracted candidate areas is weighted according to an evaluation value representing the result of evaluating the accuracy as an area corresponding to the main part, and the extracted candidate area or Processing for calculating a weighted average of the image feature amounts of the selected candidate area may be performed. Thus, even if the extracted candidate area includes a candidate area that is not a main part, the weight for the candidate area can be reduced based on the evaluation value. Can be accurately obtained by simple processing. In the invention of the fourth aspect, the selection of the candidate area when using the candidate area selected from the extracted candidate areas can be performed in the same manner as the invention of the third aspect.

Various processes can be considered as post-processing performed after evaluating the accuracy of the extracted candidate region as a region corresponding to the main part, but erroneous extraction of the candidate region and erroneous evaluation of the candidate region are possible. When there is, the degree of influence on the processing result of the post-processing greatly differs depending on the type of the post-processing. Therefore, according to the invention of claim 5, in the invention of claim 1, after the accuracy of the extracted candidate area as an area corresponding to the main part is evaluated, the extracted candidate area corresponds to the main part based on the evaluation result. When performing predetermined image processing including at least one of selection of a candidate region having high probability as a region to be performed and weighting of the extracted candidate region, the evaluation criterion is changed according to the type of the predetermined image processing. Alternatively, the selection or the weighting criterion is changed.

According to the fifth aspect of the present invention, for example, when the predetermined image processing is an image processing in which the processing result is greatly affected by erroneous extraction of a candidate area or erroneous evaluation of a candidate area, As described in claim 6, the criterion of the evaluation is changed so that the criterion of the evaluation of the accuracy as the area corresponding to the main part is strict, or the accuracy of the area corresponding to the main part is higher. The criterion of the selection is changed so that only the selected candidate region is selected, or the weighting criterion is set so that the weight for the candidate region evaluated as having low accuracy as the region corresponding to the main part is relatively small. Can be changed. In this way, in the post-processing, there is a possibility that a part of the candidate area corresponding to the main part among the extracted candidate areas may not be selected or the weight may be reduced. Even if the candidate area is erroneously evaluated, it is possible to prevent the processing result of the predetermined processing from being greatly affected.

For example, when the predetermined image processing is an image processing in which the influence of the processing result is small even if the candidate area is erroneously extracted or the candidate area is erroneously evaluated, the present invention is described in, for example, claim 7. Alternatively, the evaluation criterion may be changed so that the criterion of the evaluation of the accuracy as the area corresponding to the main part is loosened, or a candidate area evaluated as having low accuracy as the area corresponding to the main part may also be selected. In this manner, the selection criterion can be changed, or the weighting criterion can be changed such that the weight of the candidate area evaluated as having low accuracy as the area corresponding to the main part is relatively large. As a result, in the case where a candidate region is erroneously extracted or a candidate region is erroneously evaluated, in the post-processing, a candidate region which is not actually a main part may be selected or the weight may be increased. The effect on the result is small, and on the other hand, it is more likely that all candidate regions corresponding to the main part among the extracted candidate regions are selected, or that the candidate region corresponding to the main part is given a large weight. An appropriate processing result can be obtained.

As described above, according to the fifth aspect of the present invention, when the predetermined image processing is performed after the accuracy of the extracted candidate area as the area corresponding to the main part is evaluated, the predetermined image processing Depending on the type, change the evaluation criteria,
Alternatively, since the selection or weighting standard is changed, an appropriate processing result can always be obtained regardless of the type of predetermined image processing.

For example, when an image of a person photographed under backlighting conditions is recorded on a recording material, a high-luminance background portion in the image is defined as an area corresponding to the main part (in this case, the face of the person). If an erroneous determination is made and the recording density is controlled based on the density of the area, the quality of the recorded image becomes extremely low. Further, for example, when recording an image having a low luminance background portion on a recording material, the low luminance background portion corresponds to a main portion (such as a face of a person photographed without emitting a strobe under backlighting lighting conditions). Also, when the recording density is erroneously determined as the area to be controlled and the recording density is controlled based on the density of the area, the quality of the recorded image is extremely low.

Based on the above, the invention according to claim 8 is based on the invention according to claim 1, wherein the density of the candidate area is higher than the first predetermined value or lower than the second predetermined value. It is characterized in that the evaluation of the accuracy as a region corresponding to the main part is reduced, or the selection criterion for selecting a candidate region with high accuracy as the region corresponding to the main part is raised.

According to the invention of claim 8, the density in the candidate area is higher than the first predetermined value, or the density in the candidate area lower than the second predetermined value, that is, the density in the candidate area is extremely high. For extremely low candidate regions, the evaluation of accuracy as a region corresponding to the main part is reduced or the criterion of selection when selecting a candidate region with high accuracy as the region corresponding to the main part is increased, In the case of lowering the evaluation of the accuracy as a region corresponding to the main part, a region where the density is extremely high or extremely low and is erroneously extracted as a candidate region is determined as a region corresponding to the main part in post-processing. It is possible to prevent the region from being selected as a candidate region with high accuracy and from being given a large weight.

If a process including the selection of a candidate region having a high degree of certainty as a region corresponding to the main part is performed as the post-processing, the criterion for selecting a candidate region having a high degree of certainty as a region corresponding to the main part is determined. By increasing the value, it is possible to prevent a region in which the density is extremely high or extremely low and is erroneously extracted as a candidate region from being selected as a candidate region having a high probability as a region corresponding to a main part. .
Therefore, even when a region having an extremely high or extremely low density is erroneously extracted as a candidate region, it is possible to prevent the processing result of the post-processing from becoming inappropriate.

According to a ninth aspect of the present invention, there is provided an image processing apparatus, comprising: extracting means for extracting a candidate area estimated to correspond to a main part in an image represented by the image data, based on the image data; Calculating means for calculating a degree of overlap with the other candidate area for a candidate area overlapping with another candidate area on the image among the candidate areas; The evaluation of the accuracy as a region corresponding to the main part in becomes higher than the above evaluation for a candidate region that does not overlap with other candidate regions,
Evaluation means for evaluating the accuracy of the extracted candidate region as a region corresponding to the main part, so that the degree of overlap with another candidate region is higher as the degree of overlap with the other candidate region is higher,
Therefore, similarly to the first aspect of the present invention, it is possible to reduce the influence of erroneous extraction of a region estimated to correspond to a main part in an image.

[0029] The recording medium according to claim 10 is
A first step of extracting, based on the image data, a candidate area estimated to correspond to a main part in the image represented by the image data, of the extracted candidate areas, overlapping with another candidate area on the image; A second step of obtaining the degree of overlap with the other candidate area for the candidate area, and evaluating the likelihood of the candidate area overlapping with the other candidate area as the area corresponding to the main part in the image. The main part of the extracted candidate area is higher than the evaluation for a candidate area not overlapping with another candidate area, and further higher as the degree of overlap with another candidate area is higher. And a program for causing a computer to execute a process including a third step of evaluating the accuracy as an area corresponding to.

The recording medium according to the tenth aspect of the present invention executes the processing including the first to third steps, that is, the processing according to the image processing method according to the first aspect of the present invention on a computer. Since the program for causing the computer to read the program is recorded on the recording medium, the computer reads and executes the program recorded on the recording medium. The effect of erroneous region extraction can be reduced.

[0031]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 shows an image processing system 10 to which the present invention is applied. The image processing system 10 includes a scanner 12, an image processing device 14, and a printer 16 connected in series.

The scanner 12 captures a film image recorded on a photographic photosensitive material (hereinafter simply referred to as a photographic film) such as a photographic film (for example, a negative film or a reversal film), and visualizes the image by developing the object. And outputs image data obtained by the reading. The light emitted from the light source 20 and the light amount unevenness of which is reduced by the light diffusion box 22 is set on the film carrier 24. Irradiated on a photographic film 26 such as a negative film or a reversal film, and transmitted through the photographic film 26, is received by a CCD sensor 30 (which may be an area sensor or a line sensor) via a lens 28. It is configured to be imaged on a plane.

The film carrier 24 is a photographic film 2
6 is the light source 20 where the film image is recorded.
The photographic film 26 is conveyed so as to be sequentially positioned on the optical axis of the light emitted from the camera. Thus, the film images recorded on the photographic film 26 are sequentially read by the CCD sensor 30, and the CCD sensor 30 outputs a signal corresponding to the film image. The signal output from the CCD sensor 30 is converted into digital image data by the A / D converter 32 and input to the image processing device 14.

The line scanner correction unit 36 of the image processing apparatus 14 performs a dark scan for subtracting the dark output level of the cell corresponding to each pixel from the input scan data (R, G, B data input from the scanner 12). Density conversion to logarithmically convert the corrected and dark corrected data to data representing the density value,
Shading correction for correcting data after density conversion according to unevenness in the amount of light illuminating the photographic film 26, and data of cells (so-called defective pixels) from which data corresponding to the amount of incident light is not output among the data subjected to the shading correction Are interpolated from the data of the surrounding pixels, and each process of the defective pixel correction newly generated is sequentially performed. Line scanner correction unit 36
Is connected to the input terminal of the I / O controller 38, and the data subjected to each of the above processes in the line scanner correction unit 36 is input to the I / O controller 38 as scan data.

The input terminal of the I / O controller 38 is also connected to the data output terminal of the image processor 40, from which image data subjected to image processing (details will be described later) is input. Also, I / O
The input terminal of the controller 38 is the personal computer 4
2 is also connected. The personal computer 42 has an expansion slot (not shown). The expansion slot includes a driver (not shown) for reading / writing data from / to an information storage medium such as a memory card or a CD-R, and other components. A communication control device for communicating with the information processing device is connected. When file image data is input from outside via the expansion slot, the input file image data is input to the I / O controller 38.

An output terminal of the I / O controller 38 is connected to a data input terminal of the image processor 40, an auto setup engine 44, and a personal computer 42, respectively.
6 is connected. The I / O controller 38 selectively outputs the input image data to each of the devices connected to the output terminal.

In the present embodiment, the scanner 12 performs two readings at different resolutions on each film image recorded on the photographic film 26. In the first reading at a relatively low resolution (hereinafter, referred to as pre-scan), even when the density of the film image is very low (for example, a negative image with exposure under in a negative film),
The reading conditions (R, R of light to be applied to the photographic film 26) determined by the CCD sensor 30 so as not to cause saturation of the accumulated charges.
The reading of the entire surface of the photographic film 26 is performed using the light amounts of the respective wavelength ranges of G and B (the charge accumulation time of the CCD sensor 30). Data (prescan data) obtained by this prescan is input from the I / O controller 38 to the auto setup engine 44.

The auto setup engine 44 includes a CP
U46, RAM48 (for example, DRAM), ROM50
(For example, a rewritable ROM) and an input / output port 52, which are connected to each other via a bus. The auto setup engine 44
The frame position of the film image is determined based on the pre-scan data input from the I / O controller 38, and data (pre-scan image data) corresponding to the film image recording area on the photographic film 26 is extracted. Further, based on the pre-scanned image data, the size of the film image is determined, and image feature amounts such as density are calculated.
Determines the reading conditions when re-reading at a relatively high resolution (hereinafter, referred to as fine scan). Then, the frame position and the reading conditions are output to the scanner 12.

The auto setup engine 44
Is a main part in the film image based on the prescanned image data (for example, an area corresponding to a human face (face area))
Of image features including the extraction of image data, and automatically determine processing conditions for various types of image processing on image data (fine scan image data) obtained by the scanner 12 performing fine scan (setup calculation) ), The determined processing conditions are applied to the image processor 4
Output to 0.

A display, a keyboard, and a mouse are connected to the personal computer 42 (all are not shown). The personal computer 42 captures the pre-scan image data from the auto setup engine 44, captures the processing conditions of the image processing determined by the auto setup engine 44, and images the fine scan image data based on the captured processing conditions. Image processing equivalent to the image processing performed by the processor 40 is performed on the prescanned image data to generate simulation image data.

The generated simulation image data is converted into a signal for displaying an image on a display, and a simulation image is displayed on the display based on the signal. In addition, the operator checks the displayed simulation image for image quality and the like,
When the information for instructing the correction of the processing conditions is input via the keyboard as the test result, the information is output to the auto setup engine 44. As a result, the auto setup engine 44 performs processing such as recalculation of the processing conditions of the image processing.

On the other hand, image data (fine scan image data) input to the I / O controller 38 by performing fine scan on the film image by the scanner 12 is input to the image processor 40 from the I / O controller 38. Is done. Image processor 4
0 denotes color / density correction processing including gradation conversion and color conversion, pixel density conversion processing, hypertone processing for compressing the gradation of an ultra-low frequency luminance component of an image, and hyper-sharpness for enhancing sharpness while suppressing graininess. An image processing circuit that performs various image processing such as processing is provided, and performs various image processing on input image data according to processing conditions determined and notified for each image by the auto setup engine 44. .

The image processing that can be executed by the image processor 40 includes, in addition to the above, for example, sharpness correction or soft focus processing on the entire image or a part thereof (for example, a region corresponding to a human face), or intentionally changing the image tone. Image processing (image processing to finish the output image in monotone, image processing to finish the output image in portrait tone,
Image processing for finishing the output image in sepia tone, etc.), image processing for processing the image (for example, image processing for finishing a person existing in the original image to be thin on the main image, image processing for correcting red eye, etc.), , LF (film with lens), LF aberration correction processing for correcting distortion of the LF lens, geometric distortion of the image due to chromatic aberration of magnification, and color shift, and the periphery of the LF lens LF lens such as peripheral dimming correction processing for correcting a decrease in the brightness of the peripheral portion of an image due to light reduction, and defocus correction processing for correcting a decrease in image sharpness due to the characteristics of the LF lens And various types of LF aberration correction processing for correcting the deterioration of the image quality of the output image due to the above characteristics.

When the image data processed by the image processor 40 is used for recording an image on photographic paper, the image data processed by the image processor 40 is transmitted from the I / O controller 38 to the I / O controller 38. / F circuit 54
Is output to the printer 16 as image data for recording via the. When the image data after the image processing is output to the outside as an image file, the image data is output from the I / O controller 38 to the personal computer 42. As a result, the personal computer 42 outputs the image data input from the I / O controller 38 for output to the outside to the outside (the driver, the communication control device, or the like) as an image file via the expansion slot.

The printer 16 has image memories 58, R,
G and B laser light sources 60 and a laser driver 62 for controlling the operation of the laser light sources 60 are provided. The image data for recording input from the image processing device 14 is stored in the image memory 58.
Is read out after being stored once, and is used for modulating the R, G, B laser light emitted from the laser light source 60.
The laser light emitted from the laser light source 60 is scanned on a printing paper 68 via a polygon mirror 64 and an fθ lens 66, and an image is exposed and recorded on the printing paper 68. The photographic paper 68 on which the image has been exposed and recorded is sent to the processor section 18 and subjected to color development, bleach-fixing, washing and drying.
Thus, the image recorded on the printing paper 68 by exposure is visualized.

Next, as an operation of the present embodiment, the scanner 1
A description will now be given of the face area extraction / density calculation processing performed after the pre-scan data is input to the image processing apparatus 14 from the image processing unit 2 and the auto setup engine 44 performs processing such as clipping of image data from the pre-scan data.

This face area extraction / density calculation processing is processing to which the image processing method according to the present invention is applied, and is realized by executing a face area extraction / density correction program by the CPU 46 of the auto setup engine 44. Is done. The face area extraction / density correction program includes a program for causing the CPU 46 to execute other processing.
Initially, it is stored in the information storage medium 72 (see FIG. 1). Although the information storage medium 72 is shown as a floppy disk in FIG. 1, it may be constituted by a CD-ROM, a memory card, or the like. An information storage medium 72 is stored in an information reading device (not shown) connected to the personal computer 42.
Is loaded, and an instruction to transfer (install) a program from the information storage medium 72 to the image processing apparatus 14 is issued.
A face area extraction / density correction program and the like are read from the information storage medium 72 by the information reading device, and stored in the rewritable ROM 50.

When the timing to execute the face area extraction / density correction processing comes, the face area extraction / density correction program is read from the ROM 50, and the face area extraction / density correction program is executed.
The density correction program is executed by the CPU 46.
Thereby, the auto setup engine 44 functions as the image processing device according to the present invention. As described above, the information storage medium 72 storing the face area extraction / density correction program and the like corresponds to the recording medium according to claim 10.

Hereinafter, the face area extraction / density correction processing will be described with reference to the flowchart of FIG. In step 100, based on the image data to be processed, a region (a face candidate region extraction process for extracting a face candidate region that is assumed to correspond to a face of a person in the image is performed as a main part in the image represented by the image data. As an extraction method for performing the face candidate area extraction processing, a face candidate area extraction method in which an area estimated to correspond to a face of a person in an image is determined and the area is extracted as a face candidate area, There is a background part removal method that determines an area (background area) estimated to correspond to the middle background and extracts an area other than the background area as a face candidate area, and the like. The face candidate region extraction processing can be performed by employing at least one of the face candidate extraction method and the background removal method.

[Example 1 of Face Candidate Area Extraction Method] Based on data (image data) obtained by dividing an image into a number of measurement points and decomposing each measurement point into R, G, and B colors. Then, it is determined whether or not each measurement point is included in the range of the skin color on the color coordinates, and an area in which a cluster (group) of the measurement points determined to be within the range of the skin color is extracted as a face candidate area. (JP-A-52-156624, JP-A-52-156625)
JP-A-53-12330, JP-A-53-145620, JP-A-53-145621, JP-A-53-145622, etc.).

[Example 2 of Face Candidate Area Extraction Method] Based on the image data, a histogram for hue values (and saturation values) is obtained, and the obtained histogram is divided for each mountain.
It is determined which of the divided mountains each measurement point belongs to, and each measurement point is divided into groups corresponding to the divided mountains, and the image is divided into a plurality of regions for each group, and a person is selected from the plurality of regions. Is estimated, and the estimated area is extracted as a face candidate area (see Japanese Patent Application Laid-Open No. 4-346333).

[Example 3 of Face Candidate Area Extraction Method] Based on the image data, a shape pattern specific to each part of the person existing in the image (eg, the outline of the head, the outline of the face, the internal structure of the face, the torso) Any one of the shape patterns representing the contours of the person, and the size and direction of the detected shape pattern and the positional relationship between the predetermined portion of the person represented by the detected shape pattern and the face of the person. An area estimated to correspond to the face is set. Further, another shape pattern different from the detected shape pattern is searched for, the consistency of the previously set region as a human face is determined, and a face candidate region is extracted (Japanese Patent Laid-Open Nos. 8-122944 and 8-122944). 8-183925, JP-A-9-13
No. 8471).

[Example 1 of Background Removal Method] Based on the image data, each measurement point is a specific color (for example, sky or sea blue, lawn or tree green, etc.) that clearly belongs to the background on the color coordinates. ) Is determined to be included in the range, the area in which the cluster (group) of the measurement points determined to be within the specific color range is determined to be the background area and removed, and the remaining area is determined. It is extracted as a non-background area (an area that is likely to include an area corresponding to a human face: this is also a face candidate area of the present invention).

[Example 2 of Background Removal Method] Based on the image data, the image is divided into a plurality of regions in the same manner as in Example 2 of the main part extraction method, and then each region corresponds to the background. (The ratio of the linear portion included in the contour, the degree of line symmetry, the number of irregularities, the contact ratio with the outer edge of the image, the density contrast in the area, the presence or absence of a density change pattern in the area, etc.) It is determined whether or not each area is a background area based on the feature amount, the area determined as the background part is removed, and the remaining area is extracted as a non-background area (face candidate area)
8-122944 and JP-A-8-183925).

The above extraction method is merely an example,
It goes without saying that any method can be applied as long as it is an extraction method for extracting an area estimated to correspond to a human face from an image. In step 100, the face candidate region extraction process may be performed a plurality of times by applying a plurality of types of extraction methods, or the face candidate region extraction process may be performed a plurality of times by changing the processing conditions by a single type of extraction method. You may do so.

By the face candidate area extracting process in step 100, a plurality of face candidate areas (face candidate areas A to D in FIG. 3) are extracted from the image to be processed, as shown in FIG. 3, for example. Step 100 corresponds to the extracting means described in claim 9. In the following processing, the processing is performed on the face candidate area extracted by the face candidate area extraction processing, but the extracted face candidate area itself may be the processing target, or the extracted face candidate area may be circumscribed. A region having a fixed shape (for example, a circular region or a rectangular region) may be set as a processing target.

In step 102, all the face candidate areas extracted by the face candidate area extraction processing in step 100 are fetched, and in the next step 104, the face candidates overlapping on the image are included in the fetched face candidate areas. It is determined whether or not there is a candidate area (for example, see face candidate areas A, B, and C shown in FIG. 3). If the determination is affirmative, the process proceeds to step 106, where a single face candidate area pair overlapping each other (face candidate areas A and B or face candidate areas A and C or face candidate area B, C) are extracted as processing targets, and in the next step 108, the degree of overlap x of the face candidate area is calculated based on the size of the extracted face candidate area pair. Step 108, together with step 106, corresponds to the calculating means according to the ninth aspect.

For example, as shown in FIG. 4A, the face candidate area A and the face candidate area B overlap, and the area of the face candidate area A is S _A and the area of the face candidate area B is Assuming that S _B and the area of the overlapping region are S _AB , for example, the calculation can be performed according to the following equation (1).

X = min (S_AB/ S_A, S_AB/ S_B) (1) In equation (1), min () selects the minimum value of the numerical value in parentheses.
Is an operator indicating that the_AB/ S_AIs the face candidate area
Area S of area A_AArea S of the overlap region based on _ABIs regular
Value, S_AB/ S_BIs the area S of the face candidate area B_BBased on
And the area S of the overlap region_ABIs a normalized value of
Equation (1) is a formula for each of the overlapping face candidate area pairs A and B.
Normalize the area of the overlapping area based on the size (area)
The smaller one of the values is set as the degree of overlap x. this
Thus, the degree of overlap x is based on the area of the face candidate area.
By using the normalized value of the area of the overlapping area
Accurately quantify the degree of overlap regardless of the size of the complement area
Can be.

In the equation (1), S _AB / S _A , S
_{Since the} value obtained by _AB / S _B increases as the degree of overlap increases (as the value of area S _AB increases), in equation (1), the smaller of the two values (the smaller overlap) Is used as the degree of overlap x. Thus, for example, even when one face candidate area is included in the other face candidate area, it is possible to prevent the value of the degree of overlap x from becoming extremely large.

The degree of overlap x is calculated in accordance with the following equation (2) or (3) when the distance between the center of gravity of the face candidate area A and the center of gravity of the face candidate area B is L _AB. You may.

X = max (L_AB/ √ (S_A), L_AB/ √ (S_B))… (2) x = max (L_AB ^Two/ S_A, L_AB ^Two/ S_B) ... (3) In equations (2) and (3), max () is the value in parentheses
Is an operator indicating that the maximum value of_AB/ √
(S_A) And L_AB ^Two/ S_AIs the area S of the face candidate area A_ATo
Distance L between center of gravity_ABIs the normalized value of L_AB/ √
(S_B) And L_AB ^Two/ S_BIs the area S of the face candidate area B_BTo
Distance L between center of gravity_ABIs a normalized value of
Equations (2) and (3) are used to calculate overlapping face candidate area pairs.
Distance between centers of gravity based on the size (area) of each of A and B
The larger value of the normalized values of
I have. Thus, as the degree of overlap x, the area of the face candidate area
Using the normalized value of the distance between the centers of gravity based on
Therefore, regardless of the size of the face candidate area, the degree of overlap can be accurately calculated.
Can be quantified.

In the equations (2) and (3), L _AB
/ √ (S _A ), L _AB / √ (S _B ), or L _AB ² /
The values obtained by S _A and L _AB ² / S _B decrease as the degree of overlap increases (as the value of the distance L _AB between the centers of gravity decreases). The larger one of the values (a value representing a smaller degree of overlap) is used as the degree of overlap x. Thus, for example, when one face candidate area is included in the other face candidate area, it is possible to prevent the value of the degree of overlap x from becoming extremely small.

Equations (2) and (3) are for the case where the distance between the centers of gravity of a pair of face candidate areas is applied as the "distance of the candidate area" according to claim 2, but is not limited thereto. Instead, the distance between the representative points representing the positions of the pair of face candidate areas is applied as the “distance of the candidate area”, and the degree of overlap x may be obtained using a value obtained by normalizing the distance. Good. The representative point may be, for example, a point corresponding to the center of a figure of a predetermined shape (for example, an N-corner shape (N ≧ 3) or a circular shape (may be a perfect circle or an ellipse)) circumscribing or inscribing the face candidate area, or A point determined by a set specific calculation method can be adopted.

The points obtained by the specific calculation method set in advance include, for example, a specific vertex of an N-corner region (for example, a rectangular region) circumscribing the face candidate region, and a plurality of points of the N-corner region. A point corresponding to the center of gravity of the vertex, the length along the major axis direction of the elliptical region from the upper end (upper vertical direction of the face) along the major axis direction of the elliptical region circumscribing the face candidate region And a point corresponding to a position separated by a length corresponding to X%.

Further, the degree of overlap x is L _A , the length of the face candidate area _A along a predetermined direction (for example, the long side direction or the short side direction of the image), and the length of the face candidate area B along the predetermined direction. Is L _B , and the length of the overlapping region along a predetermined direction is L _AB ′,
The calculation may be performed according to the following equation (4).

[0067] _{x = min (L AB '/} L A, L AB' / L B) ... (4) Thus, as the degree of overlapping x, overlapping relative to the length along the predetermined direction of the face candidate region By using a value obtained by normalizing the length of the area along the predetermined direction, the degree of overlap can be accurately quantified regardless of the size of the face candidate area.

The expressions (1) and (2) (or (3)
An average value or a weighted average value of the duplication degrees x obtained using two or more of the expressions (4) and (4) may be used as the final duplication degree x. However, since the degree of overlap x obtained by the equations (2) and (3) and the degree of overlap x obtained by the equations (1) and (4) are different in the direction in which the value changes with respect to the change in the degree of overlap, the average is obtained. When calculating a value or a weighted average value, it is necessary to convert one of the values so that the change direction of the value is uniform.

In addition, in the calculation of the weighted average value, with respect to the weight given to the degree of duplication obtained by each arithmetic expression, a face area is extracted in advance using the degree of duplication obtained by each arithmetic expression, and the actual face area is extracted. Can be evaluated and determined for each arithmetic expression. For example, the inventor of the present application has found that, for Expressions (1) and (2) (or Expression (3)), the degree of coincidence with the actual face area is higher when the degree of overlap determined by Expression (1) is used. It has been confirmed by experiments that there are many, and when the weighted average value of the degree of duplication obtained by equation (1) and the degree of duplication obtained by equation (2) (or equation (3)) is the degree of duplication x,
It is preferable that the weight given to the degree of duplication obtained by the equation (1) be larger than the weight given to the degree of duplication obtained by the equation (2) (or the equation (3)).

Further, instead of using the weighted average value of the duplication degree obtained by each arithmetic expression as the duplication degree x, the duplication degree obtained by each arithmetic expression is converted into a weight score P (details will be described later). A weighted average value of the weight points P corresponding to the arithmetic expression may be set as the final weight points.

In the next step 110, the number of weight points P to be assigned to the face candidate area pair to be processed extracted in step 106 is determined based on the degree of duplication x calculated as described above. The determination of the number of weight points P is based on the previous step 10
In FIG. 8, when the degree of overlap x is calculated such that the value increases as the degree of overlap of the face candidate areas increases, the value of the degree of overlap x increases as shown in FIG. 4B as an example. Is used, and a map in which the conversion characteristic is determined so that the number of weight points P becomes higher in accordance with
Is converted into the number of weight points P. On the other hand, in step 108, when the degree of overlap x is calculated so that the value decreases as the degree of overlap of the face candidate areas increases, as shown in FIG. The weighting point P can be determined by using a map in which the conversion characteristic is determined so that the weighting point P decreases as the value increases, and converting the degree of overlap x into the weighting point P using the map. .

The conversion characteristics shown in FIGS. 4B and 4C are merely examples. For example, in the conversion characteristics shown in FIGS. There is an insensitive area where does not change, but conversion characteristics without such an insensitive area may be used. Further, a conversion characteristic in which the number of weight points P changes non-linearly with respect to the change in the degree of overlap x may be used.
May be a conversion characteristic that increases.

When the number of weight points P is determined as described above, the number of weight points P determined previously is given to the face candidate area pair to be processed (step 112). It is determined whether or not the processing of steps 106 to 112 has been performed on all the face candidate area pairs that have been executed. If the determination is negative, step 106
And the processing of steps 106 to 112 is repeated for the other face candidate area pairs overlapping each other. As a result, the weight point number P is assigned to all the face candidate area pairs overlapping each other.

If the determination in step 114 is affirmative, the process proceeds to step 116, in which the number of weight points P assigned to each face candidate area is integrated for each face candidate area, and the integrated value is weighted for each face candidate area. Set as the score P. For example, when the face candidate areas A to D shown in FIG. 3 are extracted from the image to be processed, the weight score P is not set for the face candidate area D because it does not overlap with other face candidate areas (the weight score P _D =
0) Since the face candidate areas A to C overlap with the two face candidate areas, respectively, two weight points P are added and set. Among them, face candidate region C overlapping degree is small with other face candidate region is weighted score P _C is the smallest, a large face candidate region B (Fig than overlapping degree face candidate region A of the face candidate region C the actual degree of coincidence between the face region is weighted number P _B of the highest face candidate region) it becomes a maximum at 3.

The number of weight points P set for each face candidate area in step 116 corresponds to an evaluation value representing the evaluation of the accuracy as an area corresponding to a person's face. And the weight score P for the face candidate area overlapping with the face candidate area becomes higher than the weight score for the face candidate area not overlapping with the other face candidate area, and the degree of overlap with the other face candidate area increases. And the value becomes even higher. Step 116, together with steps 110 and 112, corresponds to the evaluation means of the present invention.

By the way, if there is a face candidate area with extremely high or extremely low density in the face candidate area extracted in step 100, and the face candidate area is not actually a face area, There is a high possibility that the density of the face candidate area adversely affects the processing result of the post-processing performed after the face area is extracted. Therefore, in the next step 118, it is determined whether or not there is a face candidate area in which the density M in the area is extremely high or extremely low in the face candidate area extracted in step 100.

The density M is the average density (absolute value) in the area.
Or a relative density with respect to the entire screen average density of the image to be processed.
Is a threshold value for determining whether or not the density M is an extremely high value.
This can be performed by comparing TH _HIGH with a threshold value TH _LOW for determining whether the density M is an extremely low value. The threshold value TH _HIGH corresponds to a first predetermined value described in claim 8, and the threshold value TH _LOW corresponds to a second predetermined value described in claim 8.

If all face candidate areas have density M <threshold TH _HIGH and density M> threshold TH _LOW , it can be determined that there is no face candidate area with extremely high or extremely low density. If the determination at step 118 is negative, step 1
Move to 24. On the other hand, density M> threshold TH _HIGH or density M
<If there is a face candidate area that satisfies the threshold value TH _LOW , the density of the face candidate area can be determined to be extremely high or extremely low. Therefore, the determination in step 118 is affirmed, and the process proceeds to step 120. After that, the correction is made so that the weight point number P of the face candidate area determined to have the density M is extremely high or extremely low is decreased, and then the process proceeds to step 124.

The correction of the number of weight points P is performed, for example, by referring to FIG.
This can be performed using the map shown in FIG. This map has a conversion characteristic represented by a straight line that passes through the origin and has a slope smaller than 1 on coordinates with the initial (initial) weight points on the horizontal axis and the corrected weight points on the vertical axis. are doing. By converting (decreasing downward) the number of weight points P using the map as described above, even if a face candidate area whose density M is extremely high or extremely low is not actually a face area, the face candidate The extent of the adverse effect of the region on post-processing can be reduced. Steps 118 and 120 correspond to the eighth aspect of the present invention.

On the other hand, if none of the face candidate areas extracted in step 100 overlap with each other on the image, the determination in step 104 is denied, and the routine proceeds to step 122. Then, a fixed value is set as the number of weight points P of each face candidate area, and the process proceeds to step 124.

In step 124, the number of weight points P of each face candidate area is compared with a threshold value TH _F for face area determination, and a face candidate area having a weight point number P equal to or greater than the threshold value TH _F is extracted (selected) as a face area. . When the determination in step 104 is affirmative (when there are face candidate areas overlapping each other), the weight score for the face candidate area overlapping with another face candidate area and having a high degree of overlap is increased. So
A face candidate area having a high probability of being a face area is extracted as a face area. Step 124 corresponds to the third aspect of the present invention.

In the next step 126, the following (5)
The face area density M of the image to be processed according to the equation (6)
The face is calculated, and the face area extraction / density calculation processing ends.

[0083]

(Equation 1)

Here, i is a code for identifying each face candidate area, N is the total number of face candidate areas, Mi is the density of the face candidate area i, Pi is the number of weights of the face candidate area i, and Si is the face candidate area. i is the area.

As is clear from the equations (5) and (6), the face area density Mface is a weighted average value of the density M of each face candidate area. Each face candidate area is weighted on the basis of the weighting points P and the area S in Equation (6). If the determination in step 104 is affirmative (if there are overlapping face candidate areas),
Since the weight points for the face candidate areas that overlap with other face candidate areas and have a high degree of overlap are increased, the face area density Mface is larger than the average value of the density M of each face candidate area.
The value is close to the density of the actual face area.

By the way, it is empirically known that a face candidate area having a relatively small area has a relatively low probability of being a face area. However, in the above equation (5), the area of each face candidate area is considered. Since the face area density Mface is calculated without performing the calculation, a face candidate area having a relatively small area and not actually a face area is included in the face candidate area for which the face area density Mface is calculated. In addition, the density M of the face candidate area is the face area density Mface
Has a large effect on For this reason, when calculating the face area density Mface using the equation (5) (the equation (6) may be used), for example, the area S is equal to or smaller than a predetermined value, or the area S is smaller than another face candidate area. A relatively small face candidate area is excluded from the calculation of the face area density Mface, or
The face area density Mface may be calculated by reducing the value of the weight point number P. As a result, it is possible to reduce the influence of the density M of the face candidate area having a relatively small area that is not actually a face area on the face area density Mface.

Further, in the above equation (6), since the face area density Mface is calculated so that the weight of the face candidate area having a large area S becomes relatively large, the face to be calculated of the face area density Mface is calculated. If the candidate area includes a face candidate area having a relatively large area and not actually a face area, the density M of the face candidate area has a large effect on the face area density Mface. Therefore, when calculating the face area density Mface using the equation (6), for example, an upper limit value _Smax is set for the value of the area S, and for a face candidate area in which the value of the area S is larger than the upper limit value _{Smax. May} replace (clipping) the value of the area S with the upper limit value _Smax or reduce the value of the number of weight points P. As a result, it is possible to reduce the influence of the density M of the face candidate area having a relatively large area that is not actually a face area on the face area density Mface.
It should be noted that decreasing the value of the weight score P of the face candidate region in which the value of the area S is larger than the upper limit value _Smax as described above is also applied to the case where the face region density Mface is calculated using the equation (5). It is possible.

Step 126 corresponds to the fourth aspect of the present invention. Further, the face area density Mface corresponds to the image feature amount of the main part according to claim 4. However, the image feature amount of the main part is not limited to the density, and various known image feature amounts may be used. It goes without saying that it is applicable.

When the above-described face area extraction / density calculation processing is performed, the auto setup engine 44 further calculates the processing conditions of various image processing executed by the image processor 40. Is used for calculating the processing conditions of some image processing. For example, the face area extracted in the previous step 124 is used for image processing (for example, sharpness correction or red-eye correction for the face area) performed on the face area or only a part thereof by the image processor 40. Processing conditions are set so that the image processing is performed only on the face area. The face area density Mface calculated in step 126 is used, for example, for image processing (for example, color / density correction) for the entire image executed by the image processor 40. Processing conditions such as a density correction condition are calculated so as to obtain a predetermined density.

As described above, since the extraction of the face area and the face area density Mface are performed using the number of weight points set based on the degree of overlap x, the face candidate extracted by the face candidate area extraction processing is used. Even if a face candidate area that is not actually a face area is mixed in the area due to erroneous extraction, the probability that a face candidate area that is not actually a face area is extracted as a face area is greatly reduced, In practice, it is possible to prevent the face area density from largely changing depending on the density of the face candidate area that is not a face area. Therefore, appropriate processing conditions are obtained for each image processing in which the processing conditions are calculated using the face area extraction result or the face area density Mface, and the image processing is executed by the image processor 40 for the fine scan image data. Also, an appropriate processing result can be obtained for each image processing.

In the above description, the correction is performed so that the weight score is reduced for a face candidate region having an extremely high or extremely low density. However, the present invention is not limited to this. In the extraction of the face region, only the face candidate regions having extremely high or extremely low density, or all the face candidate regions are extracted as shown in FIG. ) the threshold value may be changed TH _F such that the threshold TH _F for determining the face region becomes higher as shown in FIG. This corresponds to "increase the selection criterion" described in claim 8. This makes it difficult to extract a face candidate region having an extremely high or extremely low density as a face region. Incidentally determination change amount of the threshold TH _F is may be a constant value, and the density M of concentration is extremely high or extremely low and determines the face candidate region, whether the extremely high or extremely low concentration M Thresholds TH _HIGH , TH
_LOW and the difference may vary the amount of change threshold TH _F in accordance with the. Alternatively, instead of the above, when converting the calculated degree of overlap x into the number of weight points P, if the density M of at least one face candidate area of the face candidate area pair to be processed is extremely high or low, The number of weight points P to be given to the face candidate area pair to be processed may be suppressed.

Further, the number of weight points P set for each face candidate area, the threshold value TH _F for face area determination, or the face area density Mfa
In ce, the weight given to the density M of each face candidate area may be changed according to the type of image processing performed using the processing result of the face area extraction / density calculation processing. The processing described below corresponds to the fifth aspect of the present invention.

For example, using the face area extraction result obtained by the face area extraction / density calculation processing, the image processor 40 locally applies an edge emphasis filter only to the extracted face area to reduce the sharpness of the face area. When the sharpness enhancement process for enhancing is performed, it depends on the degree of the sharpness enhancement and the type of filter. However, even if the sharpness enhancement is actually performed on a region other than the face region, the adverse effect on the visual is small (conspicuous). Not). In such a case, smaller than the normal value of the threshold TH _F for determination face region (i.e. to change the criteria for selection of a face candidate region), it is determined that more face candidate region is a face region You may do so. According to lower the value of the threshold TH _F for determining the face region, the probability that a face candidate region corresponding to the actual face region is erroneously determined not to be a face region is lowered, the above, with respect to the face region in the image And sharpness enhancement processing can be performed without omission.

[0094] Further, instead of changing the value of the threshold TH _F for determining the face region, to the degree of overlapping x determined by calculation typically set to a value greater than the weight score P (that is, each face candidate By changing the evaluation criteria for the areas), more face candidate areas can be determined to be face areas. In particular, in the case where the sharpness enhancement processing is performed as the sharpness enhancement processing as the number of weight points P increases,
By setting the number of weight points P as described above, the degree of sharpness enhancement can be controlled to a relatively high level.

For example, using the face area extraction result and the face area density Mface by the face area extraction / density calculation processing, the density is locally corrected only for the extracted face area based on the face area density Mface. When the density correction processing is performed, although it depends on the degree of the density correction, even if the density correction is actually performed on an area other than the face area, there is a case that the visual effect is small (not noticeable). In such a case, the value of the threshold TH _F for determining a face area smaller than usual, the more face candidate regions may be is determined as the face region. According to lower the value of the threshold TH _F for determining the face region, the probability that a face candidate region corresponding to the actual face region is erroneously determined not to be a face region is lowered, the above, with respect to the face region in the image The density correction process can be performed without omission.

Also, instead of changing the value of the threshold value TH _F for face area determination, a larger value than usual is set as the number of weight points P with respect to the degree of overlap x obtained by calculation, so that more values can be obtained. May be determined as a face area. In particular, in the case where processing for increasing the degree of density correction as the number of weight points P increases is performed as the density correction processing, the degree of correction of the density is controlled to be higher by setting the number of weight points P as described above. Is also possible.

The above description is of a case where, in the extraction of a face area, image processing is performed which has a small effect even when an area which is not actually a face area is erroneously extracted as a face area. when the image processing for receiving a tremendous impact when extracted as a face region by mistake regions not a face area is performed, or greater than normal value of the threshold TH _F for determination example the face area, the degree of overlapping x By setting a smaller value than usual as the weight point number P, it is also possible to extract only a face candidate area having a higher degree of accuracy as a face area as a face area.

As for the face area density, for example, as shown in the following equation (7), the face area density Mface obtained by the above equation (5) (or the equation (6)) and another image feature quantity D
(E.g. average density of the entire image, the average concentration of the non-face candidate region) Weighted average Mface the '(however, alpha _F is a weighting factor for the face area density Mface, alpha ₀ is a weighting factor for the image feature amount D) a When calculating as the face area density, the values of the weighting factors α _F and α ₀ are changed according to the type of image processing performed using the calculated face area density (that is, the weighting criterion for each face candidate area is relatively Weight), the weight given to the density M of each face candidate area may be changed.

Mface ′ = α _F · Mface + α ₀ · D (7) As image processing performed using the processing result of the face area extraction / density calculation processing, the processing result of the face area extraction / density calculation processing is used. A plurality of types of image processing with different requests (for example, image processing in which it is desirable that an area that is not a face area actually exists in the extracted face area, (Such as image processing in which it is desirable to include the face area), the extraction of the face area and the calculation of the face area density may be performed a plurality of times corresponding to each image processing. In the present embodiment, the degree of overlap x can be used as the reliability (probability) of each face candidate area as a face area. As described above, the criterion for setting the number of weight points for each face candidate area and the criterion for face area determination (Threshold T
H _F ), by changing at least one of the weighting criteria for each face candidate area, a result required by each image processing can be obtained as a face area extraction result or a face area density calculation result. Even when a plurality of types of image processing are performed, a very complicated and time-consuming face candidate region extraction process is performed in the same number as the number of types of image processing while changing processing conditions corresponding to the plurality of types of image processing. It is not necessary to repeat the processing, and the processing time of the face area extraction / density calculation processing can be reduced, and the performance of the image processing apparatus 14 can be improved.

In the above case, the processing conditions including the face area extraction and density calculation processing are calculated by the auto setup engine 44 based on the prescanned image data, and the actual image processing for the fine scan image data is performed by the image processor 40. However, the present invention is not limited to this, and it is also possible to sequentially perform the calculation of the processing conditions and the image processing under the calculated processing conditions on a single image data. The processing may be performed by a single processing unit.

Further, in the above description, the extraction of the face area and the calculation of the face area density are respectively performed based on the number of weight points set for each face candidate area. However, the present invention is not limited to this. Only the operation may be performed.

In the above description, the image data obtained by reading the image recorded on the photographic film has been processed. However, the present invention is not limited to this, and is not limited to this. Image data obtained by reading an image, image data obtained by imaging with a digital camera, or image data generated by a computer may be processed. In addition, it goes without saying that the present invention may be used for determining exposure conditions when a film image recorded on a photographic film is exposed and recorded on photographic paper by surface exposure.

In the above description, the case where the area corresponding to the face of the person in the image is the main part has been described, but the present invention is not limited to this. As an example, in mass production of parts, products, and the like, while imaging the situation in which the produced parts, products, and the like are being sequentially conveyed, an image representing the state of being conveyed is extracted at a predetermined timing from the imaging signal. The present invention can also be applied to a case where, for example, an area corresponding to the component or the product is extracted from the extracted image as an area corresponding to the main part. In this case, the extracted main part region can be used, for example, for automatically inspecting the produced parts and products.

[0104]

As described above, according to the first and ninth aspects of the present invention, a plurality of candidate regions which are presumed to correspond to a main part in an image represented by image data are extracted, and other candidate regions are extracted from the image. The degree of overlap is obtained for the candidate area overlapping with the area, and the evaluation of the accuracy as an area corresponding to the main part of the candidate area overlapping with another candidate area is determined as a candidate area not overlapping with another candidate area. For each candidate area, the accuracy as an area corresponding to a main part is evaluated so that the evaluation is higher than the above evaluation and the degree of overlap with another candidate area is higher. This has an excellent effect that the influence of the erroneous extraction of the region estimated to correspond to the main part can be reduced.

According to a second aspect of the present invention, in the first aspect of the present invention, the value obtained by normalizing the distance between the pair of candidate regions based on the size of each of the pair of overlapping candidate regions is used as the degree of overlap. A value obtained by normalizing the area of the overlapping region based on the size of each of the pair of candidate regions, and the size based on the size of each of the pair of candidate regions along the predetermined direction. Since at least one of the values obtained by normalizing the size of the region along the predetermined direction is used, in addition to the above-described effects, the degree of overlap of the candidate region can be accurately quantified regardless of the size of the candidate region. It has the effect of.

According to a third aspect of the present invention, in the first aspect of the present invention, each candidate area is compared with an evaluation value representing a result of evaluating the accuracy as an area corresponding to a main part with a threshold value. Since a candidate region having high accuracy as a region corresponding to the main part is selected from the candidate regions, in addition to the above effects,
Candidate areas with high accuracy as areas corresponding to the main part,
This has the effect that selection can be made accurately with simple processing.

According to a fourth aspect of the present invention, in the first aspect of the present invention, each candidate area or a candidate area selected from each candidate area is evaluated in accordance with an evaluation value representing a result of evaluating the accuracy as an area corresponding to a main part. Weighting is performed, and a weighted average of the image feature amounts of the respective candidate regions or the selected candidate regions is calculated as the image feature amount of the main part. This has the effect that it can be determined.

According to a fifth aspect of the present invention, in the first aspect, when predetermined image processing is performed after evaluating the accuracy of each candidate area as an area corresponding to a main part,
Depending on the type of predetermined image processing, change the criterion of the evaluation of the accuracy as a region corresponding to the main part, or select a candidate region with high probability as the region corresponding to the main part, or weight each candidate region Is changed, so that in addition to the above effects, there is an effect that an appropriate processing result can always be obtained regardless of the type of predetermined image processing.

According to a sixth aspect of the present invention, in the fifth aspect of the present invention, the predetermined image processing is an image processing in which the processing result is greatly affected by erroneous extraction of a candidate area or erroneous evaluation of a candidate area. In such a case, the evaluation criterion is changed so that the criterion of accuracy evaluation as a region corresponding to the main part is strict, or the selection is performed such that only candidate regions evaluated as having higher accuracy are selected. Or the weighting criterion is changed so that the weight for the candidate area evaluated as having low accuracy is relatively small. In addition to the above-described effects, erroneous extraction of the candidate area and erroneous selection of the candidate area are performed. Even if the evaluation is made, it is possible to prevent the processing result of the predetermined processing from being greatly affected.

According to a seventh aspect of the present invention, in the fifth aspect of the present invention, the predetermined image processing is an image processing in which the processing result is less affected even if a candidate area is erroneously extracted or a candidate area is erroneously evaluated. In this case, the criterion of the evaluation is changed so that the criterion of the evaluation of the accuracy as the area corresponding to the main part is loosened, or the criterion of the selection is selected such that the candidate area evaluated as having the low accuracy is also selected. Or the weighting criterion is changed so that the weight for the candidate area evaluated as having the low accuracy is relatively large. In addition to the above-described effects, erroneous extraction of the candidate area and erroneous evaluation of the candidate area are performed. In this case, there is an effect that the processing result of the post-processing is less affected, and an appropriate processing result is obtained.

According to an eighth aspect of the present invention, in the first aspect of the present invention, for a candidate area whose density in a candidate area is higher than a first predetermined value or lower than a second predetermined value,
Either lowering the evaluation of the accuracy as the region corresponding to the main part, or increasing the selection criterion when selecting a candidate region with high accuracy as the region corresponding to the main part,
In addition to the above effects, there is an effect that even when a region having an extremely high or extremely low density is erroneously extracted as a candidate region, it is possible to avoid an inappropriate result of the post-processing.

According to a tenth aspect of the present invention, there is provided a first step of extracting a plurality of candidate areas which are presumed to correspond to the main part in an image represented by image data, A second step of determining the degree of overlap for the region, the evaluation of the accuracy as a region corresponding to the main part for the candidate region overlapping with another candidate region, And a process including a third step of evaluating the accuracy of each candidate region as a region corresponding to a main part, so that the candidate region becomes higher as the degree of overlap with another candidate region becomes higher. Since the program to be executed by the computer is recorded on a recording medium, the effect of erroneous extraction of an area estimated to correspond to a main part in an image can be reduced. It has an effect.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of an image processing system according to an embodiment.

FIG. 2 is a flowchart showing the contents of a face area extraction / density calculation process according to the embodiment.

FIG. 3 is an image diagram showing an example of a processing result by a face candidate area extraction processing.

FIG. 4A is an image diagram showing an example of a pair of overlapping face candidate regions, and FIGS. 4B and 4C are diagrams showing an example of a map for setting weight points according to the degree of overlap; It is.

5A is a diagram showing a map for correcting a weight score for a face candidate region having an extremely high or extremely low density, and FIG. 5B is a diagram showing a face candidate region having an extremely high or extremely low density. FIG. 9 is a diagram illustrating a change of a threshold for determining a face area in the case where is present.

[Explanation of symbols]

 Reference Signs List 10 image processing system 14 image processing device 40 image processor 44 auto setup engine 72 information storage medium

Claims (10)

[Claims] 1. A candidate area which is presumed to correspond to a main part in an image represented by the image data is extracted based on the image data, and the extracted candidate area overlaps with another candidate area on an image. The degree of overlap with the other candidate area is determined for the candidate area that has been determined, and the evaluation of the accuracy of the candidate area overlapping with the other candidate area as the area corresponding to the main part in the image is performed. The extracted candidate region corresponds to the main part so that the evaluation is higher than the evaluation for the candidate region not overlapping with the candidate region, and further higher as the degree of overlap with the other candidate regions is higher. An image processing method for evaluating the accuracy as a region. 2. A value obtained by normalizing a distance between the pair of candidate regions based on a size of each of the pair of overlapping candidate regions as the degree of overlap, a size of each of the pair of candidate regions. A value obtained by normalizing the area of the overlapping region based on the reference, and the size along the predetermined direction of the overlapping region based on the size along the predetermined direction of each of the pair of candidate regions. 2. The image processing method according to claim 1, wherein at least one value obtained by normalizing is used. 3. An extracted value corresponding to the main part from the extracted candidate area is compared with a threshold by comparing an evaluation value representing a result of evaluating the accuracy of the extracted candidate area as an area corresponding to the main part. 2. The image processing method according to claim 1, wherein a candidate area having a high degree of certainty as the area to be selected is selected. 4. A method for weighting the extracted candidate region or a candidate region selected from the extracted candidate regions according to an evaluation value representing a result of evaluating the accuracy as a region corresponding to the main portion, 2. The image processing method according to claim 1, wherein a weighted average of image feature amounts of the extracted candidate region or the selected candidate region is calculated as the amount. 5. After evaluating the probability of the extracted candidate region as a region corresponding to the main part, a candidate region having a high probability as a region corresponding to the main part is determined based on a result of the evaluation. When performing predetermined image processing including at least one of selection and weighting of the extracted candidate area, according to the type of the predetermined image processing, the evaluation criterion is changed, or the selection or the weighting is changed. 2. The image processing method according to claim 1, wherein the reference is changed. 6. The accuracy as a region corresponding to a main part when the predetermined image processing is an image processing in which a processing result is greatly affected when a candidate region is erroneously extracted or a candidate region is erroneously evaluated. Either change the evaluation criterion so that the evaluation criterion becomes strict, or change the selection criterion so that only the candidate area evaluated as having higher accuracy is selected, or if the accuracy is low. 6. The image processing method according to claim 5, wherein the weighting criterion is changed so that the weight for the evaluated candidate area becomes relatively small. 7. The accuracy as a region corresponding to a main part when the predetermined image processing is an image processing that does not greatly affect the processing result even if erroneous extraction of a candidate region or erroneous evaluation of a candidate region is performed. Either change the evaluation criterion so that the evaluation criterion is relaxed, or change the selection criterion so that the candidate area evaluated as having low accuracy is also selected, or is evaluated as having low accuracy. 6. The image processing method according to claim 5, wherein the criterion of the weight is changed such that the weight for the candidate area becomes relatively large. 8. For a candidate region whose density in the candidate region is higher than a first predetermined value or lower than a second predetermined value, the evaluation of the accuracy as a region corresponding to the main part is reduced. 2. The image processing method according to claim 1, wherein a selection criterion for selecting a candidate area having high accuracy as an area corresponding to the main part is increased. 9. Extraction means for extracting a candidate area estimated to correspond to a main part in an image represented by the image data, based on the image data, and another candidate area on the image among the extracted candidate areas Calculating means for calculating the degree of overlap with the other candidate area with respect to the candidate area overlapping with the area; Is higher than the evaluation for a candidate region that does not overlap with another candidate region, and further increases as the degree of overlap with other candidate regions increases, so that the extracted candidate region Evaluation means for evaluating the accuracy as an area corresponding to the main part. 10. A first step of extracting, based on the image data, a candidate area estimated to correspond to a main part in the image represented by the image data, out of the extracted candidate areas, A second step of obtaining a degree of overlap with the other candidate area with respect to the candidate area overlapping with the candidate area; The accuracy of the evaluation is higher than the evaluation for a candidate region that does not overlap with other candidate regions, and further increases as the degree of overlap with other candidate regions increases, so that the extracted candidate region A recording medium storing a program for causing a computer to execute a process including a third step of evaluating the accuracy as an area corresponding to the main part.