JP3506958B2 - Image processing method, image processing apparatus, and recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, an image processing apparatus and a recording medium, and more particularly, to an image processing method for extracting an area estimated to correspond to a main part of an image, and the image processing method. The present invention relates to an image processing apparatus to which the above can be applied, and a recording medium on which a program for causing the computer to execute the image processing method is recorded.

[0002]

2. Description of the Related Art The most noticeable part of a person's photograph is a person's face. For example, an original image recorded on a photographic film is exposed and recorded on a recording material such as photographic paper (surface exposure or scanning exposure). It is desirable to control the exposure so that the color and density of the person's face become appropriate when recording the image). However, in order to realize this exposure control, an area corresponding to the person's face in the original image. It is necessary to accurately detect the tint and density of. In addition, with respect to the image data obtained by reading the image, among various image processing developed for the purpose of improving the image quality of the image represented by the image data, there is an area corresponding to the face of a person in the image or There is a method in which specific image processing (for example, local density correction or red-eye correction) is performed only on a part of it. To perform this processing, the position of the area corresponding to the face of the person in the image or It is necessary to detect the size accurately.

For this reason, various methods have been proposed in the past for extracting a region estimated to correspond to a main part such as a person's face in an image. For example, in Japanese Patent Laid-Open No. 8-184925, based on image data, a shape pattern peculiar to each part of a person present in an image (for example, outline of head, outline of face, internal structure of face, outline of body, etc.) Any one of the shape patterns indicating the human face, and the size and orientation of the detected shape pattern, and the face of the person according to the positional relationship between the predetermined part of the person represented by the detected shape pattern and the human face. While setting the area that is estimated to be equivalent, searching for another shape pattern that is different from the detected shape pattern, obtaining the consistency of the previously set area as a human face, and presuming that it corresponds to a human face. There is disclosed a method of extracting a face area for extracting an area (face area) to be extracted.

[0004]

However, none of the various methods currently proposed can always accurately extract a region corresponding to a main part such as a face of a person in an image. The area corresponding to may be erroneously extracted. Then, when the area corresponding to the main part is erroneously extracted, there is a problem that the processing result by the exposure control or the image processing performed by using the extraction result becomes improper.

For example, when a high-luminance background portion in an image is mistakenly extracted as an area corresponding to a person's face in an image in which a strobe is emitted under a backlight condition, the extracted area If the exposure amount when the image is exposed and recorded on the recording material is controlled based on the color and the density, the recorded image has an unsuitable image quality in which a region corresponding to a person's face is blackened. Further, even when the specific image processing is performed only on the extracted area or a part thereof, the specific processing is performed only on the area different from the area corresponding to the face of the person in the image. However, the image quality of the image represented by the image data may deteriorate.

The present invention has been made in consideration of the above facts, and an image processing method, an image processing apparatus, and a recording method capable of reducing the influence of erroneous extraction of a region estimated to correspond to a main part in an image. The purpose is to obtain the medium.

[0007]

The inventor of the present application selects an arbitrary extraction method from various extraction methods for extracting a region estimated to correspond to a main part such as a face of a person in an image,
Applying the selected extraction method to extract each of the regions corresponding to the main part from a large number of images can be performed by changing the processing conditions of the extraction method (for example, a threshold value for determining whether or not the main part), Experiments were repeated while changing the method itself. Then, as a result of comparing the regions extracted in each extraction, the extraction regions that match or substantially match the main part in the image overlap with other extraction regions, and the degree of overlap with other extraction regions is It was found that they were mostly large, and I thought that the accuracy of the extraction region as a region corresponding to the main part in the image has a strong correlation with the degree of overlap with other extraction regions.

[0008] Further, among various extraction methods, even if a plurality of areas are extracted by one extraction process, the extraction areas do not overlap each other due to a difference in algorithm. When multiple areas are extracted, there is a possibility that overlap may occur in each extracted area (for example, in an algorithm that traces edges in an image and extracts an area corresponding to a main part, multiple areas different from each other are extracted. If the edge tracking is performed using as a starting point, different regions that partially overlap may be extracted as regions corresponding to the main part). However, even in the latter extraction method, a plurality of regions can be extracted in one extraction process. The extracted area in which the area is extracted and overlaps with other extracted areas often coincides with or substantially coincides with the main part in the image, and as an area corresponding to the main part in the image of the extracted area. Accuracy, as described above, it was confirmed that there is a degree of overlapping strongly correlated with other extraction region.

Based on the above, the image processing method according to the invention of claim 1 extracts, based on the image data, a candidate region estimated to correspond to a main part in the image represented by the image data, and extracts the candidate region. Of the candidate regions, for a candidate region that overlaps another candidate region on the image, the degree of overlap with the other candidate region is obtained, and the main region in the image for the candidate region that overlaps the other candidate region is determined. The accuracy of the accuracy as a region corresponding to a part is higher than the evaluation for a candidate region that does not overlap with other candidate regions, and as the degree of overlap with other candidate regions becomes higher, it becomes higher.
The accuracy of the extracted candidate area as an area corresponding to the main part is evaluated.

According to the first aspect of the present invention, first, based on the image data, the candidate area estimated to correspond to the main part in the image represented by the image data is extracted. Although the number of candidate regions actually extracted by the above-mentioned extraction also depends on the content of the image represented by the image data, it is desirable that a plurality of candidate regions be extracted in the present invention. To improve the probability that a plurality of candidate regions will be extracted, for example, different extraction methods are applied to extract candidate regions multiple times,
This can be realized by extracting the candidate area multiple times while changing the processing conditions with a single extraction method. Also,
When applying an extraction method in which a plurality of candidate regions may be extracted in one extraction process and the candidate regions may overlap with each other, the candidate regions may be extracted only once.

Next, of the extracted candidate areas, the degree of overlap with other candidate areas is calculated for the candidate areas that overlap with other candidate areas on the image. The calculation of the degree of overlap can be performed, for example, when a plurality of candidate areas are extracted and there is a candidate area that overlaps with other candidate areas.

As the degree of overlap, for example, as described in claim 2, the distance between the pair of candidate regions (the distance between each pair of candidate regions is based on the size of each pair of overlapping candidate regions). It may be the distance of the center of gravity position, or the distance between the representative points representing the positions of the pair of candidate regions), and the size of each of the pair of candidate regions. A value obtained by normalizing the area of the overlapping region based on the reference, and the size of the overlapping region along the predetermined direction based on the size of each of the pair of candidate regions along the predetermined direction are set. At least one of the normalized values can be used. Further, an average value or a weighted average value of two or more values among the above respective values may be used. By using the normalized value as described above, the degree of overlap of the candidate areas can be accurately digitized regardless of the size of the candidate areas. In addition,
If the size of a pair of candidate regions that overlap with each other is different, the value representing the smaller degree of overlap is duplicated among the pair of values obtained by using the sizes of both candidate regions as references. It is preferable to use it as a degree.

According to the first aspect of the present invention, a candidate region whose accuracy evaluation as a region corresponding to a main part in an image with respect to a candidate region overlapping with another candidate region does not overlap with another candidate region is determined. For the extracted candidate area, and the accuracy as an area corresponding to the main part is evaluated with respect to the extracted candidate area so as to become higher as the degree of overlap with other candidate areas becomes higher. As a result, among the extracted candidate areas, the candidate areas confirmed to have high accuracy by the above-described experiment, which are areas corresponding to the main part in the image, that is, other candidate areas that overlap and are With respect to the candidate area having a relatively high degree of overlap with the area, the evaluation of the accuracy as the area corresponding to the main part in the image is made higher than that of the other candidate areas.

The extracted candidate areas can be evaluated, for example, by setting an evaluation value that represents the level of the evaluation as a numerical value. The evaluation value is set, for example, by setting the evaluation value to 0 or a low value for a candidate area that does not overlap with other candidate areas, and for a pair of candidate areas that overlap each other,
The evaluation value is set so that the value increases as the degree of overlap increases, and for candidate areas that overlap multiple candidate areas, the evaluation is set according to the degree of overlap for each overlapping candidate area. This can be done by integrating the values and setting.

As the post-processing utilizing the result of evaluating the accuracy as the area corresponding to the main part in the image with respect to the extracted candidate area as described above, for example, as described in claim 3, A process of selecting a candidate region with high accuracy as a region corresponding to the main part from the candidate region, and weighting the extracted candidate region (or the candidate region selected from the extracted candidate regions) as described in claim 4. Processing, etc., but even if the extracted candidate regions include a candidate region that is not a main part in the image due to erroneous extraction of a region estimated to be the main part in the image, Item 1
By performing the evaluation according to the invention,
It is possible to reduce the influence of the candidate region that is not the main part based on the result of the evaluation.

As an example, when selecting a candidate region having a high accuracy as a region corresponding to the main part, only candidate regions having a high accuracy evaluation as a region corresponding to the main part are selected from the extracted candidate regions. By selecting, it is possible to exclude candidate regions that are not actually the main part. In addition, for example, when weighting the extracted candidate regions, a high weight is given to candidate regions that have a high accuracy evaluation as a region corresponding to the main part, and the accuracy evaluation as a region corresponding to the main part is performed. By reducing the weight of the candidate region having a low value, the influence of the candidate region which is not a main part in practice can be reduced.

As described above, according to the first aspect of the present invention, the accuracy as the region corresponding to the main part is evaluated for the extracted candidate regions based on the presence / absence of overlap with other candidate regions and the degree of overlap. So, based on the results of this evaluation,
It is possible to reduce the influence of erroneous extraction of a region estimated to correspond to the main part in the image.

It should be noted that, as a post-process for utilizing the evaluation result for the extracted candidate region, when performing a process for selecting a highly reliable candidate region as a region corresponding to a main part from the extracted candidate region, the method according to claim 3 As described in, by comparing the evaluation value representing the result of evaluating the accuracy as a region corresponding to the main part to the extracted candidate region with a threshold value, as a region corresponding to the main part from the extracted candidate region. It is preferable to select a candidate region with high accuracy. As a result, even if the extracted candidate area actually includes a candidate area that is not the main part, the candidate area can be excluded based on the evaluation value, and the candidate area can be excluded. It is possible to select a highly accurate candidate region with high accuracy by simple processing, and it is also possible to correct the selection criterion of the candidate region by changing the threshold value.

Further, as post-processing using the evaluation result for the extracted candidate area, as described in claim 4,
The extracted candidate area or the candidate area selected from the extracted candidate areas is weighted according to the evaluation value representing the result of evaluating the accuracy as the area corresponding to the main part, and as the image feature amount of the main part, the extracted candidate area or You may perform the process which calculates the weighted average of the image feature-value of the said candidate area selected. As a result, even if the extracted candidate area actually includes a candidate area that is not the main part, the weight for the candidate area can be reduced based on the evaluation value. Can be accurately obtained by a simple process. In addition, in the invention of claim 4, when the candidate area selected from the extracted candidate areas is used, the candidate area can be selected in the same manner as the invention of claim 3.

By the way, various kinds of processing can be considered as the post-processing to be performed after the accuracy of the extracted candidate area as an area corresponding to the main part is evaluated. However, erroneous extraction of the candidate area and erroneous evaluation of the candidate area are possible. If there is such a problem, the extent to which the processing result of the post-processing is affected greatly depends on the type of the post-processing. Therefore, in the invention of claim 5, in the invention of claim 1, after the accuracy of the extracted candidate region as a region corresponding to the main part is evaluated, the result is equivalent to the main part based on the result of the evaluation. When performing a predetermined image processing including at least one of selection of a highly accurate candidate area as a region to be selected and weighting of the extracted candidate area, the evaluation criterion is changed according to the type of the predetermined image processing. Alternatively, the selection or the weighting criterion is changed.

According to the fifth aspect of the present invention, for example, when the predetermined image processing is an image processing in which the processing result is greatly affected by erroneous extraction of the candidate area or erroneous evaluation of the candidate area, for example, As described in claim 6, the evaluation standard is changed so that the accuracy evaluation standard as the area corresponding to the main part becomes strict, or the accuracy as the area corresponding to the main part is higher. The selection criterion so that only the selected candidate region is selected, or a weighting criterion so that the weight for the candidate region evaluated to have low accuracy as a region corresponding to the main part becomes relatively small. Can be changed. As a result, in the post-processing, there is a possibility that some of the extracted candidate areas corresponding to the main part are not selected or the weight is reduced. Even if the candidate area is erroneously evaluated, it is possible to prevent the processing result of the predetermined processing from being greatly affected.

Further, for example, when the predetermined image processing is an image processing which has a small influence on the processing result even if the candidate area is erroneously extracted or the candidate area is erroneously evaluated, as described in claim 7, for example. In addition, the criterion for evaluation of accuracy as a region corresponding to the main part is changed so as to be loose, or a candidate region evaluated as having low accuracy as a region corresponding to the main part is also selected. As described above, the selection criterion may be changed, or the weighting criterion may be changed so that the weight of the candidate region evaluated as having low accuracy as a region corresponding to the main part becomes relatively large. As a result, in the case where there is an erroneous extraction of a candidate area or an erroneous evaluation of a candidate area, in the post-processing, there is a possibility that a candidate area that is not the main part is actually selected or the weight is increased, but the post-processing Since the influence of the result is small, on the other hand, it is more likely that all the candidate regions corresponding to the main part of the extracted candidate regions will be selected or that the candidate regions corresponding to the main part will be heavily weighted. Appropriate processing results can be obtained.

As described above, according to the invention of claim 5, when the predetermined image processing is performed after the accuracy of the extracted candidate area as the area corresponding to the main part is evaluated, the predetermined image processing is performed. Change the evaluation criteria depending on the type,
Alternatively, since the selection or weighting criterion is changed, an appropriate processing result can always be obtained regardless of the type of the predetermined image processing.

By the way, for example, when recording an image of a person photographed under a backlit condition on a recording material, a high-intensity background portion in the image is set as an area corresponding to a main part (in this case, a person's face). If an erroneous determination is made and the recording density is controlled based on the density of the area, the quality of the recorded image will be extremely low. Further, for example, when recording an image with a low brightness background part on a recording material, the low brightness background part corresponds to the main part (a person's face, etc., which was photographed under a backlit illumination condition without firing a strobe). Even when the area is erroneously determined to be a certain area and the recording density is controlled based on the density of the area, the quality of the recorded image is extremely low.

Based on the above, in the invention according to claim 8, in the invention according to claim 1, the density in the candidate area is higher than the first predetermined value or lower than the second predetermined value. It is characterized in that the evaluation of the accuracy as the area corresponding to the main part is lowered, or the selection criterion for selecting the candidate area with high accuracy as the area corresponding to the main part is increased.

In the invention of claim 8, the density in the candidate area is higher than the first predetermined value, or lower than the second predetermined value, that is, the density in the candidate area is extremely high, or For an extremely low candidate area, the evaluation of accuracy as an area corresponding to the main part is lowered, or the selection criterion when selecting a highly accurate candidate area as an area corresponding to the main part is increased. When lowering the accuracy evaluation as the area corresponding to the main part, the area erroneously extracted as a candidate area whose density is extremely high or extremely low is regarded as the area corresponding to the main part in the post-processing. It is possible to prevent the candidate region from being selected with high accuracy and from being given a large weight.

Further, as a post-processing, if a process including selection of a highly accurate candidate region as a region corresponding to the main part is performed, a criterion for selecting a highly accurate candidate region as the region corresponding to the main part is used. Even by increasing the density, it is possible to prevent a region erroneously extracted as a candidate region whose density is extremely high or extremely low from being selected as a candidate region having high accuracy as a region corresponding to the main part. .
Therefore, even when an area having extremely high or extremely low density is erroneously extracted as a candidate area, it is possible to prevent the post-processing processing result from being unsuitable.

An image processing apparatus according to a ninth aspect of the present invention includes, based on the image data, extraction means for extracting a candidate area estimated to correspond to a main part in the image represented by the image data, and the extraction means. Of the candidate areas, for a candidate area that overlaps another candidate area on the image, a calculating unit that obtains the degree of overlap with the other candidate area, and the image for the candidate area that overlaps the other candidate area Accuracy evaluation as a region corresponding to the main part of the inside is higher than the evaluation for the candidate region that does not overlap with other candidate regions,
And so as to become higher as the degree of overlap with other candidate areas becomes higher, an evaluation means for evaluating the accuracy as an area corresponding to the main part with respect to the extracted candidate areas,
Since it is configured to include, it is possible to reduce the influence of erroneous extraction of the region estimated to correspond to the main part in the image, as in the invention of claim 1.

A recording medium according to the invention of claim 10 is
A first step of extracting a candidate region estimated to correspond to a main part in the image represented by the image data based on the image data, and among the extracted candidate regions, overlaps with other candidate regions on the image. The second step of obtaining the degree of overlap with the other candidate area with respect to the candidate area that is present, the accuracy of the candidate area overlapping the other candidate area is evaluated as an area corresponding to the main part in the image. , The main part with respect to the extracted candidate region, so that the evaluation is higher than the evaluation for a candidate region that does not overlap with another candidate region, and further higher as the degree of overlap with another candidate region is higher. A program for causing a computer to execute a process including a third step of evaluating accuracy as a region corresponding to is stored.

A recording medium according to a tenth aspect of the present invention causes a computer to execute a process including the first to third steps described above, that is, a process according to the image processing method of the first aspect of the invention. Since the program for performing the recording is recorded, it is estimated that the computer reads and executes the program recorded in the recording medium to correspond to the main part in the image as in the invention of claim 1. The influence of erroneous extraction of the area can be reduced.

[0031]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 shows an image processing system 10 to which the present invention is applied. The image processing system 10 is configured by connecting a scanner 12, an image processing device 14, and a printer 16 in series.

The scanner 12 visualizes a film image recorded on a photographic light-sensitive material (hereinafter, simply referred to as a photographic film) such as a photographic film (for example, a negative film or a reversal film) (a subject is photographed and then developed to be developed. Negative image or positive image) and outputs the image data obtained by the reading, and the light emitted from the light source 20 and having the uneven light amount reduced by the light diffusion box 22 is set on the film carrier 24. The photographic film 26, such as a negative film or a reversal film, which has been recorded, receives light transmitted through the photographic film 26 through a lens 28 and received by a CCD sensor 30 (which may be an area sensor or a line sensor). It is configured to form an image on a surface.

The film carrier 24 is a photographic film 2
6 is the light source 20 where the film image is recorded.
The photographic film 26 is conveyed so as to be sequentially positioned on the optical axis of the light emitted from the. As a result, the film images recorded on the photographic film 26 are sequentially read by the CCD sensor 30, and the CCD sensor 30 outputs a signal corresponding to the film image. The signal output from the CCD sensor 30 is converted into digital image data by the A / D converter 32 and input to the image processing device 14.

The line scanner correction unit 36 of the image processing device 14 subtracts the dark output level of the cell corresponding to each pixel from the input scan data (R, G, B data input from the scanner 12). Density conversion that performs logarithmic conversion of data that has undergone correction and darkness correction into data that expresses density values,
Shading correction that corrects the data after density conversion according to the light amount unevenness of the light that illuminates the photographic film 26, and data of a cell (so-called defective pixel) that does not output a signal corresponding to the incident light amount in the data that has been subjected to the shading correction. Are sequentially performed by interpolating from the data of surrounding pixels to newly generate defective pixel correction. Line scanner correction unit 36
Is connected to the input end of the I / O controller 38, and the data processed by the line scanner correction unit 36 is input to the I / O controller 38 as scan data.

The input end of the I / O controller 38 is also connected to the data output end of the image processor 40, and the image data subjected to image processing (details will be described later) is input from the image processor 40. Also, I / O
The input end of the controller 38 is the personal computer 4
It is also connected to 2. The personal computer 42 is provided with an expansion slot (not shown). In the expansion slot, a driver (not shown) for reading / writing data from / into an information storage medium such as a memory card or a CD-R, and the like are provided. A communication control device for communicating with the information processing device is connected. When file image data is input from the outside via the expansion slot, the input file image data is input to the I / O controller 38.

The output terminal of the I / O controller 38 is connected to the data input terminal of the image processor 40, the auto setup engine 44, and the personal computer 42, and the printer 1 is connected via the I / F circuit 54.
Connected to 6. The I / O controller 38 selectively outputs the input image data to each of the devices connected to the output end.

In the present embodiment, each film image recorded on the photographic film 26 is read by the scanner 12 twice at different resolutions. In the first reading with relatively low resolution (hereinafter referred to as prescan), even when the density of the film image is very low (for example, the negative image of the exposure underexposure on the negative film),
Reading conditions determined by the CCD sensor 30 so as not to cause saturation of accumulated charges (R of light irradiated on the photographic film 26,
The entire surface of the photographic film 26 is read according to the amount of light in each wavelength range of G and B and the charge storage time of the CCD sensor 30. The data (prescan data) obtained by this prescan is input from the I / O controller 38 to the auto setup engine 44.

The auto setup engine 44 is a CP
U46, RAM48 (eg DRAM), ROM50
(For example, a ROM whose stored contents can be rewritten) and an input / output port 52 are provided and are connected to each other via a bus. The auto setup engine 44
The frame position of the film image is determined based on the prescan data input from the I / O controller 38, and the data (prescan image data) corresponding to the film image recording area on the photographic film 26 is extracted. In addition, the size of the film image is determined based on the prescan image data, and the image feature amount such as the density is calculated, and the scanner 12 is used for the prescanned photographic film 26.
Determines the reading condition when performing another reading at a relatively high resolution (hereinafter referred to as a fine scan). Then, the frame position and the reading condition are output to the scanner 12.

Further, the auto setup engine 44
Is the main part of the film image based on the prescan image data (for example, the area corresponding to the face of a person (face area))
The image condition amount including the extraction of the image is calculated, and the processing conditions of various image processing for the image data (fine scan image data) obtained by the scanner 12 performing the fine scan are automatically determined by the calculation (the setup calculation. ), The determined processing conditions are applied to the image processor 4
Output to 0.

A display, a keyboard, and a mouse are connected to the personal computer 42 (all not shown). The personal computer 42 takes in the pre-scan image data from the auto-setup engine 44, takes in the processing conditions of the image processing determined by the auto-setup engine 44, and based on the taken-in processing conditions, images the fine scan image data as an object. Image processing equivalent to the image processing performed by the processor 40 is performed on the prescan image data to generate simulation image data.

Then, the generated simulation image data is converted into a signal for displaying an image on the display, and the simulation image is displayed on the display based on the signal. In addition, the operator tests the displayed simulation image for image quality,
When the information indicating the correction of the processing conditions is input through the keyboard as the verification result, the information is output to the auto setup engine 44. As a result, the auto-setup engine 44 performs processing such as recalculation of processing conditions for image processing.

On the other hand, the image data (fine scan image data) input to the I / O controller 38 by fine scanning the film image by the scanner 12 is input from the I / O controller 38 to the image processor 40. To be done. Image processor 4
0 is color / density correction processing including gradation conversion and color conversion, pixel density conversion processing, hypertone processing that compresses the gradation of the ultra-low frequency luminance component of the image, hypersharpness that emphasizes sharpness while suppressing graininess. An image processing circuit for performing various image processing such as processing is provided, and various image processing is performed on the input image data according to the processing conditions determined and notified for each image by the auto setup engine 44. .

As image processing that can be executed by the image processor 40, in addition to the above, for example, sharpness correction or soft focus processing for the entire image or a part (for example, an area corresponding to a person's face), or intentionally changing the image tone is performed. Image processing (image processing that finishes the output image in monotone, image processing that finishes the output image in portrait style,
Image processing for finishing the output image in sepia tone, etc., Image processing for processing the image (for example, image processing for finishing a person existing in the original image on the main image to be slender, image processing for correcting red eyes, etc.), , LF (film with lens), the LF aberration correction processing for correcting the distortion of the lens of LF, the geometric distortion of the image caused by the chromatic aberration of magnification, and the color shift, and the periphery of the lens of LF. For example, a peripheral light reduction correction process for correcting a decrease in brightness of an edge portion of an image due to light reduction, and a focus blur correction process for correcting a decrease in image sharpness due to a characteristic of an LF lens, etc. Various LF aberration correction processing and the like for correcting the deterioration of the image quality of the output image due to the characteristics of

When the image data processed by the image processor 40 is used for recording an image on photographic paper, the image data processed by the image processor 40 is transferred from the I / O controller 38 to the I / O controller 38. / F circuit 54
Is output to the printer 16 as recording image data. When the image data after the image processing is output to the outside as an image file, the image data is output from the I / O controller 38 to the personal computer 42. As a result, the personal computer 42 outputs the image data input from the I / O controller 38 for output to the outside to the outside (the driver, the communication control device, etc.) as an image file via the expansion slot.

The printer 16 includes an image memory 58, R,
A G and B laser light source 60 and a laser driver 62 for controlling the operation of the laser light source 60 are provided. The recording image data input from the image processing device 14 is stored in the image memory 58.
It is used for modulation of the R, G, and B laser beams emitted from the laser light source 60 after being stored once in the memory.
The laser light emitted from the laser light source 60 is scanned on the printing paper 68 via the polygon mirror 64 and the fθ lens 66, and an image is exposed and recorded on the printing paper 68. The photographic printing paper 68 on which the image has been exposed and recorded is sent to the processor section 18 and subjected to color development, bleach-fixing, water washing and drying.
As a result, the image exposed and recorded on the photographic printing paper 68 is visualized.

Next, as an operation of this embodiment, the scanner 1
The face area extraction / density calculation processing performed after the prescan data is input to the image processing apparatus 14 from 2 and the auto setup engine 44 performs processing such as cutting out the image data from the prescan data will be described.

This face area extraction / density calculation processing is processing to which the image processing method according to the present invention is applied, and is realized by the CPU 46 of the auto setup engine 44 executing the face area extraction / density correction program. To be done. The face area extraction / density correction program is a program for causing the CPU 46 to execute other processing,
Initially, it is stored in the information storage medium 72 (see FIG. 1). Although the information storage medium 72 is shown as a floppy disk in FIG. 1, it may be constituted by a CD-ROM, a memory card or the like. An information storage medium 72 is connected to an information reading device (not shown) connected to the personal computer 42.
Is loaded, and the transfer (installation) of the program from the information storage medium 72 to the image processing apparatus 14 is instructed,
A face area extraction / density correction program or the like is read from the information storage medium 72 by the information reading device and stored in the rewritable ROM 50.

Then, when the timing to execute the face area extraction / density correction processing comes, the face area extraction / density correction program is read from the ROM 50, and the face area extraction / density correction processing is executed.
The density correction program is executed by the CPU 46.
As a result, the auto setup engine 44 functions as the image processing device according to the present invention. Thus, the information storage medium 72 storing the face area extraction / density correction program and the like corresponds to the recording medium according to claim 10.

The face area extraction / density correction processing will be described below with reference to the flowchart of FIG. In step 100, based on the image data to be processed, as a main part in the image represented by the image data, a region estimated to correspond to the face of a person in the image (face candidate region extraction processing for extracting a face candidate region is performed. As an extraction method for performing this face candidate area extraction process, a face candidate area extraction method for determining an area estimated to correspond to a human face in an image and extracting the area as a face candidate area, and an image There is a background portion removal method that determines an area (background area) estimated to correspond to the background inside and extracts an area other than the background area as a face candidate area. At least one of the face candidate extraction method and the background removal method can be adopted to perform the face candidate area extraction processing.

[Example 1 of Face Candidate Region Extraction Method] Based on data (image data) obtained by dividing an image into a large number of measuring points and separating each measuring point into three colors of R, G and B. Determine whether or not each measurement point is included in the skin color range on the color coordinates, and extract the area where the cluster (group) of measurement points that are judged to be in the skin color range is present as the face candidate area. (JP-A-52-156624, JP-A-52-156625)
JP-A-53-12330, JP-A-53-145620, JP-A-53-145621, and JP-A-53-145622).

[Example 2 of Face Candidate Area Extraction Method] Based on the image data, a histogram of hue values (and saturation values) is obtained, and the obtained histogram is divided into mountains.
Determine which of the divided mountains each measurement point belongs to, divide each measurement point into a group corresponding to the divided mountains, divide the image into a plurality of regions for each group, and select the person among the plurality of regions. The area corresponding to the face is estimated and the estimated area is extracted as a face candidate area (see Japanese Patent Laid-Open No. 4-346333).

[Example 3 of Face Candidate Region Extraction Method] Based on the image data, a shape pattern peculiar to each part of the person existing in the image (for example, contour of head, contour of face, internal structure of face, body) Any one of the shape patterns representing the contours of the person) is searched, and the size and direction of the detected shape pattern, and the positional relationship between the predetermined part of the person represented by the detected shape pattern and the face of the person are detected. Set an area estimated to correspond to the face. Further, another shape pattern different from the detected shape pattern is searched, the matching of the previously set area as a human face is obtained, and a face candidate area is extracted (Japanese Patent Laid-Open No. 8-122944, Japanese Patent Laid-Open No. 8-183925, JP 9-13
See 8471 publication, etc.).

[Example 1 of Background Removal Method] Based on the image data, each measurement point has a specific color that clearly belongs to the background on the color coordinates (for example, blue of sky or sea, green of lawn or tree, etc.). ), The area where the cluster (group) of measurement points determined to be within the specific color range is present is determined as the background area, and the remaining area is removed. It is extracted as a non-background region (region that is highly likely to include a region corresponding to a person's face: this is also a face candidate region of the present invention).

[Example 2 of Background Removal Method] Based on the image data, the image is divided into a plurality of areas in the same manner as in Example 2 of the main part extraction method described above, and then an area corresponding to the background is created for each area. As a result, the feature amount (the ratio of the straight line portion included in the contour, the degree of line symmetry, the number of irregularities, the contact rate with the outer edge of the image, the density contrast in the area, the presence or absence of the density change pattern in the area, etc.) It is determined whether or not each area is a background area based on the feature amount, the area determined as the background portion is removed, and the remaining area is extracted as a non-background area (face candidate area).
8-122944, JP-A-8-183925, etc.).

The above extraction method is merely an example.
It goes without saying that any method can be applied as long as it is an extraction method for extracting an area estimated to correspond to a human face from an image. In step 100, the face candidate area extraction processing may be performed a plurality of times by applying a plurality of types of extraction methods, or the face candidate area extraction processing may be performed a plurality of times by changing the processing conditions by a single type of extraction method. You may do it.

By the face candidate area extraction processing in step 100, as shown in FIG. 3 as an example, a plurality of face candidate areas (face candidate areas A to D in FIG. 3) are extracted from the image to be processed. Note that step 100 corresponds to the extraction means described in claim 9. In the following process, the face candidate region extracted by the face candidate region extraction process is processed, but the extracted face candidate region itself may be the processing target, or the extracted face candidate region is circumscribed. A region having a fixed shape (for example, a circular region or a rectangular region) may be the processing target.

In step 102, all the face candidate areas extracted by the face candidate area extraction processing in step 100 are fetched, and in the next step 104, faces that overlap each other on the image are included in the fetched face candidate areas. It is determined whether there is a candidate area (see face candidate areas A, B, and C shown in FIG. 3 as an example). If the determination is affirmative, the process proceeds to step 106, and a single face candidate region pair (face candidate regions A and B, or face candidate regions A and C, or face candidate regions in the above example) that overlap each other is used. (B, C) are taken out as processing targets, and in the next step 108, the degree of overlap x of the face candidate areas is calculated based on the size of the taken out face candidate area pair. Note that step 108 corresponds to the calculating means described in claim 9 together with the above step 106.

The degree of overlap x is, for example, as shown in FIG. 4 (A), the face candidate area A and the face candidate area B overlap, the area of the face candidate area A is S _A , and the area of the face candidate area B is Assuming that S _B is the area of the overlapping region and S _AB is the area, calculation can be performed according to the following equation (1), for example.

[0059]   x = min (S_AB/ S_A, S_AB/ S_B)… (1) In equation (1), min () is the minimum value in the parentheses.
S is an operator that represents selecting_AB/ S_AIs a candidate face
Area S of area A_AArea S of the overlap area based on _ABThe regular
Value, S_AB/ S_BIs the area S of the face candidate area B_BBased on
And the area S of the overlap region_ABIs a normalized value of
Expression (1) is calculated for each of the overlapping face candidate area pairs A and B.
Normalize the area of the overlapping area based on the size (area)
The smaller one of these values is taken as the degree of duplication x. this
As shown in FIG.
By using a value that normalizes the area of the overlapping area,
Accurately quantify the degree of overlap regardless of the size of the complementary region
You can

In the equation (1), S _AB / S _A , S
Since value value obtained by _AB / S _B in accordance with degree of overlapping increases (according to the value of the area S _AB is increased) becomes larger, (1), the smaller value of both values (smaller overlap A value representing the degree) is used as the degree of overlap x. This can prevent the value of the degree of overlap x from becoming extremely large, for example, even when one face candidate area is included in the other face candidate area.

The degree of overlap x is calculated according to the following equation (2) or (3), where L _AB is the distance between the barycentric position of the face candidate area A and the barycentric position of the face candidate area B. May be.

[0062]   x = max (L_AB/ √ (S_A), L_AB/ √ (S_B))… (2)   x = max (L_AB ²/ S_A, L_AB ²/ S_B)… (3) In formulas (2) and (3), max () is the value in parentheses.
Is an operator that represents selecting the maximum value of_AB/ √
(S_A) And L_AB ²/ S_AIs the area S of the face candidate area A_ATo
Center-to-center distance L_ABIs the normalized value of L_AB/ √
(S_B) And L_AB ²/ S_BIs the area S of the face candidate area B_BTo
Center-to-center distance L_ABIs a normalized value of
Equations (2) and (3) are used for overlapping face candidate area pairs.
Distance between centers of gravity based on the size (area) of each of A and B
The larger value of the normalized values of
There is. Thus, as the degree of overlap x, the area of the face candidate region
Using a value that normalizes the distance between the centers of gravity based on
Therefore, the degree of overlap can be accurately determined regardless of the size of the face candidate area.
It can be digitized.

In equations (2) and (3), L _AB
/ √ (S _A ), L _AB / √ (S _B ), or L _AB ² /
Since the value obtained by S _A , L _AB ² / S _B becomes smaller as the degree of overlap becomes larger (as the value of the center-of-center distance L _AB decreases), both equations (2) and (3) are used. The larger one of the values (the value representing the smaller degree of duplication) is used as the degree of duplication x. As a result, it is possible to prevent the value of the degree of overlap x from becoming extremely small when, for example, one face candidate area is included in the other face candidate area.

Expressions (2) and (3) are the case where the distance between the centers of gravity of a pair of face candidate areas is applied as the "distance between candidate areas" described in claim 2, but the invention is not limited to this. Instead of applying the distance between representative points representing each position of the pair of face candidate areas as the “distance of the candidate area”, the degree of overlap x may be obtained by using a value obtained by normalizing the distance. Good. As the representative point, for example, a point corresponding to the center of a predetermined shape circumscribing or inscribing the face candidate area (for example, N-sided shape (N ≧ 3) or circular shape (may be a perfect circle or an ellipse)), or in advance It is possible to adopt a point obtained by the set specific calculation method.

As a point obtained by a specific calculation method set in advance, for example, a specific vertex of an N-shaped area (for example, a rectangular area) circumscribing the face candidate area, and a plurality of N-shaped areas A point corresponding to the center of gravity of the apex, the length along the long axis direction of the elliptical area from the upper end (upper side of the face in the vertical direction) of the elliptical area circumscribing the face candidate area And points corresponding to positions separated by a length corresponding to X% of

Further, the degree of overlap x is L _A , the length of the face candidate area _A along a predetermined direction (for example, the long side direction or the short side direction of the image), and the length of the face candidate area B along the predetermined direction. Is L _B , and the length along the predetermined direction of the overlapping region is L _AB ',
The calculation may be performed according to the following equation (4).

[0067] _{x = min (L AB '/} L A, L AB' / L B) ... (4) Thus, as the degree of overlapping x, overlapping relative to the length along the predetermined direction of the face candidate region By using a value obtained by normalizing the length of the region along the predetermined direction, the degree of overlap can be accurately digitized regardless of the size of the face candidate region.

Further, the expressions (1) and (2) (or (3)
The average value or weighted average value of the multiplicity x obtained by using two or more of the equations) and (4) may be used as the final multiplicity x. However, since the degree of overlap x obtained by the equations (2) and (3) and the degree of overlap x obtained by the equations (1) and (4) are different in the changing direction of the value with respect to the change of the degree of overlap, the average When calculating the value or the weighted average value, it is necessary to convert one value so that the changing directions of the values are aligned.

Further, regarding the weight given to the degree of overlap obtained by each arithmetic expression in the calculation of the weighted average value, face area extraction is performed in advance using the degree of overlap obtained by each arithmetic expression to determine the actual face area. The degree of coincidence can be evaluated and determined for each arithmetic expression. For example, the inventor of the present application may find that the degree of coincidence with the actual face area is higher in the equations (1) and (2) (or (3)) when the degree of overlap obtained by the equation (1) is used. It has been confirmed by experimentation that the number is large, and when the weighted average value of the multiplicity obtained by the equation (1) and the multiplicity obtained by the equation (2) (or the equation (3)) is taken as the multiplicity x,
It is preferable that the weight given to the multiplicity obtained by the equation (1) is made larger than the weight given to the multiplicity obtained by the equation (2) (or the equation (3)).

Further, instead of using the weighted average value of the degree of duplication obtained by each arithmetic expression as the degree of duplication x, the degree of duplication obtained by each arithmetic expression is converted into a weight score P (details will be described later), and A weighted average value of the number of weight points P corresponding to the arithmetic expression may be set as the final number of weight points.

In the next step 110, the number of weight points P to be given to the face candidate area pair to be processed extracted in step 106 is determined based on the degree of overlap x calculated as described above. The determination of the weight score P is performed in the previous step 10
In FIG. 8, when the overlapping degree x is calculated such that the value increases as the overlapping degree of the face candidate regions increases, the value of the overlapping degree x increases as shown in FIG. 4B as an example. A map whose conversion characteristic is determined so that the number P of weights increases with
Can be carried out by converting into the weighting point P. On the other hand, in step 108, when the degree of overlap x is calculated such that the value decreases as the degree of overlap of the face candidate area increases, as shown in FIG. 4C as an example, the value of the degree of overlap x It is possible to determine the weight score P by using a map whose conversion characteristic is determined such that the weight score P decreases as the value increases, and the degree of overlap x is converted into the weight score P using the map. .

The conversion characteristics shown in FIGS. 4B and 4C are merely examples. For example, in the conversion characteristics shown in FIGS. 4B and 4C, the number of weight points P is changed with respect to the change of the degree of overlap x. Although there is a dead region in which does not change, a conversion characteristic without such a dead region may be used. Alternatively, a conversion characteristic may be used in which the weight point P changes non-linearly with respect to the change of the degree of overlap x, and the weight point P is increased as the degree of overlap increases.
It suffices if the conversion characteristic increases.

When the number of weight points P is determined as described above, the previously determined weight points P are assigned to the face candidate area pairs to be processed (step 112), and in the next step 114, they are overlapped with each other. It is determined whether or not the processes of steps 106 to 112 have been performed for all the face candidate area pairs that are present. If the determination is negative, step 106
Then, the processing of steps 106 to 112 is repeated for other pairs of face candidate areas that overlap each other. As a result, the weighting points P are given to all the face candidate area pairs that overlap each other.

If the determination at step 114 is affirmative, the routine proceeds to step 116, where the weighted points P given to each face candidate area are added up for each face candidate area, and the integrated value is weighted for each face candidate area. Set as point P. For example, when the face candidate areas A to D shown in FIG. 3 are extracted from the image to be processed, the weight point P is not set for the face candidate area D because it does not overlap with other face candidate areas (the weight point P _D =
0), since the face candidate areas A to C overlap with the two face candidate areas, respectively, two weight points P are accumulated and set. Of these, the face candidate area C having the smallest degree of overlap with other face candidate areas has the smallest weight score P _C , and the face candidate area B having the degree of overlap with the face candidate area C larger than that of the face candidate area A (see FIG. 3, the weighting point P _B of the face candidate area having the highest degree of coincidence with the actual face area) becomes maximum.

The weight point P set for each face candidate area in the above step 116 corresponds to an evaluation value representing the evaluation of accuracy as an area corresponding to a person's face, and other face candidate areas. The weight score P for the face candidate area that overlaps with is higher than the weight score for the face candidate area that does not overlap with other face candidate areas, and the degree of overlap with other face candidate areas is high. The higher the value. The step 116 corresponds to the evaluation means described in claim 9 together with the steps 110 and 112.

By the way, when there is a face candidate area having extremely high or extremely low density in the face candidate areas extracted in step 100 and the face candidate area is not actually a face area, It is highly possible that the density of the face candidate area adversely affects the processing result of the post-processing performed after the face area is extracted. Therefore, in the next step 118, it is determined whether or not the face candidate area extracted in step 100 includes a face candidate area in which the density M in the area is extremely high or extremely low.

The density M is the average density (absolute value) in the area.
Or the relative density with respect to the average density over the entire screen of the image to be processed.
Is a threshold for determining whether or not the density M is extremely high.
This can be performed by comparing TH _HIGH and a threshold TH _LOW for determining whether or not the density M is an extremely low value. The threshold value TH _HIGH corresponds to the first predetermined value described in claim 8, and the threshold value TH _LOW corresponds to the second predetermined value described in claim 8.

If all the face candidate areas have the density M <threshold TH _HIGH and the density M> threshold TH _LOW , it can be judged that there is no face candidate area having extremely high or extremely low density. The determination of step 118 is denied and step 1
Move to 24. On the other hand, density M> threshold TH _HIGH or density M
<When there is a face candidate region that satisfies the threshold value TH _LOW , it can be determined that the density M of the face candidate region is extremely high or extremely low. Therefore, the determination at step 118 is affirmative and the process proceeds to step 120. , The density M is corrected to be extremely high or extremely low, and the weighting point P of the face candidate area is corrected so as to decrease, and then the process proceeds to step 124.

The correction of the weight point P can be performed by, for example, referring to FIG.
This can be done using the map shown in (A). This map has a conversion characteristic represented by a straight line that passes through the origin and has a slope smaller than 1 on the coordinate with the initial (original) number of weights on the horizontal axis and the corrected number of weights on the vertical axis. is doing. By converting (downward modifying) the number of weights P using the map as described above, even if the face candidate region with extremely high or extremely low density M is not actually the face region, It is possible to reduce the degree of the adverse effect of the region on the post-treatment. Note that step 118 and step 120 correspond to the invention of claim 8.

On the other hand, if there are no face candidate areas that overlap each other on the image among the face candidate areas extracted in step 100, the determination in step 104 above is denied and the process proceeds to step 122. , A constant value is set as the weighted score P of each face candidate area, and the process proceeds to step 124.

In step 124, the weighted point number P of each face candidate area is compared with the threshold value TH _F for face area determination, and the face candidate area with the weighted point number P equal to or greater than the threshold value TH _F is extracted (selected) as a face area. . When the determination in step 104 is affirmative (when there are face candidate areas that overlap each other), the weighting points for the face candidate areas that overlap with other face candidate areas and have a high degree of overlap are increased. Because
A face candidate area having a high probability of being a face area is extracted as a face area. The step 124 corresponds to the invention of claim 3.

In the next step 126, the following (5)
The face area density M of the image to be processed according to equation (6)
The face is calculated, and the face area extraction / density calculation processing ends.

[0083]

[Equation 1]

Here, i is a code for identifying each face candidate area, N is the total number of face candidate areas, Mi is the density of the face candidate area i, Pi is the weight score of the face candidate area i, and Si is the face candidate area. The area of i.

As is clear from the equations (5) and (6), the face area density Mface is a weighted average value of the densities M of the face candidate areas, and in the expression (5), the weighted score P of each face candidate area is P. Each face candidate area is weighted based on the equation (6), and each face candidate area is weighted based on the weight score P and the area S in the equation (6). When the determination in step 104 is affirmative (when there are face candidate areas overlapping each other),
Since the face candidate area that overlaps with other face candidate areas and has a high degree of overlap has a high weight point, the face area density Mface is higher than the average value of the density M of each face candidate area.
The value is close to the actual density of the face area.

By the way, it is empirically known that a face candidate region having a relatively small area has a relatively low probability of being a face region, but in the above equation (5), the area of each face candidate region is considered. Since the face area density Mface is calculated without performing the calculation, the face candidate area for which the face area density Mface is calculated includes a face candidate area that is relatively small in area and is not actually a face area. And the density M of the face candidate area is the face area density Mface
Has a great influence on. Therefore, when the face area density Mface is calculated by using the expression (5) (the expression (6) may be used), for example, the area S is equal to or less than a predetermined value, or the area S is smaller than other face candidate areas. The relatively small face candidate area is excluded from the calculation target of the face area density Mface, or
The face area density Mface may be calculated by reducing the value of the number of weight points P. As a result, it is possible to reduce the influence of the density M of the face candidate area, which has a relatively small area and is not actually a face area, on the face area density Mface.

In the above equation (6), since the face area density Mface is calculated so that the weight of the face candidate area having a large area S is relatively large, the face to be calculated for the face area density Mface is calculated. When the candidate area includes a face candidate area having a relatively large area and not actually a face area, the density M of the face candidate area has a large influence on the face area density Mface. Therefore, when the face area density Mface is calculated using the equation (6), for example, an upper limit value S _max is set for the value of the area S, and a face candidate area in which the value of the area S is larger than the upper limit value S _{max. May} replace the value of the area S with the upper limit value S _max (clipping), or may reduce the value of the number of weight points P. As a result, it is possible to reduce the influence of the density M of the face candidate area, which has a relatively large area and is not actually a face area, on the face area density Mface.
It should be noted that reducing the value of the weighting point P of the face candidate area in which the value of the area S is larger than the upper limit value S _max as described above is also applied to the case of calculating the face area density Mface using the equation (5). It is possible.

The step 126 corresponds to the invention of claim 4. Further, the face area density Mface corresponds to the image feature amount of the main part described in claim 4, but the image feature amount of the main part is not limited to the density, and various known image feature amounts may be used. It goes without saying that it is applicable.

When the face area extraction / density calculation processing described above is performed, the auto setup engine 44 further calculates processing conditions for various image processing executed by the image processor 40. The processing result of is used for the calculation of the processing condition of a part of the image processing. For example, the face area extracted in the previous step 124 is used for calculation of image processing (for example, sharpness correction or red-eye correction for the face area) executed by the image processor 40 for only the face area or a part thereof. The processing condition is set so that the image processing is performed only on the face area. The face area density Mface calculated in step 126 is used for image processing (for example, color / density correction) for the entire image executed by the image processor 40. Processing conditions such as density correction conditions are calculated so as to obtain a predetermined density.

As described above, since the face area extraction and the face area density Mface are performed using the weighted points set based on the degree of overlap x, the face candidate extracted by the face candidate area extraction processing is performed. Even if a face candidate region that is not actually a face region is mixed in the region due to erroneous extraction, the probability that a face candidate region that is not actually a face region is actually extracted as a face region is greatly reduced, It is also possible to prevent the face area density from changing significantly depending on the density of the face candidate area that is not actually the face area. Therefore, appropriate processing conditions are obtained even for each image processing in which the processing conditions are calculated using the face area extraction result or the face area density Mface, and the image processing is executed by the image processor 40 for fine scan image data. Appropriate processing results can be obtained for each image processing.

In the above description, the face candidate area having extremely high or extremely low density is corrected so that the weighting point is reduced. However, the present invention is not limited to this, and the face candidate area is changed to the face area. In the extraction of the face area, only the face candidate areas having extremely high or extremely low density, or all the face candidate areas are extracted as shown in FIG. ), The threshold TH _F may be changed so that the threshold TH _F for face area determination becomes higher. This corresponds to “raising the selection criterion” described in claim 8. As a result, it is possible to make it difficult for a face candidate region having extremely high or extremely low density to be extracted as a face region. The amount of change in the threshold TH _F may be a constant value, and it is determined whether the density M of the face candidate region whose density is judged to be extremely high or extremely low and whether the density M is extremely high or extremely low. threshold TH _HIGH, TH for
_The amount of change in the threshold TH _F may be changed according to the difference between _LOW and _LOW . Alternatively, instead of the above, when converting the calculated degree of overlap x into the weighted score P, if the density M of at least one face candidate region of the face candidate region pair to be processed is extremely high or low, You may make it suppress the weighting point P given to the said face candidate area pair of a process target.

The number of weight points P set for each face candidate area, the threshold TH _F for face area determination, or the face area density Mfa
The weight given to the density M of each face candidate area in ce may be changed according to the type of image processing performed using the processing result of the face area extraction / density calculation processing. The processing described below corresponds to the invention of claim 5.

For example, by utilizing the face area extraction result by the face area extraction / density calculation processing, the image processor 40 locally applies an edge enhancement filter only to the extracted face area to increase the sharpness of the face area. When the sharpness enhancement processing is performed, it depends on the degree of sharpness enhancement and the type of filter, but even if the sharpness enhancement is actually applied to a region that is not the face area, the visual effect is small (conspicuous). No) In such a case, the threshold value TH _F for face area determination is set smaller than usual (that is, the reference for selecting face candidate areas is changed), and more face candidate areas are determined to be face areas. You may do it. The lower the threshold value TH _F for face area determination, the lower the probability that a face candidate area corresponding to an actual face area will be erroneously determined to be not a face area. It is possible to perform sharpness enhancement processing without omission.

Further, instead of changing the value of the threshold value TH _F for face area determination, a larger value than usual is set as the weighting point P for the degree of overlap x calculated by calculation (that is, each face candidate). It is also possible to determine that more face candidate areas are face areas by changing the evaluation criteria for the areas). In particular, as the sharpness enhancement processing, when the processing of increasing the sharpness enhancement degree as the number of weight points P increases,
By setting the number of weight points P as described above, it is possible to strongly control the degree of sharpness enhancement.

Further, for example, using the face area extraction result by the face area extraction / density calculation processing and the face area density Mface, the density is locally corrected based on the face area density Mface only for the extracted face area. When the density correction process is performed, depending on the degree of the density correction, even if the density correction is actually performed on a region other than the face region, the adverse effect on the eyes may be small (not noticeable). In such a case, the value of the threshold TH _F for face area determination may be set smaller than usual so that more face candidate areas are determined to be face areas. The lower the threshold value TH _F for face area determination, the lower the probability that a face candidate area corresponding to an actual face area will be erroneously determined to be not a face area. The density correction processing can be performed without omission.

Further, instead of changing the value of the threshold value TH _F for face area determination, a larger value than usual is set as the weighting point P for the multiplicity x obtained by calculation, so that more It is also possible to determine that the face candidate area is a face area. In particular, as the density correction processing, when the processing for increasing the density correction degree as the weighting point number P increases, the density correction degree is strongly controlled by setting the weighting point number P as described above. Will also be possible.

In the above description, when the face area is extracted, the image processing that has a small influence is performed even when the area that is not actually the face area is erroneously extracted as the face area. When image processing that is greatly affected when a region that is not a face region is mistakenly extracted as a face region is performed, for example, the value of the threshold TH _F for face region determination may be set higher than usual, or the degree of overlap x On the other hand, by setting the weight point P to a value smaller than usual, it is possible to extract only the face candidate area having higher accuracy as the face area as the face area.

Regarding the face area density, for example, as shown in the following expression (7), the face area density Mface obtained by the above expression (5) (the expression (6) may be used) and other image feature amounts D
(For example, the average density of the entire image, the average density of the non-face candidate area, etc.) and the weighted average value Mface ′ (where α _F is a weighting coefficient for the face area density Mface and α ₀ is a weighting coefficient for the image feature amount D). When calculating as the face area density, the values of the weighting factors α _F and α ₀ are changed according to the type of image processing performed using the calculated face area density (that is, the weighting reference for each face candidate area is set relative to each other). The weight given to the density M of each face candidate area may be changed.

Mface ′ = α _F · M face + α ₀ · D (7) Further, as the image processing performed using the processing result of the face area extraction / density calculation processing, the processing result of the face area extraction / density calculation processing is performed. Multiple types of image processing with different requests (for example, image processing where it is desirable that the extracted face area does not actually include areas that are not face areas) When the image processing, etc. in which it is desirable to include the face area is performed, the extraction of the face area and the calculation of the face area density may be performed a plurality of times corresponding to each image processing. In the present embodiment, the degree of overlap x can be used as the reliability (accuracy) of each face candidate region as a face region, and as described above, the weight score setting criterion and the face region determination criterion for each face candidate region are used. (Threshold T
H _F ), by changing at least one of the weighting criteria for each face candidate area, the results required by each image processing can be obtained as the face area extraction result and the face area density calculation result. Even when a plurality of types of image processing are respectively performed, face candidate area extraction processing that is extremely complicated and time-consuming is performed in the same number as the number of types of image processing while changing the processing conditions corresponding to the plurality of types of image processing. It is not necessary to repeat the processing, the processing time for the face area extraction / density calculation processing can be shortened, and the performance of the image processing apparatus 14 can be improved.

In the above, when the auto-setup engine 44 calculates the processing conditions including the face area extraction / density calculation processing based on the prescan image data, and the image processor 40 performs the actual image processing for the fine scan image data. However, the present invention is not limited to this, and calculation of processing conditions for single image data and image processing under the calculated processing conditions may be performed in order, and a series of these processings may be performed. It may be performed by a single processing unit.

Furthermore, in the above description, the face area is extracted and the face area density is calculated based on the weighted points set for each face candidate area. However, the present invention is not limited to this. You may make it do only.

Further, in the above, the image data obtained by reading the image recorded on the photographic film was processed, but it is not limited to this, and it is recorded on other recording material such as paper. Image data obtained by reading an image, image data obtained by imaging with a digital camera, or image data generated by a computer may be the processing target. Further, it goes without saying that the present invention may be used for determining exposure conditions when a film image recorded on a photographic film is exposed and recorded on a photographic paper by surface exposure.

Further, although the case where the main part is the area corresponding to the face of the person in the image has been described above, the present invention is not limited to this. As an example, in mass production of parts, products, etc., while imaging the situation in which the produced parts, products, etc. are conveyed in sequence, an image representing the conveyed situation is extracted from the imaging signal at a predetermined timing. It is also possible to apply the present invention to the case of extracting a region corresponding to the part or product as a region corresponding to a main part from the extracted image. In this case, the extracted main part region can be used, for example, for automatically inspecting the produced parts or products.

[0104]

As described above, according to the inventions of claims 1 and 9, a plurality of candidate regions estimated to correspond to the main part in the image represented by the image data are extracted, and other candidate regions are extracted on the image. The degree of overlap is calculated for the candidate area that overlaps the area, and the accuracy of the accuracy of the candidate area that overlaps the other candidate areas as the area corresponding to the main part is the candidate area that does not overlap the other candidate areas. Since it is higher than the above-mentioned evaluation with respect to each of the candidate regions, and is higher as the degree of overlap with other candidate regions is higher, the accuracy as the region corresponding to the main part is evaluated for each candidate region. It has an excellent effect that it is possible to reduce the influence of erroneous extraction of a region estimated to correspond to the main part.

According to a second aspect of the present invention, in the first aspect of the invention, a value obtained by normalizing the distance between the pair of candidate regions based on the size of each of the pair of overlapping candidate regions as the degree of overlap. , A value obtained by normalizing the area of the overlapping area based on the size of each of the pair of candidate areas, and overlapping based on the size of each of the pair of candidate areas in the predetermined direction Since at least one of the values obtained by normalizing the size of the region in the predetermined direction is used, in addition to the above effect, the degree of overlap of the candidate regions can be accurately digitized regardless of the size of the candidate regions. Has the effect.

According to the invention of claim 3, in the invention of claim 1, by comparing an evaluation value representing a result of evaluating the accuracy of each candidate area as an area corresponding to a main part with a threshold value, Since a highly accurate candidate area is selected from the candidate areas as an area corresponding to the main part, in addition to the above effect,
A highly accurate candidate area as an area corresponding to the main part,
It has an effect that it can be selected with high accuracy by a simple process.

According to the invention of claim 4, in the invention of claim 1, according to an evaluation value representing a result of evaluating the accuracy of each candidate area or a candidate area selected from each candidate area as an area corresponding to a main part. Since the weighted average of the image feature amount of each candidate region or the selected candidate region is calculated as the image feature amount of the main part, the image feature amount of the main part is accurately calculated by a simple process in addition to the above effect. It has the effect of being able to seek.

According to a fifth aspect of the invention, in the first aspect of the invention, when predetermined image processing is performed after evaluating the accuracy of each candidate region as a region corresponding to a main part,
Depending on the type of predetermined image processing, change the criterion of accuracy evaluation as the area corresponding to the main part, or select a highly accurate candidate area as the area corresponding to the main part or weight each candidate area In addition to the above effects, there is an effect that an appropriate processing result can always be obtained regardless of the type of predetermined image processing.

According to a sixth aspect of the present invention, in the fifth aspect of the invention, the predetermined image processing is an image processing that has a great influence on the processing result if the candidate area is erroneously extracted or the candidate area is erroneously evaluated. In this case, the evaluation criteria are changed so that the accuracy evaluation criteria as the area corresponding to the main part becomes strict, or the selection is performed so that only the candidate areas evaluated to have higher accuracy are selected. Or the weighting reference is changed so that the weight for the candidate area evaluated to have low accuracy is relatively small. Even if there is an evaluation, it is possible to prevent the processing result of the predetermined processing from being greatly affected.

According to a seventh aspect of the present invention, in the fifth aspect of the present invention, the predetermined image processing is an image processing which is less affected by the processing result even if the candidate area is erroneously extracted or the candidate area is erroneously evaluated. In this case, the evaluation criterion is changed so that the accuracy evaluation criterion as the area corresponding to the main part becomes loose, or the selection criterion is selected so that the candidate area evaluated to have low accuracy is also selected. Or the weighting criterion is changed so that the weight for the candidate region evaluated to have a low accuracy is relatively large. Therefore, in addition to the above effects, erroneous extraction of the candidate region or erroneous evaluation of the candidate region may occur. In that case, the effect of the post-processing result is small, and an appropriate processing result can be obtained.

The invention according to claim 8 is the invention according to claim 1, wherein the density in the candidate region is higher than the first predetermined value or lower than the second predetermined value.
To reduce the evaluation of accuracy as a region corresponding to the main part, or to increase the selection criteria when selecting a highly accurate candidate region as a region corresponding to the main part,
In addition to the above effects, it is possible to prevent the post-processing processing result from becoming inappropriate even when a region having extremely high or extremely low density is erroneously extracted as a candidate region.

According to a tenth aspect of the present invention, the first step of extracting a plurality of candidate areas estimated to correspond to the main part of the image represented by the image data, the candidate overlapping the other candidate areas on the image The second step of obtaining the degree of overlap for an area, the evaluation of the accuracy as an area corresponding to the main part for a candidate area that overlaps with another candidate area, the evaluation for a candidate area that does not overlap with another candidate area Processing including a third step of evaluating the accuracy of each candidate area as an area corresponding to the main part, so that it becomes higher as the degree of overlap with other candidate areas becomes higher. Since the program to be executed by the computer is recorded on the recording medium, it is possible to reduce the influence of erroneous extraction of the region estimated to correspond to the main part of the image. It has an effect.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of an image processing system according to an embodiment.

FIG. 2 is a flowchart showing the contents of face area extraction / density calculation processing according to the present embodiment.

FIG. 3 is an image diagram showing an example of a processing result of face candidate area extraction processing.

FIG. 4A is an image diagram showing an example of a pair of overlapping face candidate regions, and FIGS. 4B and 4C are diagrams showing an example of a map for setting weight points according to the degree of overlap. Is.

FIG. 5A is a diagram showing a map for correcting a weight score for a face candidate area having extremely high or extremely low density, and FIG. 5B is a face candidate area having extremely high or extremely low density. FIG. 10 is a diagram illustrating a change in the threshold value for face area determination in the case where there is an object.

[Explanation of symbols]

10 Image processing system 14 Image processing device 40 image processor 44 Auto Setup Engine 72 Information storage medium

Claims (10)

(57) [Claims] 1. A candidate area estimated to correspond to a main part of an image represented by the image data is extracted based on the image data, and the extracted candidate area overlaps with other candidate areas on the image. For the candidate area that is being obtained, the degree of overlap with the other candidate area is obtained, and the accuracy of the candidate area overlapping the other candidate area as an area corresponding to the main part in the image is It is higher than the evaluation for a candidate area that does not overlap with a candidate area, and is higher as the degree of overlap with another candidate area is higher, and corresponds to the main part with respect to the extracted candidate area. An image processing method for evaluating the accuracy as a region. 2. A value obtained by normalizing the distance between the pair of candidate regions based on the size of each of the pair of overlapping candidate regions as the degree of overlap, and the size of each of the pair of candidate regions. A value obtained by normalizing the area of the overlapping region based on the above, and the size of the overlapping region along the predetermined direction based on the size of each of the pair of candidate regions along the predetermined direction. The image processing method according to claim 1, wherein at least one of the normalized values is used. 3. Corresponding to the main part from the extracted candidate region by comparing an evaluation value representing a result of evaluating the accuracy of the extracted candidate region as a region corresponding to the main part with a threshold value. The image processing method according to claim 1, wherein a candidate area having a high degree of accuracy as an area to be selected is selected. 4. The image feature of the main part is obtained by weighting the extracted candidate region or a candidate region selected from the extracted candidate regions according to an evaluation value representing a result of evaluation of accuracy as a region corresponding to the main portion. The image processing method according to claim 1, wherein a weighted average of image feature amounts of the extracted candidate region or the selected candidate region is calculated as the amount. 5. A candidate area having a high accuracy as an area corresponding to the main part based on the result of the evaluation after evaluating the accuracy of the extracted candidate area as an area corresponding to the main part. When performing a predetermined image processing including at least one of the selection and weighting of the extracted candidate region, according to the type of the predetermined image processing, change the evaluation criteria, or the selection or the weighting 2. The image processing method according to claim 1, wherein the criterion is changed. 6. The accuracy as an area corresponding to a main part when the predetermined image processing is an image processing in which a processing result is greatly affected by erroneous extraction of a candidate area or erroneous evaluation of the candidate area. Of the evaluation criterion is changed so that the evaluation criterion becomes stricter, or the selection criterion is changed so that only the candidate regions evaluated to have higher accuracy are selected, or the accuracy is low. The image processing method according to claim 5, wherein the weighting criterion is changed so that the weight of the evaluated candidate region becomes relatively small. 7. The accuracy as an area corresponding to a main part when the predetermined image processing is an image processing having a small influence on a processing result even if a candidate area is erroneously extracted or erroneously evaluated. Change the evaluation criteria so that the evaluation criteria becomes loose, or change the selection criteria so that the candidate area evaluated to have a low accuracy is also selected, or the accuracy is evaluated to be low. 6. The image processing method according to claim 5, wherein the weighting reference is changed so that the weight for the candidate region becomes relatively large. 8. For a candidate area whose density in the candidate area is higher than a first predetermined value or lower than a second predetermined value, the evaluation of accuracy as an area corresponding to the main part is lowered. 2. The image processing method according to claim 1, further comprising increasing a selection criterion when selecting a candidate region having high accuracy as a region corresponding to the main part. 9. Extracting means for extracting a candidate region estimated to correspond to a main part in an image represented by the image data based on the image data, and another candidate on the image among the extracted candidate regions. With respect to a candidate area that overlaps with an area, a calculating unit that obtains the degree of overlap with the other candidate area, and the accuracy as an area corresponding to the main part in the image with respect to the candidate area that overlaps with the other candidate area The evaluation is higher than the evaluation for a candidate area that does not overlap with another candidate area, and is higher as the degree of overlap with another candidate area is higher, with respect to the extracted candidate area. An image processing apparatus including: an evaluation unit that evaluates the accuracy as an area corresponding to the main part. 10. A first step of extracting a candidate region estimated to correspond to a main part in an image represented by the image data, based on the image data; The second step of obtaining the degree of overlap with the other candidate area for the candidate area overlapping with the candidate area, as the area corresponding to the main part in the image for the candidate area overlapping with the other candidate area The accuracy of the evaluation is higher than the evaluation for a candidate area that does not overlap with other candidate areas, and is higher as the degree of overlap with other candidate areas is higher. On the other hand, a recording medium in which a program for causing a computer to execute a process including a third step of evaluating accuracy as an area corresponding to the main part is stored.