JPH11316845A - Method and device for extracting main object - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a precise main object with excellent efficiency in accordance with images in various states by performing auxiliary object extraction whose characteristic is different when a synthesized result does not satisfy a decision criterion and extracting the main object by using the synthesized result and an extraction result by means of the auxiliary object extraction. SOLUTION: A synthesis and result deciding part 82 synthesizes face candidate areas which are extracted by a flesh-colored and circular extracting part 78 and a face outline and circular extracting part 80, and further decides whether a result of face extraction is appropriate or inappropriate for the synthesized face (candidate) area. When it is decided that an extraction result is inappropriate, a body and eye part extracting part 86 further performs face extraction by body extraction and circular extraction and face extraction by eye part extraction and circular extraction as auxiliary object extraction. The extraction result of a face area in the part 86 is sent together with a face area synthesized by the part 82 to a main part deciding part 88.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention belongs to the technical field of a main subject extracting method and apparatus for extracting a main subject from an original image such as an image photographed on a film, which is suitably used for a photo printer or the like.

[0002]

2. Description of the Related Art At present, an image photographed on a photographic film (hereinafter referred to as a film) such as a negative film or a reversal film is printed on a photosensitive material (photographic paper) by projecting an image of the film onto the photosensitive material. Surface-exposing the photosensitive material,
So-called direct exposure (analog exposure) is the mainstream.

On the other hand, in recent years, a printing apparatus using digital exposure, that is, an image recorded on a film is photoelectrically read, the read image is converted into a digital signal, and then various image processing is performed. 2. Description of the Related Art Digital photo printers have been put to practical use, in which image data (latent images) are recorded by scanning and exposing a photosensitive material with recording light modulated in accordance with the image data, and the resulting images are printed.

In a digital photo printer, exposure conditions at the time of printing can be determined by image data processing using an image as digital image data, so that image skipping and blurring caused by backlight, strobe photography, and the like can be corrected, and sharpness can be reduced. By suitably performing (sharpening) processing, correction of color or density feria, etc., it is possible to obtain high-quality prints that cannot be obtained by conventional direct exposure. Further, synthesis of a plurality of images, image division, and synthesis of characters can also be performed by image data processing, and a print that has been freely edited / processed according to the intended use can be output. Moreover,
According to a digital photo printer, not only can an image be output as a print (photo), but also image data can be supplied to a computer or the like or stored on a recording medium such as a floppy disk. It can be used for various purposes other than photography.

[0005] Such a digital photo printer basically includes a scanner (image reading device) for photoelectrically reading an image recorded on a film, and image data (exposure conditions) for processing the read image and outputting the image. An image input device having an image processing device, a printer (image recording device) for scanning and exposing a photosensitive material in accordance with image data output from the image input device to record a latent image, and an exposed photosensitive material And an image output device having a processor (developing device) that applies a developing process to the printed image.

In a scanner, reading light emitted from a light source is incident on a film to obtain projection light carrying an image photographed on the film, and the projection light is transmitted to an image sensor such as a CCD sensor by an imaging lens. The image is read by forming an image and performing photoelectric conversion, and after performing various types of image processing as necessary, the image is sent to the image processing apparatus as image data (image data signal) of the film. The image processing apparatus sets image processing conditions from image data read by a scanner, performs image processing according to the set conditions on the image data, and outputs the image data to a printer as output image data (exposure conditions) for image recording. send. In the printer, for example, if the device uses light beam scanning exposure, the light beam is modulated according to the image data sent from the image processing device,
The light beam is deflected in the main scanning direction and the photosensitive material is conveyed in a sub-scanning direction orthogonal to the main scanning direction, thereby exposing (burning) the photosensitive material with the light beam carrying an image to form a latent image. Then, the processor performs a developing process or the like in accordance with the photosensitive material to obtain a reproduced print (photograph) of the image photographed on the film.

[0007]

Regardless of such a digital photo printer or a general analog photo printer (hereinafter, collectively referred to as a photo printer), when a person exists in an image, A person's face is regarded as the most important, and its finish greatly affects the quality and evaluation of a reproduced image, that is, a print.

For this reason, in a conventional photo printer, a face (face area) of a person is extracted from an image photographed on a film, and the exposure condition, that is, the exposure condition, that is, the face is preferably finished.
A digital photo printer determines image processing conditions for obtaining image data for output, and an analog photo printer determines an exposure light amount, an insertion amount of a color filter, and the like. In particular, in a digital photo printer, as described above, image processing with very high flexibility can be performed by image data processing. Therefore, face information is used to set optimal image processing conditions according to an image. It becomes important information, and by performing appropriate face extraction, a high-quality print can be created more stably.

Under such circumstances, various methods of extracting a subject for extracting not only a face but also various main subjects have been proposed. For example, a method for extracting hue and saturation of an image has been proposed. Extraction of a flesh color, contour extraction by detecting an edge portion of an image, and the like are known. However, the state of an image photographed on film varies depending on various photographing conditions such as strobe photographing, backlit scene, over (excessive exposure), under (underexposure), color feria, and the like. It is difficult to perform face extraction stably and with high accuracy corresponding to the image of (1). On the other hand,
It is possible to increase the accuracy of face extraction by combining multiple subject extraction methods.However, in order to stably and accurately perform face extraction corresponding to images in various states, a large number of subjects are required. You need to combine extraction methods,
Inevitably, an increase in processing time for subject extraction cannot be avoided, resulting in a decrease in print production efficiency and the like.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems of the prior art. In digital and analog photo printers and the like, highly accurate extraction of a main subject corresponding to images in various states is performed. An object of the present invention is to provide a main subject extracting method and apparatus which can perform the processing with good efficiency.

[0011]

In order to achieve the above object, the present invention extracts a plurality of subjects having different characteristics,
By combining the subjects extracted in each subject extraction, if the combined result satisfies a predetermined criterion, a main subject is extracted from the combined result, and if the combined result does not satisfy the criterion, Further, the present invention provides a main subject extracting method, which comprises extracting auxiliary subjects having different characteristics, and extracting a main subject by using the synthesis result and the extraction result by the auxiliary subject extraction.

[0012] Preferably, the criterion is at least one of an area overlapping degree when combining the extracted images obtained by extracting the respective subjects, and a weighting score when combining the extracted images obtained by extracting the respective subjects. When the main subject is extracted from the synthesis result and the image feature amount of the extracted main subject exceeds a predetermined value, the auxiliary subject is extracted and the synthesis result and / or the main subject extraction is performed. It is preferable to extract the main subject using the result and the extraction result by the auxiliary subject extraction.
When a main subject is extracted from the synthesis result, the extracted main subjects are plural, and when there is a difference exceeding a predetermined value in the image feature amount between the plural main subjects, the auxiliary subject extraction is performed. Preferably, the main subject is extracted using the extraction result of the auxiliary subject extraction, and the subject extraction is at least two selected from skin color extraction, face contour extraction, and circular shape extraction. Is preferably one or more selected from hair part extraction, face internal structure extraction, body extraction, non-background part extraction, and circular shape extraction.

Further, the present invention provides a subject extraction unit having a plurality of extraction subunits for performing subject extraction with different characteristics, which are processed in parallel, and a condition determination unit for determining the subject extraction condition based on the result of each extraction subunit. A plurality of extraction units including at least one extraction unit are connected in cascade, and the condition determination unit weights or determines the extraction area of each subject extracted by the extraction subunit, and shifts the result. A main subject extracting apparatus, wherein at least one of the extracting units is selected.

Here, the extraction unit is a unit having one extraction sub-unit for extracting a subject and a condition determining unit for determining a condition for extracting the subject based on a result of the extraction sub-unit, or the extraction sub-unit. It is preferable to include at least one unit having only a condition determination unit that performs a condition determination of the extracted subject based on the result of the preceding cascade-connected extraction unit. In addition, the extraction sub-unit includes a skin color extraction unit, a face contour extraction unit, a circular shape extraction unit, a body extraction unit, an eye extraction unit,
Hair part extraction part, face internal structure extraction part, non-background part extraction part, skin color extraction, face outline extraction, circular shape extraction, body extraction,
It is preferable that the number is two or more selected from the extraction units that perform at least two types of extractions: eye extraction, hair extraction, face internal structure extraction, and non-background extraction. Further, the condition determination unit combines candidates of subjects having different characteristics extracted by a plurality of extraction subunits to be processed in parallel, and obtains a result of the subject extraction from an extracted region of the combined plurality of candidates. It is preferable to include a combining / result determining unit for determining whether the combination is appropriate. Further, it is preferable that the condition determination unit of the last extraction unit of the cascade-connected extraction units is a main subject determination unit that extracts a main subject.

Further, the present invention provides a subject having a plurality of types of extraction sub-steps for extracting subjects having different characteristics to be processed in parallel and a condition judgment sub-step for judging the subject extraction condition based on the result of each extraction sub-step. An extraction step including at least one extraction step is cascade-connected in a plurality of stages. In the condition determination sub-step, weighting or determination is performed on the extraction area of each subject extracted in the extraction sub-step, and The main subject extraction method is characterized in that at least one of the extraction steps at the subsequent stage of the transition to the above is selected.

Here, the extracting step includes a kind of extracting sub-step for extracting a subject, a condition determining sub-step for determining a condition for extracting the subject based on a result of the extracting sub-step, and the extracting sub-step. It is preferable to include at least one of the steps having only a condition determination sub-step of performing the condition determination of the extracted subject candidate based on the result of the previous cascade-connected extraction step.
In addition, the extraction sub-step includes skin color extraction, face outline extraction, circular shape extraction, body extraction, eye extraction or face internal structure extraction,
It is preferable to include two or more types selected from a hair part extraction, a non-background part extraction, and a combination of at least two types of extraction. The condition determination sub-step may include synthesizing candidates of subjects having different characteristics extracted in the plurality of extraction sub-steps to be processed in parallel, and extracting a result of the subject extraction from the synthesized plurality of candidate extraction areas of the subject. It is preferable to include a combining / result determining step of determining whether or not is appropriate. Preferably, the condition determination sub-step of the last extraction step of the cascade-connected extraction steps is a main subject determination step of extracting a main subject.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, a method and apparatus for extracting a main subject according to the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

FIG. 1 is a block diagram showing an example of a digital photo printer using the main subject extracting method and apparatus of the present invention. The digital photo printer (hereinafter, referred to as a photo printer 10) shown in FIG. 1 basically includes a scanner (image reading device) 12 that photoelectrically reads an image captured on a film F, and read image data ( An image processing device 14 that performs image processing of image information and the entire operation and control of the photo printer 10, and images a photosensitive material (photographic paper) with a light beam modulated according to image data output from the image processing device 14. A printer 16 that exposes, develops, and outputs (finished) prints. The image processing apparatus 14 also includes a keyboard 18a and a mouse 1 for inputting (setting) various conditions, selecting and instructing processing, and inputting instructions such as color / density correction.
An operation system 18 having an 8b and a display 20 for displaying an image read by the scanner 12, various operation instructions, a condition setting / registration screen, and the like are connected.

The method and apparatus for extracting a main subject according to the present invention not only use such a digital photo printer, but also photoelectrically read an image photographed on a film.
A conventional method of setting exposure conditions such as an exposure light amount such as an aperture amount and a color adjustment amount such as an insertion amount of a color filter from obtained image data, and exposing a photosensitive material with projection light of a film according to the exposure conditions. The present invention can be suitably used for an analog photo printer by direct exposure.

The scanner 12 is a device for photoelectrically reading an image photographed on a film F or the like one frame at a time.
A variable aperture 24, a diffusion box 28 for making the reading light incident on the film F uniform in the surface direction of the film F, an imaging lens unit 32, R (red), G (green), and B
An image sensor 34 having a line CCD sensor corresponding to each image reading of (blue), an amplifier 36,
An A / D (analog / digital) converter 38.

In the photo printer 10, the type and size of a film such as a new photographic system (Advanced Photo System) or a 135-size negative (or reversal) film, the form of the film such as strips and slides, and the like are determined. In addition, a dedicated carrier that can be mounted on the main body of the scanner 12 is prepared, and by changing the carrier, it is possible to cope with various films and processes. An image (frame) photographed on a film and provided for printing is conveyed to a predetermined reading position by this carrier. When reading an image photographed on the film F in such a scanner 12, the reading light emitted from the light source 22 and the light amount of which is adjusted by the variable aperture 24 is transferred to the film F positioned at a predetermined reading position by the carrier. , And is transmitted therethrough to obtain projection light carrying an image photographed on the film F.

As shown schematically in FIG. 2A, the carrier 30 in the illustrated example extends the line CCD sensor of the image sensor 34 (main scanning direction) while the film F is positioned at a predetermined reading position. Direction), a pair of transport rollers 30a and 30b arranged to sandwich the reading position in the sub-scanning direction and transport the film F in a sub-scanning direction perpendicular to the sub-scanning direction. And a mask 40 having a slit 40a extending in the main scanning direction and positioned in a main scanning direction corresponding to the reading position.
The film F receives the reading light while being transported in the sub-scanning direction while being positioned at the reading position by the carrier 30. As a result, the film F is two-dimensionally slit-scanned by the slit 40a extending in the main scanning direction, and the image of each frame captured on the film F is read.

Reference numeral 44 in the figure denotes a code reader for reading a bar code such as a DX code, an extended DX code, and an FNS code optically recorded on a film. Further, a magnetic recording medium is formed on the film of the new photographic system, and a carrier corresponding to the film (cartridge) of the new photographic system reads information recorded on the magnetic recording medium, and A magnetic head for recording information is provided. When reading an image on the film F, magnetic information is read by the magnetic head, and necessary information is sent from the scanner 12 to the image processing device 14.

As described above, the reading light is transmitted through the film F held by the carrier 30 and becomes projection light for carrying an image. The projection light is formed on the light receiving surface of the image sensor 34 by the imaging lens unit 32. Imaged. FIG. 2 (B)
The image sensor 34 includes a line CCD sensor 34R for reading an R image, a line CCD sensor 34G for reading a G image, and a line CCD sensor 34B for reading a B image. Color CCD sensor for each line CC
The D sensor extends in the main scanning direction as described above.
The projection light of the film F is separated into three primary colors of R, G and B by the image sensor 34 and read photoelectrically. The output signal of the image sensor 34 is
The digital signal is amplified by the A / D converter 38, converted into a digital signal by the A / D converter 38, and sent to the image processing device 14.

The scanner 12 reads an image photographed on the film F by two image readings: a prescan for reading at a low resolution and a main scan for obtaining image data of an output image. The pre-scan is performed under pre-scan reading conditions set in advance so that the image of all the films targeted by the scanner 12 can be read without saturation of the image sensor 34. On the other hand, the main scan is performed under the main scan reading conditions set for each frame from the pre-scan data so that the image sensor 34 saturates at a density slightly lower than the minimum density of the image (frame). The output signals of the pre-scan and the main scan are basically the same data except that the resolution and the output level are different.

In the digital photo printer using the method and apparatus for extracting a main subject according to the present invention, the scanner is not limited to such a slit scanning reading, and the reading light is applied to the entire surface of one frame image. The reading may be performed at a time, using surface reading. In this case, for example, using an area CCD sensor, providing means for inserting R, G, and B color filters between the light source and the film F, inserting a color filter, and reading an image with the area CCD sensor. Is sequentially performed by each of R, G, and B color filters, and the image photographed on the film is separated into three primary colors, and sequentially performed.

As described above, the digital signal output from the scanner 12 is output to the image processing device 14 (hereinafter, referred to as the processing device 14). FIG. 3 shows a block diagram of the processing device 14. The processing device 14 includes a data processing unit 48,
Log converter 50, prescan (frame) memory 5
2. It has a main scan (frame) memory 54, a pre-scan processing unit 56, a main scan processing unit 58, and a condition setting unit 60. FIG. 3 mainly shows a portion related to image processing.
A CPU for controlling and managing the entire photo printer 10 including the processing device 14, a memory for storing information necessary for the operation of the photo printer 10, and the like are arranged.
And the display 20 are connected to each part via the CPU or the like (CPU bus).

The R, G, and B digital signals output from the scanner 12 are subjected to predetermined data processing such as darkness correction, defective pixel correction, and shading correction in a data processing unit 48, and then are Log-converted. Is converted by the device 50 into digital image data (density data),
The pre-scan data is stored (stored) in the pre-scan memory 52 and the main scan data is stored in the main scan memory 54. The pre-scan data stored in the pre-scan memory 52 is stored in an image data processing unit 62 (hereinafter, referred to as an image data processing unit 62).
A pre-scan processing unit 56 having an image data converting unit 64 and a main scan memory 5
The main scan data stored in the image data processing unit 66 is referred to as an image data processing unit 66 (hereinafter, referred to as a processing unit 66).
8 is read out and processed by the main scan processing unit 58 having.

A processing section 62 of the prescan processing section 56;
A processing unit 66 of the main scan processing unit 58 is a unit that performs predetermined image processing on an image (image data) read by the scanner 12 according to processing conditions set by a condition setting unit 60 described later. , Except that the resolution is different. The image processing by both processing units is not particularly limited, and various known image processings are exemplified. For example, gray balance adjustment using a LUT (look-up table), gradation correction, and density (brightness)
Adjustment, correction of shooting light source type by matrix (MTX), image saturation adjustment (color adjustment), other, low-pass filter,
Electronic scaling processing, dodging processing (compression / expansion of density dynamic range), sharpness (sharpness) using an adder, LUT, MTX, or the like, and using averaging processing or interpolation calculation or the like in which these are appropriately combined. Process) is exemplified. These image processing conditions are set in a condition setting unit 60 described later using the prescan data.

The image data conversion unit 68 converts the image data processed by the processing unit 66 into, for example, 3D (three-dimensional) data.
-Convert using an LUT or the like, and supply to the printer 16 as image data corresponding to image recording by the printer 16. The image data conversion unit 64 thins out the image data processed by the processing unit 62 as necessary,
The image data is converted using a 3D-LUT or the like, and converted into image data corresponding to the display on the display 20.
To supply. The processing conditions in both cases are set by the condition setting unit 60 described later using the prescan data.

The condition setting section 60 includes a pre-scan processing section 5
6 and various processing conditions in the main scan processing section 58 and the reading conditions for the main scan are set. The condition setting section 60 includes a setup section 72, a key correction section 74, and a parameter integration section 76.

The setup section 72 includes a main section extraction section 72A (hereinafter, referred to as an extraction section 72A) for extracting a main section (main subject) of the image using the prescan data, and a main section extracted by the extraction section 72A. According to the information, the pre-scan data, and the operator's instruction performed as necessary, the main scan reading conditions are set and supplied to the scanner 12, and the images of the pre-scan processing unit 56 and the main scan processing unit 58 A setting unit 72B that creates (calculates) a processing condition and supplies it to the parameter integrating unit 76;
The extraction unit 72A constitutes a main subject extraction device according to the present invention that performs the main subject extraction method according to the present invention.

FIG. 4 is a block diagram showing an embodiment of an extracting section 72A which is a main subject extracting apparatus of the present invention for implementing a main subject extracting method of the present invention for extracting a human face as a main part. The invention is not limited to this.
The extracting unit 72A includes a skin color / circular shape extracting unit 78, a face contour / circular shape extracting unit 80, a combining / result determining unit 82, a switching unit 84,
It has a torso / eye part extraction unit 86 and a main part determination unit 88.

The skin color / circular shape extraction unit 78 extracts a face region as a main part by performing skin color extraction and circular shape extraction. That is, the skin color / circular shape extraction unit 78 detects the hue and saturation of each pixel from the pre-scan data (thinning may be performed as necessary), and determines a pixel area (skin color) that can be estimated as a skin color corresponding to human skin. Region). Since the human face is generally elliptical, the skin color / circular shape extraction unit 7
In step 8, a (elliptical) circular shape which is presumed to be a human face is extracted from the extracted skin color region and is set as a face candidate region.

The face contour / circular shape extraction unit 80 extracts a face region as a main part by performing edge extraction and circular shape extraction. That is, the face outline / circular shape extraction unit 80
First, binarizes pre-scan data (can be thinned out as necessary) with a predetermined threshold, and outputs an area composed of pixels having a density lower than the threshold (if the film F is negative, it is output Since the density is high on a print, this area is divided into a “black area” and an area composed of pixels having a density higher than the threshold (also a “white area”).
Next, the contour of each black region and white region is traced, the curvature of the contour is calculated, and the concave portion and the convex portion are extracted. Alternatively, edge detection by differential filtering and binarization by a predetermined threshold may be performed, and the contour may be traced based on the edge. Next, from the traced contour line, a line (specific line) that is estimated to correspond to the side of the human face is extracted from the line length, curvature, unevenness, and even symmetry with other lines, Sets the axis of symmetry of a pair of symmetric specific lines, sets the ratio of the interval and length of the corresponding lines (velocity ratio), and the line symmetry similarity of the lines obtained from image data, binarized data, etc. , Lines that can be assumed to form both sides of the human face (assumed lines) are extracted. Furthermore, it is determined whether there is a line having a vector in a direction substantially orthogonal to the hypothetical line at the top and bottom of the hypothetical line, that is, at a position corresponding to the boundary between the face, the hair, and the torso (neck and shoulder). If detected and present,
Considering the previously determined line symmetry similarity as necessary,
This line is determined as a line forming both sides of the human face. Next, in a region surrounded by the line, an elliptical shape corresponding to the above-mentioned history ratio is extracted and set as a face candidate region.

The combining / result determining section 82 combines the face candidate areas extracted by the skin color / circular shape extracting section 78 and the face contour / circular shape extracting section 80, and further, from this combined face (candidate) area. Then, it is determined whether the result of the face extraction is appropriate or inappropriate. In the illustrated example, two types of faces with different characteristics are extracted to synthesize and determine a face region. However, the present invention is not limited to this, and three or more types of face extraction with different characteristics are extracted. May be performed to perform synthesis and determination.

As shown in FIG. 5, for example, as shown in FIG. 5, the combining / result determining section 82 includes an area A extracted by the skin color / circular extracting section 78 and an area B extracted by the face contour / circular extracting section 80. Is obtained, and if the area ratio C / A and the area ratio C / B both exceed a predetermined threshold (Th), it is determined that the face extraction result is appropriate, and In the case of, it is determined to be inappropriate. Compositing / result determining unit 8
As another method of the determination method in 2, there is exemplified a method of performing weighting according to the reliability of the extracted area according to the extraction method and the like, and determining based on the total point. For example,
Referring to FIG. 5, one pixel of area A extracted by flesh color / circular shape extraction unit 78, one point of pixel of area B extracted by face outline / circular shape extraction unit 80, and both areas overlap. Assuming that the pixels in the region C are three points, the total point is calculated. If the total point exceeds a certain threshold, the face extraction result is determined to be appropriate, and if the total point is equal to or less than the threshold, the face extraction result is determined to be inappropriate. In the combining / result determining unit 82,
When both the determination based on the overlapping area and the determination based on the total points of the weighted points are performed, and when both are determined to be appropriate, or when either one is determined to be appropriate,
It may be determined that the face extraction result is appropriate.

In the synthesizing / result determining section 82, the criterion (the threshold in the above example) of the face extraction result may be changed as necessary. For example, in the case where it is considered that the area of the main part occupying the image is large due to the close-up of a person's face or the like (it may be determined based on the above-described skin color area), or in a flash shooting scene, it is necessary to obtain a high-quality reproduced image. Since it is preferable to perform face extraction with high accuracy, the criterion may be strict. In this case, the criterion may be set stepwise according to the skin color area or the like. Conversely, in the case of a normal portrait or the like, if the criterion is too strict, the face area to be extracted may be dropped. Therefore, the criterion may be loosened. The adjustment of the criterion may be performed by the extraction unit 72A (synthesis / result determination unit 82) according to the analysis result of the image (data), or the operator may determine and input an instruction. In the case of a film, a strobe shooting scene or the like may be detected from magnetic information and adjusted.

When the combining / result determining section 82 determines that the extraction result is proper, the combined face area (information thereof) is sent to the main section determining section 88, and the main section of the image is determined. Is done. The method of determining the main part (main subject) of the image in the main part determining unit 88 is not particularly limited. For example, the entire region extracted by the skin color / circular shape extracting unit 78 and the face contour / circular shape extracting unit 80 is used as the main part. May be determined,
The overlapping part of the extraction result by both may be determined as the main part. Information on the main part (main part area) of the image determined by the main part determination unit 88 is supplied to the setting unit 72B.

On the other hand, when the combining / result determining unit 82 determines that the extraction result is inappropriate,
In the torso / eye part extraction unit 86, face extraction by torso extraction and circular shape extraction, and face extraction by eye part extraction and circular shape extraction are performed as auxiliary subject extraction. When the combining / result determining unit 82 determines that the combined face extraction result is incorrect, the auxiliary subject extraction to be further performed is not limited to two types, and may be less than or more than this. A number of face extractions may be performed.

As described above, the accuracy of face extraction can be improved by combining a large number of subject extraction methods. However, if many subject extraction methods are performed, computation and image processing take time. Efficiency decreases. On the other hand, the main subject extraction method of the present invention first performs a combination of a plurality of subject extraction methods having different characteristics, thereby ensuring sufficient accuracy and improving efficiency in normal images. Only when the main subject is extracted and sufficient accuracy cannot be ensured according to the image state or the like, the main subject is extracted with higher accuracy by using another subject extraction together. Therefore, according to the present invention, the main subject can be extracted with good efficiency and sufficient accuracy. In particular, face extraction using the above-described skin color extraction, face contour extraction, and circular shape extraction is a low resolution and relatively simple image analysis,
Relatively high-accuracy face extraction can be performed, and in the case of a normal image, a combination of both extraction methods can achieve sufficient-accuracy face extraction.

In the torso / eye extraction unit 86, face extraction by torso extraction and circular shape extraction is performed, for example, as follows. First, in the same manner as in the above-described face outline extraction, a white area and a black area are discriminated by binarizing an image, and the outline of both areas is traced. Next, from the traced contour line, a set of lines whose end points are close to each other and whose angle is within a predetermined range is regarded as corresponding to the neck to shoulder of the torso, the shoulder to the arm or the lower part of the torso. It is extracted as a set of lines that can be one side. Next, the two lines are extended until they are connected to generate a line that is a candidate for one side of the body contour (one side candidate line), and a line that may form a pair of each one side candidate line and the body contour (a symmetric line). ) To extract. This extraction may be performed based on, for example, whether or not there is a possibility of forming a recess facing the one-side candidate line. Further, a line symmetry axis between each one-side candidate line and the symmetry line is set, and it is determined whether an angle formed by both lines and the line-symmetric axis is within a predetermined angle. From the result, a one-side candidate that can be determined as a body contour is obtained. A pair of a line and a symmetric line is extracted. Lastly, the position, size,
On the basis of the interval and the direction of the axis of line symmetry, a region estimated to form an elliptical shape corresponding to the human face having the body contour is extracted as a face candidate region.

On the other hand, in the torso / eye extraction unit 86, extraction of a face internal structure such as eye extraction and face extraction by circular shape extraction are performed, for example, as follows. First, a white region and a black region are binarized by binarizing an image in the same manner as in the above-described face outline extraction, and an elliptical shape having a long axis / single axis ratio in a predetermined range obtained from general human eyes is obtained. A black region is extracted as a region that may correspond to the eye, which is one of the face internal structures (referred to as an eye candidate region). Next, an angle of the major axis of the extracted eye candidate region is obtained, an elliptical black region in which the angle difference of the major axis is within a predetermined range is obtained, and an eye candidate region forming a pair with the previously extracted eye candidate region. Extract as For each pair of eye candidate regions extracted in this manner, a line symmetry axis perpendicular to the line connecting the centers of the binocular candidate regions is set, the line symmetry similarity is determined, and the eye A black region that can be estimated as a part is extracted. Finally, based on the position, size, and direction of the axis of symmetry of the black region that can be estimated as the extracted eye,
A region estimated to form an elliptical shape corresponding to the face of a person who looks at the black region as an eye is extracted as a face candidate region. The extracted face internal structure is preferably an eye, but the present invention is not limited to this. The mouth, the nose, the ear, and the like may be extracted in addition to the eye.

The result of extraction of the face region by the torso / eye portion extraction section 86 is sent to the main portion determination section 88 together with the face area synthesized by the synthesis / result determination section 82 described above. The main part determination unit 88 includes a face region synthesized by the synthesis / result determination unit 82, a face region extracted by the torso / eye extraction unit 86 by eye extraction and circular shape extraction, a torso extraction and circular shape The extracted face area is combined with the extracted face area, and an area that satisfies a predetermined criterion is set as the face area of the image, and the information is supplied to the setting unit 72B. For this determination, for example, a method of determining only an overlapping region of all extraction methods as a face region, or setting a weighted score for each extraction method, and defining a region of pixels whose total score is equal to or greater than a predetermined score as a face region A determination method is exemplified.

In the same manner as in the operation performed by the combining / result determining section 82, the face regions extracted by the full extraction method are combined to determine the suitability of the extracted result. The determination of the area may be performed. If it is determined that the extraction result is not appropriate, measures such as setting image processing conditions for not performing face extraction and redoing image reading may be performed.

In the present invention, the method of extracting a subject is not limited to this, and for example, extraction of a hair portion, extraction of a non-background portion, and the like can be suitably used. These are preferably exemplified as auxiliary information for auxiliary subject extraction and face extraction.

As a method of extracting the hair portion, for example, the following method is exemplified. First, a white area and a black area are divided by binarizing an image in the same manner as in the face outline extraction described above, and the outline of the black area is traced to calculate the curvature of the outline. Next, from this curvature, a black region including both a convex portion that is considered to correspond to the top of a person and a concave portion that is considered to correspond to the boundary between the face and the hair portion is extracted, and the characteristic amount of the concave portion and the convex portion is extracted. For example, by calculating a circular curvature, using the circular curvature, a contour that can be estimated to be a person's hair part from the degree of circularity, a ratio of a concave part to a convex part with respect to the contour perimeter, the position and direction of the contour, and the like. Is extracted, and an area that is presumed to form an elliptical shape corresponding to the face of the person having the hair is extracted as a face candidate area.

As a method of extracting the non-background portion, for example, the following method is exemplified. First, a white region and a black region are divided by binarizing an image in the same manner as in the above-described face outline extraction, and the outline of each region is traced to calculate the ratio of straight lines in the outline. A weight coefficient is assigned to the region. Then
A line symmetry axis extending in the vertical direction of the image is set for each contour, the degree of line symmetry is calculated, and a weight coefficient is assigned to an area where the degree of line symmetry is equal to or less than a predetermined value. Next, the contact rate of each contour with the outer edge of the image is calculated, and a weight coefficient is assigned to an area where the contact rate is equal to or more than a predetermined value. Next, the density contrast inside each area is calculated, and a negative weight coefficient is given to the area where the contrast is equal to or less than a predetermined value. Further, the density value inside each area is differentiated in a plurality of different predetermined directions, and an area in which any of the differential values along the plurality of directions changes regularly, that is, the density changes in a predetermined pattern, Alternatively, a weight coefficient is assigned to a region where this is repeated. All of the regions to which the weight coefficient is given by the above operation are regions where the probability of being a person is low, and the region where the weight coefficient is equal to or more than a predetermined value can be determined to be the background portion of the image. This can be used for face extraction.

It should be noted that the various subject extraction methods described above
It is described in detail in Japanese Patent Application Laid-Open No. 8-184925, etc. of the present applicant. In addition, in the present invention, various other known subject extraction methods can be used. For example, Japanese Patent Application Laid-Open Nos. 4-346332, 4-346333, and 4
-346334, 5-100328, 5-15
Nos. 8164, 5-165119 and 5-16512
No. 0, No. 6-67320, No. 6-160992, No. 6-160993, No. 6-160994, No. 6-1
No. 60995, No. 8-122944, No. 9-8065
No. 2, 9-101579, 9-138470,
Various types of subject extraction methods disclosed in each publication of JP-A-9-138471 are exemplified.

In various subject extraction methods, the accuracy of face extraction tends to be lower in flash photography and backlight scenes, and the adverse effect on image quality due to erroneous extraction is larger than usual. It is preferable to perform face extraction with high accuracy. Therefore, in the main subject extracting method of the present invention, the combining / result determining unit 82 determines that the extraction result is appropriate, and uses the face area combined by the combining / result determining unit 82 to determine the main part determining unit 88. Even when the main part is determined in the above, if the image feature amount of the determined main part is out of the predetermined range, if the image density of the main part exceeds the predetermined range, the average of the image density of the main part and the entire image density If the value difference exceeds the specified range,
By performing auxiliary subject extraction performed by the eye extraction unit 86 and the like,
It is preferable to determine the main part in consideration of the extraction result.

Similarly, when the combining / result determining section 82 determines that the extraction result is appropriate and the main portion determining section 88 determines the main part using the combined face area, a plurality of areas are determined. Also, when there is a difference in the density value between the main parts that is equal to or more than a predetermined value, the auxiliary subject extraction performed by the torso / eye part extraction unit 86 and the like is performed, and the main part is extracted in consideration of the extraction result. Is preferably determined. In the case where the density difference between the dark part and the light part is large in flash photography or the like, and when the main part is extracted in both the light part and the dark part, the image processing conditions are usually set by simply averaging both parts. . However, in such a case, it is often the case that only one of the bright part and the dark part is appropriate, and erroneous extraction of the main part adversely affects the image quality. Therefore, in this case as well, it is preferable to perform face extraction with high accuracy, and it is preferable to perform auxiliary subject extraction.

FIG. 4 shows a specific embodiment of the main subject extracting apparatus of the present invention for implementing the main subject extracting method of the present invention.
The main part extraction unit 72A of the setup unit 72 shown in FIG. 7 is basically configured as described above, but the present invention is not limited to this, and the main subject extraction device having the configuration shown in FIGS. May be applied to the main part extraction unit 72A. The main subject extraction device 90 constituting the main portion extraction unit 72A shown in FIG. 6 includes a start (start) unit 91 and N (N
, Cascade-connected N extraction units 92 (92 ₁ ), 92 ₂ ,..., 92 _N and an end unit 93. Here, the start unit 91
This is a part for transferring necessary image data to the following N-stage extraction units 92 to read out pre-scan data from the pre-scan memory 52 and start various subject extraction algorithms. The ending unit 93 obtains information on the main subject obtained by performing the main subject extraction method according to the present invention, and sets the setting unit 7 of the setup unit 72.
This is a part for sending to 2B. The extraction unit 92 is a part for extracting a subject serving as a main part, and what kind of subject is extracted is not particularly limited, and various kinds of main subject may be extracted as necessary. The extraction units 92 ₁ , 92 ₂ ,
……, 92 _N need to be cascaded,
It is preferable that the main subject is extracted by a different subject extraction algorithm.

As shown in FIG. 7, each extraction unit 92 extracts Mn (k (k is an integer of 0 or more in the illustrated example)) kinds of extraction subunits 94 (94 ₁ ) for extracting subjects having different characteristics to be processed in parallel. ),..., 94 _k, and a condition determination unit 96 for determining a condition for subject extraction based on the result of each extraction subunit 94. The number Mn of the extraction subunits 94 in each extraction unit 92 represents the number of the extraction subunits 94 of the n-th stage extraction unit 92 _n. May be. That is, the extraction unit 92 includes at least one unit having at least two types of extraction sub-units 94 to be processed in parallel and a condition determination unit 96 for performing a condition determination for subject extraction based on the result of each extraction sub-unit 94. However, in addition to this, it may include at least one unit having only one extraction subunit 94 or one unit having one extraction subunit 94 and the condition determination unit 96, or may not include at least one extraction subunit 94. At least one unit having no sub-unit and having only the condition determination unit 96 for determining the condition of the extracted subject based on the result of the cascade-connected extraction unit 92 in the preceding stage may be included.

Here, the extraction sub-unit 94 is not particularly limited as long as it extracts a subject having different characteristics, and may use any method or algorithm, or any subject. May be extracted. For example, when a human face is extracted as a main part, the extraction subunit 94 includes a skin color extraction unit, a face outline extraction unit, a circular shape extraction unit, a body extraction unit, an eye extraction unit, or a face internal structure extraction unit. A hair part extraction part, a non-background part extraction part, and at least two kinds of extractions of skin color extraction, face contour extraction, circular shape extraction, body extraction, eye extraction or face internal structure extraction hair part extraction and non-background part extraction Although an extraction unit to be used can be mentioned, preferably, the extraction unit 92 has two or more selected from these various extraction units.

Incidentally, as in the extraction unit 72A shown in FIG.
The extraction unit includes a skin color / circle extraction unit 78 and a face outline / circle extraction unit 80 in one extraction unit, and the torso / eye extraction unit 86 is one extraction subunit. It may be configured to be provided in a unit. In the present invention, as shown in FIG.
The skin color / circular shape extracting unit 78, the face contour / circular shape extracting unit 80, and the torso / eye unit extracting unit 86 that perform two types of extraction steps or algorithms may be used as one extraction subunit. Of course, a skin color extraction unit, a face outline extraction unit, a circular shape extraction unit, a body extraction unit, an eye extraction unit, a face internal structure extraction unit, a hair extraction unit that performs one type of extraction process or algorithm,
Each of the non-background part extraction units may be used as one extraction subunit. Alternatively, it is preferable that at least two types of extraction subunits that perform a main subject extraction step, such as a skin color extraction unit, a face contour extraction unit, and a circular shape extraction unit, are configured to be processed in parallel in the same extraction unit. It may be a component of the extraction unit connected to the upper stage. On the other hand, hair extraction, eye extraction, face internal structure extraction,
At least two extraction sub-units that perform auxiliary subject extraction steps such as torso extraction, non-background part extraction, and circular shape extraction unit
The seeds are preferably configured to be processed in parallel in the same extraction unit, and may be a component of the extraction unit downstream of the extraction unit having the extraction subunit that performs the main subject extraction process. In the above-described example, as a plurality of types of extraction subunits, an extraction unit that performs different extraction steps or algorithms is described as a representative example. However, the present invention is not limited to this. May be used in multiple stages, and a plurality of extraction units applying the same algorithm may be used.

The condition judging section 96 includes an extraction sub-unit 94.
(94 ₁ ,..., 94 _k ) to determine whether the candidate of each subject is appropriate as a main part or a main subject, or an extraction area of each subject, particularly each subject determined to be appropriate. , And the determination result or the weighting result is shifted, that is, at least one extraction unit 92 in the subsequent or next stage to which the extraction process is applied is selected based on the result. In the example illustrated in FIG. 7, the condition determination unit 96 has selected to shift to the extraction unit 92 at the i-th stage. in this way,
The plurality of extraction units 92 connected in cascade can be transferred not only to the next-stage extraction unit 92 but also to a subsequent-stage extraction unit 92. In the example shown in FIG. 6, the extraction unit 92 ₁ and 92 ₂ are selected each transition to the extraction unit 92 of the i stage and the j stage. By the way, the condition determination unit 9 of the last extraction unit 92 of the cascade-connected extraction units 92
Numeral 6 is preferably a main subject determination unit for finally extracting a main subject.

The condition judging section 96 determines the weight of the extraction area of each subject extracted by each extraction sub-unit 94, determines whether or not each subject candidate is appropriate, and shifts the result to one or more extraction units at the subsequent stage. Any type of determination, weighting, and selection of a subsequent extraction unit may be performed as long as the selection is made to select 92. Further, an operator may intervene in the condition determination of the condition determining unit 96. In the case of the extracting unit 72A shown in FIG. 4, as the condition determining unit, the characteristics extracted by the skin color / circular shape extracting unit 78 and the face contour / circular shape extracting unit 80, which are a plurality of extraction subunits to be processed in parallel. And a combining / result determining unit 82 for determining whether or not the result of subject extraction is appropriate from a plurality of candidate subject extraction regions that have been combined. A main part determination unit 88 for determining a subject can be given. The main part determination unit 88
Last extraction unit 92 without extraction subunit
Can be an example of the condition determination unit. The main subject extracting apparatus of the present invention for implementing the main subject extracting method of the present invention applied to the extracting section 72A of the setup section 72 is basically configured as described above.

Further, the main subject extracting method of the present invention can be carried out by the main subject extracting apparatus shown in FIGS. That is, in the main subject extraction method performed here, the above-described extraction steps (processes) are respectively performed in the above-described extraction units 92 ₁ , 92 ₂ ,..., 92 _N connected in cascade. The extraction result of each extraction unit 92 is applied to at least one of the subsequent extraction steps cascaded one after another. In each extraction unit 92, the above-described extraction sub-steps are performed in parallel in the above-described extraction sub-units 94 ₁ ,..., 94 _k . Thereafter, in the condition determination unit 96 of the extraction unit 92, a condition determination sub-step such as determination of an extraction area in each extraction sub-step, weighting, and selection of a subsequent extraction unit 92 to which these results are applied is performed. Details of each processing step such as each extraction step, each extraction sub-step, and condition determination sub-step performed in each extraction unit 92, each extraction sub-unit 94, and condition determination unit 96 are as described above. Description is omitted. In the main subject extraction method of the present invention as described above, all of the processing steps can be configured as software processing, or the above-described main processing of the present invention can be configured as hardware processing including all processing units that perform all processing steps. It can be configured as a main subject extraction device, or it can be configured such that software processing and hardware processing coexist, depending on whether processing is performed by software or unit configuration hardware for each processing step. .

The setting section 72B of the setup section 72 reads out the pre-scan data from the pre-scan memory 52 and creates a density histogram, average density, highlight (lowest density), shadow (highest density), etc. from the pre-scan data. Of the main scan of the image, and the main scanning part of the image extracted by the extraction unit 72A, as well as an operator performed as necessary, in addition to the density histogram and the image characteristic amount. Various kinds of operations in the pre-scan processing unit 56 and the main scan processing unit 58, such as creation of an LUT for performing gray balance adjustment, gradation correction and density adjustment, creation of an MTX operation formula, calculation of a sharpness correction coefficient, etc. Set image processing conditions.

The key correction unit 74 is provided with keys for adjusting the density (brightness), color, contrast, sharpness, saturation, etc., set on the keyboard 18a, and various instructions input with the mouse 18b. The adjustment amount of the image processing condition is calculated and supplied to the parameter integration unit 76. The parameter integrating unit 76 receives the image processing conditions set by the setting unit 72B, sets the image processing conditions in the pre-scan processing unit 56 and the main scan processing unit 58, and further, according to the adjustment amount calculated by the key correction unit 74, Correct (adjust) the image processing conditions set for the part, or reset the image processing conditions.

Hereinafter, the operation of the scanner 12 and the processing device 14 will be described. The operator who is requested to make a print of the film F operates on the carrier 3 corresponding to the film F.
0 is loaded into the scanner 12, the film F (cartridge) is set at a predetermined position of the carrier 30, and necessary instructions such as a print size to be created are input, and then a print creation start is instructed. Thereby, the aperture value of the variable aperture 24 of the scanner 12 and the accumulation time of the image sensor (line CCD sensor) 34 are set according to the pre-scanning reading conditions. Thereafter, the carrier 30 pulls out the film F from the cartridge and presses the film F. The film F is conveyed in the sub-scanning direction at a speed corresponding to the can, and the pre-scan is started. At a predetermined reading position, the film F is slit-scanned, and the projection light forms an image on the image sensor 34 as described above. An image captured at F is separated into R, G, and B and photoelectrically read. When the film F is transported, magnetic information is read out, and the code reader 4
A bar code such as a DX code is read by 4 and necessary information is sent to a predetermined portion.

The pre-scan and the main scan include
The prescan and the main scan may be performed one frame at a time, or continuously for all frames or for a predetermined plurality of frames. In the following example, for simplicity, 1
The description will be given by taking the image reading of a frame as an example.

The output signal of the image sensor 34 due to the pre-scan is amplified by the amplifier 36 and is output from the A / D converter 3.
8 and converted into a digital signal. The digital signal is
The data is sent to the processing device 14, subjected to predetermined data processing by the data processing unit 48, converted into digital image data by the Log converter 50, and stored in the prescan memory 52.

When the pre-scan data is stored in the pre-scan memory 52, the setup unit 72 of the condition setting unit 60 reads out the pre-scan data, and the extraction unit 72A and the setting unit 72
B. In the extraction unit 72A, the face candidate area is extracted by the skin color / circular shape extraction unit 78 and the face contour / circular shape extraction unit 80, respectively, as described above. Are combined, and the extraction result is determined. If the extraction result is appropriate, the switching unit 84 sends the extracted result to the main part determination unit 88 to determine the main part. The information of the determined main part is transmitted to the setting unit 72B. Sent. On the other hand, if the combining / result determining unit 82 determines that the extraction result is inappropriate, the switching unit 84 sends the combining result and image data to the torso / eye extracting unit 86, and extracts the torso and the eye. Is further extracted, the extraction result is sent to the main part determination unit 88, the main part is determined, and information on the main part is sent to the setting unit 72B.

The setting section 72B creates a density histogram of an image from the pre-scan data, calculates an image feature amount such as highlight and shadow, sets read conditions for the main scan, and supplies the scan conditions to the scanner 12. Further, in addition to the density histogram and the calculated image feature amounts, the image processing conditions in the processing unit 66 are set by taking into account the main parts extracted by the extraction unit 72A and the instructions of the operator performed as necessary. It is supplied to the integration unit 76. The parameter integrating unit 76 sets the received image processing conditions in predetermined portions (hardware) of the pre-scan processing unit 56 and the main scan processing unit 58.

When performing the test, the pre-scan data is read from the pre-scan memory 52 by the processing unit 62, processed by the processing unit 62, and processed by the image data conversion unit 64.
And displayed on the display 20 as a simulation image. The operator operates the display 20
, The image, that is, the processing result is checked (verified), and the color, density, gradation, and the like are adjusted as necessary using the adjustment keys and the like set on the keyboard 18a. The input of this adjustment is sent to the key correction unit 74, and the key correction unit 74
Calculates the correction amount of the image processing condition according to the adjustment input, and sends it to the parameter integration unit 76. Parameter integration unit 7
6 corrects the LUT, MTX, and the like of the processing units 62 and 66 according to the correction amount sent as described above. Therefore, the image displayed on the display 20 also changes according to the correction, that is, the adjustment input by the operator.

When the operator determines that the image of this frame is proper (OK), the operator instructs the start of printing using the keyboard 18a or the like. As a result, the image processing conditions are fixed, the aperture value of the variable aperture 24 is set in the scanner 12 in accordance with the reading conditions of the main scan, and the carrier 30 moves the film F at a speed corresponding to the main scan. Then, the main scan is started. If the verification is not performed, the image processing conditions are determined when the setting of the image processing conditions in the main scan processing unit 58 by the parameter integration unit 76 is completed, and the main scan is started.

The main scan is performed in the same manner as the pre-scan except that the read conditions such as the aperture value of the variable aperture 24 are set, and the output signal from the image sensor 34 is amplified by the amplifier 36. Then, the digital signal is converted into a digital signal by the A / D converter 38, processed by the data processing unit 48 of the processing device 14, converted into main scan data by the Log converter 50, and sent to the main scan memory 54. When the main scan data is sent to the main scan memory 54, the main scan data is read out by the main scan processing unit 58, subjected to image processing under the image processing conditions determined in the processing unit 66, and then converted by the image data conversion unit 68 and output. Is output to the printer 16.

The printer 16 exposes a photosensitive material (printing paper) in accordance with the supplied image data to record a latent image, and a printer (printing device) that performs a predetermined process on the exposed photosensitive material to perform printing. Processor (developing device) that outputs as
And In a printer, for example, after cutting a photosensitive material into a predetermined length corresponding to a print, a back print is recorded, and then three types of light beams of R exposure, G exposure, and B exposure according to the spectral sensitivity characteristics of the photosensitive material. Processing unit 1
The light beam is conveyed in the sub-scanning direction orthogonal to the main scanning direction by modulating the light-sensitive material in the main scanning direction and modulated in accordance with the image data output from the photodetector 4, thereby two-dimensionally transforming the photosensitive material by the light beam. The latent image is recorded by scanning exposure and supplied to the processor. The processor that has received the photosensitive material performs a predetermined wet development process such as color development, bleach-fixing, and washing with water, and forms a print by drying, and sorts and accumulates it into a predetermined unit such as one film.

By the way, in the above-described example, the case where the face of a person is extracted as the main part has been described. However, the present invention is not limited to this.
For example, pets, animals such as horses, cars, mountains, lakes, etc. may be extracted. In this case, an extraction algorithm using colors and shapes suitable for the selected main part may be selected. Further, in the above-described example, the case where the main subject in the image is extracted in one frame has been described. However, the present invention is not limited to this, and the extraction of the subject to the extraction area (area) in the condition determination unit of the extraction unit is described. In the weighting, the similarity to the subject extraction area of another frame in the same matter may be used. That is, by using the similarity between the main subject extraction region in the previous frame image and the main subject extraction region in the current frame image to be extracted, the main subject extraction region in the current frame image is extracted. May be weighted. For example, after normalizing the size of the extraction area,
You can see similarities in template matching. Alternatively, the similarity of the density distribution can be checked by taking a histogram of the extraction area. In addition, the similarity of images can be checked by image feature amounts such as colors. As described above, by using the similarity with other frames in the same case, the calculation time required for extracting the main subject can be further reduced.

Although the main subject extracting method and apparatus of the present invention have been described in detail above, the present invention is not limited to the above embodiments, and various improvements and changes can be made without departing from the scope of the present invention. Of course, you may.

[0072]

As described above in detail, according to the present invention, in a photo printer such as a digital photo printer, highly accurate extraction of a main subject can be performed with good efficiency.
It can be performed stably in response to images in various states,
A high-quality print in which a high-quality image is reproduced can be created with good efficiency.

[Brief description of the drawings]

FIG. 1 is a block diagram of a digital photo printer using an example of a main subject extraction method and apparatus of the present invention.

2A is a schematic perspective view for explaining a carrier mounted on the digital photo printer shown in FIG. 1, and FIG. 2B is a conceptual diagram of an image sensor of the digital photo printer shown in FIG. is there.

FIG. 3 is a block diagram of an image processing apparatus of the digital photo printer shown in FIG.

4 is a block diagram of an extraction unit of the image processing device shown in FIG.

FIG. 5 is a diagram for explaining an example of an extraction result determination method in the extraction unit shown in FIG.

6 is a block diagram of an embodiment of a main subject extracting apparatus according to the present invention applied to the extracting unit of the image processing apparatus shown in FIG. 3;

FIG. 7 is a block diagram of an embodiment of the extraction unit of the main subject extraction device of the present invention shown in FIG. 6;

[Explanation of symbols]

10 (Digital) Photo Printer 12 Scanner 14 (Image) Processing Device 16 Printer 18 Operation System 20 Display 22 Light Source 24 Variable Aperture 28 Diffusion Box 30 Carrier 32 Imaging Lens Unit 34 Image Sensor 34R, 34G, 34B Line CCD Sensor 36 Amplifier 38 A / D converter 40 Mask 44 Code reader 48 Data processing unit 50 Log converter 52 Prescan (frame) memory 54 Main scan (frame) memory 56 Prescan processing unit 58 Main scan processing unit 60 Condition setting unit 62, 66 ( Image data) processing unit 64, 68 image data conversion unit 72 setup unit 72A (main part) extraction unit 72B setting unit 74 key correction unit 76 parameter integration unit 78 skin color / circle shape extraction unit 80 face contour / circle shape extraction unit 2 synthesis and result determining section 86 fuselage and eye portion extracting section 88 the main part determination unit 90 main subject extracting section 91 start portion _{92,92 1, 92 1, ... 92} N extraction unit 93 ends portions 94,94 _1, ... 94 _k Extraction subunit 96 Condition judgment unit

Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI H04N 1/40 G06F 15/70 310 H04N 1/40 F

Claims (15)

[Claims] 1. A method for extracting a plurality of subjects having different characteristics, combining the subjects extracted in each subject extraction, and extracting a main subject from the combined result when the combined result satisfies a predetermined criterion. If the synthesis result does not satisfy the determination criterion, an auxiliary subject having a different characteristic is further extracted, and a main subject is extracted using the synthesis result and the extraction result obtained by the auxiliary subject extraction. Characteristic main subject extraction method. 2. The method according to claim 1, wherein the determination criterion is at least one of a region overlapping degree when combining the extracted images obtained by extracting the respective subjects and a weighting score when combining the extracted images obtained by extracting the respective subjects. Subject extraction method. 3. When a main subject is extracted from the synthesis result, if the image feature amount of the extracted main subject exceeds a predetermined value, the auxiliary subject is extracted.
The main subject extraction method according to claim 1, wherein a main subject is extracted using the synthesis result and an extraction result obtained by extracting an auxiliary subject. 4. When the main subject is extracted from the synthesis result and there are a plurality of extracted main subjects and there is a difference in image feature amount between the plurality of main subjects exceeding a predetermined value, the auxiliary A subject is extracted, and a main subject is extracted using an extraction result obtained by extracting an auxiliary subject.
3. The method for extracting a main subject according to any one of 3. 5. The subject extraction is at least two selected from skin color extraction, face outline extraction, and circular shape extraction, and the auxiliary subject extraction is hair extraction, face internal structure extraction, torso extraction, non-background part. 5. The main subject extraction method according to claim 1, wherein the method is one or more selected from extraction and circular shape extraction. 6. At least one subject extraction unit having a plurality of extraction subunits for performing subject extraction with different characteristics and a condition determination unit for performing a condition determination for the subject extraction based on the result of each extraction subunit, which is performed in parallel. A plurality of extraction units including a plurality of extraction units are cascade-connected. The condition determination unit weights or determines the extraction region of each subject extracted by the extraction subunit, and applies an extraction process based on the result. A main subject extraction device, wherein at least one extraction unit at a subsequent stage is selected. 7. An extraction unit for extracting a subject.
The number of extraction subunits and a unit having a condition determination unit for performing the condition determination of the subject extraction based on the results of the extraction subunits, or the result of the cascade-connected previous extraction unit without the extraction subunits, 7. The main subject extraction device according to claim 6, wherein the main subject extraction device includes at least one unit having only a condition determination unit for performing a condition determination on the extracted subject. 8. The extraction subunit includes a skin color extraction unit, a face outline extraction unit, a circular shape extraction unit, a body extraction unit, an eye extraction unit,
Hair part extraction part, face internal structure extraction part, non-background part extraction part, skin color extraction, face outline extraction, circular shape extraction, body extraction,
The main subject extraction device according to claim 6 or 7, wherein at least two types are selected from an extraction unit that performs at least two types of extraction including eye extraction, hair extraction, face internal structure extraction, and non-background extraction. 9. The condition judging section synthesizes candidates of subjects having different characteristics extracted by a plurality of extraction sub-units to be processed in parallel, and extracts the subject from an extracted region of the synthesized plurality of candidates. 9. The main subject extracting device according to claim 6, further comprising a combining / result determining unit that determines whether the result of the above is appropriate. 10. The main subject extraction according to claim 6, wherein the condition determination unit of the last extraction unit of the cascade-connected extraction units is a main subject determination unit for extracting a main subject. apparatus. 11. An object extracting step which includes a plurality of types of extracting sub-steps for extracting subjects having different characteristics and a condition judging sub-step for judging the subject extracting condition based on a result of each of the extracting sub-steps. The extraction step including one is cascade-connected in a plurality of stages. In the condition determination sub-step, weighting or determination is performed on the extraction area of each subject extracted in the extraction sub-step, and extraction is performed based on the result. A method for extracting a main subject, wherein at least one extraction step at a later stage to which the processing is applied is selected. 12. The extracting step includes a kind of extracting sub-step of extracting a subject, a step of determining a condition of extracting the subject based on a result of the extracting sub-step, and a step of determining a condition of extracting the subject. 12. The main subject extraction according to claim 11, wherein the main subject extraction includes at least one step having only a condition determination sub-step of performing a condition determination of an extracted subject candidate based on a result of the preceding extraction step connected in cascade. Method. 13. The extraction sub-step includes: skin color extraction, face contour extraction, circular shape extraction, body extraction, eye extraction or face internal structure extraction, hair extraction, non-background extraction, and at least two of these. 13. The main subject extraction method according to claim 11, wherein the main subject extraction method includes two or more types selected from a combination of extraction. 14. The condition judging sub-step combines candidate subjects of different characteristics extracted in a plurality of extracting sub-steps which are processed in parallel, and extracts the subject from an extracted region of the synthesized plural candidates. 14. The main subject extracting method according to claim 11, further comprising a combining / result determining step of determining whether an extraction result is appropriate. 15. The condition determination sub-step of the last extraction step of the cascade-connected extraction steps is a main subject determination step of extracting a main subject.
15. The method for extracting a main subject according to any one of 14.