JP2001218020A - Picture processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the appearance of especially a woman in an output picture by an automatic processing. SOLUTION: In a picture processing method, a picture processing is conducted on inputted picture data and it is made to be picture data for output. An area equivalent to the face of a person is extracted from input picture data and the area of a lip part is detected. The sex of the object person is estimated based on the color of the lip part and the picture processing is conducted based on the estimated result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for performing image processing on input image data to obtain output image data, and more particularly, to extracting a face area of a person in an original image and extracting the area. And an image processing method for producing a color tone suitable for the image processing.

[0002]

2. Description of the Related Art Conventionally, printing an image photographed on a photographic film (hereinafter, referred to as a film) such as a negative film or a reversal film onto a photosensitive material (photographic paper) involves projecting an image of the film onto the photosensitive material. This is performed by so-called direct exposure in which the photosensitive material is exposed on the surface.

On the other hand, in recent years, a printing apparatus using digital exposure, that is, an image recorded on a film is photoelectrically read, the read image is converted into a digital signal, and then various image processing is performed. 2. Description of the Related Art Digital photo printers have been put to practical use in which a photosensitive material is scanned and exposed by recording light modulated according to the image data to record an image (latent image) and print (photograph) the image data for recording.

[0004] In a digital photo printer, an image can be converted into digital image data and exposure conditions for printing can be determined by image data processing.
Correction of color feria and density feria, correction of underexposure and overexposure, correction of insufficient peripheral light, sharpness (sharpening) processing, compression / expansion processing of density dynamic range (addition of dodging effect by image data processing)
For example, various types of image processing that were impossible or difficult with conventional direct exposure can be performed with a high degree of freedom, and a very high-quality print can be obtained as compared with direct exposure.
In addition, it is possible to combine a plurality of images, divide an image, combine characters, and the like by image data processing, and output a print that is freely edited / processed according to the intended use.

According to a digital photo printer,
In addition to outputting images as prints, image data can be supplied to a computer or the like or stored on a recording medium such as a floppy disk, so use the image data for various purposes other than photography. Can also.
As described above, according to the digital photo printer, it is possible to perform image processing with a higher degree of freedom by image data processing, and it is possible to output a print with higher commercial value.

By the way, in a portrait photograph, the face is the most noticeable part, and the face of the person needs to be properly reproduced in the output image. For that purpose, first, an area corresponding to the face of a person must be accurately extracted from the image. For example, when printing on a photosensitive material such as photographic paper, the amount of exposure must be determined so that the color of a person's face is printed in an appropriate color.

On the other hand, the present applicant has already disclosed in Japanese Unexamined Patent Publication No.
In Japanese Patent No. 184925, image data is taken in,
The image is divided into a plurality of regions by binarization or another method, and a region having the highest probability, which is a region corresponding to a human face, is extracted from the plurality of regions, and the face density of the extracted face region is extracted. A method has been proposed in which an exposure amount is determined so as to be appropriate and image processing is performed.

[0008]

However, according to the technique disclosed in Japanese Patent Application Laid-Open No. HEI 8-184925, even if the face area is accurately extracted, the subsequent processing is performed uniformly regardless of gender. Because concentration control is performed,
When the subject person is a woman, in terms of reproducibility as a woman's face, an appropriate image is not always obtained, and the general demand for improving the appearance of a woman in an output image is satisfied. There was a problem that there was room for improvement. The present invention has been made in view of the above-described conventional problems, and has as its object to provide an image processing method that improves the appearance of a specific person in an output image and increases the amusement of a photograph by automatic processing. And

[0009]

According to a first aspect of the present invention, there is provided an image processing method for performing image processing on input image data to obtain image data for output. And extracting, from the input image data, an area corresponding to the face of the person, detecting the area of the lip portion, estimating the gender of the subject person based on the color of the lip portion, and determining the estimation result. And an image processing method for performing image processing based on the image processing method.

It is preferable that the skin portion is locally corrected to a predetermined color, a predetermined gradation, or a predetermined smoothing process is applied based on the estimation result.

[0011] Similarly, in order to solve the above problems,
A second aspect of the present invention is an image processing method for performing image processing on input image data to obtain image data for output, wherein an area corresponding to a face of a person is input from the input image data. Along with the extraction, each part of the face is detected, and the extracted face area and the face structure estimated from each part of the detected face are selected from a plurality of preset makeup patterns. An image processing method is provided wherein image processing is performed based on a selected makeup pattern.

[0012] Similarly, in order to solve the above problems,
A third aspect of the present invention is an image processing method for performing image processing on input image data to generate image data for output, wherein an area corresponding to a person's face is extracted from input image data. Extracting and detecting the area of the part where the difference between male and female appears from the face area, estimating the gender of the subject person based on the area of the part where the difference between male and female appears, based on the estimation result, An image processing method characterized by performing image processing is provided.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image processing method according to the present invention will be described based on a preferred embodiment shown in the accompanying drawings.
This will be described in detail.

FIG. 1 is a block diagram schematically showing a digital photo printer including an image processing apparatus for implementing an image processing method according to the present invention, which enhances the appearance of a specific person. A digital photo printer (hereinafter, referred to as a photo printer) 10 shown in FIG. 1 is a scanner (image reading device) 12 that photoelectrically reads an image photographed on a film F.
And electronic image scaling processing of image data read by the scanner 12, image processing such as edge detection, sharpness enhancement (sharpness enhancement), and smoothing processing (grain suppression) of the image data, and overall operation of the photo printer 10. And an image processing apparatus 14 for performing control and the like, and an image exposure of a photosensitive material (printing paper) with a light beam modulated according to the image data output from the image processing apparatus 14, and a development processing (finished) image is formed. And an image recording device 16 for outputting as a print. The image processing apparatus 14 includes a keyboard 18a and a mouse 18 for inputting various conditions such as input, setting, selection and instruction of processing, and instructions such as color / density correction.
and an operation system 18 having an image b, a monitor 20 for displaying an image read by the scanner 12, various operation instructions, a setting / registration screen for various conditions, and the like.

The scanner 12 is a device for photoelectrically reading an image photographed on a film F or the like one frame at a time.
A variable aperture 24, a diffusion box 26 for making the reading light incident on the film F uniform in the surface direction of the film F, a carrier 28 of the film F, and an imaging lens unit 30
, An image sensor 32 having a three-line CCD sensor corresponding to reading of image densities of R (red), G (green), and B (blue), an amplifier (amplifier) 33, and an A / D (analog / digital). ) A converter 34.

In the photo printer 10, a dedicated carrier 28 that can be freely mounted on the main body of the scanner 12 includes a film F such as a new photographic system (advanced photo system) or a 135 size negative (or reversal) film.
Are prepared in accordance with the type and size of the film, the form of the film such as strips and slides, and the like, and can be adapted to various films and processes by replacing the carrier 28. An image (frame) photographed on a film and provided for printing is conveyed by the carrier 28 to a predetermined reading position. Also, as is well known, a magnetic recording medium is formed on the film of the new photographic system, and a cartridge ID, a film size, an ISO sensitivity, and the like are recorded. Various data such as date and time, exposure level, model of camera and developing machine can be recorded. The magnetic information reading means is arranged on the carrier 28 corresponding to the film (cartridge) of the new photographic system. The magnetic information is read when the film is transported to the reading position, and the various kinds of information are subjected to image processing. It is sent to the device 14.

When reading an image photographed on the film F in such a scanner 12, uniform reading light emitted from the light source 22 and adjusted in light amount by the variable aperture 24 and the diffusion box 26 is transmitted by the carrier 28. Incident on the film F located at the predetermined reading position,
By transmitting the light, projection light carrying an image photographed on the film F is obtained. The color image signal is not limited to a signal input by reading the light transmitted through the film as described above, and may be a reflection original or an image captured by a digital camera. That is, a scanner 1 that reads an image on a film
2. In addition to the above, imaging means such as a digital camera and a digital video camera, an image reading device for reading an image of a reflection original,
An image (digital image signal) from an image source R such as a communication means such as a LAN (Local Aera Network) or a computer communication network, a memory card such as a smart media, or a medium (recording medium) such as an MO (magneto-optical recording medium).
Can be input to the image processing apparatus 14.

In the illustrated example, the carrier 28 is a 24-sheet carrier.
It corresponds to a long film F (strips) such as a 35-size film or a cartridge of a new photo system. The film F is positioned at the reading position by the carrier 28, and receives the reading light while being conveyed in the sub-scanning direction orthogonal to the main scanning direction, which is the extending direction of the RGB three-line CCD sensor. As a result, the film F is two-dimensionally slit-scanned, and the image of each frame captured on the film F is read.

The projection light of the film F is imaged on the light receiving surface of the image sensor 32 by the imaging lens unit 30. R, G and B output from the image sensor 32
Are amplified by an amplifier 33, sent to an A / D converter 34, and converted into, for example, 12-bit RGB digital image data in the A / D converter 34, respectively. Is output to

In the scanner 12, when reading an image photographed on the film F, a prescan (first image reading) for reading at a low resolution and a fine scan (first image reading) for obtaining image data of an output image are performed. (2nd image reading). Here, the pre-scan is performed under preset pre-scan reading conditions so that the image sensor 32 can read all images of the film F to be scanned by the scanner 12 without saturation. On the other hand, the fine scan is performed under the fine scan reading conditions set for each frame from the pre-scan data so that the image sensor 32 is saturated at a density slightly lower than the minimum density of the image (frame). The prescan and fine scan output image signals are basically the same image data except that the resolution and the output image signal level are different.

Note that the scanner 12 used in the photo printer 10 is not limited to the one that performs the slit scanning reading, and may be the one that performs the surface reading that reads the entire surface of one frame of the film image at a time. Good. In this case, for example, an area sensor such as an area CCD sensor is used, and means for inserting R, G, and B color filters are provided between the light source 22 and the film F, and inserted into the optical path of the light emitted from the light source 22. The reading light transmitted through the color filters is irradiated onto the entire surface of the film F, and the transmitted light is imaged on the area CCD sensor to read the entire image of the film by sequentially switching the R, G, and B color filters. Accordingly, the image photographed on the film F is separated into three primary colors and read.

As described above, the digital image data signal output from the scanner 12 is used in the image processing apparatus 1 for implementing the image processing method for improving the appearance of a woman according to the present invention.
4 is output. The R, G, and B image signals input from the scanner 12 to the image processing device
Data correction of read image data such as DC offset correction, dark time correction, defective pixel correction, and shading correction is performed to correct the sensitivity variation and dark current of each pixel of RGB digital image data caused by the CD sensor. Thereafter, the image data is subjected to logarithmic conversion processing and gradation conversion, and is converted into digital image density data.

The digital image density data is separately stored (stored) for each of the pre-scan image data and the fine scan image data. The prescanned image data is subjected to predetermined image processing and displayed on the monitor 20. Also,
From the pre-scan image data, a density histogram is created, and image feature amounts such as average density, LATD (large area transmission density), highlight (lowest density), shadow (highest density), etc. are calculated, and fine scan reading is performed. Conditions and image processing conditions are set. The fine scan image data is subjected to normal image processing and image processing for improving the appearance of a woman according to the present invention according to the set image processing conditions. The image is output from the image recording device 16 as an optimal and high-quality image. The digital photo printer including the image processing apparatus that performs the image processing method for improving the appearance of a woman according to the present invention is basically configured as described above.

Hereinafter, the operation of the first embodiment of the present invention will be described. As described above, the image processing device 14 converts the density, color and gradation of the image data, and converts the saturation of the fine scan image data read based on the reading conditions set based on the prescan image data. First, the gender of the subject is confirmed based on extraction of the main subject, which is a premise of image processing for improving the appearance of a specific person, along with processing such as conversion and electronic scaling.

Hereinafter, the image processing for improving the appearance of a specific person will be described with reference to the flowchart of FIG. In step 100, a face area of a person is extracted. First, image data is binarized using a predetermined threshold. As a result, the original image has a higher density area (black area) than the threshold value.
And a region having a density equal to or lower than the threshold (white region).
From this black area, a black area corresponding to the head of a person is extracted. To do this, first, one of the plurality of black areas is taken out, the contour of the area is traced, and the curvature of the contour is calculated. Then, based on the calculated curvature, as a shape pattern representing the contour of the head of the person, it is estimated that the contour corresponds to the concave portion estimated to correspond to the boundary between the head and face of the person and the contour corresponds to the top of the head of the person. A black region having a convex portion to be extracted is extracted. Next, a predetermined characteristic amount of the concave portion and the convex portion is calculated. Then, using this feature amount, it is determined whether or not the black area can be determined to be the head of the person, that is, whether or not the probability that the black area is the head of the person is high. In this determination, the concave portion and the convex portion each having the characteristic amount within the predetermined range are respectively extracted from the outline of the black region, and the ratio of the length of the concave portion and the convex portion to the peripheral length of the outline of the black region is within the predetermined range. And when the consistency evaluation is higher as the head of the person than the position and the direction, the result is affirmative.

When the determination is affirmative, most of the persons have a concave hair shape at the boundary between the human hair part and the face. An elliptical region inscribed in the concave portion is set as a face candidate region based on the size and the center position thereof. Further, it is presumed that a line is obtained by dividing the outline of each region obtained by the binarization, and a shape pattern representing the outline of the person's face corresponds to the side of the person's face from the line. Extract pairs of lines. If it is determined that the line pair is a line corresponding to both sides of the person's face, an elliptical face candidate region having a predetermined size is set in a region sandwiched by the line pair.

A positive weighting coefficient is assigned to the face candidate area that has just been set. Next, a face area is set based on the internal structure of the face. This is to enhance the accuracy as a face region by extracting, for example, a pair of black regions estimated to correspond to a pair of eyes of the face. Further, in order to further ensure that the face region is a face region, as a shape pattern representing the contour of the torso, a pair of lines estimated to correspond to a contour that is continuous from the neck to the shoulder of the person, from the shoulder to the arm, or to the lower torso. Is extracted. Each time such an eye pair or a line pair is extracted, a positive weighting coefficient is assigned to the face candidate area, and finally the area is determined as the area having the highest probability of being a person's face area by this weighting coefficient. The extracted region is extracted as a face region.

Next, at step 110, the mouth is detected. This is because, when the face region is extracted as described above, in order to increase the accuracy of the face region, the extraction of each part of the face such as the eyes and the mouth, which is the internal structure of the face, is performed at the same time. Is also good. For example, the eyes are extracted as a pair of black regions in the face region as described above, and the eyebrows are extracted as black lines along the eyes. Then, the mouth is detected by a predetermined color tone search for the positional relationship in the face region and the lips. The search for the color of the lips is performed somewhat broadly in consideration of various differences such as gender and race.

In step 120, the lip color is checked to determine the gender of the subject person. That is,
If the saturation is higher than the standard color, which is the original lip color, it is determined that lipstick is used, and the subject person is determined to be a woman. Here, as a standard color, the color of an original lip without using lipstick is measured for a large number of people, and a color obtained in advance as an average color is used. In addition, the judgment of being a woman is not limited to the color of the lips as described above. If there is a difference between men and women, use it as a judgment standard or an auxiliary standard for the color of the lips. It is possible. For example, it is conceivable to detect the color around the eyes and determine based on whether or not an eye shadow is used, whether or not accessories such as earrings are worn, and the like. Alternatively, the presence or absence of makeup may be determined using the color of the face (skin), and this may be added to the gender determination data. As described above, a region where a difference occurs between a man and a woman may be detected from the face region, and the gender may be determined based on the region.

If it is determined in step 120 that the subject is not a woman, the flow advances to step 140 to perform normal image processing. When it is determined that the subject is a woman, in step 130, processing for improving the appearance of the person is performed.

The processing for improving the appearance of a specific person (for example, a woman) is not particularly limited. For example, processing for making the skin look fresh or removing wrinkles can be considered. In order to make the skin look fresh, for example, in the image 50 of the woman shown in FIG.
Emphasize the highlight part of 3 and give a feeling of "tekari",
Alternatively, a process such as dyeing the cheek 53 in a slightly pink color is conceivable. In particular, the wrinkles 58 at the corners of the eyes that women care about are made inconspicuous by a smoothing process. Further, the outline 54 of the face and the outline 56 of the torso emphasize sharpening, emphasize edges, and sharpen the outline.
At this time, the strength of edge enhancement or smoothing enhancement may be changed according to the shape of the cut-out edge. For example, if the degree of edge enhancement is stronger for an elliptical portion such as the contour 54 of the face than for a straight portion such as the contour 56 of the body, the face becomes sharper. Further, when the entire body is made to be slender, it looks even more beautiful. By such a process for improving the appearance, it is possible to obtain a beautiful photographic image with no wrinkles, fine and smooth skin, and a sharp outline. The gender estimation accuracy may be scored (for example, as a continuous amount), and the above-described processes may be performed according to the degree of the accuracy, so that the degree of each process may be set to an intermediate value. .

In the above-described first embodiment, all the processes are automatically performed. However, the above-described processes may be performed only on a designated frame upon request of a customer. . In this case as well, the processing after face extraction is performed automatically, but may be performed manually by an operator. Further, in the processing after the next frame, the range of searching for the face area may be narrowed using the color of the lipstick extracted earlier as a clue.

Next, a second embodiment of the image processing method according to the present invention will be described. In the second embodiment, when the subject is a specific person, for example, a woman, makeup is applied to the face. Hereinafter, the processing of this embodiment will be described with reference to the flowchart of FIG. First, in step 200, a face region is extracted, and in step 210, eyes, a mouth, and the like are detected. These are the same as the face area extraction processing in the first embodiment described above. In the flowchart, as in the first embodiment, the steps are divided into two steps. However, in the actual processing, as described above, in order to enhance the extraction accuracy of the face area, the eyes, It is efficient to detect the contour lines of the mouth and cheek simultaneously with the extraction of the face area.

Next, in step 220, the gender of the subject is determined. This determination method is not particularly limited. For example, similar to the determination in the first embodiment,
It can be done according to the color of the lips. If the result of the determination is that it is not a woman, normal image processing is performed in step 240. If it is determined that the person is a woman, a makeup process is performed in step 230. When the subject person is determined to be a male, a process of emphasizing the masculineness, for example, a process of increasing the color of tanned skin may be applied. In the processing according to the gender of the subject person, it is more preferable that the setting of the processing content can be changed according to the country and regionality of the person.

In order to perform a makeup process,
A plurality of makeup patterns are stored in a database or the like, and a makeup pattern is automatically selected from the plurality of makeup patterns based on a face structure such as a face contour, eye size, or lip shape. . Alternatively, the customer himself
A makeup pattern may be selected. As a criterion for selecting a makeup pattern, as a face structure,
Characteristics of each face part alone as described above, for example, size,
Aspect ratio, color, etc. can be cited.In addition, the makeup pattern is determined based on a combination of the distance between the eyes, the distance between the center of both eyes and the mouth, the distance from the chin to the forehead, and the curve characteristics of the cheek outline. You may make it select.

The makeup patterns include, for example, those relating to the setting of eyes such as eye shadow, eyelashes, eyebrows, etc., those relating to the setting of lipsticks such as lipstick color, gloss, etc.
Various settings can be made for the foundation, hairstyle, and the like. Also, a weighted average combining some of these patterns may be used. As a special case, a mode such as a makeup pattern by a famous hairdresser may be settable.

Further, as a special case, a setting incorporating a play element such as a kabuki shade or a professional wrestling paint wrestler's face is also possible. Further, it can also be used for business purposes such as wearing glasses or wearing accessories such as piercings and earrings. It is also possible to apply the method of the present invention to so-called Purikura images.

As described above in detail, according to the present embodiment, by performing processing according to the gender of the subject person in the portrait, it is possible to further enhance the beauty corresponding to the person. In addition, the user can freely make makeup according to his / her preference on the image, thereby greatly increasing the recreational value of the photograph.

Although the image processing method of the present invention has been described in detail above, the present invention is not limited to the above-described embodiment, and various improvements and changes are made without departing from the gist of the present invention. Of course it is good.

[0040]

As described above, according to the present invention,
By performing processing according to the gender of the subject person in the portrait, it is possible to enhance the beauty according to the person more, and to freely make the desired makeup on the image, Can greatly increase the entertainment.

[Brief description of the drawings]

FIG. 1 is a block diagram schematically illustrating a digital photo printer including an image processing apparatus that performs an image processing method according to the present invention.

FIG. 2 is a flowchart showing a flow of processing according to the first embodiment of the present invention.

FIG. 3 is an explanatory diagram showing an example of an image subjected to image processing according to the first embodiment.

FIG. 4 is a flowchart showing a flow of processing according to a second embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Digital photo printer 12 Scanner 14 (Image) processing device 16 Image recording device 18 Operation system 18a Keyboard 18b Mouse 20 Monitor 22 Light source 24 Variable aperture 26 Diffusion box 28 Carrier 30 Imaging lens unit 32 Image sensor 34 A / D converter 50 Person image

Continued on the front page F-term (reference) 5B057 AA20 BA19 BA26 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE05 CE09 CE11 CE17 DA08 DB02 DB06 DB09 DC16 DC22 DC25 5C076 AA26 BA06 5C077 LL16 LL19 LL19 MM03 MP08 PP02 PP15 PP19 SS SS05 SS06 TT02 TT09

Claims (4)

[Claims] 1. An image processing method for performing image processing on input image data to obtain image data for output, wherein an area corresponding to a face of a person is extracted from the input image data. An image processing method, comprising: detecting an area of a lip portion; estimating the gender of a subject person based on the color of the lip portion; and performing image processing based on the estimation result. 2. The method according to claim 1, wherein the skin portion is locally corrected to a predetermined color, a predetermined gradation, or a predetermined smoothing process based on the estimation result. Image processing method. 3. An image processing method for performing image processing on input image data to obtain image data for output, wherein an area corresponding to a face of a person is extracted from the input image data. Detecting each part of the face, according to the extracted face area, and the makeup of the face estimated from each part of the detected face, a makeup selected from a plurality of preset makeup patterns An image processing method comprising performing image processing based on a pattern. 4. An image processing method for performing image processing on input image data to obtain image data for output, wherein an area corresponding to a person's face is extracted from the input image data. Detecting an area of a difference between the male and female from the face area, estimating the gender of the subject person based on the area of the difference between the male and female, and performing image processing based on the estimation result An image processing method comprising: