JP2007088993A - Image correcting method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image correcting method and apparatus in which complexions of the same human figure can be reproduced in similar colors regarding a plurality of images obtained by repeatedly photographing a plurality of human figures, complexions can be reproduced in proper color densities and an image of stable quality can be obtained as a whole. <P>SOLUTION: Face authentication is performed upon face areas of human figures in a photographed image to identify each of the human figures, a relative difference of complexions between human figures in each image is calculated, a correlative relationship of complexions regarding the plurality of human figures photographed in the plurality of images is defined from the relative difference of complexions between human figures calculated for each of the plurality of images and on the basis of the correlative relationship of the complexions, the image is corrected. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention belongs to the technical field of image processing, and particularly relates to an image correction method and apparatus for correcting an image in which a person is photographed.

  When creating a photographic print from digital image data obtained by photographing with a digital camera or digital image data obtained by photoelectrically reading an image photographed on a photographic film, The image (image data) is corrected so as to be reproduced at the density. In particular, in the case of an image obtained by photographing a person, it is important that the skin color of the person is reproduced beautifully.

As a human skin color correction method, the face area of a person is detected, the color of the detected face area is compared with a reference color set in advance as a skin color, and the color and density of the entire image so as to approach the reference color There are known methods for correcting the above.
However, with the correction method using the reference color, it is difficult to correct appropriately when races with different skin colors are used as subjects, and it is difficult to cope with differences in face colors due to differences in age and gender.

In contrast, Patent Document 1 detects a face image of a subject included in an input image, estimates the race, age, and gender of the face image, and corrects the face image according to the result. It is described that the content is determined. In addition, the person who is the subject is estimated from the feature amount of the face image, and the subject is corrected with the favorite color or method of the subject. It also describes accepting input of information indicating.
JP 2004-246456 A

By the way, in photographs that have been taken by duplicating multiple persons in the same environment, such as school photographs, wedding receptions, parties, etc., the face color of the same person is reproduced in the same color in each photograph. At the same time, it is desirable that everyone's face color be properly reproduced.
However, since the technique of Patent Document 1 cannot know the relationship between individual facial colors of a plurality of persons photographed in one image, correction of the facial color of one person is appropriate or according to preference. Even if the process is performed, the face color of another person cannot always be made appropriate, and the quality of the image after correction is not stable due to variations in face color due to race differences or individual differences.

  Further, in the above technique, for a plurality of subjects photographed in one image, correction processing content can be manually set for each subject, and each subject can be individually corrected. It is not possible to automatically correct the skin color of all face areas having different values at once. Furthermore, it is not possible to automatically reproduce the same person photographed over a plurality of sheets with the same skin color.

  An object of the present invention is to solve the above-mentioned problems of the prior art and to reproduce the face color of the same person in a similar color for a plurality of images in which a plurality of persons are photographed in duplicate. It is an object of the present invention to provide an image correction method and apparatus capable of reproducing an image with an appropriate color density and obtaining an image having a stable quality as a whole.

In order to solve the above problems, the present invention is a method for correcting a plurality of images in which a plurality of persons are photographed,
Extracting a facial region in the image;
Identifying the person in the image by face authentication for the extracted face region;
Calculate the relative color difference between the persons in the same image,
From the relative difference in facial color between the persons calculated for each of the plurality of images, defining a correlation of facial colors for the plurality of persons photographed in the plurality of images,
An image correction method is provided, wherein the image is corrected based on the correlation between the face colors.

  Here, the absolute value of the face color in the image is estimated by comparing the correlation of the face color with predefined face color distribution information, and the image is corrected based on the estimated absolute value of the face color. It is preferable to do this.

  Further, it is preferable that the corrected image is displayed on a display device, and the image is corrected based on the correlation of the face color in accordance with a correction amount input to the displayed image.

The present invention also includes a face authentication / identification unit that extracts a face area in an image and identifies a person in the image by face authentication for the extracted face area.
Calculating a relative difference in face color between the persons in the same image, and a plurality of the persons photographed in the plurality of images from the relative difference in face color between the persons calculated for each of the plurality of images; A face color correlation definition section for defining the correlation between the face colors of
An image correction apparatus comprising: an image correction unit that corrects the image based on the correlation between the face colors defined by the face color correlation definition unit.

Here, further, a storage unit for storing pre-defined facial color distribution information;
A face color absolute value estimation unit that estimates the absolute value of the face color in the image by comparing the correlation of the face color defined by the face color correlation definition unit with the distribution information of the face color stored in the storage unit And
Preferably, the image correction unit corrects the image based on the absolute value of the face color estimated by the face color absolute value estimation unit.

Furthermore, a display unit that displays the image corrected by the image correction unit;
A correction amount input unit that receives an input of a correction amount for the displayed image,
It is preferable that the image correction unit corrects the image based on the correlation between the face colors in accordance with the correction amount input by the correction amount input unit.

  According to the present invention, with the above configuration, for a plurality of persons photographed in each image, individual persons are identified by face authentication, the relative color difference between the persons is grasped, and based on the correlation Since the image is corrected, the face color of the same person captured in multiple images can be reproduced in the same color, and even if there are people with different skin colors as subjects, It can be reproduced to an appropriate color density. As a result, an image having a stable quality as a whole can be obtained.

  An image correction method and apparatus according to the present invention will be described in detail below based on a preferred embodiment shown in the accompanying drawings.

FIG. 1 is a block diagram showing an embodiment of an image correction apparatus of the present invention that implements the image correction method of the present invention.
An image correction apparatus 10 shown in FIG. 1 identifies individual persons by face authentication for each of a plurality of input captured images, analyzes the relative facial color differences of the persons in the image at the same time, and By estimating and correcting the variation, correction is performed so that the face color of each person is reproduced with an appropriate color, and the corrected image is output to a photo printer or the like that performs digital exposure. The image correction apparatus 10 includes a face authentication / identification unit 12, a face color correlation definition unit 14, a face color distribution information storage unit 16, a face color absolute value estimation unit 18, an image correction unit 20, a display unit 22, and a correction amount input unit 24. ing.

  The image correction apparatus 10 is connected directly or indirectly to an image input machine, a print order receiving machine, etc. (hereinafter collectively referred to as an image input machine). This image input device includes a media driver for reading out image data from various media on which image data acquired by photographing with a digital camera or the like is recorded, and a network connection device for acquiring image data through a communication line such as the Internet. A terminal for directly connecting to a digital imaging device such as a digital camera or a camera-equipped mobile phone, a scanner for photoelectrically reading an image taken on a photographic film and obtaining image data, for example, a print order A plurality of captured images (image data) collected in units or the like are acquired.

Among a plurality of captured images acquired by the image input device, images captured in the same environment or similar environments are made into one group and input to the image correction apparatus 10 for each group.
Images taken in the same environment or similar environments are, for example, images taken at school, wedding receptions, parties, and the like. In such an image, a plurality of persons are usually photographed in an arbitrary combination and overlapped with a plurality of sheets.

Image grouping is performed automatically by, for example, setting information for identifying a group in a camera or the like at the time of shooting, and referring to identification information attached to the image in an image input device connected to the image correction apparatus 10. Can be done. Further, in the image input device, by referring to the shooting date and time information, a plurality of images shot within a predetermined time period may be determined as images shot in the same environment, and may be grouped. You may group according to an input.
The image correction apparatus 10 may include a unit that performs the grouping process.

  In addition, when the image data acquired by the image input device is image data from a digital camera or the like, usually, the digital camera or the like is subjected to the minimum image processing necessary for reproduction as it is. The image data may be directly input to the image correction apparatus 10. On the other hand, in the case of image data obtained by reading from a photographic film, the image is input to the image correction apparatus 10 after normal image processing is performed so that the entire image is reproduced almost properly.

The face authentication / identification unit 12 extracts a face area in each of a plurality of input images, and identifies an individual by face authentication on the extracted face area image (face image).
The method for extracting the face area is not particularly limited, and various known techniques such as a method for extracting a pixel group area in the skin color range as a face area, a method by shape pattern search, and the like can be used.
The face authentication method is not particularly limited, and a known technique such as a method using pattern matching can be used. For example, as in the method described in Japanese Laid-Open Patent Publication No. 2004-259253, a template face image of a specific person is held, the photographed face image is compared with the template face image, and the same person is determined according to the degree of coincidence. You may use the method of judging whether it is.

Further, in the present embodiment, since the face authentication only needs to be able to identify the faces of the same person photographed in a plurality of images, each face image in the photographed image is not retained without holding a template face image of a specific person. The face of the same person in the plurality of images may be authenticated by analyzing the pattern and performing pattern matching over the plurality of images.
For example, the face authentication / identification unit 12 specifies the persons A to E by performing pattern matching of each face image on the three images in FIGS.

The face color correlation definition unit 14 calculates a relative difference in face color between individuals included in the same image based on the information of each individual identified by the face authentication / identification unit 12.
That is, in the example of FIG. 2A, the relative difference of the face color is calculated for each combination of persons A to C, such as person A and person B, person A and person C, and person B and person C. Similarly, in the example of FIG. 2B, the relative differences of the face colors are calculated for the persons A to D and in the example of FIG. 2C for the persons A to E, respectively.

In addition, the face color correlation definition unit 14 defines the correlation between the face colors of persons photographed in a plurality of images from the relative differences in face colors between individuals calculated for all of the plurality of input images.
Specifically, for example, for each of the images in FIGS. 2A to 2C, the relative difference of the face color calculated previously is adjusted so that the same person in each image has almost the same face color. The relative relationship of the face colors of the persons A to D is determined. Thereby, the correlation between the face colors of the persons A to D is represented by a relative positional relationship on the chromaticity coordinates.
In the illustrated example, the person E is only shown in FIG. 2C and cannot be used for defining the correlation. However, after obtaining the correlation between the persons A to D, the person E is shown in FIG. ) Using the relative difference in face color between the person E and other persons in the above relationship.

FIG. 3 is a schematic diagram showing an example of the face color distribution of FIGS. 2A to 2C on the L ^* a ^* b ^* color system chromaticity diagram, and the distribution X is the person A in the correction target image. The facial color distribution of ~ E is shown. In the chromaticity diagram of FIG. 3, R represents red, G represents green, Y represents yellow, and B represents blue.
Each point of the distribution X has a relative positional relationship corresponding to the relative relationship of each face color obtained as described above, but its absolute value is a value based on the current image data (pre-correction image data).

In the definition of the correlation between the face colors above, the extracted face area in the image includes a face that looks down and a shadow, etc. It may not fit. In such a case, a face detected as an inconsistent face is calculated from the calculation of the relative difference of the face color in one image or the definition of the correlation between the face colors in a plurality of images so that there are fewer conflicting faces. It is preferable to exclude.
In addition, when defining the correlation between the face colors, if there is a face color relative difference of the image that greatly increases the amount of adjustment when combining the face color relative differences of the images, that image is the image that failed to shoot. It may be estimated that there is, and the face color relative difference of the image may be excluded.

  The face color distribution information storage unit 16 is a memory for storing face color distribution information defined in advance. In the illustrated example, the face color distribution information storage unit 16 is provided inside the image correction apparatus 10, but may be an external memory connected to the image correction apparatus 10.

The face color distribution information is statistical data indicating the distribution of the human face color (skin color), and indicates a preferable range for reproduction as a skin color on a photo print or the like. As facial color distribution information, for example, a skin color standard distribution image described in Japanese Patent Application Laid-Open No. 6-114013 is previously collected. Can be used. In the present embodiment, for example, it is represented by a distribution Z on the L ^* a ^* b ^* color system chromaticity diagram of FIG.

  Since the human skin color varies depending on the race, the variation in the skin color distribution varies depending on the race living in each region as a population and the ratio of a plurality of races. Therefore, the face color distribution information storage unit 16 has at least skin color distribution information corresponding to the race in the area where the image correction apparatus 10 is used.

In addition, the face color distribution information storage unit 16 includes data for a single race, data with different race ratios, etc. so that it can be used in various regions and images taken in various regions. Skin color distribution information corresponding to each region may be stored so that data used for comparison can be selected as appropriate.
Furthermore, as skin color distribution information, facial color distribution data is prepared for each race, gender, age, and other characteristic groups, stored in the facial color distribution information storage unit 16, and data to be compared according to the selection is stored. It may be changed.

The face color absolute value estimation unit 18 compares the correlation of the face color defined by the face color correlation definition unit 14 with the face color distribution information stored in the face color distribution information storage unit 16, thereby obtaining the absolute value of the face color in the image. Is estimated.
That is, when the facial color distribution information stored in the facial color distribution information storage unit 16 is represented by the distribution Z in the chromaticity diagram of FIG. 3, the distribution X and the distribution Z are compared, and each point in the distribution X is compared. The absolute value in the face color distribution Z of the persons A to E included in the distribution X, that is, the position that should be in the distribution Z of.

For example, the facial color absolute value estimation unit 18 first obtains the offset amount of the origin of the chromaticity diagram so that the region of the distribution X is included in or substantially coincides with the region of the distribution Z. Then, the absolute value of each face color is estimated by adding the obtained offset amount to the value of each face color. When calculating the offset amount, the offset amount may be further adjusted in consideration of the median value and the center of gravity of the distribution.
In this way, the face color absolute value estimation unit 18 obtains the absolute value of the face color so as to correspond to the face color distribution information while maintaining the correlation between the face colors. The absolute value of the face color thus obtained becomes the target value for image correction for face reproduction.

The image correction unit 20 corrects each of the plurality of images based on the absolute value of each face color estimated by the face color absolute value estimation unit 18. Since each image is corrected so that the face of the same person approaches the absolute value estimated for the face, the same person can be reproduced with substantially the same face color in a plurality of images.
Further, when the correction amount is input from the correction amount input unit 24, the image correction unit 20 corrects all the images while maintaining the correlation between the face colors of the plurality of images according to the correction amount.

As a preferred form, the image correction apparatus 10 includes a display unit 22 and a correction amount input unit 24, displays an image corrected by the image correction unit 20 on the display unit 22, and sets a correction amount for the displayed image. Accept input.
The display unit 22 is an image display device having a display screen, and the correction amount input unit 24 is an instruction input device such as a keyboard, a mouse, or a touch panel incorporated in the display unit 22.

  Note that the face authentication / identification unit 12, the face color correlation definition unit 14, the face color absolute value estimation unit 18, and the image correction unit 20 described above can be configured by hardware or software that executes predetermined calculation processing.

  Next, the image correction process performed in the image correction apparatus 10 will be described along the flowchart shown in FIG.

  First, a plurality of photographed images in which a plurality of persons are photographed in a similar environment such as a school, a wedding reception, or a party venue are input to the image correction apparatus 10 (step S101).

  Next, the image correction apparatus 10 extracts a face area for each of a plurality of input images by the face authentication / identification unit 12 (step S102), and performs face authentication for the extracted face area image. An individual is identified (step S103). Thereby, individual persons photographed over a plurality of images are specified, and the same person is identified. For example, the persons A to E are specified in the three images of FIGS.

  Subsequently, the face color correlation defining unit 14 calculates the relative difference in face color between individuals in each image (step S104), and a plurality of images photographed in a plurality of images from the face color relative difference data for all images. Correlation of facial color for a person is defined (step S105). For example, each point in the distribution X of FIG. 3 is obtained.

When the correlation between the face colors in the plurality of images is obtained, the face color absolute value estimation unit 18 compares the face color distribution X with the face color distribution information stored in the face color distribution information storage unit 16 (step S106). The absolute value of the face color is estimated for each person in the image (step S107).
The estimated absolute value of the face color is sent to the image correction unit 20 together with the image, and the image correction unit 20 corrects all the images using the absolute value of the face color (step S108).

  When all the images have been corrected, the corrected image is output to the display unit 22 and displayed on the monitor (step S109). The operator can evaluate the correction result with respect to the correction processed automatically by looking at the displayed image. As a result of the evaluation, when it is determined that further correction is necessary, a correction value (correction value of the correction amount) is manually input from the correction amount input unit 24 (Yes determination in step S110). On the other hand, when it is determined that the correction result is good, the correction value correction value is not input (No determination in step S110). At this time, an instruction indicating that the correction is OK may be input.

  Here, the input of the correction value may be performed for any one person, but if a person with a high appearance frequency is selected, the operator's judgment becomes easy. What is necessary is just to input a correction value for any one person. Since the correction amount of the face color of one person is determined, the correction amount of the face color of the other person is automatically calculated from the correlation of the face colors defined in step S105. It is because it is required.

  If all the corrected images are displayed on the display unit 22, the operator's evaluation work becomes complicated, so it is preferable to select one or several images. At this time, any one of a plurality of images may be selected. However, it is preferable to select an image in which a person with a high appearance frequency is shown. It is more preferable to select the one shown in the center.

Further, in the face color correlation definition in the face color correlation definition unit 14 in step S105, groups that are independent in the face color distribution (for example, a yellow race group, a black race group, a white race group). If there is, a face having a high appearance frequency may be extracted for each group, and one image in which the extracted face is captured for each group may be displayed on the display unit 22 for evaluation. .
Thereby, it is possible to correct face colors having different colors to appropriate colors while maintaining the correlation between the face colors.

In step S105, when there are face color groups having no correlation, a plurality of face color correlations corresponding to each face color group are defined. For example, there are six people, people A to F, in multiple photos, but there are groups of people A, B, and C and people D, E, and F, and the members of the two groups are together. If the image is not shown in FIG. 2, two face color correlations independent of each other are defined corresponding to each group.
In this case, in step S109, a face with a high appearance frequency may be extracted for each independent group, and one image obtained by photographing a face with a high appearance frequency for each group may be used for evaluation.

  When the correction value is input in step S110, that is, when the correction amount is changed manually, the image correction apparatus 10 interprets that the target value is shifted, and applies the same change to all the images. That is, returning to step S108, correction is performed using the correction values input to all images, and the corrected image is displayed again on the display unit 22 (step S109).

If no correction value is input in step S110, that is, if it is determined that the correction result is good, a corrected image is output.
The corrected image output from the image correction apparatus 10 is sent to a digital photographic printer for print creation. Further, it may be sent to a recording device such as a display device or a media driver to display an image or store image data.

As described above, according to the present invention, the person is identified by face authentication, and the image is corrected so as to keep the relative difference by grasping the relative difference of the face color by the combination of the persons who are captured at the same time. In a plurality of images taken in the same environment, it is possible to output stable images in which face reproduction of the same person does not vary and each face color is appropriately corrected.
Further, since the relative difference of the face colors is used, the entire shift can be corrected by verifying any one face color.

In the above-described example, as a preferred form, the correlation between the face colors defined by the face color correlation definition unit 14 in step S105 is compared with the face color distribution information stored in the face color distribution information storage unit 16 in step S106, and the estimated face color The correction is performed using the absolute value of the target value as a target value.
As described above, the face color distribution included in the image for one case (for one group) is compared with the distribution of the general face color to determine the correction amount, so that faces with different skin colors are mixed. Even in this case, each face color can be reproduced to an appropriate color density, and a preferable skin color can be reproduced as a whole.

  However, the present invention is not limited to this, and a preferable facial color distribution can be obtained without using facial color distribution information, such as when the number of images in the same group is sufficiently large, or when it is known that the predetermined shooting conditions have been met. If it can be obtained, the correlation between the face color distribution information is not compared and the correlation between the face colors is defined in step S105. Then, the process proceeds to step S108, and all images are displayed based on the defined correlation between the face colors. It is good also as a form corrected. In this case, the image correction apparatus 10 may not include the face color distribution information storage unit 16 and the face color absolute value estimation unit 18.

  In the above-described example, as a preferable mode, after all images are corrected in step S108, a corrected image is displayed, an input of a correction value for correction amount is received, and correction is performed again with the input correction value. did. According to this, the correction amount can be adjusted more finely and correction according to the taste of the customer or user can be performed.

  However, the present invention is not limited to this, and after the initial correction in step S108, the process may proceed to step S111 without outputting a display or a correction value, and a corrected image may be output. In this case, the image correction apparatus 10 does not have to include the display unit 22 and the correction amount input unit 24.

  The image correction method and apparatus according to the present invention have been described in detail above, but the present invention is not limited to the various embodiments described above, and various improvements and modifications can be made without departing from the spirit of the present invention. Of course it is good.

It is a block diagram which shows one Embodiment of the image correction apparatus of this invention. (A)-(c) is a schematic diagram which shows the example of a picked-up image. It is a schematic diagram which shows the example of face color distribution on a L ^* a ^* b ^* color system chromaticity diagram. It is a flowchart of the image correction process implemented in the image correction apparatus of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image correction apparatus 12 Face authentication / identification part 14 Face color correlation definition part 16 Face color distribution information storage part 18 Face color absolute value estimation part 20 Image correction part 22 Display part 24 Correction amount input part

Claims (6)

A method for correcting a plurality of images in which a plurality of persons are photographed,
Extracting a facial region in the image;
Identifying the person in the image by face authentication for the extracted face region;
Calculate the relative color difference between the persons in the same image,
From the relative difference in facial color between the persons calculated for each of the plurality of images, defining a correlation of facial colors for the plurality of persons photographed in the plurality of images,
An image correction method, wherein the image is corrected based on the correlation between the face colors. By comparing the correlation of the face color with predefined face color distribution information, the absolute value of the face color in the image is estimated,
The image correction method according to claim 1, wherein the image is corrected based on the estimated absolute value of the face color. Display the corrected image on a display device;
The image correction method according to claim 1, wherein the image is corrected based on the correlation between the face colors in accordance with a correction amount input to the displayed image. A face authentication / identification unit that extracts a face area in an image and identifies a person in the image by face authentication with respect to the extracted face area;
Calculating a relative difference in face color between the persons in the same image, and a plurality of the persons photographed in the plurality of images from the relative difference in face color between the persons calculated for each of the plurality of images; A face color correlation definition section for defining the correlation between the face colors of
An image correction apparatus comprising: an image correction unit that corrects the image based on the correlation between the face colors defined by the face color correlation definition unit. Furthermore, a storage unit for storing pre-defined facial color distribution information;
A face color absolute value estimation unit that estimates the absolute value of the face color in the image by comparing the correlation of the face color defined by the face color correlation definition unit with the distribution information of the face color stored in the storage unit And
The image correction apparatus according to claim 4, wherein the image correction unit corrects the image based on an absolute value of the face color estimated by the face color absolute value estimation unit. A display unit for displaying the image corrected by the image correction unit;
A correction amount input unit that receives an input of a correction amount for the displayed image,
The image correction device according to claim 4, wherein the image correction unit corrects the image based on the correlation of the face color according to the correction amount input by the correction amount input unit.

Images