JP3497801B2 - Face image display method and face image processing device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention takes a face of a customer, performs a makeup process on the face image to simulate makeup for makeup such as foundation, eyebrows, and lipstick, and displays the face image on a display. It is used for a makeup advice system that gives advice such as makeup methods.

The make-up advice system includes a face of a customer, a hairstyle simulation, a hair color simulation, and a clothing or accessory simulation.

The present invention, in such a system,
The present invention relates to a correction method for displaying a face image of a customer (target person) with a brightness suitable for displaying the skin color of the face image, and a processing device therefor.

[0004]

2. Description of the Related Art Demonstration sales have hitherto been widely used for selling cosmetics in stores. Its general form is
The salesperson listens to the request of the target person who is a customer, selects a makeup method and cosmetics, and actually applies makeup to the target person's face. In the demonstration sales, makeup is applied using the face of the target person, so it is an effective means to inform the target person of the effect, but if the finish does not meet the request of the target person, remove makeup and try again. , I need to start over.

As described above, in the conventional demonstration sales, it takes a lot of time to apply makeup to the target person and to apply makeup again, and it is difficult to perform efficient sales.

Therefore, the applicant of the present application has proposed a technique of displaying a face image of a subject on a display device and making full use of the image processing technique by a computer device to perform makeup makeup simulation in Japanese Patent Application No. 10-373975. Wish 2000-5
4034 (all unpublished at the time of filing this application). The technique will be briefly described below.

First, the subject selects a desired makeup method and cosmetics by taking a photograph of makeup on the model. Next, by applying makeup data of the model selected by the subject to the image of the face of the subject, it is possible to simulate a state where the subject applies makeup.

At this time, it is difficult to directly combine the makeup data applied to the model with the image of the face of the subject. Therefore, the makeup data is changed from the face shape of the model so as to match the image of the face of the subject. It transforms into the shape of the subject's face.

With this, the subject can confirm the finish condition of the makeup in advance by the photograph of the model to which the makeup is applied and the state when the makeup is applied to his / her face. It is possible to prevent the salesperson from applying makeup different from what the target person desires due to the difference in feeling between the salesperson and the target person. Therefore, it is possible to accurately grasp the finish condition of makeup and efficiently sell cosmetics.

Regarding the technique of face image processing, refer to References (Nobuichi Ichimatsu, Yoichi Muraoka, "Sensitivity and Information Processing" edited by Interdisciplinary Conference of Japan, The Institute of Electronics, Information and Communication Engineers, AUG, 1997.Vol.J80-A.
No.8, pp.1231-1249).

Further, by preliminarily classifying the makeup data according to the face type, the type of cosmetics and other factors and storing the makeup data in the memory, the makeup data desired by the subject can be quickly selected from a large number of makeup data. can do.

[0012]

When performing the makeup simulation of the above makeup, it is necessary to control the exposure amount so that the average value of the luminance signal of a certain portion in the image becomes an appropriate level. For example, a desirable image brightness standard is set such that the luminance signal level of a person's flesh color is 55 to 75 IRE, and the entire image or a part of the region predicted to be occupied by the face in the center of the image or the color signal is used. It is necessary to control the exposure amount with a diaphragm or the like so that the average luminance of the portion determined to be a skin color matches the reference luminance level. However, since a so-called fair white or dark black, or a skin color different for each person is used as a reference, a specific color is not always output at the same level.

Further, it is necessary to prevent the saturation of the high luminance and low luminance areas and the deterioration of the contrast by keeping the luminance distribution of the image within the output range of the camera. However, when there is a remarkably bright area or a dark area depending on clothes, the brightness of the face changes.

Further, the exposure amount is set so that the reflectance (brightness) of 3 to 96% (black to white), which is the lightness range of the colorimetric value of an object in nature, substantially matches the latitude (effective exposure area) of the film. Need to control. However, when the highlight area is included in the effective exposure area, the luminosity of the flesh color is expressed slightly darker than the luminosity that is considered to give a preferable impression to the viewer.

As described above, when the face image of the customer used for the makeup simulation is photographed by the camera, if the skin color of the face in the displayed image is too dark or too bright, the makeup simulation (foundation, lipstick, The color reproducibility of (eyebrows) deteriorates.

When displaying the photographed face image on the display, if the skin color part of the face in the displayed image is too dark,
Or, if it's too bright, a “look” that looks good to the customer
Since it does not give the impression of, it may not be used for makeup advice.

Therefore, according to the present invention, a face image display method and a face image display method capable of always correcting an image displayed on a display to a constant brightness even when conditions such as brightness during photographing change. An object is to provide a display device. SUMMARY OF THE INVENTION It is an object of the present invention to provide a face image display method and a face image display device capable of correcting the skin color of the face of an image to a desirable “look”.

[0018]

In the same environment where a face image is captured, a camera is used to capture a color chart with known colorimetric brightness (reflectance). The illuminance correction coefficient Q1 under the shooting environment is calculated from the RGB values of the color chart image input from the camera to the computer device by this shooting, the taken face image is output to the computer device, and the entire image is processed with the illuminance correction coefficient Q1. The RGB component of each pixel is converted into a conversion output level converted to the brightness of the colorimetric value, and the illuminance is corrected.

By the illuminance correction coefficient Q1 under the photographing environment,
Colorimetric value conversion to luminance The face image converted to the output level is corrected to the output level from the camera using the previously corrected illuminance correction coefficient Q2 that "the face (skin color) looks like preferable brightness". Output to the display. When the illuminance correction coefficient Q1 of the shooting environment is equal to Q2, the camera output may be output to the display as it is.

As a result, even when conditions such as the brightness at the time of photographing change, the image displayed on the display can be constantly corrected to have a constant brightness. Therefore, it is possible to correct the flesh color of the face to a desirable "look".

That is, a first aspect of the present invention is to photograph a face of a subject, and subject the photographed face image to a makeup process, a hair-make process, and sometimes a makeup process including clothes and ornaments to display means. A method of displaying a digital color face image to be displayed, wherein the brightness of the face image displayed by the display means is monotonic with respect to the reflectance Y of the colorimetric value of the object to be photographed, the Green pixel value of the RGB value of the image is monotonous. It is determined by a correction function that satisfies the increasing relationship, and black of the image of the Macbeth color checker No. 22 is set to 0 and white is set to 255 25
Green pixel value of RGB value of 6 gradations is 145 or more 17
Brightness equivalent to 0 or less (or 57% of the maximum brightness value)
The brightness is corrected to 67% or less). Preferably, RG of Macbeth Color Checker No. 22
The brightness is corrected so that the Green pixel value of B is 150 or more and 165 or less. As shown in Table 1, when the RGB Green pixel value of Macbeth Color Checker No. 22 is less than 145, it is felt that the skin color of the subject's face is too dark. If it exceeds 170, it is felt that the skin color of the subject's face is too bright.

Under the same shooting conditions as when shooting the subject,
A correction coefficient (Q1) for photographing the Macbeth color checker No. 22 and correcting the luminance value of the photographed target person's face image to the brightness under the standard photographing conditions from the photographed Green pixel value of the Macbeth color checker No. 22 The stage of seeking
Based on this correction coefficient (Q1), the step of converting the brightness value of the photographed target person's face image into the brightness under the standard photographing condition, and the brightness value of the target person's face image displayed on the display means are calculated by the standard photographing. From the brightness under the conditions, the Green pixel value of the RGB value of 256 gradations in which the black of the image captured by the Macbeth color checker No. 22 is 0 as the brightness for display and the white as 255 is 145 or more and 170 or less. Brightness (or
It is desirable to include a step of correcting the brightness value to a brightness value of 57% or more and 67% or less of the maximum brightness value.

A second aspect of the present invention is a camera for photographing a face image of a subject, and an image processing means for performing brightness correction processing of the photographed face image of the subject and simulation of make-up, hair make-up and the like. And a display means for displaying the face image of the subject who has undergone this image processing. The image processing means photographs the gray chart of the reference color checker with the camera, obtains its RGB values, and R
Means for obtaining a correction coefficient such that the GB value is the brightness under the reference photographing condition, means for converting the luminance value of the photographed image of the subject inputted based on this correction coefficient into the luminance value for the reference condition, and image processing And a means for correcting the luminance value in the bright direction based on a predetermined correction coefficient for display of the face image of the target person after performing the above.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The configuration of a face image processing apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of a face image processing apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the present invention is a camera 1 for photographing a face image of a subject, and an image for performing brightness correction processing and make-up and hair-make simulation on the photographed face image of the subject. The image processing device 2 includes a processing device 2 and a display device 3 that displays the face image of the subject who has performed this image processing. The image processing device 2 captures the Macbeth color checker No. 22 with the camera 1 and displays the RGB values of the area. Illuminance correction coefficient calculation unit 4 that calculates and obtains a correction coefficient Q1 such that the RGB values of the pixels of the image become the brightness of the standard shooting conditions.
A luminance value conversion unit 5 for converting the luminance value of the photographed image of the subject input based on the correction coefficient Q1 into the luminance value of the reference condition, and the subject simulation after the makeup simulation by the makeup simulation unit 6. The face image is the Green of the 256 gradation RGB values of the image of the Macbeth color checker No. 22.
The brightness value is changed in the bright direction based on the correction coefficient Q2 for correcting the brightness value corresponding to the brightness where the n pixel value is 145 or more and 170 or less (or the brightness of 57% or more and 67% or less of the maximum brightness value). It is a face image processing device including a brightness value conversion unit 5 for correction. Hereinafter, examples of the present invention will be described in more detail.

In order to correct the brightness of the output image of the camera, the output characteristic function of the camera used for photographing is obtained in advance. For example, the Macbeth color checker is photographed in a state where the surface reflection light of the object is removed using a polarizing plate, and the relationship between the luminance value (known) of the gray chart (# 19 to 24) and the camera output level is examined. It is confirmed that the characteristic function can be approximated by, for example, a quadratic equation, and the characteristic functions of RGB are obtained by the least square method. The shooting condition at this time is set as a reference shooting condition. FIG. 2 shows the relationship between the green output level Ig of the image from the camera 1 and the colorimetric value luminance Oi of the gray chart.

The illumination light is installed toward the Macbeth color checker or the person to be advised. For example, two lights are used, and the brightness of each lighting light is 1000 Lx at the illuminance at the face position of the subject.
Adjust so that it is about the same level (2 lights 1800-2000
Lx). The height of the camera is set so as to be substantially equal to the height of the line of sight of the Macbeth color checker or the adviser. The distance between the light and the subject is about 130
cm, and the distance between the camera and the subject is about 100 cm. The space above and to the left and right of the person being advised should be covered with booths to reduce the effects of outside light during shooting.

After the photographing conditions are set, the white balance of the camera is set. The shooting environment is 20 ~ in summer.
30 ° C (reference value for setting: 26-27 ° C), humidity 40
-80% (standard 50-60%), temperature 15-25 in winter
The temperature is set to ° C (reference 20 to 22 ° C) and the humidity is set to 20 to 60% (reference 40 to 50%). In the interim period, both temperature and humidity are set to values midway between summer and winter. For example, the temperature and humidity at the time of shooting in early June are 30 ° C,
It was 40%.

A Macbeth color checker is installed at a predetermined position and a picture is taken with a camera. The captured image is output to the image processing device 2. The illuminance correction coefficient calculation unit 4 calculates the illuminance correction coefficient Q1 after calculating the average value of the Green pixels in the regions corresponding to the color charts 21, 22, and 23 in the image.

Nos. 21, 22 and 2 obtained from photographed images
Average value Igi (i) of the green pixel of the 3rd camera output
= 21,22,23) is converted into the output level Igi 'converted into the green component of the incident light quantity by the inverse function of the output characteristic function. Figure 3 shows the converted output level Ig 'and the image Green
The relationship with the component output level Ig is shown. Igi '= (-C1 + √ (C1 ² -4C2 (C0-Ig
i))) / (2C2) (where C2, C1 and C0 are coefficients of the 2, 1, 0th order of the characteristic function of the Green component)

Colorimetric luminance Oi-converted output level I of color chart
In the g ′ space, the slope α of a straight line passing through {G21 ′, G22 ′, G23 ′} is obtained from the least square method. This α and the slope α 0 of a straight line obtained in the same manner under the shooting conditions for which the output characteristic function is obtained
Of the illuminance correction coefficient Q (corresponding to the ratio of light quantity) and α / α0
Set to 1. In order to obtain better color reproducibility, it is desirable to adjust the exposure amount of the camera so that the illuminance correction coefficient Q1 becomes a value close to 1.0. FIG. 4 shows the relationship between the inclination of the straight line and the aperture value. In the example of FIG. 4, the aperture F8.0 is the aperture F
Since it is closer to the inclination of the reference shooting condition than in the case of 5.6, it is desirable that the aperture value be F8.0.

Subsequently, a face image of the advise target person is photographed. The captured image is output to the image processing device 2. The values Ir, Ig, Ib of each pixel of the image are converted into converted output levels Ir ', Ig', Ib 'using the inverse function of each characteristic function and the previously determined illuminance correction coefficient Q1. FIG. 5 shows an example of correcting the converted output level Ig ′ by the illuminance correction coefficient Q1. The thin solid line in FIG. 5 is the actually photographed output level, and this output level is multiplied by 1 / Q1 in order to approach the output level of the reference photographing condition. That, Ik '= ((C1k + √ (C1k 2 -4C2k (C0k
-Ik))) / (2C2k)) * m / Q1k = R, G, B. By this correction, the brightness of the converted image becomes equal to the brightness captured under the reference shooting condition (the condition when the output characteristic function is obtained) regardless of the illuminance at the time of shooting and the exposure amount of the camera.

Makeup simulation is performed on the face image whose illuminance has been corrected. That is, for (1) face image,
Determine the positions of feature points on the eyebrows, eyes, nose, and lips. (2) Perform eyebrow simulation. (3) Make a foundation makeup simulation. (4) Perform lipstick simulation.

The face image (brightness conversion output levels Ir ', Ig', Ib ') after the make-up simulation is converted into R by using a predetermined illuminance correction coefficient Q2 for display.
The output level Ir of the camera from the characteristic function of each GB,
Convert to Ig, Ib. In order to determine the illuminance correction coefficient Q2, a subject is photographed, an image subjected to illuminance correction for display is displayed on the display device 3 after the makeup simulation,
The impression of the displayed image was evaluated by the subject himself or a third party. The results are shown in Table 1.

[0035]

[Table 1]

From the above results, the Green pixel value of Macbeth color checker No. 22, which is in the range from "brightness that can be seen by any person of any complexion" to "brightness that is felt slightly bright", is 150 or more and 165 or less. Brightness (illuminance correction coefficient Q2 is 1.9 ≦ Q2 ≦ 2.1)
Is the brightness for display.

The "preferable brightness" of a digital color image is, for example, the brightness of an 18% gray plate having a reflectance almost equal to that of Macbeth Color Checker No. 22 in silver halide photography, and the center of the latitude of the film (BMP gradation). 0 to
It is 20% or more brighter than the brightness when it is adjusted to the center 128 of 255). In addition, the brightness that the skin color of a person (Japanese), which is a guide for adjusting the exposure amount with the automatic exposure control device of the camera, is "IRE = 55 to 75" is the RG of BMP.
It is considered to correspond to a Green value of B of about 110 to 150, but the above-mentioned “preferable brightness” is brighter than this. This is considered to give an impression of "preferable" because the dullness of skin and stains that can be confirmed when photographed realistically become inconspicuous by being corrected brightly.

The image output level from the camera 1 is shown in FIG.
The relationship with the calculated force level Ig 'is shown. That is, Ik = C0k + C1k (Ik '· Q2 / m) + C2k
(Ik '· Q2 / m)^Two k = R, G, B

As for this illuminance correction coefficient Q2, the Green pixel value of Macbeth color checker No. 22 is about 150.
A range in which the above is converted to 165 or less (when the correction formula of the camera used in the embodiment is used, 1.9 ≦ Q2 ≦ 2.
It is desirable to decide in 1). FIG. 7 shows an example of correcting the converted output level Ig ′ by the illuminance correction coefficient Q2. As shown in the example of FIG. 5, the output level converted to the reference shooting condition at the time of shooting by the illuminance correction coefficient Q1 is displayed by being multiplied by Q2 by the illuminance correction coefficient Q2 at the time of display.

The image whose illuminance is corrected by the illuminance correction coefficient Q2 is output to the display. At this time, the display conditions (color temperature, γ, etc.) are fixed to fixed conditions (default values of color temperature, γ, etc. are values at which the face image looks preferable).

By the method as described above, even when the condition such as the brightness at the time of photographing changes, the image displayed on the display can be automatically corrected to a constant brightness.
Further, when the correction function is used, if the illuminance correction coefficient at the time of display is set to 1.9 to 2.1, the face image can be displayed so that the flesh color of the face has a preferable "look". At this time, Gre of the cheek part of the average Japanese face
The en pixel value is approximately 163 to 180. Further, the value of the illuminance correction coefficient described here is based on the assumption of a target person of yellow race such as Japanese, but in the case of a target person of white color having a higher flesh color, the correction coefficient is 0.8 to 0. .9 times to reduce the brightness of the image, and for a black target person with low flesh color brightness, increase the brightness of the image by increasing the correction coefficient by 1.2 to 1.5 times. Can be corrected to a brightness suitable for.

Although the makeup simulation system has been described in the above embodiments, a makeup simulation for a hairstyle and a makeup simulation for a hair color are performed, and a makeup simulation for clothes and ornaments is performed to display the result and give advice. The illuminance correction described in this embodiment can also be applied to the coordinate system. With this, by displaying the face image so that the flesh color of the face has a preferable "look",
As in the case of the makeup simulation system, it is possible to give a good impression to the customer (target person).

[0043]

As described above, according to the present invention,
Even when conditions such as the brightness at the time of shooting change, the image displayed on the display can be constantly corrected to have a constant brightness. Further, it is possible to correct the flesh color of the face to a desirable "look".

[Brief description of drawings]

FIG. 1 is a block configuration diagram of a face image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a relationship between a green output level of an image from a camera and a colorimetric value luminance of a gray chart.

FIG. 3 is a diagram showing a relationship between a converted output level and a green component output level of an image.

FIG. 4 is a diagram showing the relationship between the inclination of a straight line and the aperture value.

FIG. 5 is a converted output level Ig ′ according to an illuminance correction coefficient Q1.
FIG.

FIG. 6 is a diagram showing a relationship between an image output level from a camera and a converted output level.

FIG. 7 is a converted output level Ig ′ according to an illuminance correction coefficient Q2.
FIG.

[Explanation of symbols]

1 camera 2 Image processing device 3 display devices 4 Illuminance correction coefficient calculator 5 Luminance value converter 6 Makeup Simulation Department

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI H04N 7/18 H04N 1/46 Z (72) Inventor Nobutoshi Kojima 2-3-1, Fumika, Sumida-ku, Tokyo Kao Research Co., Ltd. In-house (56) References JP-A 5-103336 (JP, A) JP-A 2000-132155 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 9/64 G06T 1 / 00 340 G06T 1/00 510 G06T 5/00 100 H04N 1/46 H04N 7/18

Claims (3)

(57) [Claims] 1. A method for displaying a digital color face image in which a face of a subject is photographed, and the photographed face image is subjected to makeup processing and hair / makeup processing by image processing and displayed on a display means. The brightness of the face image is expressed as the green of the RGB value of the image with respect to the reflectance of the colorimetric value of the object to be photographed.
A correction function that satisfies the relationship that the n pixel value monotonically increases is defined, and G of the RGB value of 256 gradations where black is 0 and white is 255 in the image photographed by the Macbeth color checker No. 22.
A method for displaying a face image, comprising correcting the brightness so that a reen pixel value is 145 or more and 170 or less. 2. The luminance value of the face image of the subject photographed from the Green pixel value of the photographed Macbeth color checker 22 under the same photographing conditions as when photographing the subject. And a step of obtaining a correction coefficient (Q1) for correcting the brightness to the brightness under the standard shooting conditions, and based on the correction coefficient (Q1), the brightness value of the face image of the photographed subject is calculated based on the brightness under the standard shooting conditions. And the brightness value of the face image of the target person displayed on the display means from the brightness under the standard shooting conditions, and the blackness of the image taken by the Macbeth color checker No. 22 as the display brightness is set to 0, White 2
The method for displaying a face image according to claim 1, further comprising a step of correcting the luminance pixel to a brightness value such that the Green pixel value of RGB values of 256 gradations of 55 is equivalent to 145 or more and 170 or less. 3. A camera for photographing a face image of a subject, image processing means for performing brightness correction processing of the photographed face image of the subject and simulation of makeup, hair make-up, etc., and performing the image processing. And a display unit for displaying the face image of the target person, wherein the image processing unit captures a gray chart of the reference color checker with the camera to obtain RGB values thereof, and the RGB values are the reference imaging conditions. A means for obtaining a correction coefficient to obtain the brightness, a means for converting the brightness value of the photographed image of the subject inputted based on this correction coefficient into a brightness value of the reference condition, and a means for performing the image processing on the subject. Macbeth Color Checker 2 based on a predetermined correction coefficient for displaying face images
When shooting No. 2, set black to 0 and white to 255
Green pixel value of 256 gradation RGB value is 145 or more
A face image processing apparatus, comprising: a unit that corrects the brightness to 170 or less .