JP4506243B2 - Skin simulation image forming method - Google Patents

Description

  The present invention relates to a method of changing color unevenness while maintaining naturalness as a texture or skin color in a skin simulation image.

  A skin simulation image is formed in order to make it possible to grasp, as an image, the skin in which pores, spots, etc. have become inconspicuous by basic makeup or makeup, and the skin that has been stained by sunburn or the like.

  As a method for forming a simulation image of skin, a database of changes in colorimetric values that can be changed by basic makeup for various skins, and the colorimetric values of facial images of the customer's bare skin are changed by image processing based on the database. A method has been proposed (Patent Document 1).

  Moreover, in the construction of a database, a method of using the amount of melanin component and the amount of hemoglobin component of the internally reflected light of the skin has been proposed (Patent Document 2).

  On the other hand, with the progress of computer graphics technology, it has become possible to reproduce human skin on an image with a more natural texture than before, taking into consideration the layer structure of the skin and the reflection characteristics based on it. However, the actual situation is that reproduction of the spatial distribution of skin, such as spots and acne, found in actual skin is performed manually. That is, in the formation of a simulation image of the skin, color adjustment such as making the skin color brown or lightening as a whole according to the information on the melanin content and hemoglobin content of the skin is automatically performed by image processing using a computer. However, spots and the like must be manually distributed on the image.

JP 2002-203238 A JP 2002-200050 A

  Therefore, an object of the present invention is to make it possible to form a simulation image in which color unevenness such as a stain is increased or decreased while maintaining a natural texture and color of skin by image processing using a computer.

  The present inventor extracts an image contributed by the melanin component from the internally reflected light image of the skin, decomposes it into a plurality of spatial frequencies, and specifies according to the size of color unevenness such as pores and spots. When the contrast of the image in the frequency band is changed and the image is synthesized again, it has been found that color unevenness due to pores, spots, etc. can be increased or decreased while maintaining the natural texture and color of the skin on the simulation image.

That is, the present invention acquires an internal reflection light image and a surface reflection light image for the subject's skin,
The internal reflected light image is separated into a melanin internal reflected light image and an internal reflected light image other than the melanin component by independent component analysis,
The melanin component internally reflected light image is decomposed into multiple spatial frequency bands,
The contrast is selectively changed with respect to the melanin component internally reflected light image decomposed into the plurality of spatial frequency bands,
Next, the melanin component internally reflected light image of the plurality of spatial frequency bands is synthesized to form a melanin component modified internally reflected light image,
A composite internal reflection image is formed by combining a melanin component modified internal reflection image and an internal reflection image other than the melanin component,
Provided is a skin simulation image forming method for forming a simulation image in which skin color unevenness is changed by combining a combined internal reflection light image and the surface reflection light image.

The present invention also provides an image forming means capable of forming an internally reflected light image and a surface reflected light image of the skin using polarized light,
A calculation means for forming a skin simulation image based on an image acquired by the image forming means, and a skin simulation image forming system comprising a display for displaying the simulation image,
The computing means is
The internal reflected light image is separated into a melanin internal reflected light image and an internal reflected light image other than the melanin component by independent component analysis,
The melanin component internally reflected light image is decomposed into multiple spatial frequency bands,
Selectively change the contrast for the melanin component internally reflected light image decomposed into multiple spatial frequency bands,
Next, a melanin component internally reflected light image of a plurality of spatial frequency bands is synthesized to form a melanin component modified internally reflected light image,
A composite internal reflection image is formed by combining a melanin component modified internal reflection image and an internal reflection image other than the melanin component,
Provided is a skin simulation image forming system that forms a simulation image in which color unevenness of skin is changed by synthesizing a synthesized internal reflected light image and the surface reflected light image.

  According to the present invention, the processing of a computer incorporating a specific calculation means increases or decreases the color unevenness due to pores, spots, etc. on the subject's skin image without damaging the texture of the skin or the natural color of the background. A simulated image can be formed.

  Further, although there are various sizes of color unevenness due to pores, spots, etc. in actual skin, according to the present invention, color unevenness of a specific size can be selectively generated, increased or eliminated. Can do.

  Therefore, according to the present invention, it is not necessary to manually distribute pores, spots and the like on the simulation image.

  Hereinafter, the present invention will be described in detail with reference to the drawings. In each figure, the same numerals indicate the same or equivalent components.

  FIG. 1 is a process diagram of a skin simulation image forming system of one embodiment of the present invention and a skin simulation image forming method performed there.

  This system includes an image forming unit 1 capable of forming an internal reflection light image and a surface reflection light image of skin using polarized light, and a calculation unit that forms a skin simulation image based on an image acquired by the image formation unit 1. 2 and a display 3 for displaying a simulation image.

  More specifically, the image forming means 1 can be composed of a light source 4 for illumination, a digital camera 5, and polarizing plates 6 and 7 that are detachably provided on the front surface of the light source 4 and the digital camera 5, respectively.

  As the calculation means 2, a personal computer having an image processing function to be described later can be used. The personal computer may be connected to a display 3 capable of appropriately switching or simultaneously displaying a normal image, an internal reflection light image, a surface reflection light image, a simulation image, and the like of a subject, and may further be connected to a printer.

  In this system, skin simulation images are formed by first obtaining an internal reflection light image and a surface reflection light image for the subject's skin.

  The internally reflected light image is formed by mounting the polarizing plate 7 on the front surface of the digital camera 5 so that the polarization direction is orthogonal to the polarizing plate 6 on the front surface of the light source 4 and removing the surface reflected light component. Can do. There is no limitation on the skin part to be simulated, and any part such as a face, arm, or chest can be used. Further, the skin area where the image is acquired and the simulation image is formed may be the entire image forming area or a part thereof.

  The surface reflected light image includes an image obtained by attaching the polarizing plate 7 on the front surface of the digital camera 5 so that the polarizing direction is the same as that of the polarizing plate 6 on the front surface of the light source 4, and the polarizing plate 7 with the polarizing plate 6. It can be formed from the difference from the image obtained by mounting so that the polarization directions are orthogonal.

  In addition, it is preferable that the image forming unit 1 captures images without using the polarizing plates 6 and 7 and obtains a normal image for reference.

  Next, an independent component analysis is performed on the internally reflected light image by the computing means 2, and the internally reflected light image is divided into a melanin component internally reflected light image and an internally reflected light image other than the melanin component (specifically, a hemoglobin component). The internal reflection light image and the shadow component internal reflection light image are separated.

  Here, independent component analysis refers to the skin layer structure, the epidermal layer containing melanin as the main pigment component, the dermis layer containing hemoglobin as the main pigment component, and the subcutaneous tissue containing other pigment components. This is an analysis method that separates and extracts the signals of each layer from the image signal, considering that the signal is emitted independently from each layer and the mixture of them is an image signal.

  More specifically, for RGB of the image signal, -log (R), -log (G), and -log (B) are assigned to the x-axis, y-axis, and z-axis, respectively, and the skin color of the flat portion of the skin is there. When the color space mapping is performed, the distribution is almost flat as shown in FIG. 2A, and it can be seen that two components contribute to the skin color. This independent two-component signal intensity is considered to correspond to the amount of melanin or hemoglobin, respectively, and as shown in FIG. 2B, the skin color is closer to the component vector of melanin (-log (B)) ) And the hemoglobin component vector (the one closer to -log (G)). Therefore, a signal representing the amount of melanin or a signal representing the amount of hemoglobin is extracted from the signal of the internally reflected light image of each subject, and a distribution image of the melanin component amount and a distribution image of the hemoglobin component amount are output. However, if there is undulations or illumination unevenness, the amount of both melanin and hemoglobin components obtained by projecting onto a plane containing both the melanin and hemoglobin vectors and removing the shadow component along the direction of the vector of the light source components acquired in advance is used. obtain. Further, by subtracting the distribution image of the melanin component amount and the distribution image of the hemoglobin component amount from the internal reflection light image, an internal reflection light image of the shadow component is obtained.

  Details of such analysis processing and image processing are described in Vol. 16, No. 9 / September 1999 / J. Opt. Soc. Am. A 2169, and commercially available image analysis software (for example, This can be done by installing Adobe Photoshop).

  Next, the melanin component internally reflected light image is decomposed into a plurality of spatial frequency bands.

  A method of decomposing an image into a plurality of spatial frequency bands is called subband coding. Subband coding is a technique in which a single image is hierarchized into frequency components with multiple resolutions, and independent processing can be applied to each frequency component, or frequency components are recombined (JWWoods, SDO). 'Neil, IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-34, 1278-1288 (1986)).

  Subband coding includes image pyramid method (DJHeeger, JRBergen, COMPUTER GRAPHICS PROCEEDINGS, p229-238 (1995)), wavelet multi-resolution analysis method (Kouichi Niijima wavelet image analysis science and technology publication), etc. As an example, the wavelet multi-resolution analysis method, which has been completed as above and is easy to obtain software and the like, is taken up. In decomposing the melanin component internally reflected light image into a plurality of spatial frequency bands in the wavelet multi-resolution analysis, a preferred way of dividing the spatial frequency band can be appropriately determined according to the size of the target color unevenness, for example, If there are three types of skin color unevenness of interest: pores with a diameter of about 0.5 mm, fine spots with a diameter of about several millimeters, and large spots with a diameter of about several millimeters to several tens of millimeters, each increase or decrease on the simulation image. In this case, it is preferable to obtain a high-resolution image that can sufficiently reproduce the size of the finest pores and divide the image into three bands corresponding to the size of the target color unevenness on the image.

  Thus, the images with different spatial frequency bands decomposed by the wavelet multi-resolution analysis hold information on color unevenness of different sizes, each derived from a melanin pigment. Therefore, in the present invention, the contrast of an image in a specific spatial frequency band is selectively changed in accordance with the magnitude of color unevenness to be increased or decreased in a skin simulation image. For example, in a simulation image, when emphasizing color unevenness due to pores, the contrast of an image in a high frequency band is increased, and when it is reduced, the contrast is decreased. Further, when emphasizing color unevenness due to large spots, the contrast of an image in a low frequency band is increased, and when it is reduced, the contrast is decreased.

  After selectively changing the contrast for the melanin component internally reflected light images of multiple spatial frequency bands, the melanin component internally reflected light images of multiple spatial frequency bands are synthesized to synthesize the melanin component internally reflected light images. Forming a composite internal reflected light image by combining this image and an internal reflected light image other than the melanin component (specifically, a hemoglobin component internally reflected light image and a shadow component internally reflected light image), By combining the synthesized internal reflected light image and the surface reflected light image, a simulation image in which the color unevenness of the skin is changed is formed.

  The skin simulation image thus obtained is a natural texture image in which only the color unevenness is increased or decreased while retaining the skin texture and skin color of the original image.

  On the other hand, without separating the melanin component internally reflected light image into a plurality of spatial frequency bands, even if the contrast is changed to form a simulation image (that is, the contrast of the entire melanin component internally reflected light image is increased or decreased, Combined with other component images to form a composite internally reflected light image, and then combine a surface reflected light image to form a simulation image), in the simulation image only pores and spots of a specific size It cannot be increased or decreased. Therefore, when the target spot in the simulation image is increased, color unevenness (pores or the like) other than the target spot is also increased. Also, without separating the melanin component internally reflected light image from the internally reflected light image, the entire internally reflected light image is decomposed into a plurality of spatial frequency bands, and the brightness contrast of each spatial frequency band image is selectively increased or decreased. In addition, the size of nasal cavity and eyelashes other than pores and stains increase and decrease, and natural simulation images cannot be obtained. Furthermore, if the melanin component internally reflected light image is not separated from the internally reflected light image, the spatial frequency band decomposition is not performed, and only the brightness of the image is changed, only the pores and spots are changed in the simulation image. Can not. When these change, the nasal cavity and eyelashes also change, resulting in unnatural skin that is far from the original image.

  As a method of using the simulation image of the skin formed according to the present invention, for example, a database is constructed for specific cosmetics and the degree of conspicuous pores and stains that can be changed by the use thereof, or the continuous use days of specific cosmetics and their Cosmetics such as constructing a database about changes in the degree of conspicuous pores and spots, and presenting simulation images of skin when using specific cosmetics to subjects based on these databases and recommending the cosmetics Can be recommended and sold.

Example 1
The subject's cheeks are photographed using a polarizing plate on a digital camera, the internal reflection light image and the surface reflection light image are acquired, and the internal reflection light image is subjected to independent component analysis, whereby the melanin component internal reflection light image and hemoglobin are obtained. The component internal reflection light image and the shadow component internal reflection light image were obtained, and the melanin component internal reflection light image was decomposed into nine layers having different resolutions by wavelet multiresolution analysis. Under the shooting conditions of the present embodiment, among the nine layers, the images of the third to fourth layers, the images of the fourth to fifth layers, and the images of the sixth to seventh layers from the high frequency side are 0.7 to Corresponding to color irregularities of 1.5 mm (pores, etc.), color irregularities of 1.5-3.0 mm in diameter (fine stains, etc.), and color irregularities of 6-12 mm in diameter (large stains, etc.).

  Of these, in the cases where the contrast of the intermediate frequency band image (images of the fourth to fifth layers) is increased by 100% (2 times) and when the contrast is reduced by 100% (0 times), the contrast is changed. The image and the remaining band image are combined to form a melanin component modified internal reflection image, and the hemoglobin component internal reflection image and shadow component internal reflection image are combined to form a combined internal reflection image. Two types of simulation images were formed by combining this and the surface reflected light image. The original image and the obtained simulation image are shown in FIGS. 3A to 3E.

  In this simulation image, it was possible to increase / decrease only the color unevenness due to the target stain while maintaining the texture and background color of the original image.

Comparative Example 1
The melanin component internally reflected light image is a melanin component modified internally reflected light image obtained by raising the contrast by 100% or lowering by 100% without being decomposed into three spatial frequency bands. Two types of simulation images were formed. The obtained simulation image is shown in FIG. 3C.

  In this simulation image, color unevenness such as pores was increased or decreased in addition to the target stain.

Comparative Example 2
The internal reflected light image is decomposed into three spatial frequency band images without being separated into a melanin component internally reflected light image, a hemoglobin component internally reflected light image, and a shadow component internally reflected light image. Two types of simulation images were formed in the same manner as in Example 1 except that the brightness contrast of the (intermediate resolution image) was increased by 100% or decreased by 100%, and the melanin component corrected internal reflection light image was used. The obtained simulation image is shown in FIG. 3D.

  In this simulation image, in addition to pores and stains, the contrast of the contours of the eyelashes and nasal cavity increased and decreased, resulting in an unnatural image relative to the original image.

Comparative Example 3
The internal reflection light image is not separated into a melanin component internal reflection image, a hemoglobin component internal reflection image, and a shadow component internal reflection image, and is not decomposed into three spatial frequency bands, and the brightness contrast is increased by 100%. Or 100% lowered, was used as a composite internally reflected light image, and other types of simulation images were formed in the same manner as in Example 1. The obtained simulation image is shown in FIG. 3E.

  In this simulation image, the contours of eyes and nose were further emphasized or blurred as compared with the simulation image of Comparative Example 2.

Example 2
The subject's cheek part is photographed to obtain the internal reflection light image and the surface reflection light image, and the internal reflection light image is subjected to independent component analysis, whereby the melanin component internal reflection light image, hemoglobin component internal reflection light image, and shadow component are obtained. An internal reflection light image is obtained, and the melanin component internal reflection light image is decomposed into nine layers having different resolutions by wavelet multi-resolution analysis, and from these nine layers, a high frequency band image (high resolution image), an intermediate A frequency band image (intermediate resolution image) and a low frequency band image (low resolution image) were selected.

  FIG. 4 shows the original image. As shown in the figure, this original image has color unevenness A due to pores of 0.7 to 1.5 mm, color unevenness B due to fine stains having a diameter of 1.5 to 3.0 mm, and color due to large stains having a diameter of 6 to 12 mm. There is unevenness C, which corresponds to the above-described high frequency band image, intermediate frequency band image, and low frequency band image, respectively.

  When the contrast of the high frequency band image is increased by 100%, when decreased by 100%, when the contrast of the intermediate frequency band image is increased by 100%, when decreased by 100%, when the contrast of the low frequency band image is increased by 100%, For a total of 6 patterns when lowered by 100%, the image of the band with the changed contrast and the image of the remaining band are combined to form a melanin component corrected internal reflection image, and this and the hemoglobin component internal reflection image And a shadow component internal reflection light image were combined to form a combined internal reflection light image, and this and the surface reflection light image were combined to form six types of simulation images.

  Six beauty specialists compare the simulation image and the original image when the contrast in the high frequency band is raised and lowered with respect to the conspicuousness of the pore A, score as follows, and score the image with increased contrast The average value of the six persons was determined for the difference between the score of the image and the lowered image, and this was taken as the “unevenness noticeable change”. This “unevenness conspicuous change amount” indicates that the greater the numerical value, the greater the difference in the conspicuousness of color unevenness when the contrast is increased and when the contrast is decreased.

Score 5 When the color unevenness of the simulation image is conspicuous with respect to the original image. Score 4 When the color unevenness of the simulation image is conspicuous with respect to the original image. Score 3 When the color unevenness of the original image and the simulation image are conspicuous. When the color unevenness of the simulation image is not conspicuous with respect to the image Score 1 When the color unevenness of the simulation image is not very conspicuous with respect to the original image Similarly, the contrast in the high frequency band is increased or decreased for fine stains B and large stains C. The “unevenness conspicuous change amount” between each simulation image and the original image was calculated. The results are shown in FIG.

  In addition, “unevenness noticeable change amount” between each simulation image and the original image when the contrast of the intermediate frequency band and the low frequency band is raised and lowered were obtained for pore A, fine stain B, and large stain C. The results are shown in FIGS. 5 (b) and (c).

  From these figures, the conspicuousness of the pores A changes greatly by raising and lowering the contrast of the image in the high frequency band, and the conspicuousness of fine spots B changes greatly by raising and lowering the contrast of the image in the intermediate frequency band. It can be seen that the conspicuousness of the large stain C changes greatly by raising and lowering the contrast of the image in the low frequency band.

  INDUSTRIAL APPLICABILITY The present invention is useful in various fields for forming a skin simulation image, such as recommendation and sales of cosmetics.

It is process drawing of the simulation image formation method of skin. It is explanatory drawing of independent component analysis. 3 is an original image taken in Example 1. FIG. 2 is a simulation image obtained in Example 1. FIG. 6 is a simulation image obtained in Comparative Example 1. 10 is a simulation image obtained in Comparative Example 2. 10 is a simulation image obtained in Comparative Example 3. 6 is an original image taken in Example 2. FIG. FIG. 5 is a relationship diagram between unevenness change amount and unevenness size in a simulation image in which the contrast of a specific spatial frequency band image is raised and lowered.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image formation means 2 Calculation means 3 Display 4 Light source 5 Digital camera 6, 7 Polarizing plate

Claims (4)

An image obtained by making the polarization direction of the polarizing plate on the front surface of the light source for illumination and the polarizing plate on the front surface of the digital camera perpendicular to each other to form an internal reflected light image of the skin and making the polarization direction of the polarizing plate the same Image forming means capable of forming a surface reflected light image from the difference from the image obtained by orthogonalizing the polarization direction of the polarizing plate;
A calculation means for forming a skin simulation image based on an image acquired by the image forming means, and a skin simulation image forming system including a display for displaying the simulation image,
The computing means is
The internal reflected light image is separated into a melanin internal reflected light image and an internal reflected light image other than the melanin component by independent component analysis,
The melanin component internally reflected light image is decomposed into multiple spatial frequency bands,
Among the melanin component internally reflected light images decomposed into a plurality of spatial frequency bands, select the image of the spatial frequency band according to the size of the color unevenness to be increased or decreased in the simulation image, change the contrast,
Next, the melanin component internally reflected light image of the plurality of spatial frequency bands is synthesized to form a melanin component modified internally reflected light image,
A composite internal reflection image is formed by combining a melanin component modified internal reflection image and an internal reflection image other than the melanin component,
A skin simulation image forming system for forming a simulation image in which skin color unevenness is changed by combining a composite internally reflected light image and the surface reflected light image.   2. The skin simulation image forming system according to claim 1, wherein the calculation means decomposes the melanin component internally reflected light image into a plurality of spatial frequency bands by wavelet multi-resolution analysis.   Simulation is performed by changing the color unevenness of pores in the simulation image by changing the contrast of the melanin component internally reflected light image in the high frequency band and by changing the contrast of the melanin component internally reflected light image in the low frequency band. The skin simulation image forming system according to claim 1, wherein the color unevenness of the stain larger than the pores in the image is changed. A skin simulation image forming method by the skin simulation image forming system according to claim 1,
Image forming means to form an internal reflection light image and the surface reflection light image for skin of the subject,
Computing means separates the internally reflected light images other than the melanin component internally reflected light image and the melanin component by ICA internally reflected light image by the image forming means has formed,
The melanin component internally reflected light image is decomposed into multiple spatial frequency bands,
The contrast of the selected image is selected based on the color unevenness to be increased or decreased in the simulation image for the melanin component internally reflected light image decomposed into the plurality of spatial frequency bands. Change
Next, the melanin component internally reflected light image of the plurality of spatial frequency bands is synthesized to form a melanin component modified internally reflected light image,
A composite internal reflection image is formed by combining a melanin component modified internal reflection image and an internal reflection image other than the melanin component,
A skin simulation image forming method of forming a simulation image in which color unevenness of skin is changed by synthesizing a synthesized internal reflected light image and the surface reflected light image.

Images