JP4385925B2 - Image forming method - Google Patents

Description

  The present invention relates to an image forming method for controlling and displaying a fine surface irregularity state of a translucent object such as human skin, teeth, marble, and jewelry.

  A skin simulation image is formed in order to make it possible to grasp, as an image, the skin in which pores, wrinkles, spots, etc. have become inconspicuous by basic makeup or makeup, and the skin that has been stained by sunburn or the like.

  As a method for forming a skin simulation image, a method for creating a database of changes in colorimetric values that can be changed by makeup for various skins, and changing the colorimetric values of facial images of the customer's bare skin by image processing based on the database Has been proposed (Patent Document 1).

  Moreover, in the construction of a database, a method of using the amount of melanin component and the amount of hemoglobin component of the internally reflected light of the skin has been proposed (Patent Document 2).

  On the other hand, with the progress of computer graphics technology, it is possible to intentionally blur or refine an image. For example, as one of the refinement methods, a method for restoring a degraded image by performing a deconvolution operation using a point spread function (hereinafter referred to as PSF) has been proposed (Patent Document 3). .

JP 2002-203238 A JP 2002-200050 A JP 2001-43361 A

  The texture of the skin image is affected by shading caused by surface irregularities such as wrinkles and pores, and the color of pigment components such as hemoglobin and melanin. Further, since the skin is a translucent object, the skin image is likely to be blurred due to surface irregularities. For this reason, in conventional skin images, color unevenness due to melanin or hemoglobin and shadows due to surface irregularities cannot be distinguished. Therefore, the skin image can be made high-definition so that even small wrinkles and pores can be distinguished, or color It was difficult to form a simulation image in which skin properties such as wrinkles and pores were changed without changing the taste.

  On the other hand, the present invention makes it possible to control and display a fine surface irregularity state for an image of a translucent object. In particular, when the skin is a measurement object, the color by melanin or hemoglobin. Make it possible to discriminate unevenness and shading caused by surface irregularities, thereby obtaining, for example, a high-definition skin image that enables measurement of the diameter and depth of pores, and the actual skin condition It is an object of the present invention to obtain a simulation image in which the surface unevenness state of skin such as fine lines and pores is changed.

  In order to be able to discriminate between color unevenness due to melanin and hemoglobin and shadows due to surface irregularities in the skin image, the present inventors can eliminate blurring of the image due to the skin being a translucent object. For this purpose, it is effective to perform deconvolution using PSF obtained from the measurement of optical properties of the skin, and skin such as fine lines and pores without changing the skin color information. It has been found that in order to form a simulation image in which the surface irregularity state of the skin is selectively changed, a convolution calculation or a deconvolution calculation may be performed on the skin image using the skin PSF.

  That is, the present invention acquires an image of a translucent object, acquires a PSF by measuring the optical properties of the object, performs a deconvolution operation or a convolution operation on the image based on the acquired PSF, An image forming method for forming a state-controlled image is provided.

Further, the present invention acquires an internal reflection light image and a surface reflection light image of the skin,
Internal reflection light image is separated into pigment component image and shadow component image by independent component analysis,
Obtain the skin point spread function (PSF) by measuring the optical properties of the skin,
Based on the acquired PSF, a deconvolution operation or a convolution operation is performed on the separated image.
Next, a skin image forming method for recombining each image is provided.

  In the image forming method of the present invention, an image of a translucent object is acquired, a PSF is acquired by measuring optical properties of the object, and a deconvolution operation or a convolution operation is performed on the image based on the acquired PSF. Compared to the case where deconvolution calculation or convolution calculation is performed using the theoretical value or experimental value of PSF calculated based on the optical characteristics (camera numerical aperture, refractive index), etc. of the camera used for photographing. There is an advantage that it is possible to analyze the blur of light of the target object itself that has not been considered (light emission from other than the incident point).

  In particular, in the image forming method of the present invention, skin is targeted as a translucent object, an internal reflection light image and a surface reflection light image of the skin are obtained, and a dye component image and a shadow component image are obtained by independent component analysis. On the other hand, the skin PSF is acquired by measuring the optical properties of the skin, and the deconvolution operation or the convolution operation is performed on the shadow component image based on the acquired PSF. When the skin image is formed by performing deconvolution operation or convolution operation on the image based on PSF and recombining each image, color unevenness such as a stain caused by melanin, acne rash caused by hemoglobin, Without sacrificing skin texture due to color irregularities such as congestion, skin surface irregularities such as fine lines and pores, from actual skin condition Simulation image is varied and it is possible to obtain.

  Hereinafter, the present invention will be described in detail with reference to the drawings, focusing on the case where human skin is a measurement object as a translucent object. In each figure, the same numerals indicate the same or equivalent components.

  The image forming method of the present invention is a model skin, teeth, or semitransparent objects such as marble or jewelry formed with bare skin, makeup skin, silicone resin, etc. This is a method of forming a texture having various surface irregularities by controlling blurring by deconvolution calculation or convolution calculation.

  Here, convolution is also referred to as convolution integration, and is an operation of obtaining an output function by weighting an input function. On the other hand, deconvolution is an inverse operation of convolution, that is, an operation for obtaining an input function from an output function and a weight function, and is applied to restoration of a degraded image as described above.

In the present invention, it is assumed that the PSF used in the deconvolution calculation or the convolution calculation is acquired by measuring and analyzing the optical properties of the measurement target. More specifically, for example, the PSF is acquired by the following method.
(1) Irradiate the collected light to the measurement object, and obtain the PSF from the spread of the light.
(2) Obtain the equivalent scattering coefficient μs' (or scattering coefficient μs and anisotropic scattering coefficient g) and absorption coefficient μa of the measurement object, and Monte Carlo simulation (Lihong Wang, Steven L. Jacques; http: // omlc. ogi.edu/pubs/pdf/man_mcml.pdf) to obtain the PSF.
(3) Obtain the scattering particle size, the refractive index of the particles and the medium that make up the measurement object, and obtain the equivalent by Mie scattering calculation (Scott Prahl; http://omlc.bme.ogi.edu/software/mie/) The scattering coefficient μs ′ (or the scattering coefficient μs and the anisotropic scattering coefficient g) and the absorption coefficient μa are obtained, and the PSF is obtained in the same manner as (2).
(4) Obtain the equivalent scattering coefficient μs' (or scattering coefficient μs and anisotropic scattering coefficient g) and absorption coefficient μa of the measurement object, and analyze the solution (SL Jacques, A. Gutsche, JA Schwartz, LH Wang and FK Tittel: “Video-reflectometry to extract optical properties of tissue in vivo,” Medical Optical Tomography: Functional Imaging and Monitoring, IS11 of SPIE Institute Series (SPIE, Bellingham, 1993) pp. 211-226).

In the method of performing deconvolution calculation or convolution calculation on an image based on the PSF acquired in this way, when a measurement object is skin and an image in which the surface unevenness is refined is formed, a photographed image of the skin and the PSF of the skin By performing the deconvolution calculation, the influence of PSF is removed from the photographed image. This deconvolution operation is expressed by the following equation, where I is a captured image, PSF of the captured image is PSF ₀ , O is a combined image, and F is Fourier transform.

  On the other hand, in the case of forming a simulation image in which the skin surface unevenness is blurred in consideration of the transparency of the skin from the photographed image, the above-mentioned image from which the influence of the PSF is removed, and the above-mentioned PSF and the PSF having different internal scattering characteristics The convolution operation is performed with. As PSFs having different internal scattering characteristics, those obtained by measurement and analysis of the optical properties of the target object are used in the same manner as the PSF acquisition described above.

This convolution operation is expressed as follows when PSF ₁ to be newly applied is PSF ₁ , the synthesized image is O, and the Fourier transform is represented by F.

  In the present invention, as a skin image to be subjected to deconvolution calculation or convolution calculation using PSF, a photographed image of normal skin may be used as it is, but a skin internal reflection image and a surface reflection image are separated. It is preferable that the internal reflection light image is separated into a pigment component image and a shadow component image by independent component analysis, and the shadow component image thus obtained is used as a skin image for performing a deconvolution operation or a convolution operation. After deconvolution calculation or convolution calculation on the shadow component image, the simulation image of the skin having the desired surface irregularity state is synthesized by synthesizing the shadow component image and the other component images previously separated. Can be formed.

FIG. 1 is a process diagram of such image processing.
The system for performing image processing in FIG. 1 includes a PSF measuring unit 1, an image forming unit 10 capable of forming an internal reflection light image and a surface reflection light image using light, and an image acquired by the image forming unit 10. It can be comprised from the calculating means 20 which performs a calculation process, and the display 30 which displays a simulation image.

  The PSF measuring unit 1 collects and irradiates a subject with light by an optical system according to an embodiment described later, and obtains the PSF from the spread of the light.

  The image forming means 10 can be composed of, for example, an illumination light source 11, a digital camera 12, and polarizing plates 13 and 14 provided detachably on the light source 11 and the front surface of the digital camera 12, respectively.

  As the calculation means 20, a personal computer having an image processing function can be used. The personal computer may be connected to a display 30 capable of appropriately switching or simultaneously displaying a normal image of skin, an internal reflection light image, a surface reflection light image, a composite simulation image, and the like, and may further be connected to a printer. .

  In this system, in order to form a skin simulation image, first, an internal reflection light image and a surface reflection light image are acquired for the skin as a subject.

  The internally reflected light image is formed by mounting the polarizing plate 14 on the front surface of the digital camera 12 so that the polarization direction is orthogonal to the polarizing plate 13 on the front surface of the light source 11 and removing the surface reflected light component. Can do.

  The surface reflected light image is obtained by attaching the polarizing plate 14 on the front surface of the digital camera 12 so that the polarizing direction is the same as that of the polarizing plate 13 on the front surface of the light source 11, and the polarizing plate 14 with the polarizing plate 13. It can be formed from the difference from the image obtained by mounting so that the polarization directions are orthogonal.

  Note that the image forming unit 10 preferably obtains an image as a reference original image by performing normal photographing without using the polarizing plates 13 and 14.

  After obtaining the internally reflected light image by the image forming means 10, an independent component analysis is performed on the internally reflected light image by the computing means 20, and from the internally reflected light image, a dye component image such as a melanin component image and a hemoglobin component image, The shadow component image is separated. Then, a deconvolution calculation or a convolution calculation is performed on the shadow component image using the PSF obtained by the PSF measuring unit 1 to control the appearance of the uneven surface state of the skin in the simulation image.

  In addition, when evaluating skin color unevenness in consideration of internal skin scattering characteristics, deconvolution calculation or convolution calculation may be performed on a melanin component image or a hemoglobin component image as necessary. As the PSF used at this time, those obtained as PSF corresponding to the melanin component or the hemoglobin component from the skin in which the melanin component and the hemoglobin component of the skin are changed by a sunburn test or the like can be used.

  Here, independent component analysis refers to the skin layer structure, the epidermal layer containing melanin as the main pigment component, the dermis layer containing hemoglobin as the main pigment component, and the subcutaneous tissue containing other pigment components. This is an analysis method that separates and extracts the signals of each layer from the image signal, considering that the signal is emitted independently from each layer and the mixture of them is an image signal.

  More specifically, for RGB of the image signal, -log (R), -log (G), and -log (B) are assigned to the x-axis, y-axis, and z-axis, respectively, and the skin color of the flat portion of the skin is there. When the color space mapping is performed, the distribution is almost flat as shown in FIG. 2A, so that it is understood that two components contribute to the skin color. This independent two-component signal intensity is considered to correspond to the amount of melanin or hemoglobin, respectively, and as shown in FIG. 2B, the skin color is closer to the component vector of melanin (-log (B)) ) And the hemoglobin component vector (the one closer to -log (G)). Therefore, a signal representing the amount of melanin or a signal representing the amount of hemoglobin is extracted from the signal of the internally reflected light image of the skin as the subject, and a distribution image of the amount of melanin component and a distribution image of the amount of hemoglobin component are output. However, if there is unevenness or uneven illumination, the amount of both melanin and hemoglobin components obtained by projecting onto the plane containing both the melanin and hemoglobin vectors and removing the shadow component along the direction of the vector of the light source component acquired in advance is used. obtain. Further, a shadow component image of the internally reflected light image is obtained by subtracting the distribution image of the melanin component amount and the distribution image of the hemoglobin component amount from the internally reflected light image.

  Details of such analysis processing and image processing are described in Vol. 16, No. 9 / September 1999 / J. Opt. Soc. Am. A 2169, and commercially available image analysis software (for example, This can be done by installing Adobe Photoshop).

  After performing the deconvolution operation or the convolution operation on the shadow component image, and further correcting the melanin component image and the hemoglobin component image as necessary, these component images are combined, and the combined internal reflected light image and By combining the surface reflected light image, a simulation image in which the surface unevenness state is controlled is formed. The skin simulation image obtained in this way maintains the skin color and color unevenness of the original image, and the surface unevenness increases to become a skin image in which wrinkles and pores are emphasized, or the surface unevenness decreases. A smooth skin image is obtained.

  In the skin image forming method of the present invention, the skin surface unevenness state may be three-dimensionally represented by photographing the skin a plurality of times from different angles and performing three-dimensional processing. Specifically, first, the parallax with respect to corresponding points of the skin as a subject is obtained with a plurality of cameras by the stereo method, and the shape is calculated. At this time, the image acquired with each camera is deconvolved using PSF, and the corresponding point is found by clarifying the boundary of the shadow. As a result, the corresponding points are conventionally blurred on the image in a semi-transparent object, and it is difficult to accurately specify the corresponding points. Can be found accurately and easily, and the formation accuracy of the three-dimensional image can be improved. The three-dimensional processing itself based on a plurality of shootings from different angles can be performed by a known method, for example, Pixel-based matching, Area-based matching, Feature- based matching and the like.

Example 1 (Image refinement by deconvolution calculation)
First, a model skin in which 0.5 wt% of skin color pigment was contained in a silicone resin was prepared. A groove having a depth of 0.5 mm and a width of 0.5 mm was formed in the model skin.

  Next, the PSF of the model skin was determined using the optical system shown in FIG. That is, the light emitted from the white light source 2 is guided by the optical fiber 3, and the shutter speed is changed to each model skin S through the achromat lens 4, the diaphragm 5, and the pellicle half mirror 6 within a range of 1/50 seconds to 5 seconds. The condensed light was irradiated and the light spread was photographed with the digital camera 7, and the PSF was obtained from the two-dimensional coordinates and the luminance value. However, it is desirable to use multiple images with different shutter speeds due to the dynamic range of the digital camera.

  Further, as shown in FIG. 4, the internally reflected light image of the model skin S was illuminated from the light source 11 through the polarizing plate 13 at an incident angle of 45 °, and taken by the digital camera 12 through the polarizing plate 14 at 0 °. Then, an image obtained by deconvolution calculation using the internally reflected light image and the PSF of the model skin described above was formed. FIG. 5A shows this internally reflected light image and a graph in which the luminance of this image is plotted in the horizontal axis direction from the center of the image, and FIG. 5B shows an image obtained by deconvolution calculation and a graph in which the luminance of this image is plotted. . The image obtained by the deconvolution calculation in FIG. 5B has no blur and the grooves can be clearly seen, and is observed with an opaque texture as compared with the internally reflected light image in FIG. 5A.

Example 2 (Contrast between simulation image and original drawing by convolution calculation and deconvolution calculation)
Model skins with skin color pigment concentrations of 0.05 wt%, 0.1 wt%, 0.15 wt%, and 0.2 wt% were prepared in the same manner as in Example 1, and grooves similar to those in Example 1 were formed in each model skin. The PSF was determined for each model skin formed.

  A simulation image of each skin color pigment concentration is formed by using the PSF obtained for the model skin of each skin color pigment concentration and performing the convolution on the skin color pigment 0.5 wt% deconvolution image of FIG. 5B. The contrast was compared with the internally reflected light image of the model skin of each skin color pigment concentration.

  As a result, for the model skin of each skin color pigment concentration, the internally reflected light image and the image subjected to the convolution calculation were observed in the same groove depth and texture, and the groove became less noticeable as the skin color pigment concentration was lower. Thus, it was confirmed that by controlling the PSF and performing the convolution calculation or the deconvolution calculation, it is possible to form an image in which the surface unevenness state is controlled in the skin image.

  The image forming method of the present invention can be used when displaying an image by controlling the fine surface irregularity state of a semi-transparent object. For example, the improvement can be achieved by using skin as a measurement object and using a specific cosmetic. It can be used for recommending and selling cosmetics, such as showing the conspicuousness of small wrinkles and pores as simulation images and recommending the cosmetics.

It is process drawing of the skin image formation method. It is explanatory drawing of independent component analysis. It is explanatory drawing of the optical system which calculates | requires PSF. It is explanatory drawing of the optical system which takes the image of model skin. It is an internal reflection light image of model skin, and the graph of the brightness | luminance. It is the graph of the brightness | luminance and the image which carried out the deconvolution calculation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 PSF measuring means 2 White light source 3 Optical fiber 4 Achromatic lens 5 Aperture 6 Pellicle half mirror 7 Digital camera 10 Image forming means 11 Light source 12 Digital cameras 13 and 14 Polarizing plate 20 Calculation means 30 Display

Claims (2)

Acquire internal reflection image and surface reflection image of skin,
Internal reflection light image is separated into pigment component image and shadow component image by independent component analysis,
Obtain the skin point spread function (PSF) by measuring the optical properties of the skin,
Based on the acquired PSF, a deconvolution operation or a convolution operation is performed on the shadow component image,
Next, a skin image forming method for recombining each image. A melanin component image and a hemoglobin component image are separated as the pigment component image. On the other hand, a melanin component and a PSF corresponding to the hemoglobin component are obtained from the skin where the melanin component and the hemoglobin component are changed, and the melanin component is obtained using the PSF. The skin image forming method according to claim 1, wherein a deconvolution operation or a convolution operation is performed on the image and the hemoglobin component image to re-synthesize each image.

Images