JPH0755447A - Analysis system for surface state of skin - Google Patents

Abstract

PURPOSE:To make the state of pores of the skin possible to be by automatically evaluated smoothing the luminance level of each pixel constituting the image on the surface of the skin and then binarizing the difference of image before and after smoothing. CONSTITUTION:Image signals picked up from the surface of skin using a trispectral camera 70 is subjected to level conversion for each wavelength by means of a level controller 71 and the output therefrom is subjected to A/D conversion before it is temporarily stored in an image memory 72. A computor 74 reads out the data stored in the memory 72 and smoothes the illuminance level of each pixel constituting the image two-dimensionally. Subsequently, the difference between the illuminance level of the pixel constituting the smoothed image and that of the corresponding pixel prior to smoothing is determined by difference method thus outputting a differential image. Furthermore, the differential image is binarized to obtain the binary image of a pore and the area thereof is calculated as an index representative of the state of pore.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for photographing the surface of a skin and analyzing the surface condition of the skin by an image analysis method.

[0002]

2. Description of the Related Art There are various types of foundations that are applied to make spots and freckles inconspicuous, and it is desirable to make an appropriate selection according to the nature and condition of the skin. One of the important factors in this case is the size of the pores. That is, in the case of skin with large pores, a so-called "pore dimple" is likely to occur, and the pigment is clogged in the pores to deteriorate the finish. Therefore, the liquid type is more suitable than the powdery type as the foundation. Also, the density of stains and freckles is an important factor,
When it is dark, a foundation with strong covering power is suitable. Therefore, if a device capable of objectively evaluating these factors is developed, it is possible to accurately indicate the skin condition of the user of the cosmetic product and the cosmetic product suitable for the skin condition, which leads to promotion of sales.

[0003]

Therefore, the main object of the present invention is to determine the size (area) of pores existing on the surface of the skin.
An object of the present invention is to provide a system for analyzing the surface condition of the skin, which is capable of measuring and evaluating. Another object of the present invention is to provide an analysis system for the surface condition of the skin, which can obtain an objective index of the pigment density as well as the size of the pores and can output them in a comprehensive form. .

[0004]

According to the present invention, there is provided a skin surface condition analyzing system, comprising: an image pickup means for outputting an image of the surface of the skin; and a luminance level of each pixel constituting the image output by the image pickup means. Smoothing means for two-dimensionally smoothing, and calculating the difference between the brightness level of each pixel forming the smoothed image and the brightness level of the corresponding pixel of the image before smoothing to output a difference image. And a first binarizing unit that obtains an image of pores by binarizing the difference image.

The above-mentioned system further includes a pore area calculating means for calculating the area of the pore from the image of the pore output by the first binarizing means and using it as a first index, and an image output by the imaging means. Second binarizing means for dividing each pixel constituting the first pixel into a first set of pixels having a high brightness level and a second set of pixels having a low brightness level, and the first and second binarization means. An average value of the brightness levels of the pixels belonging to each of the sets is calculated, and the difference between the two is calculated as a second index, and the first index calculated by the pore area calculation unit is calculated as one of the two. It is preferable to further include an output unit that outputs a plot on a two-dimensional plane with the axis as the axis and the second index calculated by the density calculating unit as the other axis.

[0006]

[Function] Generally, the image of pores is smaller in size than the density due to stains, freckles, etc., and can be removed by smoothing. Therefore, a binary image of pores is obtained by binarizing the difference between the images before and after the smoothing. Further, the area of the pores is calculated from this binary image and used as an index representing the state of the pores, and the difference between the average value of the brightness levels of the pixels in the area of the spots / freckles and the average value of the other areas is calculated. By plotting the surface condition of the skin in a two-dimensional plane from both, as an indicator of the density of freckles,
It facilitates the optimal selection of foundation.

[0007]

[Example] Structure of the apparatus In order to extract spots / freckles and pores from a captured image, binarization of the image is required. In this case, if the shading that occurs at the time of imaging is not prevented as much as possible, it will be difficult for the software processing. Therefore, to reduce the occurrence of shading, we decided the size and shape of the flash discharge tube, mirrors, and their spatial arrangement by repeating prototypes and tests. As a result, almost no shading occurs, and a ring-shaped flash discharge tube made of explosion-proof quartz glass that secures the illumination light in the ultraviolet region (rough adjustment of the light emission is performed by changing the capacitor capacity, which does not change the color temperature, The adjustment is a voltage change and is designed so that it can be arbitrarily adjusted). A new electronic flash with a built-in electronic flash and a three-plate CCD camera (Toshiba-I
We have newly developed an imaging trispectral camera integrated with the K-T30C whose specifications have been changed to the mother body). The optical system of the camera body is shown in FIG.

The flash discharge tube 10 as a light source has a ring shape. The reflector 12 is arranged behind it. Similarly, the reflector 12 also has a ring shape as a whole, and its mirror surface is designed to uniformly project the light from the flash discharge tube 10 onto the surface 14. Flash discharge tube 1
A ring-shaped diffuser 16 is also arranged in front of 0. The diffuser 16 diffuses and transmits light having a wavelength of 350 to 750 nm. The diffuser 16 has a wavelength of 350 to 750 nm
Is fixed by being sandwiched between two ring-shaped clear filters 18 that transmit the light. The clear filter 18 is the discharge tube 10.
It also acts as a protector to prevent glass fragments from scattering when the glass is broken. The hood 20 is provided and the reflector 1
In addition to 2, the light from the light source 10 is prevented from directly entering the rear. The diffuser 16 and the clear filter 18 are fixed to the hood 20. The light guide space 22 in front of the diffuser 16 has a gradually narrower cross section, and the inner wall 24 defining the light guide space 22 is coated with barium sulfate so that the surface 14 is uniformly illuminated with flash light. Inner wall 2
4 is concave. A fisheye lens 26 is arranged on the center axis of the ring behind the ring-shaped discharge tube 10, and a CCD camera 28 is arranged behind it.

When this device is pressed against the skin to discharge the discharge tube 10, the light and the reflected light from the reflector 12 are diffused by the diffuser 16 and illuminate the skin surface evenly. The image of the surface of the illuminated skin is C by the fisheye lens 26.
By forming an image on the CCD element in the CD camera 28, a signal of an image of the skin surface without shading can be obtained.

During calibration, the calibration cap 30 is fixed to the device by fitting a pin 32 provided on the calibration cap 30 and a hole 34 provided in the housing of the device.
A tile 36 as a standard gray sample is fitted on the inner wall of the calibration cap 30, and when the cap 30 is attached to the apparatus, the surface of the tile 36 is a surface 14.
Matches

As will be described later in detail, the tile 3 obtained with the calibration cap 30 attached
By calibrating using the luminance value of the image of No. 6, it becomes possible to evaluate numerically the color intensity of stains, freckles and the like. FIG. 2 is a sectional view showing a detailed configuration of the CCD camera 28 of FIG. The light from the fisheye lens 26 enters the prism 44 through the IR cut filter 40 and the low pass filter 42. A dichroic mirror 46 is provided on one surface of the prism 44, and an angle is set so that a light component near a wavelength of 400 nm is reflected and separated on this surface. The light component not reflected by the dichroic mirror 46 further enters the prism 48. A dichroic mirror 50 is also provided on one surface of the prism 48, and an angle is set so that a light component near a wavelength of 700 nm is reflected and separated on this surface. The light component that is not reflected by the dichroic mirror 50 passes through the prism 52 and the bandpass filter 5 having a main wavelength of 550 nm and a half value width of ± 10 nm.
An image is formed on the CCD element 56 after passing through 4. The light component reflected by the dichroic mirror 46 is further reflected by the other surface of the prism 44 and has a main wavelength of 400 nm and a half value width of ± 10.
Pass through the nm bandpass filter 58 and pass through the CCD element 60
Image on top. The light reflected by the dichroic mirror 50 is further reflected by the other surface of the prism 48, and the main wavelength 7
An image is formed on the CCD element 64 after passing through the band pass filter 62 of 00 nm and half width ± 10 nm.

In the above structure, the discharge tube 10 (FIG. 1)
When is discharged, electric signals corresponding to images at 550 nm (center of visible region), 400 nm (near ultraviolet region), and 700 nm (near infrared region) are shaded on the CCD elements 56, 60, and 64, respectively. Obtained without causing. FIG. 3 shows the bandpass filters 54, 58,
The pass characteristic of 62 is shown. As is apparent from FIG. 3, the trispectral camera used in the present invention includes:
Unlike ordinary 3-plate cameras, 400nm, 550nm, 7
An image in three wavelength bands with narrow bands (hereinafter referred to as near-ultraviolet image, visible image, and near-infrared image, respectively) using a bandpass filter with a main wavelength of 00 nm and a half width of ± 10 nm. ) Is obtained at the same time. Note that, as far as the present invention is concerned, the analysis is performed only on the visible image.

The structure of the entire system is shown in FIG. The present system includes a wavelength-specific level controller 71 (described in detail later) that converts the level of the image signal from the above-described trispectral camera 70 by wavelength, and an image memory that outputs the output from the analog / digital converter and temporarily stores it in a memory. 72,
A computer 74 is provided that appropriately reads out data stored in the image memory 72 and performs various analysis processes. A keyboard 76 for inputting instructions from an operator, a monitor 78 for outputting analysis results, a video printer 80 for outputting a hard copy thereof, and image data and analysis results are stored in the computer 74. Is connected to the magneto-optical disk 82.

The calibration stain / freckle image of the image data for each wavelength is digitized into 8 bits for each of the near-ultraviolet image, the visible image and the near-infrared image and recorded in the memory. This system corrects variations in image input data due to uncontrollable fluctuations that occur when images of the same target are continuously captured over a long period of time, and quantitative image data that can be used for continuous comparison is always available. The following operations are performed for the purpose of obtaining the above and for the purpose of increasing the detection power by allocating 8 bits within the existence range of the spectral reflectance of the skin color which depends on the wavelength.

400 nm, 550 nm, 70 obtained from the skin color (cheek) measurement results of 826 Japanese women (16 to 59 years old) using Minolta CM-1000HR.
The average value and standard deviation of the spectral reflectance at 0 nm are as shown below. Wavelength Reflectance average value Standard deviation (σ) 400 nm 16.0% 2.9 550 nm 29.9% 3.5 700 nm 59.4% 2.4 Therefore, at 400 nm, reflection of 25% (average value + 3σ) or more There is almost no measurement target with a rate, so 0% to 25
It was decided to allocate 8 bits to%. In this case, assuming that there is a linear relationship between the reflectance and the output luminance, one luminance corresponds to 0.10% when converted to the reflectance, and an extremely high resolution can be obtained.

Even at 550 nm and 700 nm, bits are assigned in the same way. As a device for this allocation, a wavelength-specific level controller that converts an input voltage corresponding to the reflectance of an imaging target at 400 nm, 550 nm, and 700 nm into an output voltage corresponding to the above-described bit division was newly designed and manufactured. Even though it is a dark stain or freckle, the reflectance is not 0%, so it is possible to round up the lower limit, but we do not pay particular attention here. The relationship between the reflectance of the imaging target and the output brightness in this system under the input / output conditions obtained by interposing the above-mentioned wavelength level controller 71 (FIG. 4) is shown in matte gray coated paper (Murakami Color Technology Research (I asked the location to make it). The reflectance of matte gray coated paper was measured by Hitachi Color Analyzer 607. The results obtained are shown in FIGS. 5, 6 and 7.

The relationship between the reflectance and the luminance at each wavelength shows a relationship in the upper right, but it is not a perfect linear relationship. In addition, the linearity becomes worse when the reflectance is small. As mentioned above, the reflectance is not 0% even if it is a dark stain or freckle, the sensitivity of the CCD camera in the low reflectance region is poor, and further,
Since it is not practical to prepare a large number of calibration gray samples and perform non-linear luminance calibration, it was decided to use a small number of calibration gray samples as much as possible and perform the calibration assuming a linear relationship. This calibration method is considered necessary and sufficient for the purpose of calibration for quantitative successive comparison of image input data.

As a result, the standard gray sample for brightness calibration has no fear of deterioration and can be wiped off even if it becomes dirty. Therefore, "EVER-COLORS" (manufactured by Yoneda Glass Industrial Co., Ltd. It was decided to apply a # 3000 gold sand to a standard plate) to make it a matte surface and to use it. The finally selected “EVER-COLORS” is GRAY No. 1
000, GRAY No. 3000, and GRAY No. 6000.
To obtain the calibration values for near-ultraviolet image and visible image, GRAY N
GRAY No. 1000 and GRAY No. 6 can be used to obtain the calibration value of the near infrared image using o.1000 and GRAY No. 3000.
000 was used.

A calibration standard plate set as shown in FIG. 8 (for the calibration "EVER-COLORS" described above) is provided for the purpose of ensuring ease of calibration work and reproduction of geometric conditions of illumination and light reception.
3 sets, 2 pieces each, 6 pieces in total) were manufactured. The set of two sheets, arranged as shown in Fig. 8, is "EVER
-Because of the limitation of "COLORS" in machining and processing, it is impossible to cut out a large area. Therefore, if you try to deal with uneven illuminance that may exist in the imaging area by targeting a large area as much as possible, 2 Because it is unavoidable that it is a sheet.

As described with reference to FIG. 1, the pin 32 is attached to the opening of the camera as a guide so that the calibration can be automatically performed by pressing the shutter once. For the calibration, the value obtained by calculating the average luminance in each window (50 × 100 pixels or 100 × 50 pixels) shown by the solid line in FIG. 8 is regarded as the value indicating the system state at that time. It is executed by doing.

Table 1 shows the spectral reflectance (Hitachi Color Analyzer 607) of the calibration "EVER-COLORS" and the brightness values set by the above wavelength level controller.

[0022]

[Table 1]

The method of obtaining the calibration value is shown below by taking the case of a near-ultraviolet image as an example. As shown in Table 1, GRAY No.1000
Has a brightness of 38 in the near-ultraviolet image and a spectral reflectance of 400% at 400 nm obtained by a spectrophotometer, and GRAY No.
Assuming that the reference state is when the luminance of 3000 is 240 and the spectral reflectance is 27.0%, the relationship between the spectral reflectance X and the luminance Y in the reference state is

[0024]

[Equation 1]

[0025] GRAY No. 1 at the time of arbitrary calibration
000 has a brightness of 40, and GRAY No. 3000 has a brightness of 250. In this case, the relationship between the spectral reflectance X and the measured luminance value Y ₁ is

[0026]

[Equation 2]

[0027] Therefore, if X is deleted from the equations (1) and (2), the relationship between Y and Y ₁ is obtained, and thus the calibration value Y of the luminance in the reference state is calculated from the luminance value Y ₁ before calibration. Through the above procedure, the calibration values corresponding to the luminances 0 to 255 of the pixels to be calibrated are obtained, and the calibration values are referred to to calibrate all the pixels. The visible image and the near-infrared image are calibrated by the same method.

Binarization of Image Data by Wavelength First, in order to know whether or not all the collected images can be binarized with the same threshold value, samples were taken from 143 persons in June 1992 (n = 143). ) The frequency distribution of the lowest luminance, the luminance of the automatic binarization and the highest luminance within 192 × 192 pixels cut out from the visible image in the original image was obtained.

As a result, it was found that the three distributions overlap each other and the same threshold value cannot be adopted. Therefore, we decided to binarize each image. There are several proposals for the binarization method, but the method proposed by Otsu, which uses the slice level as the value that bisects the best distribution in the given concentration distribution (Otsu Nobuyuki) ,
The Institute of Electronics and Communication Engineers, 63, 4, 349 (1980)),
As a result of processing, it was found that all collected images were binarized with a slightly higher brightness. In other words, the area of stains / freckles was greatly extracted.

Therefore, for a large number of images, from the threshold value for binarization obtained by the method of Otsu, how much lower the threshold value is to be binarized, it is visually recognizable as stains / freckles. The threshold was set to see if a similar binary image could be obtained. As will be described later, the density of the spots / freckles is digitized using the brightness data of the visible image. The optimum binarization level in the visible image was determined as follows.

First, the collected original image (visible, n = 14)
3) was binarized under the following 8 conditions. Binarization with automatic binarization (Automatic binarization value-3) Binarization with (Automatic binarization value-6) Binarization with (Automatic binarization value-8) (Automatic binarization value- 9) Binarization (automatic binarization value -10) binarization (automatic binarization value -11) binarization (automatic binarization value -12) binarization The hard copy of the value image is output using a video printer, and under what conditions the image is processed to extract the region that is most faithful to pigmentation with the naked eye. Let

As a result, 117 out of 143 cases (82
%), The image processed under the condition of was evaluated to be optimum. In addition, 14 cases were evaluated as optimal as a binary image in which the image processed under the condition that the divisor is smaller than -6 and 12 cases are large. From this result, for the visible image, (threshold value for automatic binarization-6) was set as the slice level for binarization.

Evaluation of Darkness of Pigmented Portion FIG. 9 shows an example of a histogram of the brightness of each pixel forming a visible image. As shown in FIG. 9, when the threshold value of “automatic binarization value−6 by Otsu's method” is determined, a set of pixels having a luminance value higher than the value (corresponding to a background portion) and a set of pixels having a lower luminance value ( (Corresponding to the pigmented portion), the average value of the luminance is calculated, and the difference between them is used as an index of the density of the pigmented portion.

That is, this is indexed based on the idea that when the brightness difference between the background and the pigmented portion is large, the stain is noticeable (that is, it feels dark), and when the difference is small, it is not noticeable (that is, it feels light). It is a thing. Evaluation of Pore Size FIG. 10 is a flowchart showing the procedure of the process of evaluating the state of the pores. Image data is input from the image memory 72 (FIG. 4) (step a), and the 8-neighbor smoothing filter (1
1 × 11) is executed twice for smoothing (step b).
The 8-neighbor smoothing filter is a filter that simply averages the luminance of pixels in a square area centered on the pixel of interest to obtain the luminance of that pixel. In the present embodiment, this is used for 11 × 11 squares. I did. Note that 11 pixels correspond to 1.14 mm in the image pickup target. Next, the brightness of the corresponding pixel before smoothing is subtracted from the brightness of each pixel after the smoothing process to obtain a difference image (step c). In the difference image, the brightness of the pixel corresponding to the pore remains with a small positive value, and the brightness is almost zero in other cases. Since this differential image has a narrow change range, log conversion is performed so that the difference image is widely distributed in the range of luminance 0 to 255 (step d). Since this image is a screen in which the positions of the pores are bright and the background is dark, a process of inverting the brightness is performed for easy viewing (step e), but this process is essentially unnecessary. Next, automatic binarization processing is performed on the inverted screen (step f). Here again, as in the case described above, the binary image obtained by the Otsu's method tended to represent pores a little too large, so binarization is performed by (automatic binarization value-60). Finally, the total number of pixels of the pixels determined to be pores is calculated (step g) and used as an index indicating the state of pores.

Output of Results FIG. 11 shows an example of display output of evaluation results. The horizontal axis shows the total area of pores on a scale of 0 to 4000, and the vertical axis shows the difference between the average value of the background density and the average value of the pigmented area on a scale of 0 to 255. The intersections of the two are marked with a circle. In addition, the above numerical value "400
"0" is the maximum value determined from the index calculated by selecting the one with the most noticeable pores by the naked eye from the data collected about 400.

On this screen, the skin of the subject is positioned on a plane having two axes of "pigmentation state" and "pore state" as axes.
Furthermore, it can be used as a reference to suggest what kind of maintenance directions are possible.

[0037]

As described above, according to the present invention,
A system is provided that can automatically assess the condition of skin pores and the condition of pigmentation.

[Brief description of drawings]

FIG. 1 is a cross-sectional view of a trispectral camera used in the present invention.

FIG. 2 is a CCD camera 28 in a trispectral camera.
3 is a detailed cross-sectional view of FIG.

FIG. 3 is a diagram illustrating a pass characteristic of a bandpass filter.

FIG. 4 is a block diagram showing a configuration of a system according to an exemplary embodiment of the present invention.

FIG. 5 is a diagram showing a relationship between reflectance and brightness at 400 nm of matte gray coated paper.

FIG. 6 is a diagram showing a relationship between reflectance and brightness at 550 nm of matte gray coated paper.

FIG. 7 is a diagram showing the relationship between reflectance and luminance at 700 nm of matte gray coated paper.

FIG. 8 is a diagram showing a calibration standard plate.

FIG. 9 is a diagram showing an example of a luminance histogram of each pixel forming a visible image.

FIG. 10 is a flowchart showing a process of evaluating the state of pores.

FIG. 11 is a diagram illustrating an example of a display output of evaluation of the surface state of skin.

[Explanation of symbols]

 10 ... Flash discharge tube 12 ... Reflector 16 ... Diffuser 26 ... Fisheye lens 28 ... CCD camera 44, 48, 52 ... Prism 46, 50 ... Dichroic mirror 54, 58, 62 ... Bandpass filter 56, 60, 64 ... CCD element 70 ... Trispectral camera 76 ... Keyboard 78 ... Monitor 80 ... Video printer

Claims (3)

[Claims] 1. An image pickup means for outputting an image of the surface of the skin, a smoothing means for two-dimensionally smoothing the brightness level of each pixel forming the image output by the image pickup means, and the smoothing. Difference calculation means for calculating the difference between the brightness level of each pixel forming the image and the brightness level of the corresponding pixel of the image before smoothing, and outputting a difference image; and binarizing the difference image. A first binarizing means for obtaining an image of pores, and a system for analyzing the surface condition of the skin. 2. A logarithmic conversion means for performing a logarithmic conversion on the luminance level of each pixel forming the difference image output by the subtraction means and supplying it to the first binarization means as a difference image. The system according to 1. 3. A pore area calculation means for calculating a pore area from a pore image output by the first binarization means and using it as a first index, and an image output by the imaging means. Second binarizing means for dividing the pixel into a first set of pixels having a high brightness level and a second set of pixels having a low brightness level, and for each of the first and second sets The average value of the brightness levels of the pixels to which the pixels belong is calculated, the difference between the two is calculated as a second index, and the first index calculated by the pore area calculation unit is used as one axis, 3. The system according to claim 1, further comprising an output unit that outputs a plot on a two-dimensional plane with the second index calculated by the density calculation unit as the other axis.