JP3219550B2 - Analysis system for skin surface condition - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for photographing the surface of skin and analyzing the surface condition of the skin by an image analysis technique.

[0002]

2. Description of the Related Art There are various types of foundations to be applied to make spots and freckles less noticeable, and it is desirable to make an appropriate selection according to the nature and condition of the skin. One of the important factors in this case is the pore size. That is, in the case of skin having a large pore size, a so-called "pore fall" is likely to occur. In such a case, the pores are clogged with pigment and the finish is deteriorated. Therefore, a liquid type is more suitable as a foundation than a powdery type. In addition, the depth of spots and freckles is also an important factor,
When it is dark, a foundation with strong covering power is suitable. Therefore, if a device capable of objectively evaluating these factors is developed, the skin condition of the cosmetics user and the cosmetics suitable for the cosmetics can be accurately shown, which leads to sales promotion.

[0003]

SUMMARY OF THE INVENTION Accordingly, a main object of the present invention is to determine the size (area) of pores existing on the surface of the skin.
It is an object of the present invention to provide a system for analyzing the surface condition of the skin, which can measure and evaluate the skin surface condition. Another object of the present invention is to provide a skin surface state analysis system capable of obtaining an objective index of the pigment density together with the pore size and outputting them in an integrated form. .

[0004]

According to the present invention, there is provided a system for analyzing the condition of the surface of a skin, comprising: an imaging unit for outputting an image of the surface of the skin; and a luminance level of each pixel constituting the image output by the imaging unit. Means for two-dimensionally smoothing the image, calculating a difference between a luminance level of each pixel constituting the smoothed image and a luminance level of a corresponding pixel of the image before smoothing, and outputting a difference image And a first binarization unit for binarizing the difference image to obtain an image of the pores.

The above-mentioned system further comprises a pore area calculating means for calculating a pore area from a pore image outputted by the first binarizing means and using the pore area as a first index, and an image outputted by the imaging means. A second set of binarizing means for dividing each pixel constituting the first and second pixels into a first set of pixels having a high brightness level and a second set of pixels having a low brightness level; The average value of the luminance levels of the pixels belonging to each of the sets is calculated, and the difference between the two is calculated as the second index, and the first index calculated by the pore area calculating means is used as one of the two indexes. It is preferable that the apparatus further comprises output means for outputting a value obtained by plotting the second index calculated by the density calculation means on the two-dimensional plane as the other axis as an axis.

[0006]

In general, an image of pores is smaller in size than light and shade due to spots, freckles, etc., and can be removed by smoothing. Therefore, a binary image of the pores is obtained by binarizing the difference between the images before and after the smoothing. Further, the area of the pore is calculated from the binary image and used as an index indicating the state of the pore, and the difference between the average value of the luminance level of the pixel in the spot / freckle area and the average value of the other areas is calculated. By plotting the surface condition of the skin in a two-dimensional plane from both, as an index representing the depth of freckles,
Optimal selection of the foundation is facilitated.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In order to extract spots, freckles and pores from a picked- up image of an apparatus, binarization of the image is required. In this case, it is difficult to perform soft processing unless shading that occurs during imaging is prevented as much as possible. Therefore, in order to reduce the occurrence of shading, the size and shape of the flash discharge tube and mirror, and their spatial arrangement were determined by repeating trial production and testing. As a result, there is almost no shading and an explosion-proof quartz glass ring-shaped flash discharge tube that secures illumination light in the ultraviolet region. An electronic flash with a built-in adjustment is designed so that it can be arbitrarily adjusted by changing the voltage. A three-chip CCD camera (Toshiba I.
KT30C was changed to the mother body) and an integrated imaging trispectral camera was newly developed. FIG. 1 shows the optical system of the camera body.

The flash discharge tube 10 as a light source has a ring shape. Behind the reflector 12 is disposed. The reflector 12 also has a ring shape as a whole, and its mirror surface is designed so as to uniformly project the light from the flash discharge tube 10 onto the surface 14. Flash discharge tube 1
A ring-shaped diffuser 16 is also arranged in front of 0. The diffuser 16 diffuses and transmits light having a wavelength of 350 to 750 nm. The diffuser 16 has a wavelength of 350 to 750 nm.
Is fixed by being sandwiched between two ring-shaped clear filters 18 that transmit the light. The clear filter 18 is connected to the discharge tube 10
It also acts as a protector to keep glass fragments from scattering when the glass breaks. A hood 20 is provided, and the reflector 1
Together with 2, light from the light source 10 is prevented from directly entering rearward. The diffuser 16 and the clear filter 18 are fixed to the hood 20. The light guide space 22 in front of the diffuser 16 has a gradually narrowing in cross section, and an inner wall 24 that defines the light guide space 22 is coated with barium sulfate to uniformly apply flash light to the surface 14. Inner wall 2
4 has a concave surface. A fish-eye lens 26 is arranged on the center axis of the ring behind the ring-shaped discharge tube 10, and a CCD camera 28 is arranged behind the fish-eye lens 26.

When this device is pressed against the skin to discharge the discharge tube 10, the light and the reflected light from the reflector 12 are diffused by the diffuser 16 to illuminate the skin surface evenly. The image of the illuminated skin surface is converted to a fisheye lens C
By forming an image on the CCD element in the CD camera 28, a signal of an image of the skin surface without shading is obtained.

At the time of calibration, the calibration cap 30 is fixed to the device by fitting a pin 32 provided on the calibration cap 30 with a hole 34 provided in the housing of the device.
A tile 36 as a standard gray sample is fitted on the inner wall of the calibration cap 30. When the cap 30 is attached to the apparatus, the surface of the tile 36 is
Matches.

As will be described later in detail, the tile 3 obtained with the calibration cap 30 attached thereto is obtained.
By calibrating using the luminance value of the image of No. 6, it is possible to evaluate numerical values of the color density of spots, freckles and the like. FIG. 2 is a sectional view showing a detailed configuration of the CCD camera 28 of FIG. Light from the fisheye lens 26 is incident on the prism 44 via the IR cut filter 40 and the low-pass filter 42. A dichroic mirror 46 is provided on one surface of the prism 44, and an angle is set so that a light component having a wavelength of about 400 nm is reflected and separated on this surface. Light components not reflected by the dichroic mirror 46 further enter the prism 48. A dichroic mirror 50 is also provided on one surface of the prism 48, and the angle is set so that light components having a wavelength of around 700 nm are reflected and separated on this surface. The light component not reflected by the dichroic mirror 50 passes through the prism 52 and has a main wavelength of 550 nm and a half-width ± 10 nm.
4 to form an image on the CCD element 56. The light component reflected by the dichroic mirror 46 is further reflected by the other surface of the prism 44, and has a main wavelength of 400 nm and a half width of ± 10.
pass through the band-pass filter 58 of
Image on top. The light reflected by the dichroic mirror 50 is further reflected by the other surface of the prism 48, and the main wavelength 7
The light passes through a band-pass filter 62 having a wavelength of 00 nm and a half-value width of ± 10 nm to form an image on a CCD element 64.

In the above configuration, the discharge tube 10 (FIG. 1)
Is discharged, electric signals corresponding to images at 550 nm (center of the visible region), 400 nm (near the ultraviolet region), and 700 nm (near the infrared region) are shaded on the CCD elements 56, 60, and 64, respectively. Obtained without occurrence. FIG. 3 shows each bandpass filter 54, 58,
62 shows the passing characteristics. As is apparent from FIG. 3, the trispectral camera used in the present invention includes:
400nm, 550nm, 7
A band-pass filter having a main wavelength of 00 nm and a half-width of ± 10 nm is used, and images in three narrow wavelength bands (hereinafter, referred to as a near-ultraviolet image, a visible image, and a near-infrared image, respectively). ) Are obtained at the same time. Note that, as far as the present invention is concerned, the analysis is performed only on visible images.

FIG. 4 shows the configuration of the entire system. This system includes a wavelength-specific level controller 71 (to be described in detail later) for converting the level of the image signal from the above-described trispectral camera 70 for each wavelength, and an image memory for converting the output from analog to digital and temporarily storing it in a memory. 72,
A computer 74 is provided for appropriately reading data stored in the image memory 72 and performing various analysis processes. The computer 74 has a keyboard 76 for inputting instructions from an operator, a monitor 78 for outputting an analysis result, a video printer 80 for outputting a hard copy thereof, and storing image data and analysis results. Are connected.

Calibration spots / freckles of wavelength-specific image data are digitized in 8 bits each for near-ultraviolet images, visible images, and near-infrared images, and recorded in a memory. This system compensates for variations in image input data due to unmanageable fluctuations that occur when images of the same object are successively captured over a long period of time. The following operation is performed for the purpose of obtaining the image data and allocating 8 bits within the existing range of the spectral reflectance of the flesh color determined by the wavelength to increase the detection power.

Using Minolta CM-1000HR, 400 nm, 550 nm, 70 nm obtained from skin color (cheek) measurement results of 826 Japanese women (16 to 59 years old)
The average value and the standard deviation of the spectral reflectance at 0 nm are as follows. Wavelength Average value of reflectance Standard deviation (σ) 400 nm 16.0% 2.9 550 nm 29.9% 3.5 700 nm 59.4% 2.4 Therefore, at 400 nm, reflection of 25% or more (average value + 3σ) or more 0% to 25%
8 bits are assigned to%. In this case, assuming that there is a linear relationship between the reflectance and the output luminance, one luminance corresponds to 0.10% in terms of reflectance, and an extremely high resolution can be obtained.

At 550 nm and 700 nm, bits are allocated based on the same concept. As an apparatus for this assignment, a wavelength-specific level controller for converting an input voltage corresponding to the reflectance of the imaging target at 400 nm, 550 nm, and 700 nm into an output voltage corresponding to the above-mentioned bit division was newly designed and manufactured. Even though dark spots and freckles are present, the reflectance is not necessarily 0%, and the lower limit can be rounded up. However, no special consideration is given here. The relationship between the reflectance of the imaging target under the input / output conditions obtained by interposing the above-described wavelength-specific level controller 71 (FIG. 4) and the output luminance in the present system is described using a matte gray painted paper (Murakami color technology research). Was requested to be manufactured). The reflectance of the matte gray coated paper was measured by Hitachi Color Analyzer 607. FIGS. 5, 6, and 7 show the obtained results.

The relationship between the reflectance and the luminance at each wavelength shows a relationship on the upper right, but is not a perfect linear relationship. Where the reflectance is small, the linearity becomes worse. As described above, even if dark spots and freckles are present, the reflectance is not 0%, and the sensitivity of the CCD camera in the low reflectance area is poor.
It is not practical to prepare a large number of calibration gray samples and perform non-linear brightness calibration. Therefore, calibration was performed using as few calibration gray samples as possible and assuming a linear relationship. This calibration method is considered to be a necessary and sufficient operation for the purpose of calibration for quantitative serial comparison of image input data.

As a result, the standard gray sample for the brightness calibration has no fear of deterioration and can be wiped off even if it becomes dirty. Therefore, "EVER-COLORS" (manufactured by Yoneda Glass Industrial Co., Ltd., a common reflection for photometry and colorimetry) A standard plate) was sprinkled with # 3000 sand and a matte surface was prepared and used. "EVER-COLORS" finally selected is GRAY No.1
000, GRAY No. 3000, GRAY No. 6000.
To find the calibration values for near-ultraviolet and visible images, use GRAY N
To obtain the calibration value of the near-infrared image using o.1000 and GRAY No.3000, use GRAY No.1000 and GRAY No.6.
000 was used.

In order to ensure the easiness of the calibration work and to reproduce the geometric conditions of illumination and light reception, a calibration standard plate set as shown in FIG. 8 (the above-mentioned calibration "EVER-COLORS") is used.
(3 sets, 2 sets each, 6 sets in total) were produced. As a set of two sheets, and arranged as shown in FIG.
-It is impossible to cut out a large area due to the limitations on processing and processing of "COLORS". Therefore, if it is intended to cover a large area as much as possible and to cope with uneven illuminance that may exist in the imaging area, 2 It is due to the necessity of making a piece.

As described with reference to FIG. 1, this is mounted on the camera opening with the pin 32 as a guide, and the calibration can be automatically performed only by pressing the shutter once. The calibration is performed by calculating a value obtained by calculating an average luminance in each window (50 × 100 pixels or 100 × 50 pixels) indicated by a solid line in FIG. 8 with a value indicating the state of the system at that time. It is executed by doing.

Table 1 shows the spectral reflectance (Hitachi Color Analyzer 607) of the calibration "EVER-COLORS" and the luminance value set by using the above-described wavelength-based level controller.

[0022]

[Table 1]

The method of obtaining the calibration value will be described below by taking the case of a near-ultraviolet image as an example. As shown in Table 1, GRAY No. 1000
Has a luminance in the near ultraviolet image of 38, a spectral reflectance at 400 nm determined by a spectrophotometer of 7.4%, and GRAY No.
Assuming that the luminance at 3000 is 240 and the spectral reflectance is 27.0% as the reference state, the relationship between the spectral reflectance X and the luminance Y in the reference state is

[0024]

(Equation 1)

## EQU1 ## GRAY No.1 at arbitrary calibration
It is assumed that the luminance of GRAY No. 3000 is 250 and the luminance of 000 is 40. In this case, the relationship between the measured value Y ₁ of the spectral reflectance X and brightness

[0026]

(Equation 2)

## EQU1 ## Therefore, if X is eliminated from the equations (1) and (2), the relationship between Y and Y ₁ is obtained, whereby the luminance calibration value Y in the reference state is calculated from the luminance value Y ₁ before calibration. With the above procedure, a calibration value corresponding to the luminance 0 to 255 of the calibration target pixel is obtained in advance, and calibration is performed for all the pixels by referring to this as a calibration table. Visible images and near-infrared images are calibrated in the same way.

Binarization of Image Data by Wavelength First, in order to determine whether all the collected images can be binarized at the same threshold value, data were collected from 143 persons in June 1992 (n = 143). ) The frequency distribution of the minimum luminance, the luminance of the automatic binarization, and the maximum luminance in 192 × 192 pixels cut out from the visible image in the original image was obtained.

As a result, it was found that the three distributions overlapped, and the same threshold value could not be adopted. Therefore, binarization is performed for each image. Although there are several proposals for the binarization method, the method proposed by Otsu to use the slice level as a value that divides into two at the best resolution in a given concentration distribution (Nobuyuki Otsu) ,
Journal of the Institute of Electronics and Communication Engineers, 63, 4, 349 (1980))
As a result of the processing, it was found that all the collected images were binarized at a slightly higher luminance. That is, the area of spots and freckles was extracted largely.

Therefore, for a large number of images, if the threshold value is lowered from the threshold value for binarization obtained by the Otsu's method, and then binarized, the blemishes and freckles can be visually recognized. The threshold was determined to see if a similar binary image could be obtained. As described later, the density of spots and freckles is quantified using luminance data of a visible image. The optimal binarization level in the visible image was determined as follows.

First, the collected original image (visible, n = 14
3) was binarized under the following eight conditions. Automatic binarization (Auto binarization value-3) Binarization (Auto binarization value-6) Binarization (Auto binarization value-8) Binarization (Auto binarization value- 9) binarization (auto binarization value -10) binarization (auto binarization value -11) binarization (auto binarization value -12) binarization A hard copy of the value image was output using a video printer, and the setters (three) evaluated under what conditions the processed image most accurately extracted the area where pigmentation was perceived by the naked eye. I let it.

As a result, of 143 cases, 117 cases (82 cases)
%), The image processed under the conditions of was evaluated to be optimal. In 14 cases, the image processed under the condition that the decrement was smaller than -6, and in 12 cases, the image processed under the large condition was evaluated as the optimal binary image. From this result, for the visible image, (threshold value for automatic binarization −6) was set as the slice level for binarization.

[0033] An example of a histogram of the luminance of the pixels constituting the density visible image evaluation Figure 9 pigmentation portion. As shown in FIG. 9, when the threshold value of “Auto binarization value−6 by Otsu's method” is determined, a set of pixels having a higher luminance value (corresponding to the background portion) and a set of pixels having a lower luminance value (e.g. The average value of the luminance is calculated for each of the pigmented portions, and the difference between them is used as an index of the density of the pigmented portion.

That is, this is indexed based on the idea that if the luminance difference between the background and the pigmented portion is large, the stain is conspicuous (ie, it feels dark), and if the difference is small, it is inconspicuous (ie, it feels pale). Things. Evaluation of pore size FIG. 10 is a flowchart showing the procedure of the process of evaluating the state of pores. Image data is input from the image memory 72 (FIG. 4) (step a), and an 8-neighbor smoothing filter (1) is used.
(1 × 11) is performed twice to smooth (step b).
The 8-neighbor type smoothing filter simply averages the luminance of pixels in a square area centered on the pixel of interest to obtain the luminance of the pixel. In the present embodiment, this is applied to an 11 × 11 square. Done. Note that 11 pixels correspond to 1.14 mm in the object to be imaged. Next, the difference image is obtained by subtracting the brightness of the corresponding pixel before smoothing from the brightness of each pixel after the smoothing process (step c). In the difference image, the luminance of the pixel corresponding to the pores remains as a small positive value, and the luminance becomes almost 0 in other cases. Since the difference image has a narrow change range, the difference image is subjected to log conversion so as to be widely distributed in the range of luminance 0 to 255 (step d). Since this image is a screen in which the positions of pores are bright and the background is dark, a process of inverting the brightness is performed for easy viewing (step e), but this process is essentially unnecessary. Next, automatic binarization processing is performed on the inverted screen (step f). In this case as well, the binary image obtained by the Otsu's method tends to express pores slightly too large, so that binarization is performed by (auto-binary value−60). Finally, the total number of pixels is calculated for the pixels determined to be pores (step g) and used as an index representing the state of the pores.

Output of Result FIG. 11 shows an example of the display output of the evaluation result. The horizontal axis indicates the total area of the pores on a scale of 0 to 4000, and the vertical axis indicates the difference between the average value of the density of the background portion and the average value of the density of the pigmented portion on a scale of 0 to 255. Therefore, the point of intersection between the two is indicated by a circle. The above value “400”
“0” is a maximum value determined from an index calculated by selecting the most conspicuous pores from the data collected from about 400 by visual judgment.

On this screen, the subject's skin is positioned on a plane centered on the two states of "pigmentation" and "pores".
Furthermore, it can be used as a reference for suggesting what kind of care direction can be considered.

[0037]

As described above, according to the present invention,
A system is provided that can automatically assess the condition of skin pores and the condition of pigmentation.

[Brief description of the drawings]

FIG. 1 is a sectional view of a trispectral camera used in the present invention.

FIG. 2 is a CCD camera 28 in a trispectral camera.
FIG.

FIG. 3 is a diagram illustrating pass characteristics of a band-pass filter.

FIG. 4 is a block diagram illustrating a configuration of a system according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a relationship between reflectance and luminance at 400 nm of matte gray coated paper.

FIG. 6 is a diagram showing a relationship between reflectance and luminance at 550 nm of matte gray coated paper.

FIG. 7 is a diagram showing a relationship between reflectance and luminance at 700 nm of matte gray coated paper.

FIG. 8 is a diagram showing a calibration standard plate.

FIG. 9 is a diagram illustrating an example of a histogram of luminance of each pixel constituting a visible image.

FIG. 10 is a flowchart illustrating a process of evaluating the state of pores.

FIG. 11 is a diagram illustrating an example of a display output of evaluation of a skin surface state.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Flash discharge tube 12 ... Reflector 16 ... Diffuser 26 ... Fisheye lens 28 ... CCD camera 44, 48, 52 ... Prism 46, 50 ... Dichroic mirror 54, 58, 62 ... Bandpass filter 56, 60, 64 ... CCD element 70 ... Trispectral camera 76 ... Keyboard 78 ... Monitor 80 ... Video printer

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-6-313708 (JP, A) JP-A-6-105826 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G01B 11/00-11/30 102 A61B 5/00 A61B 5/06-5/22

Claims (3)

(57) [Claims] 1. An image pickup means for outputting an image of the surface of the skin, a smoothing means for two-dimensionally smoothing the luminance level of each pixel constituting the image output by the image pickup means, Calculating a difference between a luminance level of each pixel constituting the divided image and a luminance level of a corresponding pixel of the image before smoothing and outputting a difference image; and binarizing the difference image. An analysis system for skin surface condition, comprising: a first binarizing means for obtaining an image of pores. 2. The image processing apparatus according to claim 1, further comprising a logarithmic conversion unit that performs logarithmic conversion on a luminance level of each pixel included in the difference image output by the subtraction unit and supplies the luminance level to the first binarization unit as a difference image. The system of claim 1. 3. A pore area calculation unit that calculates a pore area from a pore image output by the first binarization unit and uses the calculated pore area as a first index; and a unit that forms an image output by the imaging unit. A second binarizing means for dividing the pixels into a first set of pixels having a high brightness level and a second set of pixels having a low brightness level; and each of the first and second sets A density calculating means for calculating an average value of luminance levels of the pixels belonging thereto, calculating a difference between the two, and using the first index calculated by the pore area calculating means as one axis, The system according to claim 1, further comprising an output unit configured to output a result obtained by plotting the second index calculated by the density calculation unit on a two-dimensional plane with the other axis as the other axis.