KR101436988B1 - Method and Apparatus of Skin Pigmentation Detection Using Projection Transformed Block Coefficient - Google Patents

Abstract

The present invention relates to an apparatus and a method for detecting a skin pigmentation using a projection conversion block coefficient, and a method and apparatus for detecting skin pigmentation using projection conversion block coefficients. The method includes extracting a skin region from a target image, And a melanin component, and when the skin region is divided into blocks, a projection conversion block coefficient indicating a size of each of the hemoglobin and melanin components in each block is calculated, and pigment conversion And a result output unit for outputting a result of discriminating the deposition of the pigment, and a method therefor are provided.

Description

FIELD OF THE INVENTION [0001] The present invention relates to an apparatus and a method for detecting skin pigmentation using a projection conversion block coefficient,

The present invention relates to an apparatus and method for detecting skin pigmentation, and more particularly, to an apparatus and method for detecting skin pigmentation level using a projection conversion block coefficient.

Skin is the largest organ in the body and responds to the external and internal stimuli of the skin by changing the pigmentation pattern. Skin pigmentation refers to a phenomenon that is different from normal skin. Pigmentation itself is difficult to judge as a factor of health abnormality, but melanoma, basal cell carcinoma, and squamous cell Squamous cell carcinoma, and the like. Therefore, skin pigment detection plays a very important role not only in beauty but also in medicine.

The main causes of skin color change are melanin and hemoglobin. Skin color changes to light yellow, reddish brown or black depending on the degree of melanin contained in the skin. Hemoglobin acts as oxygen supply to blood cells Which causes the skin to become reddish. Thus, by analyzing the distribution of melanin and hemoglobin components in the skin, the degree and morphology of the skin pigment can be analyzed. The most widely used method for skin pigmentation discrimination is visual examination. This method is most commonly used by physicians and hairdressers to discriminate pigmentation, but there is a disadvantage that the test result is very subjective and not quantitative. Therefore, there is a need for quantitative determination of skin pigmentation degree.

Since the early 90's, digital image analysis techniques have been used to detect skin pigmentation, which can quantitatively measure not only the pigmented area but also the size and extent of the pigmented area. A computer vision based skin pigment detection method has been proposed. In this method, skin pigment asymmetry is set as a criterion of skin lesion, and the degree of pigmentation is determined by detecting and quantifying the skin pigment asymmetry. In addition, the fuzzy co-clustering algorithm for images (FCCI) method has been extended to detect blotch of pigmented skin lesions, and a normalized entropy function has been used to detect features of spots .

In order to obtain reliable and accurate results in the existing pigmentation detection method, there is a disadvantage that stable illumination and high-performance skin dermoscopy are required for the skin image acquisition process.

It is an object of the present invention, in view of the above, to provide an apparatus and method for detecting pigment deposition using an image processing technique instead of the conventional optical measurement technique.

According to an aspect of the present invention, there is provided an apparatus for detecting skin pigmentation, including: a video input unit for inputting a target image; A method of extracting a skin region from a target image, separating the extracted skin region into hemoglobin and melanin components, and dividing the skin region into blocks, An image processing unit for calculating a coefficient and determining pigment deposition using the calculated projection conversion block coefficient; And a result output unit for outputting a result of discriminating the deposition of the pigment.

The image processing unit constructs a Gaussian color model for a skin region through statistical analysis of a training image, extracts a skin region from the target image using the constructed Gaussian color model, performs morphological processing, And extracts the skin region by removing the noise existing in the extracted skin region.

And the image processing unit extracts the skin region using only the chrominance components (Cb and Cr) of the YCbCr color coordinate system.

The image processing unit separates the extracted skin region into hemoglobin and melanin components using an ICA (Independent Component Analysis) algorithm.

Wherein the image processing unit calculates the ratio of the magnitude of projection conversion block coefficients of melanin and hemoglobin of any one block to the ratio of magnitudes of projection conversion block coefficients of melanin and hemoglobin to the whole block, Characterized in distinguishing pigment deposits or melanin pigment deposits.

Wherein the image processor quantifies the region of pigment deposition by an area ratio of the blocks determined to be hemoglobin and melanin pigment deposition.

The degree of pigment deposition in the block determined as hemoglobin or melanin pigment deposition is calculated as the standard deviation of the pixel value in the identified block and the average pixel value of the blocks not discriminated as pigment deposition.

According to another aspect of the present invention, there is provided a skin pigmentation detection method comprising: extracting a skin region from a target image; Separating the extracted skin region into hemoglobin and melanin components; Calculating a projection transform block coefficient indicating a magnitude of each of the hemoglobin and melanin components in each block when the skin region is divided into blocks; And determining pigment deposition using the calculated projection conversion block coefficient.

Extracting the skin region comprises constructing a Gaussian color model for a skin region through statistical analysis of a training image and extracting a skin region from the target image using the constructed Gaussian color model; And removing noise existing in the extracted skin region through morphological processing.

The extracting of the skin region is performed using only the chrominance components (Cb and Cr) of the YCbCr color coordinate system.

Wherein the separating step is performed using an independent component analysis (ICA) algorithm.

Wherein the step of discriminating the pigment deposits comprises determining a ratio of the ratio of magnitudes of projection conversion block coefficients of melanin and hemoglobin of any one block to a ratio of magnitudes of projection transform block coefficients of melanin and hemoglobin to the whole block, Is used to discriminate hemoglobin pigment deposits or melanin pigment deposits.

The step of discriminating the pigment deposition is characterized by quantifying the area of the pigment deposit by an area ratio of the blocks determined by hemoglobin and melanin pigment deposition.

The degree of pigment deposition in the block discriminated by hemoglobin or melanin pigment deposition is calculated as the standard deviation of the pixel value in the discriminated block and the average pixel value of the blocks not discriminated by pigment deposition, .

As described above, according to the present invention, the skin area is derived from the target image using only the color difference components (Cb, Cr), thereby reducing the computational complexity and reducing the computation speed. In addition, it can be confirmed that the skin pigmentation detection method is robust against the change in illumination, and the detection performance is visually superior. In addition, there is an advantage of being able to quantify the pigment deposition area and extent.

1 is a cross-sectional view showing a cross-sectional structure of skin tissue.
2 is a block diagram for explaining a skin pigment deposition detection apparatus using projection conversion block coefficients according to an embodiment of the present invention.
3 is a flowchart illustrating a method of detecting a pigment deposit according to an embodiment of the present invention.
FIG. 4 is a diagram showing distribution of Y, Cb, and Cr components for a skin image photographed in various illumination environments.
Figure 5 shows reflectivity of three skin types with clustered values having a wide range.
FIG. 6 is a diagram showing GMM-EM clustering and skin detection based on morphology processing. FIG.
7 is a view for explaining the operation principle of the ICA algorithm in the optical density area according to the embodiment of the present invention.
8 is a graph for explaining projection conversion block coefficients according to an embodiment of the present invention.
9 is a diagram for explaining non-uniformity of skin intensity due to a face curve under a uniform illuminance.
FIG. 10 is a diagram illustrating a skin pigment deposition determination method using a global ratio and an area ratio. FIG.
FIG. 11 is a view showing a result of separating hemoglobin and melanin components from the skin region using the ICA algorithm.
12 is a diagram showing a pigmented area detected through a projection conversion block coefficient while varying the number of blocks.
13 is a diagram showing a test image obtained from various illumination environments.
FIG. 14 is a diagram showing a detection result according to a change in illuminance of a skin pigmentation detection method for a test image of FIG. 13 according to an embodiment of the present invention. FIG. And,
15 is a graph comparing performance of the method according to the embodiment of the present invention and the conventional method.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention. In the following description of the embodiments of the present invention, descriptions of techniques which are well known in the technical field of the present invention and are not directly related to the present invention will be omitted. This is for the sake of clarity of the present invention without omitting the unnecessary explanation. For the same reason, some of the components in the drawings are exaggerated, omitted, or schematically illustrated. Also, the size of each component does not entirely reflect the actual size.

The present invention proposes a method capable of detecting and evaluating the degree of pigmentation of a digital skin image that is robust to ambient illumination and does not require a skin magnifier. According to an embodiment of the present invention, the degree of pigment deposition is determined by using the difference in the composition of the normal skin and the deposited skin. The skin region detection based on Gaussian color model, hemoglobin by ICA (independent component analysis) Melanin pigmentation, and determination of hemoglobin and melanin pigmentation by a 2D local histogram.

In general, the method of measuring pigmentation uses a specific component of the color coordinate system. To do this, it is necessary to study the physiological structure and optical properties of the skin. 1 is a cross-sectional view showing a cross-sectional structure of skin tissue. The physiological structure and optical properties of skin and color components that can discriminate skin pigmentation and their measurement methods will be discussed.

The color of the skin is determined by the distribution of the melanin and hemoglobin components present in the skin tissue, which are present in the epidermis and the dermis, respectively, as in Fig. Melanin is produced by melanocytes that absorb and propagate visible light from the epidermis and darken skin tone. The absorption of light is determined by the unit volume of melanin, Here are the reasons for different skin colors. Hemoglobin is a natural chromophore that is present in blood cells, causing the skin to turn red. In general, skin color depends on melanin and hemoglobin. Abnormal hyperpigmentation of melanin can cause albinism and melanoma, and hemoglobin is accompanied by erythematic.

According to the present invention, a novel pigment deposition discrimination algorithm using only chrominance components (Cb and Cr) is proposed for improving the problems of the conventional pigment deposition discrimination algorithm and for reducing the amount of calculation and memory. analysis algorithm to separate the skin region into hemoglobin and melanin components, and then determine the size and degree of deposition of the pigmentation region using a local histogram. Hereinafter, an apparatus and method for detecting skin pigmentation using projection conversion block coefficients according to an embodiment of the present invention will be described in detail.

2 is a block diagram for explaining a skin pigment deposition detection apparatus using projection conversion block coefficients according to an embodiment of the present invention.

Referring to FIG. 2, a skin pigmentation detection apparatus according to an embodiment of the present invention includes an image input unit 100, an image processing unit 200, and a result output unit 300.

The image input unit 100 is for acquiring a target image for skin pigmentation detection and providing the acquired target image to the processing unit 200. The image input unit 100 may be implemented in various forms. For example, the image input unit 100 may be a camera module capable of performing a camera function. In this case, the image input unit 100 can actually photograph the target image and provide the photographed image to the processing unit 200. Alternatively, the image input unit 100 may be a communication module having a communication function, and may receive the target image from the other party through which communication is performed through the communication, and provide the target image to the processing unit 200.

The image processing unit 200 receives a target image from the image input unit 100, extracts a skin region from the target image, and detects pigment deposition in the extracted skin region. The image processing unit 200 includes a skin region extracting unit 210 and a pigment deposition detecting unit 220. The skin image extracting unit 210 constructs a Gaussian color model for a skin region through statistical analysis of a training image and extracts a skin region from a target image to be detected by the skin pigmentation. Here, the skin region extracting unit 210 can extract a skin region according to the Gaussian color model, and remove noise existing in the skin region through morphological processing, thereby obtaining a more accurate skin region.

The pigmentation detection unit 220 separates the skin region into hemoglobin and melanin components using an ICA (Independent Component Analysis) algorithm, and examines a local histogram for each of the separated hemoglobin and melanin components, .

The result output unit 300 is for providing a result of detecting pigmentation. The result output unit 300 may be implemented in various forms. For example, the result output unit 300 may be a display module having a display function, and may be used to receive the pigment deposition detection result from the image processing unit 200 and display the result. Alternatively, the result output unit 300 may be a communication module having a communication function. The result output unit 300 may receive the result of detecting the pigment deposition from the image processing unit 200, and may transmit the result of detection of the pigment deposition to the other party through communication through the network.

3 is a flowchart illustrating a method of detecting a pigment deposit according to an embodiment of the present invention.

Referring to FIG. 3, the image input unit 100 acquires a target image and inputs it to the image processing unit 200. For example, when the image input unit 100 is a camera module, the image input unit 100 can actually capture a target image and input the captured image to the processing unit 200. [ Alternatively, when the image input unit 100 is a communication module, the image input unit 100 may receive the target image from the other party through which the communication is performed through the communication, and may input the target image to the image processing unit 200.

The image processing unit 200 receiving the target image extracts the skin region in the first step. The Gaussian color model is constructed by extracting the skin region, constructing the Gaussian color model for the skin region by statistical analysis of the training image, extracting the skin region from the target image using the constructed Gaussian color model (Step 1.1), and a process of removing noise existing in the extracted skin area through morphological processing (step 1.2).

Next, the image processing unit 200 separates pigment components into hemoglobin and melanin in the skin region extracted in the second step using the ICA algorithm.

Next, the image processing unit 200 measures the pigment deposition on the pigment components separated into hemoglobin and melanin in the third step. This pigmentation measurement is performed by: i) deriving a projection conversion block coefficient indicating the size of pigment components of hemoglobin and melanin in each block (Step 3.1), dividing the skin region into blocks, and calculating the total hemoglobin projection conversion block coefficient and melanin The process of detecting the pigment deposition for each block (Step 3.2) is performed using the global ratio, which is the average ratio of the projection conversion block coefficients, and the local ratio, which is the ratio of the hemoglobin projection conversion block coefficient to the melanin projection conversion block coefficient of the specific block , And quantification of the pigment deposition (step 3.3) is performed to derive the pigment deposit area and degree of pigment deposit.

Next, the result output unit 300 outputs the processing result of the image processing unit 200 described above.

Hereinafter, each of the above-described steps will be described in detail.

1. Extraction of skin region based on GMM-EM

1.1 Construction of Gaussian color model and skin area extraction

In the proposed method according to the embodiment of the present invention, a preprocessing process for extracting a skin region before skin pigment detection is required. According to an embodiment of the present invention, a skin region is extracted using a Gaussian Mixture Model (EMM) -EM (Expectation Maximization) clustering algorithm that can set parameters according to various environments. The preprocessing process for extraction of skin region based on GMM-EM clustering will be described.

FIG. 4 is a diagram showing distributions of Y, Cb, and Cr components for 200 skin images including hands, arms, and faces photographed in various illumination environments. 4A shows the Y component of the skin images in the YCbCr coordinate system, FIG. 4B shows the Cb component, and FIG. 4C shows the Cr component distribution. In general, the skin color is mainly influenced by the chromaticity rather than the ambient illumination, and therefore, as shown in FIG. 4, the chrominance components (Cb and Cr) for the skin images have a very narrow distribution relative to the Y component . Thus, it can be confirmed that the color difference components (Cb and Cr) of the YCbCr color coordinate system are suitable for color clustering. Therefore, in the preprocessing process, adaptive GMM-based skin regions for Cb and Cr components are extracted.

개 화소의 CbCr 성분

,

이 주어지고,

개 GMM로부터 평균 벡터

가 주어졌을 때, 로그-우도비(Log-Likelihood ratio)가 최대인 변수 집합

를 찾는 것이다.

개 GMM을 위하여,

분포는 가중치

을 가지는 가우시안 분포

의 합으로 정의된다. 여기서

는

,

으로,

개의 평균과 공분산(covariance) 행렬의 변수 집합을

이라 한다. GMM-EM clustering

CbCr component of pixel

,

Given this,

The average vector

The log-likelihood ratio is the maximum.

.

For the dog GMM,

Distribution is weighted

Gaussian distribution with

. here

The

,

to,

A set of variables in the mean and covariance matrix of

Quot;

개 GMM들의 초기 변수

,

들을 복수개, 약, 200개 피부 영상(훈련 영상)들로부터 계산한 다음, 초기 우도의 대수값(log-likelihood)을 다음의 수학식 1에 따라 구한다. In the chrominance component (Cb Cr) clustering process, first,

Initial variables of GMMs

,

(Training images), and then the log likelihood of the initial likelihood is calculated according to the following equation (1).

번째 반복 횟수에서

가

번째 가우시안 분포에 포함될 확률

을 다음의 수학식 2에 따라 구한다. And

The number of repetitions

end

Probability to be included in the second Gaussian distribution

Is obtained by the following equation (2).

+1번째 횟수에서의 변수

,

을 다음의 수학식 3 및 수학식 4와 같이 예측한다. after that,

Variables in the first number of times

,

Is predicted according to the following equations (3) and (4).

+1번째 우도의 대수값(log-likelihood)을 다음의 수학식 5와 같이 구한다. And

The log-likelihood of the +1-th likelihood is obtained by the following equation (5).

일 때까지 위의 과정을 반복 수행함으로써 최종 변수 집합

들을 구한다. 실시예에서는

=7개의 GMM 기반 피부 클러스터들로 설정하였다. after that,

And the final set of variables

. In the embodiment

= 7 GMM-based skin clusters.

(

이 주어졌을 때, GMM 확률밀도함수

을 다음의 수학식 6과 같이 구한다. The skin segmentation method using the GMM-EM cluster obtained from the skin training image is implemented as follows.

(

Given a GMM probability density function

As shown in the following Equation (6).

가 문턱치

보다 클 경우

는 피부 색소로 간주되어, 가장 가까운 GMM 클러스터

로 다음의 수학식 7과 같이 분류된다. And

Threshold

Greater than

Are regarded as skin pigment, and the closest GMM cluster

As shown in the following Equation (7).

는 피부 색소로 간주되지 않는다. 위와 같은 방법에 의하여 임의의 영상에서 피부 영역이 추출된다.
If not,

Is not considered a skin pigment. The skin region is extracted from an arbitrary image by the above method.

1.2 Morphological processing removes noise in the skin area

와 재구성 연산을 제한하는 마스크(Mask)

이 필요하다. 피부 영역 이진 영상에 침식(erosion) 연산이 수행된 마커

의 초기값을

라 할 때,

번째 연산된

는 다음의 수학식 8과 같다. The skin region extracted by the above-described method is not continuous due to noise or external environment, and isolated skin pixels can be generated. In the embodiment of the present invention, the noise inside the area and the protruding external noise can be removed by using a closing operator by reconstruction. In the reconstruction-close operation, a marker indicating the starting point for the reconstruction operation,

And a mask (mask) for restricting the reconstruction operation.

Is required. The skin area binarized image has a marker erosion operation

The initial value of

In other words,

Th computed

Is expressed by the following equation (8).

는 3x3 크기의 모폴로지 SE(Structuring element)이고,

는 팽창 (dilation)연산이다. 모폴로지 재구성 연산(형태학적 처리)은

일 때까지

를 팽창한 값과 마스크인

를 비교하여 작은 값을 취한다. 이때 마커

는

을 만족한다. 마스크의 제한 점까지 마커

를 반복적으로 팽창함으로써 이진 영상의 돌출된 부분의 잡음이 제거되고, 이 영상에 역을 수행하여 같은 연산을 수행함으로써 내부의 잡음이 제거된다. 이와 같은 모폴로지 재구성 연산은 열림 연산과 유사하지만, 모양을 유지한다는 장점이 있다. here

Is a 3 x 3 sized structuring element SE,

Is a dilation operation. The morphology reconstruction operation (morphological processing)

Until

And the value of the mask

And takes a smaller value. At this time,

The

. Markers up to the limit of the mask

The noise of the protruded portion of the binary image is removed, and the noise is removed by performing the inverse operation on the image and performing the same operation. This morphology reconstruction operation is similar to the open operation, but has the advantage of maintaining the shape.

가 낮게 설정될 경우, 피부의 세기 변화에 매우 민감하여 부정 오류(false negative)로 판정된 화소들이 발생할 수 있다. 반면, 문턱치

가 클 경우, 긍정 오류(false positive)로 판정된 화소들이 발생할 수 있다. 본 발명의 실시예에서는 긍정 오류 판정된 영역이 일련의 처리 과정에서 색소 침착으로 검출되지 않아야 하므로, 문턱치

를 작게 선택하였다. 얼굴 영상 데이터베이스 및 실제 측정 실험으로부터 평균 4% 미만의 부정 오류율과 0.1% 미만의 긍정 오류율이 나타남을 확인하였다. According to the embodiment of the present invention, it has been confirmed that a high recognition rate is obtained from an experiment on a database having hundreds of facial images. This is because the face images are mostly different in color from the background and skin, and the complexity is low. However, some pixels having colors similar to skin color may be misrecognized. Threshold

Is set to a low value, it is possible to generate pixels determined to be false negative because they are very sensitive to changes in intensity of the skin. On the other hand,

The pixels determined to be false positives may occur. In the embodiment of the present invention, since the region in which the positive error has been determined should not be detected as pigment deposition in a series of processing steps,

. From the facial image database and the actual measurement experiments, it was confirmed that the negative error rate of less than 4% and the positive error rate of less than 0.1% were found.

를 다르게 설정할 수 있다. 도 5는 클러스터링된 값들이 넓은 범위를 가지는 세 가지 피부 타입의 반사도(reflectivity)를 도시한 도면이다. 피부의 색의 결정하는 반사도는 인종에 따라 분포 범위가 넓기 때문에 다른 인종의 피부색으로 검출되거나 혹은 피부 영역이 피부가 아닌 영역으로 검출될 수 있기 때문에 본 발명의 실시예에서는, 백인, 아시아인, 및 아프리카인('White', 'Normal' 및 'Black')의 피부색 분포에 따라 3가지 피부색으로 군집화하였다. 파장에 따른 각 인종의 피부 반사도는 도 5에 도시된 바와 같다. 이와 같은 문턱치에 의하여 피부 인식률이 향상됨과 동시에 배경에 대한 긍정 오류율이 낮아진다. Further, according to the embodiment of the present invention, in order to secure a higher recognition rate,

Can be set differently. Figure 5 shows reflectivity of three skin types with clustered values having a wide range. Since the reflectivity determined by the color of the skin is wide in the range of distribution depending on the race, it can be detected as a skin color of another race, or the skin area can be detected as a non-skin area. Three skin colors were clustered according to the skin color distribution of African ('White', 'Normal' and 'Black'). The skin reflectivity of each race according to wavelength is as shown in Fig. Such a threshold improves the skin recognition rate and lowers the positive error rate for the background.

FIG. 6 is a diagram showing GMM-EM clustering and skin detection based on morphology processing. FIG. Referring to FIG. 6 (b), which is a GMM-EM clustering-based skin detection region for the original image in FIG. 6 (a), some non-continuous isolated pixels may be seen. 6 (c), which is a skin detection region processed by morphological processing, which is a closure operation by reconstruction, it can be seen that the isolated pixels are removed. Referring to FIG. 6D, which is the final detected skin region, it can be seen that the skin region and the non-skin region are well separated.

2. Skin pigmentation

In the embodiment of the present invention, the chromatic components of hemoglobin and melanin are separated in the skin region extracted using an ICA (Independent Component Analysis) algorithm. 7 is a view for explaining the operation principle of the ICA algorithm in the optical density area according to the embodiment of the present invention. Here, the ICA algorithm maximizes the statistical independence of the predicted signals from signals composed of independent components even if there is no a priori information for each component, as shown in FIG. 7 There is an advantage that each component can be extracted.

According to the present invention, the following three assumptions are set to apply ICA to color skin images. 1) Skin color is determined by hemoglobin and melanin components. 2) The distribution of hemoglobin and melanin in the skin region is independent of each other. 3) The amount of skin image and hemoglobin and melanin has a linearity in the optical density region.

는 다음의 수학식 9와 같이 표현함으로써 피부 영역에서 헤모글로빈 및 멜라닌 성분을 분리할 수 있다. Based on the linearity assumption of 3) above, the optical density vector of the RGB color coordinate system

The hemoglobin and the melanin components can be separated from the skin region by expressing it as shown in the following Equation (9).

은 전치 행렬을 의미하고,

,

, 및

는 각각 RGB 색 좌표계에서 R, G, 및 B 성분의 화소값이다. 그리고 나머지 2가지 가정에 의해 광 밀도 벡터

는 다음의 수학식 10과 같이 표현될 수 있다. here

Denotes a transpose matrix,

,

, And

Are the pixel values of the R, G, and B components in the RGB color coordinate system, respectively. And the optical density vector

Can be expressed by the following Equation (10).

및

는 각각 멜라닌 및 헤모글로빈의 밀도 벡터이고,

및

는 각각 멜라닌 및 헤모글로빈 성분의 크기 값이며,

는 다른 피부 조직에 의해 영향을 받는 공간 정상 벡터(spatially stationary vector)를 나타낸다. 따라서 피부 영상의 벡터

은 다음의 수학식 11과 같이 표현될 수 있다. here

And

Are the density vectors of melanin and hemoglobin, respectively,

And

Are the magnitude values of the melanin and hemoglobin components, respectively,

Refers to a spatially stationary vector that is affected by other skin tissues. Therefore,

Can be expressed by the following Equation (11).

In this case, the sizes of the two components are expressed by Equation (12).

는 다음의 수학식 13과 같다. And

Is expressed by the following equation (13).

는

의 예측치를 의미하고,

및

는 합성 인자 (synthesis parameter)를 나타낸다. 본 발명의 실시예에서는 멜라닌 및 헤모글로빈에 대하여

및

와 같이 각각 설정한다. 도 6은 광 밀도 영역에서의 헤모글로빈 및 멜라닌 성분 크기를 보여준다. 위와 같은 방법에 의하여

크기의 피부 영역에서 각 위치에 대한 멜라닌 성분 크기

과 헤모글로빈 성분 크기

를 각각 구할 수 있다. here

The

, And "

And

Represents a synthesis parameter. In the examples of the present invention, melanin and hemoglobin

And

Respectively. Figure 6 shows hemoglobin and melanin component sizes in the light density region. By the above method

Size of melanin component for each position in skin area

And hemoglobin component size

Respectively.

According to the present invention, the distribution of melanin pigment in the skin region where hemoglobin and melanin components are present at the same time is not influenced by the hemoglobin pigment, and pigmentation occurs by the melanin component in the skin region having a small pixel value.

3. Pigmentation measurement

3.1. Derive projection transform block coefficients

8 is a graph for explaining projection conversion block coefficients according to an embodiment of the present invention. Referring to FIG. 8, in the embodiment of the present invention, the projection conversion block coefficients of the hemoglobin and melanin components are obtained in order to determine the positions of the hemoglobin and melanin components. The projection conversion block coefficients represent the average values projected in the discrete intervals of the horizontal axis X and the vertical axis Y. [ Discrete intervals divided on two axes are represented by blocks, and one block has a projection transform block coefficient composed of two coefficients projected on two axes.

및 헤모글로빈 크기

에 대한 투영 변환 블록 계수를 구한다. 피부 영역의 해상도가

이고, 블록의 크기가

일 때, 각 축에 대한 분할된 이산 구간의 개수는

,

이고, 블록의 개수는

이다. 멜라닌 크기

에 대한

번째 이산 구간의 X축 투영 계수

와

번째 이산구간의 Y축 투영 계수

는 다음의 수학식 14 및 수학식 15와 같이 정의된다. According to the present invention, the skin region is divided into blocks, and melanin sizes projected in the X and Y axis directions

And hemoglobin size

Lt; / RTI > Resolution of skin area

And the size of the block is

, The number of divided discrete intervals for each axis is

,

, And the number of blocks is

to be. Melanin size

For

Axis projection coefficient of the second discrete section

Wow

Y-axis projection coefficient of the second discrete section

Is defined by the following equations (14) and (15).

번째 블록

은 X축 및 Y축에 투영된 계수

을 가지며, 이를 투영 변환 블록 계수라 한다. 따라서 멜라닌 투영 변환 블록 계수는 다음의 수학식 16과 같다.

Th block

Is the coefficient projected on the X and Y axes

And is called a projection transform block coefficient. Therefore, the melanin projection conversion block coefficient is expressed by the following Equation (16).

Similarly, the hemoglobin projection conversion block coefficient is given by the following equation (17).

은

와 같은 방법으로 헤모글로빈 크기 값

에 의하여 구하여진다.

silver

The hemoglobin size value < RTI ID = 0.0 >

.

및 헤모글로빈 크기

를 반영하며, 구해진 투영 변환 블록 계수는 Z축의 강도(intensity)값으로 표현된다. 이러한 Z축의 강도(intensity)값은 헤모글로빈 및 멜라닌 성분에 대한 해당 블록에 속해 있는 화소들의 평균값이 될 수 있다. In other words, as described above, the projection conversion block coefficients are obtained by dividing the skin region into blocks and measuring the melanin size

And hemoglobin size

And the obtained projection conversion block coefficient is represented by an intensity value of the Z axis. The intensity value of the Z axis may be an average value of pixels belonging to the corresponding block for the hemoglobin and melanin components.

Fig. 8 shows the projection conversion block coefficients of hemoglobin and melanin for the skin region (Fig. 8 (a) is the hemoglobin component, and Fig. 8 (b) is the projection conversion block coefficient of the melanin component). The block coefficient and the pigmentation measurement depend on the number of blocks. In general, normal skin has a higher coefficient than the surrounding area, but pigmentation has a lower coefficient. By using such properties, pigmentation can be detected. According to embodiments of the present invention, the number of blocks can be adjusted to adjust the rate of detection of pigmentation.

3.2. Pigmentation detection

를 다음의 수학식 18과 같이 설정하고, 블록 내의 지역 비(local ratio)

을 다음의 수학식 19와 같이 설정한다. 9 is a diagram for explaining non-uniformity of skin intensity due to a face curve under a uniform illuminance. Generally, skin surface is not uniform, so skin images have different brightness values depending on the direction of skin surface absorbing light even under the same illumination conditions. That is, since the face or the body is not flat, the skin strength is not constant even under uniform illumination as in FIG. This effect acts simultaneously on the hemoglobin and melanin regions. Even in such uneven skin intensity distribution, pigmentation must be detected in all areas of the skin. According to an embodiment of the present invention, a global ratio of hemoglobin and melanin components is determined in order to discriminate pigmentation while minimizing unevenness due to curves.

Is set as shown in the following Equation (18), and the local ratio in the block is set to "

Is set as shown in the following Equation (19).

는

번째 블록의 멜라닌과 헤모글로빈의 투영 변환 블록 계수 크기의 비율을 나타내고,

는 전체 블록에 대한 두 성분의 투영 변환 블록 계수 크기의 평균 비율을 나타낸다. 본 발명의 실시예에 따르면, 지역 비

를 전역 비

와 비교함으로써 블록 내의 색소 침착 발생 여부를 다음의 수학식 20과 같이 판정할 수 있다.

The

Th block, the magnitude of the projection transform block coefficient of melanin and hemoglobin,

Represents the average ratio of the projection transform block coefficient magnitudes of the two components over the entire block. According to an embodiment of the present invention,

To the global ratio

It is possible to determine whether or not the pigment deposition in the block is generated as shown in the following equation (20).

의 값이 지역 비

와 문턱치

의 합한 값 보다 크면 홍점(red spot)이라고 판정하고,

가 지역 비

에서 문턱치

를 차감한 값 보다 작으면 흑점(black spot)으로 판정한다. FIG. 10 is a diagram illustrating a skin pigment deposition determination method using a global ratio and an area ratio. FIG. Referring to FIG. 10,

The value of

And threshold

, It is determined that the spot is a red spot,

Local rain

In the threshold

Is determined to be a black spot.

In the embodiment of the present invention, it is assumed that the area ratio of the normal skin area in the uniform illumination environment is almost the same as the global ratio, and these two ratios are affected only by the pigmentation. Comparing the hemoglobin and melanin components of the pigmentation images under these assumptions, the difference between the two regions visually appears to be small when the difference between the local ratio of the pigment deposition region and the local ratio of the normal skin region is close to 35, On the contrary, if the difference between the two ratios is more than 60, it can be confirmed that there is a remarkable difference in visibility. Further, in the embodiment of the present invention, the threshold value is determined according to the general difference between the pigmented area and the normal skin area.

3.3. Quantification of pigmentation

은 헤모글로빈 및 멜라닌으로 판별된 블록들의 면적 비율로 다음의 수학식 21과 같이 계산되어진다. The pigmentation obtained by the projection conversion block coefficients obtained above can be quantified in terms of area and degree. Area of pigmentation

Is the area ratio of the blocks determined to be hemoglobin and melanin, and is calculated by the following equation (21).

는 헤모글로빈 또는 멜라닌으로 판별된 블록들의 개수를 나타낸다. 색소 침착 정도는 정상 피부와의 표준 편차로 계산되어진다. 헤모글로빈 또는 멜라닌 색소 침착으로 판별된 임의의 블록

내의 색소 침착 정도는 해당 블록 내의 화소값

와 색소 침착으로 판별되지 않은 블록들의 평균 화소값

에 대한 표준 편차

로 다음의 수학식 22와 같이 정의된다. here

Represents the number of blocks identified as hemoglobin or melanin. The degree of pigmentation is calculated as the standard deviation of normal skin. Any block determined to be hemoglobin or melanin pigmentation

The degree of deposition of the pigment in the corresponding block

And the average pixel value of blocks not discriminated by pigment deposition

Standard deviation for

Is defined by the following equation (22).

This is a measure of the difference between a pixel in a pigmented area and a pixel in a normal skin, and can also be used to measure improvement in pigmentation before and after make-up.

As described above, according to the embodiment of the present invention, the skin pigmentation detection method comprises: 1. skin area extraction, 2. skin image division into hemoglobin and melanin components, and 3. pigmentation measurement step.

In step 1, the skin area extraction step, 200 skin images including human hand, arm, and face were analyzed to extract the skin area based on GMM-EM clustering.

The second step, the ICA algorithm, separates the hemoglobin and melanin components from the skin area. FIG. 11 is a view showing a result of separating hemoglobin and melanin components from the skin region using the ICA algorithm. 11 (a) is skin having a black spot, (b) melanin, (c) hemoglobin component, (d) skin having a red spot, and (e) melanin And (f) a hemoglobin component. As shown, an example of separating the melanin and hemoglobin components from other pigmentation images of black and red spots is shown. It can be seen that the proposed method accurately detects the pigmentation of the two components.

와 지역 비

가 유사한 값을 가짐을 확인하였고, 흑점 및 홍점을 포함하고 있는 피부에서는 서로 상이한 값을 가짐을 확인하였다. 도 12는 블록의 개수를 30x30과 100x100으로 가변하면서 투영 변환 블록 계수를 통해 검출된 색소 침착된 영역을 도시하는 도면이다. 도 12에서 색소 침착으로 판별된 블록들은 녹색으로 표시되어 있다. Step 3 calculates the projection transform block coefficients for the separated hemoglobin and melanin components. From this experiment,

And local rain

Were found to have similar values, and it was confirmed that they have different values in the skin including the black spot and the red spot. 12 is a diagram showing the pigmented areas detected through projection conversion block coefficients while varying the number of blocks to 30x30 and 100x100. In FIG. 12, blocks identified as pigment deposition are indicated in green.

FIG. 13 is a view showing a test image obtained from various illuminance environments, and FIG. 14 is a diagram showing a detection result according to a change in illuminance of a skin pigmentation detection method for the test image of FIG. 13 according to an embodiment of the present invention . For detection and evaluation of illumination variations, we used 60 test images generated from six illumination environments for a single light source, multiple light sources, and sunlight, as shown in FIG. The test images were taken from an auto white Nikon D90 camera. The results detected from the test image are shown in Fig. 14, and the blocks marked in green are identified as pigment deposits. The extent and extent of pigmentation for each illumination environment are shown in Table 1 below.

Dark
shadow Medium shadow Light
shadow Lights Sun Specular  Detected area 6.95% 11.52% 7.54% 9.68% 8.14% 4.97%  Standard deviation 17.04 18.07 15.54 20.07 16.61 8.54  Error rate 22.85% 52.79% 1.28% 0% 1.19% 48.7%

As shown in Table 1, it can be confirmed that the skin pigmentation detection method according to the embodiment of the present invention is robust against the change in illumination intensity and visually shows excellent detection performance.

The pigmentation detection method according to the present invention can be applied to the performance evaluation of a sunscreen agent. For this purpose, we evaluated the performance of UV - blocking agents produced by three brands that have the function of masking skin pigmentation through experiments. In the experimental evaluation, about 0.05ml of the blocker was applied to the pigmented skin, and the degree of pigmentation was measured. Also, the area where the blocker was applied was visually compared with the normal skin area and the pigmented area. At this time, the skin color applied to the sunscreen is close to normal skin. The following Table 2 shows the evaluation results for the three kinds of ultraviolet screening agents.

Skin without pigmentation Skin with pigmentation Test Sample 1 Test Sample 2 Test Sample 3 Brand and Item - - Company H. primer base M company blemish
cover balm E Company BB
magic cream The detected region 0.00% 9.68% 0.12% 0.00% 0.15% Standard Deviation 7.61 20.07 9.17 7.72 8.98

From Table 2, it was confirmed that the region of pigmentation was very small in the skin image applied with the sunscreen agent, and the standard deviation of Sample 2 was the smallest. This standard deviation indicates a close proximity to the normal skin area.

15 is a graph comparing performance of the method according to the embodiment of the present invention and the conventional method. In the conventional method (J. Lu, JH Manton, E. Kazmierczak, and R. Sinclair, "Erythema detection in digital skin images," Image Processing ( ICIP ) , pp. 2545-2548. Sep. 2010.) The skin region is separated by a Bayesian classifier, melanin and hemoglobin components are extracted based on the ICA algorithm, and erythema regions are detected by SVM (support vector machine) for melanin and hemoglobin. This method basically has a merit that it can detect the pigment deposition even in a low contrast image, but it has a disadvantage in that there is a limitation in the processing time and the ambient illumination, and the degree of pigment deposition can not be quantified. In skin region segmentation and pigmentation discrimination, a method of analyzing the characteristics of the three components of the color coordinate system is generally used, which has a disadvantage that the execution time of the algorithm is long. In the performance comparison experiment, the detection time and the detection rate of the method of the present invention and the conventional method were compared and evaluated. At this time, the detection time and detection rate of pigment deposition for 200 skin images having different resolutions were measured and shown in FIG. Conventional methods are only capable of pigmentation partitioning, and since they can not measure the amount of pigment deposition, only performance comparisons against detection time and detection rate are possible. The average detection time results per resolution are shown in the graph of FIG. 15 (a). Conventional methods detect pigment deposition on the basis of color information of three channels, and the method of the present invention detects pigment deposition by color information of two channels of CbCr and projection conversion block coefficient on spatial domain. Therefore, the present invention reduces the detection time somewhat as compared with the conventional method due to the channel reduction. In particular, it can be seen that the larger the resolution, the shorter the detection time of the proposed method is 0.5-2 times than the conventional method.

In the detection of pigmentation, the non-skin region can be mistakenly extracted into the skin region in the skin region extraction process, and the influence of illumination in the hemoglobin and melanin pigment separation process should be generally considered. In the proposed method, the influence of illumination is minimized by using the global ratio and the local ratio of the projection transform block coefficients. Figure 14 (b) shows the detection rates of the proposed method and the conventional method. As a result, as shown in FIG. 15, at the medium brightness, both methods showed a detection rate close to 100%. However, at dark or bright brightness, the method of the present invention showed a high detection rate of at least 25% higher than that of the conventional method (Lu).

The pigment deposition detection method according to an embodiment of the present invention may be implemented in a form of a readable program through various computer means and recorded on a computer readable recording medium. Here, the recording medium may include program commands, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a recording medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. For example, the recording medium may be a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical medium such as a CD-ROM or a DVD, a magneto-optical medium such as a floppy disk magneto-optical media, and hardware devices that are specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. It is obvious to those who have the knowledge of. Furthermore, although specific terms are used in this specification and the drawings, they are used in a generic sense only to facilitate the description of the invention and to facilitate understanding of the invention, and are not intended to limit the scope of the invention.

100: image input unit 200: image processing unit
210: Skin region extracting unit 220: Pigmentation detecting unit
300: Result output unit

Claims (14)

A skin pigmentation detection apparatus comprising:
A video input unit for inputting a target video;
Extracting a skin region from the target image, separating the extracted skin region into hemoglobin and melanin components,
An image processing unit for calculating a projection conversion block coefficient indicating a size of each of the hemoglobin and melanin components in each block when the skin region is divided into blocks and determining pigment deposition using the calculated projection conversion block coefficients; And
And a result outputting unit for outputting a result of discriminating the deposition of the pigment. The method according to claim 1,
The image processing unit
We constructed a Gaussian color model for the skin region through statistical analysis of the training image, extract the skin region from the target image using the constructed Gaussian color model,
The noise existing in the extracted skin region is removed through morphological processing,
Characterized in that the skin region is extracted
Skin pigmentation detection device. The method according to claim 1,
The image processing unit
Wherein the skin region is extracted using only the chrominance components (Cb and Cr) of the YCbCr color coordinate system. The method according to claim 1,
The image processing unit
Characterized in that the extracted skin region is separated into hemoglobin and melanin components by using an ICA (Independent Component Analysis) algorithm
Skin pigmentation detection device. The method according to claim 1,
The image processing unit
Using the global ratio representing the ratio of the magnitude of the projection transform block coefficients of melanin and hemoglobin of any one block and the average ratio of the magnitudes of the projection transform block coefficients of melanin and hemoglobin to the whole block
Characterized in that hemoglobin pigmentation or melanin pigmentation is distinguished
Skin pigmentation detection device. The method according to claim 1,
The image processing unit
Characterized in that the area of pigment deposition is quantified by the area ratio of the blocks determined by hemoglobin and melanin pigment deposition
Skin pigmentation detection device. The method according to claim 1,
The image processing unit
The extent of pigmentation in blocks determined by hemoglobin or melanin pigmentation
And calculating a standard deviation of a pixel value in the determined block and an average pixel value of blocks not discriminated as pigment deposits
Skin pigmentation detection device. A skin pigmentation detection method of a skin pigmentation detection apparatus,
Extracting a skin region from a target image;
Separating the extracted skin region into hemoglobin and melanin components; And
Calculating a projection transform block coefficient indicating a magnitude of each of the hemoglobin and melanin components in each block when the skin region is divided into blocks; And
And determining the pigment deposition using the calculated projection conversion block coefficient. The method for detecting skin pigmentation in a skin pigmentation detection apparatus, 9. The method of claim 8,
The step of extracting the skin region
Constructing a Gaussian color model for a skin region through statistical analysis of a training image and extracting a skin region from the target image using the constructed Gaussian color model; And
And removing noise existing in the extracted skin region through morphological processing. The method for detecting skin pigmentation in a skin pigmentation detection apparatus, 9. The method of claim 8,
The step of extracting the skin region
Wherein only the chrominance components (Cb and Cr) of the YCbCr color coordinate system are used. 9. The method of claim 8,
The separating step
Wherein the separation is performed using an ICA (Independent Component Analysis) algorithm. 9. The method of claim 8,
The step of determining the pigment deposition
Using the global ratio representing the ratio of the magnitude of the projection transform block coefficients of melanin and hemoglobin of any one block and the average ratio of the magnitudes of the projection transform block coefficients of melanin and hemoglobin to the whole block
Wherein the detection of hemoglobin pigmentation or melanin pigmentation is judged by detecting the presence of skin pigmentation in the skin pigmentation detection apparatus. 9. The method of claim 8,
The step of determining the pigment deposition
Wherein the area of the pigment deposit is quantified by the area ratio of the blocks determined to be hemoglobin and melanin pigment deposition. 9. The method of claim 8,
The step of determining the pigment deposition
Wherein the degree of pigment deposition in the block determined as hemoglobin or melanin pigment deposition is calculated as the standard deviation of the pixel value in the identified block and the average pixel value of the blocks not discriminated as pigment deposition. Skin pigmentation detection method.

Images