KR20190036401A - Method for evaluating skin and program for executing the method - Google Patents

Abstract

An embodiment of the present invention relates to a method for evaluating a skin condition based on skin images. The method comprises the steps of: generating a first characteristic vector from skin images; determining a second characteristic vector corresponding to an attribute value for at least one attribute of the skin based on the first characteristic vector and a skin recognizer; and determining a grade of the skin based on the second characteristic vector and a second skin recognizer, wherein the first skin recognizer is a data set representing a correlation between the first characteristic vector corresponding to each of the plurality of skin images and the second characteristic vector corresponding to one or more attribute values of each of the plurality of skin images, and the second skin recognizer is a data set representing a correlation between the second characteristic vector corresponding to one or more attribute values of each of the plurality of skin images and a third characteristic vector corresponding to a grade of each of the plurality of skin images. According to the present invention, it is possible to collectively provide analysis on the skin condition and a solution based on each skin condition.

Description

[0001] METHOD FOR EVALUATING SKIN AND PROGRAM FOR EXECUTING THE METHOD [0002]

Embodiments of the invention relate to methods and programs for evaluating the condition of the skin.

The skin is the largest organ of the envelope system, including skin parts such as hair, nails, hair and glands. The skin includes multiple layers of epithelial tissue that protect the lower muscles and organs. Because skin is constantly attacked by many external and internal factors, skin can become infected with many diseases. Thus, monitoring skin conditions is important from a health point of view and from a cosmetic point of view.

There are a number of ways to monitor the condition of the skin, but most methods require access to dedicated devices that are available to dermatologists or dermatologists. However, access to such physician or dedicated devices is accompanied by difficulties, inconveniences and excessive expenditure.

Therefore, there is a need for a simpler solution that allows ordinary users to easily grasp the condition of the skin without a dermatologist or a dedicated device provided for the dermatologist.

The present invention is to provide a skin condition evaluation method and program that can confirm the attribute value and the skin rating of one or more skin attributes from a skin image.

The present invention also provides a skin condition evaluation method and a program for identifying an attribute value of each skin attribute and determining an objective rating of the skin based on the attribute value, so that appropriate measures can be taken for the skin.

Furthermore, the present invention provides a skin condition evaluation method and a program that can integrally provide a skin condition analysis and a solution based on each condition.

A method of evaluating a skin condition based on a skin image according to an embodiment of the present invention includes: generating a first characteristic vector from a skin image; Determining a second feature vector corresponding to an attribute value for at least one attribute of the skin based on the first feature vector and a skin recognizer; And determining a grade of the skin based on the second feature vector and the second skin recognizer. The first skin recognizer is a data set representing a correlation between a first feature vector corresponding to each of a plurality of skin images and a second feature vector corresponding to one or more attribute values of each of the plurality of skin , The second skin recognizer may be a data set expressing a correlation between a second feature vector corresponding to one or more attribute values of each of a plurality of skin and a third feature vector corresponding to a class of each of the plurality of skin.

Wherein the attribute includes a degree of flushing of the skin, a degree of wrinkles of the skin, and a degree of flatness of the skin, and the second feature vector includes at least one vector component corresponding to the degree of flushing, . ≪ / RTI >

The grade of the skin may include at least one or more grades depending on the management need for the skin. Wherein the at least one or more classes may include a class 0 requiring no special care, a class 1 requiring general management, a class 2 requiring professional management, and a class 3 urgently required for treatment.

The generating of the first feature vector may generate a first vector based on the skin image and at least one physical quantity of the skin.

Wherein the generating the first feature vector comprises: recognizing a face region in the skin image; And generating the first feature vector based on the image corresponding to the recognized face region.

The method of evaluating skin condition according to an embodiment of the present invention may include a step of generating a skin image based on one or more training skin images in which an attribute value of at least one attribute of skin is labeled, Training the first skin recognizer; And training the second skin recognizer based on one or more training skin information that includes a property value for at least one attribute of the skin to which the skin's rating is to be tagged.

The method of evaluating skin condition according to an embodiment of the present invention may further include the step of providing at least one of a property value for each attribute corresponding to the second feature vector and a rating of the skin, ; ≪ / RTI >

According to the present invention, a method and program for evaluating a skin condition can be implemented that can confirm the attribute value and the skin rating of one or more skin attributes from a skin image.

Also, it is possible to implement a skin condition evaluation method and a program for confirming an attribute value of each attribute of the skin and determining an objective rating of the skin on the basis thereof, so that appropriate measures can be taken for the skin.

Furthermore, the present invention can implement a skin condition evaluation method and a program that can integrally provide a skin condition analysis and a solution based on each condition.

1 schematically shows a skin condition evaluation system according to an embodiment of the present invention.
2 schematically shows a configuration of a skin condition evaluating apparatus according to an embodiment of the present invention.
3 is a diagram for explaining a method of generating a skin recognizer according to an embodiment of the present invention.
FIG. 4 is a diagram for explaining a method of determining a skin grade using a skin recognizer according to an embodiment of the present invention. Referring to FIG.
5 is a view for explaining a method of generating a first vector by the skin condition evaluating apparatus according to an embodiment of the present invention.
6 is a flowchart illustrating a skin condition evaluation method performed by the skin condition evaluating apparatus according to an embodiment of the present invention.
FIG. 7 is an illustration of a screen displayed on a user terminal according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below, but may be implemented in various forms.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. In the following examples, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In the following embodiments, terms such as inclusive or possessive are intended to mean that a feature, or element, described in the specification is present, and does not preclude the possibility that one or more other features or elements may be added. In the drawings, components may be exaggerated or reduced in size for convenience of explanation. For example, the sizes and shapes of the respective components shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to those shown in the drawings.

1 schematically shows a skin condition evaluation system according to an embodiment of the present invention.

Referring to FIG. 1, a skin condition evaluation system according to an embodiment of the present invention may include a server 100, a user terminal 200, an external device 300, and a communication network 400 connecting them.

The skin condition evaluation system according to an embodiment of the present invention can allow the server 100 to receive an image of skin from the user terminal 200 and / or the external device 300, and to evaluate the status of the skin. The skin condition assessment system may also allow the server 100 to acquire one or more skin images labeled with attributes and ratings from the user terminal 200 and / or the external device 300, and train the skin recognizer based thereon. A more detailed description will be given later.

In the present invention, 'skin' is a tissue of a human body constituting the surface of a human body, and may be an object to be determined for an attribute value or a grade. The skin may be the skin of the face or the skin of various parts of the human body. Hereinafter, for convenience of explanation, it is assumed that the skin is the skin of the face.

In the present invention, the term 'skin property' may mean various factors underlying skin evaluation. For example, the properties of the skin may mean any one of the degree of flushing of the skin, the degree of wrinkling of the skin, and the flatness of the skin.

At this time, the degree of redness of the skin means that the skin is reddish, and the more reddish the skin, the higher the degree of redness. The degree of redness of the skin can have meaning as an element to judge the color of the skin.

The degree of wrinkling of the skin may mean the degree to which the skin folds finely due to a decrease in elasticity of the skin. At this time, the degree of the wrinkles of the skin is high, which may mean that the area of the skin folded finely is large or the degree of folded finely is large.

On the other hand, the flatness of the skin may mean the roughness of the skin due to factors other than the bending due to the wrinkles of the skin. For example, the flatness of the skin may mean the roughness of the skin due to the pores, or the degree of ruggedness of the skin due to scratches, dots, scars, etc. in a specific area.

However, the above-mentioned attributes of the skin are merely illustrative, and the spirit of the present invention is not limited thereto. It is obvious that the skin can be used as an attribute of the skin of the present invention if it can represent the state of the skin.

In the present invention, the 'grade of skin' may be a value determined according to the necessity of management of the skin based on the above-mentioned property of the skin. Grades are, for example, four grades, which may include a zero grade that requires no special care, a first grade that requires general care, a second grade that requires professional care, and a third grade that is urgently needed. However, such classification of grades is exemplary and skin can be classified into more or less grades. At this time, the grade of the skin can be calculated by taking into consideration the above-mentioned properties of the skin in various ways.

In the present invention, the user terminal 200 may refer to various devices capable of transmitting an image of skin to the server 100 and receiving the attribute value or skin rating of the skin from the server 100. In this case, the terminal may be the portable terminal 201 or the personal computer 202.

The user terminal 200 may include display means for displaying contents or the like, and input means for obtaining a user's input on the contents. At this time, the input means and the display means can be configured in various ways. For example, the input means may include, but is not limited to, a keyboard, a mouse, a trackball, a microphone, a button, a touch panel,

Although not shown in the figure, the user terminal 200 may be used in conjunction with a skin diagnostic device (not shown) that obtains various information about the skin.

The skin diagnosis apparatus (not shown) irradiates the skin with at least one wavelength of light, acquires an image of the skin with respect to light of each wavelength, and transmits the skin image to the server 100 through the user terminal 200. In addition, the skin diagnosis apparatus (not shown) may acquire skin images at various angles according to a request received from the user terminal 200 from the server 100, and may transmit the skin images to the server 100.

In addition to the image of the skin, the skin diagnosis device (not shown) may acquire information on the environment in which the skin is exposed and transmit the information to the server 100. For example, the skin diagnosis apparatus (not shown) may measure the temperature, humidity, illuminance, color temperature and ultraviolet ray at the time of photographing the skin image, and transmit the result to the server 100.

Of course, the skin diagnosis device (not shown) may measure the physical quantity of the skin itself and transmit it to the server 100. For example, the skin diagnosis apparatus (not shown) measures the elasticity of the skin, the degree of water content of the skin, the thickness of the keratin skin of the skin, the thickness of the makeup painted on the skin, etc. based on various sensors provided in the skin diagnosis apparatus And may transmit it to the server 100.

The skin diagnosis device (not shown) may be a device separate from the user terminal 200 as described above. For example, the skin diagnosis device (not shown) may be connected to the user terminal 200 through a wired / wireless network. In this case, the user terminal 200 may include a skin diagnosis device (not shown) Lt; / RTI >

Meanwhile, the skin diagnosis apparatus (not shown) may be an apparatus included in the user terminal 200. In other words, the user terminal 200 according to an embodiment of the present invention can operate as the aforementioned skin diagnosis apparatus. However, the present invention is not limited thereto.

The external device 300 may refer to various devices that transmit and receive data to and from the server 100 and the user terminal 200 via the communication network 400. [

According to an embodiment of the present invention, the external device 300 may be a device for providing a training skin image or training skin information for training a skin recognizer provided in the server 100. For example, the external device 300 may be a server of a skin care hospital to provide an image of the skin and diagnostic information about the skin (in particular, an evaluation or rating of the above-described skin property). The external device 300 may be a single number or a plurality.

According to another embodiment of the present invention, the external device 300 may be an apparatus for determining an attribute value or a rating of a skin image acquired by the external device 300 itself. For example, the external device 300 may be a device provided in a dermatologist or the like to provide skin information for a user. However, both of the above-described cases are exemplary, and the spirit of the present invention is not limited thereto.

In the present invention, the communication network 400 connects the server 100, the user terminal 200, and the external device 300. For example, the communication network 400 provides a connection path so that the user terminal 200 can transmit and receive packet data after connecting to the server 100. The communication network 400 may be a wired network such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs (Integrated Service Digital Networks), wireless LANs, CDMA, Bluetooth, But the scope of the present invention is not limited thereto.

In the present invention, the server 100 can receive an image of a skin to know an attribute value or a rating from the user terminal 200 and / or the external device 300, and determine an attribute value or a rating of the skin. Of course, prior to determining the attribute value or rating of the skin, the server 100 obtains the training skin image or training skin information from the user terminal 200 and / or the external device 300, . For this, the server 100 may include a skin condition evaluating apparatus according to an embodiment of the present invention.

FIG. 2 schematically shows a configuration of a skin condition evaluating apparatus 110 according to an embodiment of the present invention.

Referring to FIG. 2, a skin condition evaluating apparatus 110 according to an embodiment of the present invention may include a communication unit 111, a controller 112, and a memory 113. Also, although not shown in the figure, the skin condition evaluation apparatus 110 according to an embodiment of the present invention may further include an input / output unit, a program storage unit, and the like.

The communication unit 111 is connected to the communication unit 111 through hardware necessary for the skin condition evaluation apparatus 110 to transmit and receive a signal such as a control signal or a data signal through a wired connection with another network apparatus such as the user terminal 200 and / And software.

The control unit 112 may include any kind of device capable of processing data, such as a processor. Herein, the term " processor " may refer to a data processing apparatus embedded in hardware, for example, having a circuit physically structured to perform a function represented by a code or an instruction contained in the program. As an example of the data processing apparatus built in hardware, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an ASIC (Application-Specific Integrated Circuit, and an FPGA (Field Programmable Gate Array), but the scope of the present invention is not limited thereto.

The memory 113 performs a function of temporarily or permanently storing data processed by the skin condition evaluating apparatus 110. [ The memory may include a magnetic storage medium or a flash storage medium, but the scope of the present invention is not limited thereto.

Although the skin condition evaluating apparatus 110 is provided in the server 100 in the following description, the skin condition evaluating apparatus 110 may be provided separately from the server 100 according to the role allocation.

As described above, the server 100, that is, the skin condition evaluating apparatus 110 receives an image of a skin to determine an attribute value or a rating from the user terminal 200 and / or the external apparatus 300, Lt; RTI ID = 0.0 > and / or < / RTI > The skin condition evaluating device 110 also obtains one or more training skin images or training skin information from the user terminal 200 and / or the external device 300 with one or more attribute values and ratings, Can be trained.

Hereinafter, a description will first be given of how the skin condition evaluating apparatus 110 generates and / or updates a skin recognizer from one or more training skin images or training skin information, and based on the generated skin recognizer, I will explain how to determine the grade later.

<Method of generating / updating skin recognizer of skin condition evaluating device 110>

In the present invention, the 'skin recognizer' is a concept that includes the first skin recognizer and the second skin recognizer. Also 'generating' a skin recognizer can mean not only configuring the skin recognizer for the first time, but also updating the configured skin recognizer.

The first skin recognizer may be a data set representing a correlation between a first feature vector corresponding to each of a plurality of images of the skin and a second feature vector corresponding to one or more attribute values of each of the plurality of skin . Wherein the first feature vector may be generated from each of a plurality of skin images.

According to an embodiment of the present invention, the first skin recognizer may correspond to a first feature vector corresponding to each of a plurality of images of the skin, generated by a machine learning technique, and one or more attribute values of each of the plurality of skin (I.e., mapping information between the first feature vector and the second feature vector).

Meanwhile, the second skin recognizer may be a data set expressing a correlation between a second feature vector corresponding to one or more attribute values of each of a plurality of skin and a third feature vector corresponding to a degree of each of the plurality of skin.

Similarly to the first skin recognizer, the second skin recognizer also includes a second feature vector corresponding to one or more attribute values of each of the plurality of skin generated by the Machine Learning technique, and a second feature vector corresponding to the class of each of the plurality of skin The correlation between the three feature vectors, that is, the mapping information of the two.

The skin condition evaluating apparatus 110 may include a logistic regression, a decision tree, a nearest-neighbor classifier, a kernel discriminate analysis, a neural network, a support vector machine, a random forest, and a Boosted tree to classify a plurality of training skin images and / And / or &lt; / RTI &gt;

The skin condition evaluating apparatus 110 may use an algorithm and / or a method such as a linear regression, a regression tree, a kernel regression, a support vector regression, and a deep learning to predict an attribute value or a grade of a skin.

In addition, the skin condition evaluating apparatus 110 may use algorithms and / or methods such as Principal component analysis, Non-negative matrix factorization, Independent component analysis, Manifold learning, and SVD for vector calculation.

The skin condition evaluating apparatus 110 may use algorithms and / or methods such as k-means, hierarchical clustering, mean-shift, and self-organizing maps (SOMs) for grouping images of a plurality of skin.

The skin condition evaluating apparatus 110 may use algorithms and / or methods such as Bipartite cross-matching, n-point correlation two-sample testing, and minimum spanning tree for data comparison.

However, the above-described algorithms and / or schemes are illustrative and not intended to limit the scope of the present invention.

On the other hand, the skin recognizer may be a data set as described above. In other words, the skin recognizer may be a data set composed of a plurality of numbers such as a matrix.

FIG. 3 is a view for explaining a method of generating a skin recognizer by the skin condition evaluating apparatus 110 according to an embodiment of the present invention.

3, the controller 112 of the skin condition evaluating apparatus 110 according to an embodiment of the present invention includes one or more training skin members 331 and 332 for generating the first skin recognizer 331 and the second skin recognizer 332, An image 310 may be obtained from the user terminal 200 and / or the external device 300.

At this time, the training skin image 310 may have an attribute value for at least one attribute and a rating of the skin. In other words, the training skin image 310 is an image for the skin that already has its attribute value and rating, and may be the information used for the generation and / or training of the skin recognizer.

For example, in the case of the first training skin image 311, the attribute value 312 including the flushing degree, wrinkle degree and flatness of the skin image 311 and the skin grade 313 may be marked. In this case, the degree of flushing, the degree of wrinkling and the degree of flatness, respectively, may be numerical values indicating the strength of each property such as 0.1 and 0.8. Also, the skin may have a grade of 'first class' or 'third class' and one of the above four classes.

In addition to the attribute values for the three attributes described above, the attribute value 312 may include attribute values for skin age (for example, skin age 30 years), sex (male), race (for example, Asian), disease As shown in FIG. However, the present invention is not limited thereto.

The controller 112 according to an embodiment of the present invention may acquire the training skin image 310 from the external device 300 or acquire the training skin image 310 from the memory 113 of the skin condition evaluating device 110 . For example, the control unit 112 may acquire a training skin image 310 from a medical history stored in a server of a specific dermatologist. At this time, the medical history stored in the dermatology server may be written by an expert (for example, a dermatologist). In other words, the training skin image 310 may be based on data that an expert has evaluated the skin. In addition, the control unit 112 may acquire the training skin image 310 from the history of determining the attribute value or the grade of another skin image in the past.

The controller 112 according to an embodiment of the present invention may generate a first feature vector from each of the training skin images 310 acquired by the above-described process. For example, the control unit 112 may generate a first feature vector 321 from the first training skin image 311. At this time, the control unit 112 may generate the first feature vector 321 based on the image analysis on the skin image (311).

For example, the control unit 112 can recognize the face region in the skin image 311 for more accurate evaluation of the skin condition, and will be described with reference to FIG. The control unit 112 can determine the face area in the image 510 by using the recognition method of contour ratio and the recognition method of the contour. The control unit 112 may generate the first feature vector 520 based on the image 511 corresponding to the recognized face region. In other words, the control unit 112 can generate the first feature vector 520 only from the image 511, which is the object of the state evaluation, except for the region not subject to the state evaluation in the skin image 510. [ Thus, the present invention can exclude the elements of inaccuracy of skin condition evaluation according to the acquisition state or method of image.

Meanwhile, in generating the first feature vector, the controller 112 may generate the first feature vector by further considering the physical quantity of at least one physical quantity of the skin or the environment in which the skin is exposed.

Images only contain visual information. Thus, from the image, it may be difficult to obtain the rest of the information other than the visual information about the skin (for example, the elasticity of the skin, the moisture content of the skin, the thickness of the skin, the thickness of the makeup painted on the skin, etc.). The controller 112 according to an embodiment of the present invention may generate the first feature vector by considering at least one physical quantity of the skin in generating the first feature vector.

Also, the result may be different depending on the acquisition environment. For example, when an image is acquired under a lighting condition of a specific color, the color of the skin on the image may be different from the color of the original skin. The controller 112 according to an embodiment of the present invention may generate the first feature vector by considering the physical quantity of the environment in which the skin is exposed when the first feature vector is generated.

On the other hand, one or more physical quantities for the skin or physical quantities for the environment in which the skin is exposed may be obtained by the above-described skin diagnosis device (not shown) and / or the user terminal 200.

Thus, the present invention can evaluate the condition of the skin more accurately in consideration of various environmental factors.

Referring back to FIG. 3, the controller 112 according to an embodiment of the present invention calculates the attribute value (s) of at least one attribute of the first feature vector 321 and the training skin image 311 generated by the above- It is possible to update the first skin recognizer 331 based on the first skin recognizer 331. The attribute value 312 for at least one attribute of the skin may be in the form of a second feature vector 322. In other words, the training skin image 311 may be marked with a second feature vector 322 corresponding to an attribute value 312 for at least one attribute of the skin. Such a second feature vector 322 may be generated through a separate transformation process on the attribute value 312 for at least one attribute of the skin.

As described above, the first skin recognizer 331 includes a first feature vector corresponding to each of a plurality of images of the skin, generated by a machine learning technique, and a second feature vector corresponding to one or more attribute values of each of the plurality of skin The correlation between the second feature vectors, i.e., the mapping information of both of them.

The control unit 112 according to an embodiment of the present invention includes the first feature vector 321 generated by the above process and the attribute value 312 (i.e., the second feature vector 322) for at least one attribute of the skin, The first skin recognizer 331 can be updated in such a manner that the data set of the first skin recognizer 331 is updated so that the first skin recognizer 331 can correspond to each other. When the data set of the first skin recognizer 331 is composed of a plurality of coefficients as shown in FIG. 3, the control unit 112 outputs the first feature vector 321 and the second feature vector 322, The first skin recognizer 331 can adjust the coefficients constituting the first skin recognizer 331 based on

The control unit 112 repeatedly generates a first feature vector from a plurality of training skin images, repeatedly obtains an attribute value (i.e., a second feature vector) for at least one attribute of the skin, It is possible to acquire the first skin recognizer 331 that provides more accurate skin attribute values.

The controller 112 according to an embodiment of the present invention then updates the second skin recognizer 332 based on the second feature vector 322 and the skin grade 313 included in the training skin image information, can do. The skin grade 313 may then be in the form of a third feature vector 323. In other words, the training skin image 311 may be marked with a third feature vector 323 corresponding to the skin level. The third feature vector 323 may also be generated through a separate conversion process for the skin class 313.

As described above, the second skin recognizer 332 also includes a second feature vector corresponding to one or more attribute values of each of the plurality of skin generated by the machine learning technique, and a second feature vector corresponding to the class of each of the plurality of skin The correlation between the three feature vectors, that is, the mapping information of the two.

The control unit 112 according to an embodiment of the present invention may be configured such that the second feature vector 322 and the skin grade 313 (i.e., the third feature vector 323) The second skin recognizer 332 can be updated in such a manner that the data set of the recognizer 332 is updated. 3, the control unit 112 determines whether the second feature vector 322 and the third feature vector 323 are included in the second feature vector 322. For example, when the data set of the second skin recognizer 332 is composed of a plurality of coefficients, And adjusts the coefficients constituting the second skin recognizer 332 based on the second skin recognizer 332. [

The control unit 112 can acquire the second skin recognizer 332 that repeatedly obtains the second feature vector and the third feature vector and repeatedly updates the coefficients based thereon to provide a more accurate skin rating .

Thus, the present invention can obtain a first skin recognizer 331 that provides attribute values with high accuracy and a second skin recognizer 332 that provides a high accuracy rating.

&Lt; Method of determining the property value of skin and the degree of skin &

In the foregoing, a method of generating skin recognizers 331 and 332 by the skin condition evaluating apparatus 110 has been described with reference to FIG. Hereinafter, a method for determining the attribute values and the grades of the skin to be evaluated by the skin condition evaluating apparatus 110 generated by the above-described process will be described.

Hereinafter, it is assumed that the skin recognizers 331 and 332 are generated by the skin condition evaluating apparatus 110 according to the above-described process.

FIG. 4 illustrates a method of determining a property value and a rating of a skin to be evaluated by the skin condition evaluator 110 according to an embodiment of the present invention using the skin recognizers 331 and 332. FIG.

Referring to FIG. 4, the controller 112 of the skin condition evaluating apparatus 110 according to an embodiment of the present invention may receive the skin image 411 of the skin to be evaluated.

The control unit 112 may receive the skin image 411 from the user terminal 200 or receive the skin image 411 from the external device 300. [

The controller 112 according to an embodiment of the present invention may generate a first feature vector 421 for the skin image 411. [ As described in FIG. 3, the control unit 112 may generate the first feature vector based on the image analysis on the skin image 411. FIG. For example, the control unit 112 can recognize the face region in the skin image 411 for more accurate evaluation of the skin condition, and will be described with reference to FIG. The control unit 112 can determine the face area in the image 510 by using the recognition method of contour ratio and the recognition method of the contour. The control unit 112 may generate the first feature vector 520 based on the image 511 corresponding to the recognized face region. In other words, the control unit 112 can generate the first feature vector 520 only from the image 511, which is the object of the state evaluation, except for the region not subject to the state evaluation in the skin image 510. [ Thus, the present invention can exclude the elements of inaccuracy of skin condition evaluation according to the acquisition state or method of image.

Meanwhile, in generating the first feature vector, the controller 112 may generate the first feature vector by further considering the physical quantity of at least one physical quantity of the skin or the environment in which the skin is exposed.

Images only contain visual information. Thus, from the image, it may be difficult to obtain the rest of the information other than the visual information about the skin (for example, the elasticity of the skin, the moisture content of the skin, the thickness of the skin, the thickness of the makeup painted on the skin, etc.). The controller 112 according to an embodiment of the present invention may generate the first feature vector by considering at least one physical quantity of the skin in generating the first feature vector.

Also, the result may be different depending on the acquisition environment. For example, when an image is acquired under a lighting condition of a specific color, the color of the skin on the image may be different from the color of the original skin. The controller 112 according to an embodiment of the present invention may generate the first feature vector by considering the physical quantity of the environment in which the skin is exposed when the first feature vector is generated.

On the other hand, one or more physical quantities for the skin or physical quantities for the environment in which the skin is exposed may be obtained by the above-described skin diagnosis device (not shown) and / or the user terminal 200.

Thus, the present invention can evaluate the condition of the skin more accurately in consideration of various environmental factors.

Referring again to FIG. 4, the control unit 112 according to an embodiment of the present invention may correspond to an attribute value of at least one attribute of the skin based on the first feature vector 421 and the first skin recognizer 331 The second feature vector 422 may be determined.

As described above, the first skin recognizer 331 includes a first feature vector corresponding to each of a plurality of images of the skin, generated by a machine learning technique, and a second feature vector corresponding to one or more attribute values of each of the plurality of skin The correlation between the second feature vectors, i.e., the mapping information of both of them.

The controller 112 according to an embodiment of the present invention inputs the first feature vector 421 generated from the skin image 411 to the first skin recognizer 331 and outputs the second feature vector 422 as the output result, Can be obtained. Wherein the second feature vector 422 may comprise a vector component corresponding to each of at least one attribute of the skin. For example, the second feature vector 422 may include two vector components corresponding to the flushing degree of the skin, six vector components corresponding to the degree of skin wrinkles, and three vector components corresponding to the flatness of the skin have.

Thus, the present invention can obtain an attribute value for one or more attributes of the skin from the skin image 411. At this time, the attribute value itself can not only have a meaning as an index indicating the state of the skin, but also can have a meaning as a basis for determining the skin's level.

Then, the controller 112 according to an embodiment of the present invention can determine the skin level based on the second feature vector 422 and the second skin recognizer 332. [

As described above, the second skin recognizer 332 includes a second feature vector corresponding to one or more attribute values of each of the plurality of skin generated by the Machine Learning technique, and a second feature vector corresponding to the class of each of the plurality of skin The correlation between the three feature vectors, that is, the mapping information of the two.

Therefore, the controller 112 according to the embodiment of the present invention inputs the second feature vector 422 obtained by the above-described process to the second skin recognizer 332 and outputs the third feature vector 431 as the output result Can be obtained. At this time, the third feature vector 431 may have a meaning as a vector representing the skin's level. Of course, the third feature vector 431 may include a skin component and a corresponding vector component.

Thus, the present invention can confirm the attribute value of each attribute of the skin and determine the objective rating of the skin on the basis thereof.

The controller 112 according to an embodiment of the present invention may provide at least a part of the information corresponding to each of the second feature vector and the third feature vector determined by the above process to the user terminal 200. [ At this time, the information corresponding to the second feature vector may be an attribute value for each attribute of the skin. Further, the information corresponding to the third characteristic vector may be an evaluation value of the skin.

In an alternative embodiment, the controller 112 may provide at least some of the information corresponding to the second feature vector and the third feature vector determined by the above-described procedure to the external device 300.

In this case, the information corresponding to the third feature vector, i.e., the evaluation level of the skin, may have a meaning as an overall state of the skin, and the information corresponding to the second feature vector, It can have meaning as individual information that tells you what part of the treatment needs improvement.

In addition, the controller 112 according to an embodiment of the present invention may determine a recommendation management product based on the attribute values and the skin class for each attribute of the skin, and further provide the recommendation management product to the user terminal 200. [ At this time, the recommendation management product may include at least one of a treatment service for managing skin, a product for managing skin, and an educational content for managing skin.

Thus, the present invention can integrally provide a state solution based on each state in addition to the state analysis of the skin.

On the other hand, the control unit 112 receives the user's selection information for the recommended management product provided to the user terminal 200 by the above-described process, and based on the selection information of the user, And transmit the rating to a third server (not shown). At this time, the third server (not shown) may be the server of the management subject of the recommended management product.

For example, when receiving the selection information for 'treatment service for skin management' from the user terminal 200, the control unit 112 may be a server of the medical institution providing the treatment service, Lt; / RTI &gt; At this time, the control unit 112 may transmit the skin image together. The transmitted information can be used in a series of processes (eg, placement of a specialist) for the treatment of the skin.

 In addition, when receiving the selection information for 'product or content for management of skin' from the user terminal 200, the control unit 112 transmits the attribute value and the skin rating Lt; / RTI &gt;

Thus, the present invention not only enables accurate diagnosis of the skin from the image of the skin, but also provides an appropriate treatment method depending on the skin condition in conjunction with the subsequent procedure.

6 is a flowchart illustrating a skin condition evaluation method performed by the skin condition evaluation apparatus 110 according to an embodiment of the present invention. Hereinafter, the description of the contents overlapping with those described with reference to FIGS. 1 to 5 will be omitted. In the following, it is assumed that the skin recognizers 331 and 332 are generated by the skin condition evaluating apparatus 110 by the above-described process. In other words, it is assumed that the first skin recognizer 331 and the second skin recognizer 332 are trained by the skin condition evaluating apparatus 110 in accordance with S61 and S62.

The skin condition evaluating apparatus 110 according to an embodiment of the present invention can receive the skin image of the skin to be evaluated (S63)

4, the skin condition evaluating apparatus 110 may receive the skin image 411 from the user terminal 200 and receive the skin image 411 from the external apparatus 300 .

The skin condition evaluating apparatus 110 according to an embodiment of the present invention may generate a first feature vector 421 for the skin image 411. [ As described in FIG. 3, the skin condition evaluating apparatus 110 can generate the first feature vector based on the image analysis on the skin image 411. [

For example, the skin condition evaluating apparatus 110 can recognize the face area in the skin image 411 for more accurate evaluation of the skin condition, and will be described with reference to FIG. The skin condition evaluating apparatus 110 can determine the face area in the image 510 by using the recognition method of contour ratio and the recognition method of contour. The skin condition evaluation apparatus 110 may generate the first feature vector 520 based on the image 511 corresponding to the recognized face region. In other words, the skin condition evaluating apparatus 110 can generate the first feature vector 520 only from the image 511 to be subjected to the condition evaluation, excluding the region not subject to the condition evaluation in the skin image 510 have. Thus, the present invention can exclude the elements of inaccuracy of skin condition evaluation according to the acquisition state or method of image.

Meanwhile, in generating the first feature vector, the skin condition evaluating apparatus 110 may generate the first feature vector by further considering the physical quantity of at least one physical quantity of the skin or the environment in which the skin is exposed.

Images only contain visual information. Thus, from the image, it may be difficult to obtain the rest of the information other than the visual information about the skin (for example, the elasticity of the skin, the moisture content of the skin, the thickness of the skin, the thickness of the makeup painted on the skin, etc.). The skin condition evaluating apparatus 110 according to an embodiment of the present invention may generate the first feature vector by considering at least one physical quantity of the skin in generating the first feature vector.

Also, the result may be different depending on the acquisition environment. For example, when an image is acquired under a lighting condition of a specific color, the color of the skin on the image may be different from the color of the original skin. The skin condition evaluating apparatus 110 according to an embodiment of the present invention may generate the first feature vector considering the physical quantity of the skin exposed environment in generating the first feature vector.

On the other hand, one or more physical quantities for the skin or physical quantities for the environment in which the skin is exposed may be obtained by the above-described skin diagnosis device (not shown) and / or the user terminal 200.

Thus, the present invention can evaluate the condition of the skin more accurately in consideration of various environmental factors.

Next, the skin condition evaluating apparatus 110 according to an embodiment of the present invention calculates a skin condition based on the first feature vector 421 and the first skin recognizer 331 based on the attribute values of at least one attribute of the skin, The feature vector 422 can be determined (S65)

As described above, the first skin recognizer 331 includes a first feature vector corresponding to each of a plurality of images of the skin, generated by a machine learning technique, and a second feature vector corresponding to one or more attribute values of each of the plurality of skin The correlation between the second feature vectors, i.e., the mapping information of both of them.

Therefore, the skin condition evaluating apparatus 110 according to an embodiment of the present invention inputs the first feature vector 421 generated from the skin image 411 to the first skin recognizer 331, Gt; 422 &lt; / RTI &gt; Wherein the second feature vector 422 may comprise a vector component corresponding to each of at least one attribute of the skin. For example, the second feature vector 422 may include two vector components corresponding to the flushing degree of the skin, six vector components corresponding to the degree of skin wrinkles, and three vector components corresponding to the flatness of the skin have. Thus, the present invention can obtain an attribute value for one or more skin attributes from a skin image. At this time, the attribute value itself can not only have the meaning as an index indicating the state of the skin, but also can have a meaning as a basis for determining the skin's level.

Then, the skin condition evaluating apparatus 110 according to an embodiment of the present invention can determine the skin level based on the second feature vector 422 and the second skin recognizer 332. (S66)

As described above, the second skin recognizer 332 includes a second feature vector corresponding to one or more attribute values of each of the plurality of skin generated by the Machine Learning technique, and a second feature vector corresponding to the class of each of the plurality of skin The correlation between the three feature vectors, that is, the mapping information of the two.

Therefore, the skin condition evaluating apparatus 110 according to an embodiment of the present invention inputs the second feature vector 422 obtained by the above-described procedure to the second skin recognizer 332 and outputs the third feature vector 422 431). At this time, the third feature vector 431 may have a meaning as a vector representing the skin's level. Of course, the third feature vector 431 may include a skin component and a corresponding vector component.

Thus, the present invention confirms the attribute value of each attribute of the skin, and based on this, an objective rating of the skin is determined so that appropriate measures can be taken for the skin.

The skin condition evaluating apparatus 110 according to an embodiment of the present invention may provide at least a part of the information corresponding to each of the second feature vector and the third feature vector determined by the above process to the user terminal 200 (S67) At this time, the information corresponding to the second feature vector may be an attribute value for each attribute of the skin. Further, the information corresponding to the third characteristic vector may be an evaluation value of the skin.

In an alternative embodiment, the skin condition evaluating apparatus 110 may provide at least part of the information corresponding to each of the second feature vector and the third feature vector determined by the above-described process to the external device 300.

In this case, the information corresponding to the third feature vector, i.e., the evaluation level of the skin, may have a meaning as an overall state of the skin, and the information corresponding to the second feature vector, It can have meaning as individual information that tells you what part of the treatment needs improvement.

In addition, the skin condition evaluating apparatus 110 according to an embodiment of the present invention determines a recommendation management product on the basis of the attribute value and the skin rating of each attribute of the skin described above, and further provides the recommendation management product to the user terminal 200 . At this time, the recommendation management product may include at least one of a treatment service for managing skin, a product for managing skin, and an educational content for managing skin.

Thus, the present invention can provide an integrated solution for skin condition analysis as well as a solution based on each state.

On the other hand, the skin condition evaluating apparatus 110 receives the user's selection information on the recommended management product provided to the user terminal 200 by the above-described process, and based on the selection information of the user, Value and skin rating to a third server (not shown). At this time, the third server (not shown) may be the server of the management subject of the recommended management product.

For example, when the selection information for 'treatment service for skin management' is received from the user terminal 200, the skin condition evaluating unit 110 is a server of the medical institution providing the treatment service, And skin rating. At this time, the skin condition evaluating apparatus 110 may transmit the skin image together. The transmitted information can be used in a series of processes (eg, placement of a specialist) for the treatment of the skin.

 In addition, when receiving the selection information on 'product or content for management of skin' from the user terminal 200, the skin condition evaluating device 110 determines whether the attribute value And skin rating.

Thus, the present invention not only enables accurate diagnosis of the skin from the image of the skin, but also provides an appropriate treatment method depending on the skin condition in conjunction with the subsequent procedure.

FIG. 7 is an illustration of a screen 700 displayed on a user terminal 200 according to an embodiment of the present invention.

7, the screen 700 includes a region 710 for displaying an image of skin to be evaluated by the user, a region 720 for displaying brief information about the skin and a rating, an attribute value An area 730 for displaying a recommendation management product, and areas 740 and 750 for displaying a recommended management product.

Brief information about the skin and the grade displayed in the area 720 in which the grade is displayed may be determined by the control unit 112 based on the second feature vector and the second skin recognizer. Meanwhile, the gender and estimated age to be displayed together may be derived from image analysis or input from the user terminal 200.

One or more attributes displayed in the area 730 where the attribute values for each attribute of the skin are displayed may be based on the second feature vector that the controller 112 determines based on the first feature vector and the first skin recognizer.

The recommended management goods and services displayed in the areas 740 and 750 in which the recommended management goods are displayed may be those determined by the control unit 112 based on the attribute values and the skin class for each attribute of the skin.

Accordingly, the user can easily grasp his / her skin condition through the screen 700, and can determine an appropriate product suitable for his skin condition and purchase.

The embodiments of the present invention described above can be embodied in the form of a computer program that can be executed on various components on a computer, and the computer program can be recorded on a computer-readable medium. At this time, the medium may be a computer-executable program. Examples of the medium include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floptical disk, And program instructions including ROM, RAM, flash memory, and the like.

Meanwhile, the computer program may be designed and configured specifically for the present invention or may be known and used by those skilled in the computer software field. Examples of computer programs may include machine language code such as those produced by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as "essential "," importantly ", etc., it may not be a necessary component for application of the present invention.

Accordingly, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all ranges that are equivalent to or equivalent to the claims of the present invention as well as the claims .

100: Server
110: skin condition evaluation device
111:
112:
113: Memory
200: user terminal
300: External device
400: communication network

Claims (9)

A method of evaluating a condition of a skin based on a skin image,
Generating a first characteristic vector from a skin image;
Determining a second feature vector corresponding to an attribute value for at least one attribute of the skin based on the first feature vector and a skin recognizer; And
Determining a grade of the skin based on the second feature vector and the second skin recognizer,
The first skin recognizer
A data set representing a correlation between a first feature vector corresponding to each of a plurality of skin images and a second feature vector corresponding to one or more attribute values of each of the plurality of skin,
The second skin recognizer
Wherein a correlation between a second feature vector corresponding to one or more attribute values of each of a plurality of skin and a third feature vector corresponding to a grade of each of the plurality of skin is expressed. The method of claim 1, wherein
The attribute
The degree of flushing of the skin, the degree of wrinkling of the skin, and the flatness of the skin,
The second feature vector
And at least one vector component corresponding to each of the degree of flushing, the degree of wrinkling and the degree of flatness, respectively. The method of claim 1, wherein
The grade of the skin
And at least one or more classes according to a management need for the skin. The method of claim 3, wherein
The at least one or more classes
A first grade requiring no special care, a first grade requiring general management, a second grade requiring professional management, and a third grade urgently required for treatment. The method of claim 1, wherein
The step of generating the first feature vector
And generates a first vector based on the skin image and at least one physical quantity of the skin. The method of claim 1, wherein
The step of generating the first feature vector
Recognizing a face region in the skin image; And
And generating the first feature vector based on the image corresponding to the recognized face region. The method of claim 1, wherein
The skin condition evaluation method
Before the step of generating the first feature vector
Training the first skin recognizer based on at least one Training Skin image having an attribute value for at least one attribute of the skin labeled;
And
Training the second skin recognizer based on one or more training skin information that includes a property value for at least one attribute of the skin to which the skin's rating is labeled, . The method of claim 1, wherein
The skin condition evaluation method
After determining the grade of the skin,
Providing at least one of an attribute value for each attribute corresponding to the second feature vector and a rating of the skin. 9. A computer program stored on a medium for carrying out the method of any one of claims 1 to 8 using a computer.

Images