WO2022124746A1

WO2022124746A1 - Skin condition prediction and skin care system and method

Info

Publication number: WO2022124746A1
Application number: PCT/KR2021/018402
Authority: WO
Inventors: 우혜정
Original assignee: 우혜정
Priority date: 2020-12-09
Filing date: 2021-12-07
Publication date: 2022-06-16
Also published as: KR102285912B1

Abstract

A skin condition prediction and skin care system according to an embodiment of the present invention comprises: a skin measurement device which may include elements for checking skin condition, has a built-in camera module to capture images indicating skin condition, produces skin condition information by means of the captured skin images, and, in order to manage skin condition information for each subject, transmits subject information and the skin condition information to a diagnostic server; and a prediction server which is provided with skin condition information from the skin measurement device to diagnose skin condition, may calculate predicted skin change values, is configured so as to provide a skin care program corresponding to the predicted skin change values, and stores, in a database, and provides management programs for each subtopic for skin care.

Description

Skin condition prediction and skin care system and method

The present invention relates to a skin condition prediction and skin care system, and more particularly, to a skin condition prediction and skin care system that diagnoses the skin condition by photographing it with a camera and provides a user-customized skin care program suitable for the skin condition. it's about

In general, the skin pores and pigmentation state are analyzed by processing the skin image. However, since the analyzed result is derived as the state information of pores and/or the state information of pigmentation, it is expected that it will be difficult for the user to directly utilize the analyzed result.

That is, it will be necessary to increase the user's utility by allowing the user to easily know skin condition information from the state of pores and the state of pigmentation.

The skin has elasticity and flexibility. In particular, skin elasticity is an index used to examine skin aging or to analyze health according to constitution. Therefore, in order to examine skin aging and accurately analyze health according to constitution, it is necessary to accurately examine skin elasticity.

In addition, since the existing skin care program only compares the results before and after the skin condition diagnosis, the future skin condition is predicted to provide the skin care that I will receive in the future (future skin condition prediction), and a corresponding skin care program is developed. there is a need to provide

The present invention has been devised to solve the above-mentioned problems, by calculating skin condition information such as skin elasticity, wrinkles, etc. from a skin image, and calculating a skin change prediction value for each age of the subject based on a prediction algorithm, and a corresponding skin care program An object of the present invention is to provide a skin condition prediction and skin care system and a method for proposing a skin condition.

The skin condition prediction and skin care system according to an embodiment of the present invention may include a configuration for checking the skin condition, a camera module is built-in to capture the skin condition, and the skin condition using the captured skin image a skin measuring device that calculates information and transmits subject information and the skin condition information to a diagnosis server to manage skin condition information for each subject; It is possible to receive skin condition information from the skin measuring device to diagnose a skin condition, calculate a predicted skin change value, set to provide a skin care program corresponding to the predicted skin change value, and manage each detailed item for skin care It includes a prediction server that stores and provides programs in a database.

The skin condition information is characterized in that it includes at least one of oil, moisture, temperature, skin pH, pigmentation, and elasticity of the skin.

The prediction server may include: a collecting unit for receiving skin condition information and subject information from a skin measuring device and storing the received skin condition information and subject information in a database; a prediction unit that predicts and calculates skin changes according to age according to age using an artificial intelligence-based prediction algorithm based on skin condition information collected for each subject; and a skin care unit that provides a management program suitable for the subject according to the predicted skin change value of the prediction unit, and provides a subdivided management program according to the type of skin care.

The prediction algorithm is characterized in that it predicts a skin change value according to an increase in the age of the skin condition of the subject based on the current skin condition information of the subject and the average skin condition value of the corresponding age.

The prediction algorithm is characterized in that it compares the actually measured skin change value for each subject with the predicted skin change value and feeds back as much as an error to perform learning.

As for the prediction algorithm, the GAN algorithm is applied to predict the future skin condition from the face image of the subject by image learning according to the change in the aging state, but by learning with the age condition by the conditional GAN algorithm, the skin condition according to the face aging It is characterized in that it is implemented to be predictable.

The skin care program includes elasticity management, facial features/balanced face/face reduction management including V-line lifting, whitening management, wrinkle management/care, moisture management, sensitive soothing management, acne management, skin regeneration, wedding management, swelling management, double chin It is characterized in that it includes at least one of management.

In the skin condition prediction and skin care system, data encryption/decryption technology is applied to the transmission/reception of information in order to protect the information exchanged between the prediction server, the skin measuring device, or the mobile terminal from the risk of hacking from the outside. and a protection unit for performing a lightweight encryption algorithm using identification information as a private key (private key) by giving identification information, which can prove identity to the skin measuring instrument.

A skin condition prediction and skin care method using a skin condition prediction and skin care system according to an embodiment of the present invention includes: photographing a skin image from the skin measuring device, and calculating skin condition information from the skin image; transmitting the skin condition information and subject information to the prediction server; calculating, by the prediction server, a predicted skin condition change value using a prediction algorithm based on the skin condition information and the subject information; and providing a skin care program set according to the calculated predicted skin change value.

The method further includes; using a mobile terminal possessed by a subject who has been treated by an operator according to the skin care program, providing post-treatment evaluation feedback (after treatment) to a prediction server.

In the calculating of the predicted skin condition change value, the prediction unit calculates the predicted change for the skin care factors by age based on the data on the skin condition information collected using the simple linear regression model and the multiple regression model. do it with

When calculating the predicted change for the age-specific skin care factors, a correction value (

) is calculated, and through this data, it is possible to compare before and after skin care procedures, and the correction value is characterized by including management program execution factors, cosmetic use factors, and management factors for environmental indexes.

The skin condition prediction and skin care system of the present invention calculates skin condition information such as skin elasticity, wrinkles, etc. from a skin image, calculates a skin change prediction value for each age of the subject based on a prediction algorithm, and proposes a corresponding skin care program By doing so, there is an advantage that user-customized skin care is possible.

In addition, the prediction accuracy is improved by comparing the predicted value with the actual skin measurement value, and the error is reflected in the prediction algorithm to perform learning. It has the effect of reducing the risk of the degree of effectiveness of the procedure.

1 is a block diagram showing the configuration of a skin condition prediction and skin care system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing the internal configuration of the prediction server of FIG. 1 in detail.

3 is a flowchart of a skin condition prediction and skin care method according to an embodiment of the present invention.

4 is a view showing the change in elasticity by age by way of example.

5 is a diagram illustrating an example of changes in pigmentation according to age.

6 is a view showing the change in pH value by age by way of example.

7 is a view exemplarily showing changes in wrinkles by age.

8 is a diagram illustrating a change in face size according to the degree of eating habits.

9 is a diagram illustrating a change in face length according to a sleep level by way of example.

10 is a block diagram illustrating an algorithm for predicting the amount of skin change using the simple linear regression model and the multiple regression model of the prediction unit.

11 is a graph showing changes due to correction values used when predicting the amount of skin change according to the algorithm of FIG. 10 .

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the spirit of the present invention may add, change, delete, etc. other elements within the scope of the same spirit, and may use other degenerative inventions or the present invention. Other embodiments included within the scope of the invention may be easily proposed, but this will also be included within the scope of the invention. In addition, components having the same function within the scope of the same idea shown in the drawings of each embodiment will be described using the same reference numerals.

1 is a block diagram showing the configuration of a skin condition prediction and skin care system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing the internal configuration of the prediction server 200 of FIG. 1 in detail.

As shown in FIG. 1 , the skin condition prediction and skin care system according to an embodiment of the present invention is connected to the skin measuring device 100 and the skin measuring device 100 through a communication network 400 capable of wired and wireless communication to change the skin. It includes a prediction server 200 for predicting and a block chain server 300 for data management.

The skin measuring device 100 may include a configuration for checking a skin condition, a camera module is built-in to capture a skin condition, and calculates skin condition information using the captured skin image.

The skin condition information may include skin oil content, moisture, temperature, skin pH, pigmentation, elasticity, etc., and the calculation result of the skin condition information may be calculated in the form of a graph as shown in FIGS. 4 to 8 .

In addition, the skin measuring device 100 may receive the age of a subject, a name, a customer identification code, etc. as subject information in order to manage skin condition information for each subject.

In addition, the skin measuring instrument 100 may further include a display unit for displaying and visually confirming the calculated skin condition information, and communication for transmitting the skin condition information and the subject information to the prediction server 200 through the communication network 400 . Modules can be embedded.

Furthermore, the camera module of the skin measuring device 100 includes AdaBoost, Support Vector Machine (SVM), Linear Disciminant Analysis (LDA), main components to increase the recognition rate when photographing a face (skin). An algorithm such as Principal Component Analysis (PCA) may be built-in.

All of these algorithmic techniques identify the region to be recognized based on the appearance, and the face region is detected using a model trained by a set of captured images to be used for training, and various surrounding constraints are overcome through training. As a result, face recognition accuracy and reliability can be improved.

The prediction server 200 may receive skin condition information from the skin measuring device 100 to diagnose or predict a future skin condition, and may build a database 250 to provide a necessary skin care program according to the prediction result.

To this end, the prediction server 200 further includes a collection unit 210 , a prediction unit 220 , a skin care unit 230 , a protection unit 240 , and a database 250 as shown in FIG. 2 .

The collection unit 210 receives skin condition information and subject information from the skin measuring device and stores it in the database 250 , and the skin condition information may be managed to be separately stored for each subject.

The prediction unit 220 may predict and calculate age-specific skin changes according to the increase in age based on skin condition information collected for each subject. Prediction algorithms used for this purpose include, for example, a supporter vector machine advantageous for pattern change learning, CNN. , RNN, and artificial intelligence-based algorithms such as generative adversarial networks (GANs).

In addition, the skin change value by age according to the increase in age is calculated separately for each numerical value (pH, elasticity, pigmentation, wrinkles, face area/length, etc.) included in the skin condition information, so that prediction results for detailed items can be evaluated. can do.

In addition, the prediction algorithm may predict a skin change value according to an increase in the age of the skin condition of the subject based on the current skin condition information of the subject and the average skin condition value of the corresponding age. To this end, the average skin condition numerical statistical value for each age may be stored in the database 250 .

In addition, the prediction algorithm can then perform learning by comparing the actual measured skin change value and the predicted skin change value for each subject and feedback as much as the error.

4 is a view showing the change in elasticity by age, and it can be seen that the elasticity of the skin by age gradually decreases in inverse proportion to age. In addition, since the risk of skin elasticity by age increases rapidly from the age of 40, it can be seen that it is a period that requires intensive management. By reflecting such an average elasticity value, the predicted skin elasticity change value of the subject can be calculated, and when predicting the elasticity change value, it can be adjusted using the correction value, and external factors (customer's current skin condition) , life pattern, amount of sleep, amount of exercise, exposure to sunlight, degree of pollution, smoking, occupational risk, frequency of massage, weight, pore size, amount of sebum secretion, skin texture, skin color, moisture content, etc.) may be included.

5 is a view showing changes in pigmentation by age, it can be seen that the change in pigmentation increases relatively in proportion to age, and it increases rapidly from the 20s, the lower the amount of water in the body, the more severe, and regardless of age, the degree of exposure to sunlight Since pigmentation may appear quickly depending on the condition, when calculating the predicted skin pigmentation change value, the amount of exposure to sunlight and the amount of moisture in the body should be reflected as correction values.

6 is a view showing changes in pH values by age, and shows relatively flat values by age, but increases sharply in teens, and then decreases slightly again from the 20s. If the pH value is 5 to 6, it means normal acidity skin, if it is 6 to 7 it means dry skin, and if it is 4.5 to 4.6, it means weakly acidic skin. Although the predicted pH change value is calculated, the correction value as described above may be reflected and calculated.

7 is a view showing the changes in wrinkles by age, and the wrinkles increase in proportion to the changes in age, but shows that the wrinkles increase rapidly in the 30s and 40s, and the management program provided according to the changes in wrinkles by age is lifting management , elasticity management, and skin regeneration management may be included, and further, it may be recommended to use lifestyle modification, eye cream, nutrition cream, moisture cream, and collagen cream.

8 is a view showing changes in the size of the face according to the degree of eating habits, the y-axis is expressed as a number from 1 to 5 according to the good and bad eating habits, and the change in the face area is shown on the x-axis accordingly. A typical graph is derived.

9 is a diagram showing the change in face length according to the degree of sleep, and the degree of sleep is expressed as a number from 1 to 5 on the y-axis according to good or bad, and the change in face length (unit: mm or cm) is shown on the x-axis, Based on the original face length (the number is 1), it can be seen that the worse the sleep level (the amount of sleep), the shorter the face length, and the better the sleep level, the longer the face length.

The management program according to facial changes can suggest face reduction management, head management, moisture cream, moisture essence, sunscreen, mask pack, improvement of eating habits, and back management.

Furthermore, referring to FIG. 10 , the prediction unit 220 may calculate a predicted change for skin care factors by age based on data about skin condition information collected using a simple linear regression model and a multiple regression model.

Here, the skin care elements may include elements shown as graphs in FIGS. 4 to 9, and changes in elasticity, pigmentation, pH values, wrinkles, eating habits, and sleep level, as well as changes in pore size, sebum secretion, and skin density, which are shown in graphs. , skin texture (roughness) change, skin color change, moisture content, etc. may be further included.

The prediction unit 220 may collect skin measurement data (skin condition information) and use it as an input variable in a simple linear regression model to calculate a change graph for skin care elements, and arithmetic formula 1 for this is as follows can indicate

[Equation 1]

(Here, x is the dependent variable, age, and y is the independent variable, skin care factor (pH, pore size, sebum secretion, skin density, elasticity, etc.)

In addition, the following correction values (

) is calculated by Equation 2, and through this data, it is provided so that before/after skin care procedures can be compared.

[Equation 2]

here

represents the rate of change for the management program, and specifically, it is a numerical value including the management program execution factor, cosmetic use factor, and environmental index management factor.

Management program execution factors may include elasticity factor, fascia factor, periosteum factor, lifting factor, sticker factor, etc. Cosmetic use factors include moisture cream factor, hyaluronic acid factor, collagen factor, oil factor, vitamin factor, sunscreen factor, etc. may be included, and management factors for the environmental index may include sleep time factors, external work exposure factors, alcohol consumption factors, smoking factors, water intake factors, coffee consumption factors, and the like.

In addition, referring to FIG. 11 , when the skin change factors of the average age group are displayed on a straight line graph, the skin condition data of each subject (user to receive skin care) is displayed as a dot, at this time the correction value (

), it can be confirmed that the amount of change is close to the average value, and the numerical value of change, which is comparative data, can be fed back as an input variable of a simple linear regression model or multiple regression model together with skin measurement data for the next procedure.

In addition, whether a skin care program treatment is required can be confirmed through the subject's data. If a dot is located at the bottom of the graph representing the skin change factors of the average age, a management procedure must be performed, and after the management procedure, the correction value (

), it can be confirmed that the amount of change is changed to be close to the average value.

In addition, in the case of a well-managed subject or management procedure that does not require treatment, a dot is placed at the top of the graph, and a separate skin care program is not provided, or a light management prescription for skin maintenance is provided. The line is provided to the operator.

Furthermore, the prediction unit 220 may apply a GAN algorithm to predict a future skin condition from a face image of a subject by image learning according to a change in the aging state when the prediction algorithm is applied. By learning, it can be implemented to predict the skin condition according to face aging.

Specifically, the conditional GAN algorithm is an algorithm that induces the output to be calculated in a desired direction by giving a random condition together with noise. By putting it in a generator and learning it, the desired output can be produced.

In the case of face aging, it is necessary to estimate the old appearance of a specific person by age and not to lose the identity of the face of the original image. By proposing, the conditional GAN can perform identity-preserving of the input image, where condition y is the age.

When reconstructed by concating vector z0, which is just encoding the input image, with condition y0 (age), a recon image that is slightly different from the face of the original image is generated However, if the reconstruction is performed by concating z* and y0 that have undergone Identity Preserving Optimization, an optimized image almost similar to the original can be regenerated. Although the images generated through this are somewhat different from the original image, it is an image that is almost similar to the original image, so that image changes according to age can be derived.

The skin care unit 230 provides a management program suitable for the subject according to the predicted skin change value of the prediction unit 220 , and the management program may be subdivided and provided according to the type of skin care. Skin care types include elasticity management, facial features/balanced face/face reduction management including V-line lifting, whitening management, wrinkle management/care, moisture management, sensitive soothing management, acne management, skin regeneration, wedding management, swelling management, double chin management etc. are included. In addition, the type of skin care may be managed differently for each body part, such as the neck, upper body, lower body, and abdomen. In addition, since these skin care programs are often performed directly by the operator (skin care professional), the prediction server 200 is specified in the form of a message transmitted to the skin measuring device 100 and the management terminal (not shown) possessed by the operator. A management program may be suggested (recommended), and the operator may be guided so that the operation is performed according to the proposed program.

For the skin care program, for example, in the case of people over the age of 40, when the decrease in elasticity becomes severe, management programs for elasticity management, lifting, and moisture management are provided. A management program may be provided to recommend

For those in their 30s and 40s who have severe pigmentation, whitening management, nutrition management, improvement of eating habits (intake of inorganic tea, change in eating patterns, etc.), moisture/nutrition cream management, etc. can be provided as management programs.

For those in their 20's to 30's whose pH value is alkaline, the management program may include improving eating habits, regenerating skin care (maintaining pH balance), head management, moisture essence/cream, and moisture pack management programs.

In addition, the skin care unit 230 provides a management program set in advance according to the predicted change value of the prediction unit 220, but receives evaluation feedback information of the subject for the provided management program, and there is an error in the predicted change value. It may be set to provide a reward when it does not fit or the effect is insignificant.

The reward set value can be quantitatively quantified and reflected by receiving evaluation feedback information as a numerical value such as satisfaction. The skin care unit 230 may communicate with a mobile terminal (not shown) possessed by the subject in order to receive evaluation feedback information from the subject, and for this purpose, a skin care app interworking with the server may be provided to the mobile terminal.

The protection unit 240 provides information (eg, subject information, skin condition information, feedback information, etc.) exchanged between the prediction server 200, the skin measuring device 100, or the mobile terminal from the risk of hacking from the outside. To protect, data encryption/decryption technology can be applied to transmission/reception of information. More specifically, the prediction server 200 performs a lightweight encryption algorithm using the identification information as a private key by giving identification information that can prove identity to the skin measuring instrument 100 . . The lightweight encryption algorithm is an encryption technology designed to be implemented in a limited environment such as a smart device such as the skin measuring device 100. The symmetric key encryption algorithm HIGHT (HIGh security and light weigHT), LEA (Lightweight Encryption), and the hash function LSH (Lightweight Secure Hash), etc. can be used. By encrypting/decrypting information using such a lightweight encryption algorithm, it is possible to prevent illegal control due to external leakage or external hacking of the corresponding data.

The database 250 may store and manage skin condition information, subject information, feedback information, predicted skin change values, etc. for each subject, and may be distributed and stored and managed using the block chain server 300 for data protection. .

First, a skin image is taken from the skin measuring device 100, and skin condition information including oil, moisture, temperature, skin pH, pigmentation degree, elasticity, etc. of the skin is calculated from the skin image (S100, S102).

In addition, the skin condition information calculated by the skin measuring device 100 is transmitted to the prediction server 200 through the communication network 400, and the prediction server 200 predicts by age by a prediction algorithm based on the skin condition information collected for each subject. A skin change value is calculated (S104, S106).

Then, a skin care program such as wrinkle management/whitening management/face reduction management set according to the calculated predicted skin change value is provided (S108).

The subject, who has been treated by the operator according to the skin care program, may provide the evaluation feedback (after the treatment) to the prediction server 200 using a portable terminal having the post-performance evaluation (S110).

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

[Explanation of code]

100 ; skin measuring instrument

200 ; prediction server

210; collector

220 ; predictor

230 ; skin care department

240 ; protection department

250 ; database

300 ; blockchain server

Claims

It may include a configuration for checking the skin condition, and a camera module is built-in to capture the skin condition, calculate skin condition information using the captured skin image, and manage the skin condition information for each subject. , a skin measuring device for transmitting the skin condition information to a diagnosis server;

It is possible to receive skin condition information from the skin measuring device to diagnose a skin condition, calculate a predicted skin change value, set to provide a skin care program corresponding to the predicted skin change value, and manage each detailed item for skin care Prediction server that provides programs by storing them in the database

A skin condition prediction and skin care system comprising a.
The method of claim 1,

The skin condition information is skin condition prediction and skin care system, characterized in that it includes at least one of oil, moisture, temperature, skin pH, pigmentation, and elasticity of the skin.
According to claim 1,

The prediction server

a collecting unit for receiving skin condition information and subject information from the skin measuring device and storing the received skin condition information and subject information in a database;

a prediction unit that predicts and calculates skin changes according to age according to age using an artificial intelligence-based prediction algorithm based on skin condition information collected for each subject;

a skin care unit that provides a management program suitable for a subject according to the predicted skin change value of the prediction unit, and provides a subdivided management program according to a skin care type;

A skin condition prediction and skin care system comprising a.
4. The method of claim 3,

The prediction algorithm is a skin condition prediction and skin care system, characterized in that it predicts the skin change value according to the increase in the age of the skin condition of the subject based on the skin condition information of the current subject and the average skin condition value of the corresponding age .
4. The method of claim 3,

The prediction algorithm compares the actual measured skin change value and the predicted skin change value for each subject thereafter, and feeds back as much as an error to learn the skin condition prediction and skin care system.
4. The method of claim 3,

The prediction algorithm is

Apply the GAN algorithm to predict the future skin condition from the subject's face image by image learning according to the change in aging state, but learn with the age condition by the conditional GAN algorithm to predict the skin condition according to face aging A skin condition prediction and skin care system, characterized in that it is implemented.
4. The method of claim 3,

The skin care program

At least one of elasticity management, facial features/balanced face/face reduction management including V-line lifting, whitening management, wrinkle management/care, moisture management, sensitive soothing management, acne management, skin regeneration, wedding management, swelling management, double chin management Skin condition prediction and skin care system comprising a.
4. The method of claim 3,

Data encryption/decryption technology is applied to the transmission/reception of information to protect the information exchanged between the prediction server, the skin measuring device, or the mobile terminal from the risk of hacking from the outside, and identification is possible with the skin measuring device. The skin condition prediction and skin care system further comprising a protection unit that grants identification information and performs a lightweight encryption algorithm using the identification information as a private key.
In the skin condition prediction and skin care method using the skin condition prediction and skin care system according to any one of claims 1 to 8,

taking a skin image from the skin measuring device and calculating skin condition information from the skin image;

transmitting the skin condition information and subject information to the prediction server;

calculating, by the prediction server, a predicted skin condition change value using a prediction algorithm based on the skin condition information and the subject information;

providing a skin care program set according to the calculated predicted skin change value;

A skin condition prediction and skin care method comprising a.
10. The method of claim 9,

using a mobile terminal possessed by a subject who has been treated by an operator according to the skin care program, and providing post-treatment evaluation feedback (after treatment) to a prediction server;

A skin condition prediction and skin care method further comprising a.
10. The method of claim 9,

Calculating the predicted skin condition change value comprises:

Skin condition prediction and skin care method, characterized in that the predictor calculates the predicted change for the skin care factors by age based on the data on the skin condition information collected using the simple linear regression model and the multiple regression model.
12. The method of claim 11,

When calculating the predicted change for the age-specific skin care factors,

Correction value (
), and using this data to compare before/after skin care procedures,

A skin condition prediction and skin care method, characterized in that the correction value includes a management program execution factor, a cosmetic use factor, and a management factor for an environmental index.