KR20230110973A - Mobile-based personalized skin diagnosis service system - Google Patents

Abstract

It relates to a mobile-based personalized skin diagnosis service system that analyzes skin images uploaded by mobile with an artificial intelligence (AI) engine to predict skin conditions and skin diseases, and provides skin analysis results and treatment methods based on the prediction results and DNA analysis results so that users can manage their skin using mobile. A mobile-based personalized skin diagnosis service system is built, including a skin diagnosis engine that predicts skin conditions and skin diseases based on the results, and a skin image diagnosis result provider that displays the analyzed skin diagnosis results and treatment methods on the screen.

Description

Mobile-based personalized skin diagnosis service system

The present invention relates to a mobile-based personalized skin diagnosis service system, and more particularly, to a mobile-based personalized skin diagnosis service system that analyzes a skin image uploaded to a mobile device with an artificial intelligence (AI) engine to predict skin conditions and skin diseases, and provides skin analysis results and treatment methods based on the predicted results and DNA analysis results so that users can manage their skin using mobile devices.

With the arrival of the 4th industrial revolution era and the acceleration of artificial intelligence technology development, various image data can be precisely processed and processed according to the purpose of use, and application technologies based on identification of images containing body information such as facial recognition and motion recognition are also emerging.

On the other hand, in the field of skin beauty, a skin care method that recommends cosmetics suitable for the skin to the user according to the condition estimated based on the measurement of the user's skin condition is being used.

In order to utilize this method, there are many cases in which cost burdens arise, such as the user having to separately purchase a measuring device with a skin care device or the like or paying a considerable fee for use.

In order to solve this problem, as in Korean Registered Patent Publication No. 10-2211884, a technique for transmitting a Baumann skin type questionnaire (BSTQ) to a user terminal, receiving survey result data for the Baumann skin type questionnaire from the user terminal, and diagnosing the skin type using big data-based artificial intelligence machine learning has been proposed.

However, this prior art is a method of learning patterns inherent in skin conditions, used products, and environmental information and recommending products through skin care analysis using artificial intelligence machine learning based on questionnaire data, and since it is not a method using skin images of actual users, but a method of analyzing skin conditions based on survey data, there is a disadvantage in that accuracy in skin analysis is low.

In addition, there is a disadvantage in that it can only be analyzed for skin types and cannot be analyzed for skin diseases.

Korean Registered Patent No. 10-2211884 (registered on January 28, 2021) (method of providing skin care recommendation service using big data-based artificial intelligence machine learning)

Therefore, the present invention has been proposed to solve various problems occurring in the general skin care method and the prior art as described above, and the skin image uploaded to the mobile is analyzed with an artificial intelligence (AI) engine to predict skin conditions and skin diseases, and based on the predicted results and DNA analysis results, skin analysis results and treatment methods are provided to provide a mobile-based personalized skin diagnosis service system that enables users to manage their skin using mobile devices.

In order to achieve the above object, the "mobile-based personalized skin diagnosis service system" according to the present invention,

an image input unit for uploading a skin image for diagnosis;

A skin diagnosis engine that predicts skin conditions and skin diseases based on image pattern analysis that compares and analyzes learned features using an artificial intelligence engine loaded with respect to images input through the image input unit and DNA analysis results provided by a DNA analysis agency; and

It is characterized in that it comprises a skin image diagnosis result providing unit that displays the skin diagnosis result and treatment method analyzed by the skin diagnosis engine on a screen.

In the above image input unit,

It is characterized in that the diagnostic skin image is provided for real-time analysis and at the same time provided as learning data to the skin care web server.

In the above, the DNA analysis agency analyzes DNA stored in DNA test kits provided by the user, uploads the DNA analysis result to the skin care web server, and the skin care web server transmits the uploaded DNA analysis result to the skin diagnosis engine.

In the above skin diagnosis engine,

It is characterized in that a skin diagnosis engine is updated by downloading a deep learning learning model from a skin care web server.

In the skin care web server,

It is characterized by storing skin images uploaded from mobile as big data for learning, analyzing artificial intelligence learning images to extract features, and installing engine code optimized for the extracted features in the mobile app to update the skin diagnosis engine.

In the above, the skin diagnosis engine is characterized in that it transmits skin condition and skin disease prediction information to the skin care web server, receives a treatment method generated based on expert knowledge from the skin care web server, and displays it on the screen.

In the above skin diagnosis engine,

a skin image interface module that interfaces the skin image input through the image input unit;

a real-time image processing module for real-time pre-processing of the skin image interfaced through the skin image interface module;

a data filtering module extracting skin image data for analysis by filtering the skin image data preprocessed by the real-time image processing module;

a deep learning inference engine that analyzes the skin image data for analysis extracted from the data filtering module using a CNN-based deep learning inference model;

A skin condition diagnosis and action module for generating skin condition diagnosis and action information in conjunction with an expert knowledge base based on the analysis result from the deep learning inference engine and the DNA analysis result received from the skin care web server;

It is characterized in that it comprises a skin condition diagnosis and action information display module that displays the skin condition diagnosis and action information generated by the skin condition diagnosis and action module on a screen.

In addition, the skin diagnosis engine,

a plasma generation energy level setting module configured to set a plasma generation energy level according to the action information generated by the skin condition diagnosis and action module;

It is characterized by further comprising a plasma generation energy control module for generating plasma generation energy control data according to the plasma generation energy level set in the plasma generation energy level setting module and controlling the output through a Bluetooth module.

In addition, the skin diagnosis engine,

In conjunction with the skin care web server, it receives the engine code optimized for the extracted features, updates the inference model of the deep learning inference engine, receives the received treatment method based on expert knowledge, and delivers it to the deep learning inference engine. Characterized in that it further includes a deep learning API.

In the above, the deep learning inference engine,

It is characterized by analyzing skin image data using a CNN deep learning inference model that introduces Residule Block.

In addition, the "mobile-based personalized skin diagnosis service system" according to the present invention,

It is characterized in that it further comprises a plasma generator connected to the skin diagnosis engine through Bluetooth communication and generating plasma for skin disease improvement based on the received plasma generation energy control data.

According to the present invention, the skin image uploaded to the mobile is analyzed using an artificial intelligence (AI) engine to predict skin conditions and skin diseases, and skin analysis results and treatment methods are provided based on the skin image analysis results and DNA analysis results. There is an effect of promoting the user to manage the skin using a mobile device.

In addition, according to the present invention, a plasma generator for generating plasma and a skin diagnosis engine mounted on a mobile are interlocked through Bluetooth communication, so that even a plasma generator without a skin diagnosis engine can generate optimal plasma for improving skin disease based on control data based on the skin diagnosis result transmitted from the mobile device, thereby improving the user's skin disease.

1 is a conceptual diagram of a mobile-based personalized skin diagnosis service in the present invention;
2 is an exemplary view of diagnosing a skin condition based on a skin image taken in the present invention;
Figure 3 is an example of analyzing the risk of skin disease using user DNA in the present invention;
4 is a configuration diagram of a mobile-based personalized skin diagnosis service system according to a preferred embodiment of the present invention;
5 is a block diagram of a skin disease risk prediction algorithm using a skin image in the present invention;
6 is a conceptual diagram of skin disease risk prediction using DNA analysis technology in the present invention;
7 is a conceptual diagram of predicting a skin condition by analyzing a skin image using deep learning AI technology in the present invention;
8 is an exemplary view of a customized skin care report provided through mobile in the present invention.

Hereinafter, a mobile-based personalized skin diagnosis service system according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The terms or words used in the present invention described below should not be construed as being limited to conventional or dictionary meanings, and the inventors may appropriately define the concept of terms in order to explain their invention in the best way. Based on the principle, it should be interpreted as meanings and concepts consistent with the technical spirit of the present invention.

Therefore, the embodiments described in this specification and the configurations shown in the drawings are only preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention, so that they can be substituted at the time of the present application. It should be understood that there may be equivalents and modifications.

1 is a conceptual diagram of a mobile-based personalized skin diagnosis service in the present invention, combining AI image analysis technology and DNA analysis technology to predict a personal skin disease, and to improve skin disease based on the predicted personal skin disease information Provides personalized recommendation and management information.

Here, personalized recommendation and management can recommend treatment methods through AI recommendation, expert opinion and recommendation, and doctor/hospital recommendation around the user, and clinically and effect-proven functional products (eg, BIO peptides and functional materials) can be recommended for improvement of skin diseases, and continuous management is performed through continuous management and monitoring.

2 is a configuration diagram of a “mobile-based personalized skin diagnosis service system” according to the present invention, which includes a mobile 100 such as a smartphone with a skin diagnosis engine installed, a plasma generator 200, and a skin care web server 300.

The skin care web server 300 stores the skin image uploaded from the mobile 100 as big data for learning, analyzes the artificial intelligence learning image to extract features, and installs the engine code optimized for the extracted features in the skin care application (app) built in the mobile 100 to update the skin diagnosis engine 120.

In addition, the skin care web server 300 analyzes DNA stored in DNA test kits provided by the user from a professional DNA analysis institution, uploads the DNA analysis result, stores it, and transmits the uploaded DNA analysis result to the skin diagnosis engine of the mobile 100.

The mobile 100 includes an image input unit 110 that uploads a skin image for diagnosis, an image pattern analysis that compares and analyzes features learned using an artificial intelligence engine mounted on an image input through the image input unit 110, and a skin diagnosis engine 120 that predicts skin conditions and skin diseases based on DNA analysis results provided by a DNA analysis agency, and a skin image diagnosis result providing unit that displays skin diagnosis results and treatment methods analyzed by the skin diagnosis engine 120 on a screen 130) may be included.

The image input unit 110 serves to provide the diagnostic skin image for real-time analysis and to the skin care web server 300 as learning data.

To this end, a skin photographing camera and an image processing module may be included.

The skin diagnosis engine 120 downloads a deep learning learning model from the skin care web server 300 to update the skin diagnosis engine.

In addition, the skin diagnosis engine 120 transmits skin condition and skin disease prediction information to the skin care web server 300, receives a treatment method generated based on expert knowledge from the skin care web server 300, and displays it on the screen.

The skin diagnosis engine 120 includes a skin image interface module 121 that interfaces the skin image input through the image input unit 110, a real-time image processing module 122 that pre-processes the skin image interfaced through the skin image interface module 121 in real time, a data filtering module 123 that extracts skin image data for analysis by filtering skin image data preprocessed in the real-time image process module 122, and a skin image for analysis extracted from the data filtering module 123. A deep learning inference engine 124 that analyzes data using a CNN-based deep learning inference model, a skin condition diagnosis and action module 125 that generates skin condition diagnosis and action information in conjunction with an expert knowledge base based on the analysis result from the deep learning reasoning engine 124 and the DNA analysis result received from the skin care web server 300, and a skin condition diagnosis and action information display module 127 that displays the skin condition diagnosis and action information generated by the skin condition diagnosis and action module 125 on a screen. include

In addition, the skin diagnosis engine 120 may further include a plasma generation energy level setting module 126 for setting the plasma generation energy level according to the action information generated by the skin condition diagnosis and action module 125, and a plasma generation energy control module 128 for generating plasma generation energy control data according to the plasma generation energy level set in the plasma generation energy level setting module 126 and controlling it to be output through a Bluetooth module.

In addition, the skin diagnosis engine 120 may further include a deep learning API 129 that receives engine code optimized for the extracted features in conjunction with the skin care web server 300 to update the inference model of the deep learning inference engine 124, receives a treatment method received based on expert knowledge, and transmits it to the deep learning inference engine 124.

The deep learning inference engine 124 may analyze skin image data using a CNN deep learning inference model in which Residule Block is introduced.

The plasma generator 200 is connected to the skin diagnosis engine 120 through Bluetooth communication, and serves to generate plasma for skin disease improvement on the user's skin based on the received plasma generation energy control data.

The operation of the "mobile-based personalized skin diagnosis service system" according to the present invention configured as described above will be described in detail with reference to FIGS. 2 to 8 of the accompanying drawings.

First, a user who wants to self-diagnose the skin installs a skin diagnosis application on the mobile 100 carried.

Here, the skin diagnosis application includes the skin diagnosis engine 120 installed from the skin care web server 300 .

The skin care web server 300 stores skin images uploaded from multiple mobile devices as big data for learning, analyzes AI learning images to extract features, and installs an engine code optimized for the extracted features in a skin care application (app) to update the skin diagnosis engine.

Therefore, when the user installs the skin diagnosis application, the skin diagnosis engine is also installed.

For self-diagnosis of the skin condition, the user can diagnose the condition of his or her own skin through two methods: using a skin image and using DNA analysis.

For skin condition diagnosis using DNA analysis, a user is registered in the skin care web server 300 using the mobile 100, and the registered contents are sent to a DNA analysis agency (DNA Test Lab). Upon receiving the user information, the DNA analysis agency identifies the user in conjunction with the skin care web server 300 and, if the user is registered, sends a DNA diagnosis kit to the user based on the address information registered by the user.

Upon receipt of the DNA diagnosis kit, the user collects DNA using the DNA diagnosis kit according to the provided DNA collection method, and then sends the DNA diagnosis kit to the DNA analysis institution.

As shown in FIG. 3, the DNA analysis agency analyzes the DNA collected through the DNA diagnosis kit by a predetermined analysis method, and uploads the DNA analysis result (skin gene analysis result) to the skin care web server 300. The DNA analysis result is used as information for predicting the risk of skin disease in advance. Here, DNA analysis progress information is transmitted to the user mobile 100 so that the user can also recognize the DNA analysis progress process.

In FIG. 3, DNA analysis is performed by a specialized DNA analysis agency, and risk analysis and personalized prediction are performed by a mobile skin diagnosis engine.

The skin care web server 300 analyzes DNA collected from DNA Test Kits provided by the user from the professional DNA analysis institution, and when the DNA analysis result is uploaded, it is matched with user information and stored in the database, and the uploaded DNA analysis result is transmitted to the skin diagnosis engine of the mobile 100.

In addition, the user transmits a skin image for diagnosis to the skin diagnosis engine 120 using the image input unit 110 for skin diagnosis. That is, the image input unit 110 includes a skin photographing camera and an image processing module, uses the skin photographing camera to photograph the skin to be diagnosed, and the image processing module preprocesses the skin image measured by the skin photographing camera and transmits it to the skin diagnosis engine 120.

In addition, the image input unit 110 provides the pre-processed skin image for diagnosis to the skin diagnosis engine 120 for real-time analysis and simultaneously provides it to the skin care web server 300 as learning data.

The skin diagnosis engine 120 predicts skin conditions and skin diseases based on image pattern analysis that compares and analyzes learned features using an artificial intelligence engine installed for diagnosis skin images input through the image input unit 110 and DNA analysis results obtained through the skin care web server 300 and provided by a DNA analysis agency.

The skin diagnosis engine 120 downloads a deep learning learning model from the skin care web server 300 to update the skin diagnosis engine.

For example, the skin diagnosis engine 120 works with the skin care web server 300 through the deep learning API 129 to receive engine codes optimized for the extracted features, and updates the inference model of the deep learning inference engine 124. The deep learning inference engine 124 may analyze skin image data using a CNN deep learning inference model in which Residule Block is introduced.

Figure 2 is an analysis process for analyzing and diagnosing skin condition, taking a picture of the skin through a mobile camera, capturing a picture taken through an image processing module, analyzing the skin condition with artificial intelligence/deep learning technology in a skin diagnosis engine, and diagnosing the skin condition with artificial intelligence/deep learning analysis technology.

A detailed description of this is as follows.

As shown in FIGS. 5 and 7, the skin image interface module 121 interfaces the skin image (500 × 500 RGB) input through the image input unit 110, and the interfaced skin image is subjected to real-time preprocessing (Data Augmentation) through the real-time image processing module 122. Here, pre-processing is a process of pre-processing a video image, and is an image pre-processing technique generally performed in conventional image processing techniques.

Next, the skin image data for analysis is extracted by filtering the skin image data (Augmented Image (50×50)) preprocessed in the real-time image processing module 122 in the data filtering module 1230. A primary sliding window, a skin detection technique (Binary Classification), and a secondary sliding window can be used to extract the skin image data for analysis.

Subsequently, the skin image data for analysis extracted by the data filtering module 123 is analyzed by the deep learning inference engine 124 using a CNN-based deep learning inference model. Here, the CNN-based deep learning inference model predicts the type of dermatitis by inferring the skin image into six preset types using the Softmax classification model.

Next, the skin condition diagnosis and action module 125 interlocks with the expert knowledge base based on the skin image analysis result from the deep learning reasoning engine 124 and the DNA analysis result received from the skin care web server 300 to generate skin condition diagnosis and action information.

Skin condition diagnosis using a skin image diagnoses dermatitis through inference using the predicted dermatitis type information.

In addition, skin condition diagnosis using DNA analysis results, as shown in FIG. 6, analyzes variation between individuals, and calculates skin disease risk using a preset skin disease risk calculation algorithm using the analysis result. Next, a preventive measure for skin disease improvement is recommended in conjunction with the skin care web server 300 using the calculated skin disease risk prediction value.

For preventive measures and measures for improving skin diseases, refer to the personalized recommendation and management information in FIG. 1 described above.

In addition, the skin condition diagnosis and action information display module 127 transmits the skin condition diagnosis and action information generated by the skin condition diagnosis and action module 125 to the skin image diagnosis result providing unit 130 to be displayed on the screen.

The skin image diagnosis result providing unit 130 displays the skin diagnosis result and treatment method analyzed by the skin diagnosis engine 120 on the screen so that the user can recognize his or her skin diagnosis condition.

Here, the skin diagnosis engine 120 transmits skin condition and skin disease prediction information to the skin care web server 300, and from the skin care web server 300, treatment methods and skin disease improvement plans generated based on expert knowledge are received and displayed on the screen.

8 is an example of a user-customized skin diagnosis result report. The customized skin diagnosis result report may include skin health information, skin age information, detailed skin health information, and the like.

The user can recognize his own skin condition through the skin diagnosis result report displayed on the screen, and can improve his skin disease by taking corresponding measures using the dental disease improvement plan provided.

Here, plasma can be used as one method for improving skin diseases.

For example, a user has a plasma generator 200, but the plasma generator 200 can generate plasma for skin disease improvement, but does not have a function of diagnosing the user's own skin condition.

Therefore, in order to compensate for the disadvantages of the plasma generator 200, if the plasma generator 200 and the mobile 100 are linked via Bluetooth, the skin condition of the mobile 100 is accurately diagnosed, and plasma generation control information according to the diagnosis result is generated and provided to the plasma generator 200 to optimally control plasma generation, skin diseases can be effectively improved.

For example, the plasma generation energy level setting module 126 of the skin diagnosis engine 120 sets the plasma generation energy level according to action information generated by the skin condition diagnosis and action module 125 . It is assumed that the plasma generation energy level for skin disease improvement is stored in the database in advance in the form of a table according to the skin condition diagnosis result.

The plasma generation energy level set in this way is generated as plasma generation energy control data through the plasma generation energy control module 128, and the generated plasma generation energy control data is transmitted to the plasma generator 200 through the Bluetooth module.

The plasma generator 200 is connected to the skin diagnosis engine 120 through Bluetooth communication, and generates plasma for skin disease improvement on the user's skin based on the received plasma generation energy control data, thereby improving the user's skin disease. Performs the function.

Therefore, even a simple plasma generator without a skin diagnosis engine can optimally improve a user's skin disease by linking with a mobile device.

According to the present invention described above, the skin image uploaded to the mobile is analyzed by the AI engine to learn the pattern of the image on the mobile to predict the skin condition and skin disease, and based on the skin image analysis result and the DNA analysis result, skin analysis results and treatment methods are provided so that users can manage their skin using mobile devices.

In addition, according to the present invention, a plasma generator that simply generates only plasma and a skin diagnosis engine mounted on a mobile are interlocked through Bluetooth communication, so that even a plasma generator without a skin diagnosis engine can generate optimal plasma for improving skin disease based on control data of plasma generation energy transmitted from the mobile device, thereby improving the user's skin disease.

Although the invention made by the present inventors has been specifically described according to the above embodiments, the present invention is not limited to the above embodiments and can be changed in various ways without departing from the gist of the art. It is obvious to those skilled in the art.

100: mobile
110: image input unit
120: skin diagnosis engine
121: skin image interface module
122: real-time image processing module
123: data filtering module
124: deep learning inference engine
125: skin condition diagnosis and action module
126: Plasma generation energy level setting module
127: skin condition diagnosis and action information display module
128: Plasma generation energy control module
130: skin image diagnosis result providing unit
200: plasma generator
300: skin care web server

Claims (11)

an image input unit for uploading a skin image for diagnosis;
A skin diagnosis engine that predicts skin conditions and skin diseases based on image pattern analysis that compares and analyzes learned features using an artificial intelligence engine loaded with respect to images input through the image input unit and DNA analysis results provided by a DNA analysis agency; and
A mobile-based personalized skin diagnosis service system comprising a skin image diagnosis result provider that displays skin diagnosis results and treatment methods analyzed by the skin diagnosis engine on a screen.
In claim 1, the image input unit,
A mobile-based personalized skin diagnosis service system characterized by providing diagnostic skin images for real-time analysis and at the same time providing them as learning data to a skin care web server.
The mobile-based personalized skin diagnosis service system of claim 1, wherein the DNA analysis institution analyzes DNA collected through DNA test kits provided by the user, uploads the DNA analysis result to the skin care web server, and the skin care web server transmits the uploaded DNA analysis result to the skin diagnosis engine.
In claim 1, the skin diagnosis engine,
A mobile-based personalized skin diagnosis service system characterized by updating a skin diagnosis engine by downloading a deep learning learning model from a skin care web server.
In claim 1, the skin care web server,
A mobile-based personalized skin diagnosis service system characterized by storing skin images uploaded from mobile as big data for learning, analyzing artificial intelligence learning images to extract features, and installing engine code optimized for the extracted features in a mobile app to update a skin diagnosis engine.
The mobile-based personalized skin diagnosis service system of claim 1, wherein the skin diagnosis engine transmits skin condition and skin disease prediction information to the skin care web server, receives a treatment method generated based on expert knowledge from the skin care web server, and displays the result on a screen.
In claim 1, the skin diagnosis engine,
a skin image interface module that interfaces the skin image input through the image input unit;
a real-time image processing module for real-time pre-processing of the skin image interfaced through the skin image interface module;
a data filtering module extracting skin image data for analysis by filtering the skin image data preprocessed by the real-time image processing module;
a deep learning inference engine that analyzes the skin image data for analysis extracted from the data filtering module using a CNN-based deep learning inference model;
A skin condition diagnosis and action module for generating skin condition diagnosis and action information in conjunction with an expert knowledge base based on the analysis result from the deep learning inference engine and the DNA analysis result received from the skin care web server; and
A mobile-based personalized skin diagnosis service system comprising a skin condition diagnosis and action information display module for displaying the skin condition diagnosis and action information generated by the skin condition diagnosis and action module on a screen.
In claim 7, the skin diagnosis engine,
a plasma generation energy level setting module configured to set a plasma generation energy level according to the action information generated by the skin condition diagnosis and action module; and
Further comprising a plasma generation energy control module for generating plasma generation energy control data according to the plasma generation energy level set in the plasma generation energy level setting module and controlling it to be output through a Bluetooth module. Mobile based personalized skin diagnosis service system.
In claim 7, the skin diagnosis engine,
A mobile-based personalized skin diagnosis service system further comprising a deep learning API for receiving an engine code optimized for the extracted features in conjunction with the skin care web server to update an inference model of the deep learning inference engine, receiving a treatment method received based on expert knowledge, and passing it to the deep learning inference engine.
In claim 7, the deep learning inference engine,
A mobile-based personalized skin diagnosis service system characterized by analyzing skin image data using a CNN deep learning inference model incorporating Residule Block.
In claim 1,
A mobile-based personalized skin diagnosis service system further comprising a plasma generator connected to the skin diagnosis engine through Bluetooth communication and generating plasma for improving skin disease based on the received plasma generation energy control data.

Images