KR20220051564A - Method and system for providing of user customized cosmetic based on artivicial intelligence - Google Patents

Abstract

According to an embodiment of the present disclosure, a method for providing user-customized cosmetic based on artificial intelligence comprises: a step of receiving information about one or more images in which facial skin of a user is captured; a step of receiving information about a skin characteristic of the user determined by analyzing a gene of the user; and a step of determining one or more user-customized cosmetics based on information about one or more images and information about the skin characteristic of the user.

Description

A method and system for providing customized cosmetics based on artificial intelligence

The present disclosure relates to a method and system for providing customized cosmetics to a user based on artificial intelligence, and specifically, to a method and system for providing customized cosmetics based on artificial intelligence that recommends cosmetics suitable for a user by utilizing a captured image and genetic analysis it's about

Cosmetics are articles used on the human body to clean and beautify the human body to add attractiveness, to brighten the appearance, or to maintain or promote the health of skin and hair. Recently, as interest in functional cosmetics has increased, intensive research is being conducted on areas to improve skin condition beyond simply making the appearance of the skin beautiful.

On the other hand, customized cosmetics refers to a form of diagnosing the skin condition of individual customers based on accumulated skin diagnosis data to produce and provide the most suitable cosmetics for each individual skin.

For customized cosmetics, customized cosmetics using a skin measurement system that checks the skin condition of customers by performing diagnosis through a skin measurement device and skin diagnosis through a questionnaire, and presents cosmetics to customers accordingly, and skin diagnosis through a questionnaire. According to the skin type determined after diagnosis, it can be classified into customized cosmetics using a questionnaire that presents cosmetics to customers. With respect to customized cosmetics using a skin measurement system, the use of a skin measurement device is limited to a relatively simple test, and an individual's genetic analysis is also used, but the technology is only at a rudimentary level. In addition, with respect to customized cosmetics using a paper questionnaire, there is a problem in that it is difficult to provide optimal cosmetics for each individual skin because it is usually in the form of a questionnaire through a consumer survey.

Republic of Korea Patent Registration No. 10-2074178 'Personalized information provision analysis system and method'

The present disclosure provides a method and system for providing user-customized cosmetics based on artificial intelligence for solving the above problems.

Embodiments described in this specification provide a method and system for providing customized cosmetics to a user based on artificial intelligence that recommends customized cosmetics suitable for a user by utilizing an image of the user's facial skin and genetic analysis.

The present disclosure may be implemented in various ways including a method, a system, or a computer program stored in a computer-readable recording medium.

A method for providing customized cosmetics to a user based on artificial intelligence according to an embodiment of the present disclosure includes the steps of receiving information on one or more images of the user's facial skin, and analyzing the user's genes to determine the user's skin characteristics. Receiving the information, and determining one or more user-customized cosmetics based on the information on the one or more images and the information on the skin characteristics of the user.

According to an embodiment, the method may further include receiving evaluation information on cosmetics crawled on a network.

According to an embodiment, the determining of the user-customized cosmetics may include determining a plurality of user-customized cosmetics candidates based on information on one or more images and information on the user's skin characteristics, and the determined plurality of user-customized cosmetics. It may include selecting one or more user-customized cosmetics from among the candidates based on evaluation information on the cosmetics.

According to an embodiment, the determining of the user-customized cosmetics includes inputting information about one or more images, user's skin characteristics, and evaluation information on cosmetics into an artificial neural network model trained to provide customized cosmetics, and customized to the user. It may include the step of outputting cosmetics.

According to an embodiment, the receiving of the information on the one or more images of the user's facial skin includes receiving information on the plurality of images of the user's facial skin captured at a plurality of predetermined times can do.

According to an embodiment, the plurality of images are captured at a plurality of predetermined times according to each of a plurality of instructions, wherein the plurality of instructions include an instruction to image the user's facial skin before applying one or more cosmetics and one or more It may include an instruction to image the skin of the user's face after applying cosmetics.

According to an embodiment, the plurality of images are captured at a plurality of predetermined times according to each of a plurality of instructions, wherein the plurality of instructions include an instruction to image the user's facial skin before applying one or more cosmetics and one or more It may include an instruction to image the skin of the user's face after applying cosmetics.

According to an embodiment, the skin characteristics of the user may include information on at least one of ingredients of cosmetics that are not suitable for the user and current or future skin troubles of the user.

According to an embodiment, the artificial intelligence-based method for providing customized cosmetics to a user is compatible with receiving information on specific cosmetics and receiving information on skin characteristics and specific cosmetics of the user and information on skin characteristics and cosmetics. It may include inputting the artificial neural network model trained to provide the degree, and outputting the degree of compatibility.

According to an embodiment, a method for providing customized cosmetics to a user based on artificial intelligence includes the steps of: receiving information about weather; and inputting the user's skin characteristics and the received weather information into the artificial neural network model trained to provide a makeup method when the weather information and skin characteristics are input, and outputting the makeup method. .

According to another embodiment of the present disclosure, the artificial intelligence-based user-customized cosmetic providing system receives information about one or more images of the user's facial skin from the imaging device, and analyzes the user's genes from the genetic analysis server a communication module configured to receive information on the skin characteristics of the user determined by is configured to transmit information on the determined customized cosmetics to the user terminal.

According to some embodiments of the present disclosure, by comprehensively considering the current state of the user's facial skin and the user's skin characteristics through genetic analysis, cosmetics with high skin suitability for each user and/or cosmetics that suit their taste are recommended. As a result, the skin condition can be improved, and the satisfaction with the use of cosmetics can be increased, and skin troubles can be prevented in advance due to the low skin suitability.

According to some embodiments of the present disclosure, information on whether the cosmetic product is suitable for the user may be provided by calculating the user's component compatibility with the cosmetic currently being used by the user.

According to some embodiments of the present disclosure, it is possible to provide a makeup method suitable for the weather information to the user, thereby helping the user to put on makeup suitable for the weather.

The effect of the present disclosure is not limited to the above-mentioned effects, and other effects not mentioned are those of ordinary skill in the art to which the present disclosure belongs from the description of the claims (hereinafter referred to as 'person of ordinary skill') can be clearly understood by

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, in which like reference numerals denote like elements, but are not limited thereto.
1 is an exemplary diagram illustrating a screen of a user terminal on which a user-customized cosmetics recommendation is made through an artificial intelligence-based user-customized cosmetics recommendation service according to an embodiment of the present disclosure.
2 is a diagram illustrating a system for providing user-customized cosmetics based on artificial intelligence according to an embodiment of the present disclosure.
3 is a schematic diagram illustrating a configuration in which a customized cosmetic providing server is connected to communicate with a plurality of user terminals in order to provide artificial intelligence-based customized cosmetics according to an embodiment of the present disclosure.
4 is a block diagram illustrating an internal configuration of a server for providing customized cosmetics according to an embodiment of the present disclosure.
5 is a diagram illustrating a system for performing crawling according to an embodiment of the present disclosure.
6 is an exemplary view showing evaluation information on cosmetics received by crawling according to an embodiment of the present disclosure.
7 is an exemplary diagram showing information on a skin image possessed by each user according to an embodiment of the present disclosure.
8 is an exemplary diagram showing information on a user's skin characteristics determined by a genetic analysis result according to an embodiment of the present disclosure.
9 is a diagram illustrating an example of an artificial neural network model according to an embodiment of the present disclosure.
10 is a flowchart illustrating a method for providing user-customized cosmetics based on artificial intelligence according to an embodiment of the present disclosure.
11 is an exemplary view showing a genetic analysis result screen according to an embodiment of the present disclosure.

Hereinafter, specific contents for carrying out the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, the same or corresponding components are assigned the same reference numerals. In addition, in the description of the embodiments below, overlapping description of the same or corresponding components may be omitted. However, even if descriptions regarding components are omitted, it is not intended that such components are not included in any embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become apparent with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the present disclosure to be complete, and the present disclosure provides those skilled in the art with the scope of the invention. It is provided for complete information only.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail. Terms used in this specification have been selected as currently widely used general terms as possible while considering the functions in the present disclosure, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the content throughout the present disclosure, rather than the simple name of the term.

References in the singular herein include plural expressions unless the context clearly dictates the singular. Also, the plural expression includes the singular expression unless the context clearly dictates the plural. In the entire specification, when a part includes a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In addition, the term 'module' or 'unit' used in the specification means a software or hardware component, and 'module' or 'unit' performs certain roles. However, 'module' or 'unit' is not meant to be limited to software or hardware. A 'module' or 'unit' may be configured to reside on an addressable storage medium or configured to reproduce one or more processors. Thus, as an example, a 'module' or 'unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, and properties. , procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays or at least one of variables. Components and 'modules' or 'units' are the functions provided therein that are combined into a smaller number of components and 'modules' or 'units' or additional components and 'modules' or 'units' can be further separated.

According to an embodiment of the present disclosure, a 'module' or a 'unit' may be implemented with a processor and a memory. 'Processor' should be construed broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, and the like. In some contexts, a 'processor' may refer to an application specific semiconductor (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), or the like. 'Processor' refers to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such configuration. You may. Also, 'memory' should be construed broadly to include any electronic component capable of storing electronic information. 'Memory' means random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erase-programmable read-only memory (EPROM); may refer to various types of processor-readable media, such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. A memory is said to be in electronic communication with the processor if the processor is capable of reading information from and/or writing information to the memory. A memory integrated in the processor is in electronic communication with the processor.

1 is an exemplary diagram illustrating a screen of a user terminal 110 in which a user-customized cosmetics recommendation is made through an artificial intelligence-based user-customized cosmetics recommendation service according to an embodiment of the present disclosure.

The artificial intelligence-based user-customized cosmetic recommendation service may be provided to the user through an application for service provision installed in the user terminal 110 , wherein the user terminal 110 includes interface devices of the terminal possessed by the user. (for example, a touch display, keyboard, mouse, touch pen or stylus, microphone, motion recognition sensor) can be used to access the artificial intelligence-based user-customized cosmetic service system. For example, in response to a user input, the user terminal may access the online profile search system by executing a web page address and an application provided by the artificial intelligence-based user-customized cosmetic recommendation service system. In this case, the user may be a consumer who uses an artificial intelligence-based user-customized cosmetic recommendation service.

When the user accesses the artificial intelligence-based user-customized cosmetic recommendation service system using the interface device of the user terminal 110 , the user terminal 110 includes a first area 120 for retrieving the user's skin image, the user's The second area 122 for retrieving skin characteristics (gene analysis results, etc.), and the third area for selecting the type of cosmetics (skin, lotion, cream, essence, moisture cream, sunscreen, etc.) that the user wants to receive recommendation. 124 may be output. For example, the user selects 'load' of the first area 120 and is stored in the user terminal 110 or an imaging device provided in the user terminal 110 . You can import one or more images of your face skin taken directly from (camera, etc.). In addition, the user may select 'Load' of the second region 122 to retrieve information on skin characteristics according to his/her genetic analysis result stored in the gene analysis server. In addition, the user may select at least one of the types of cosmetics presented in the third area 124 .

In an embodiment, the facial skin image through 'loading' of the first region 120 may be an image of the user's facial skin captured at a predetermined time according to an instruction. For example, the user may photograph his or her own skin according to an instruction to photograph the facial skin before applying cosmetics and an instruction to photograph the facial skin after applying cosmetics. When the skin image import is completed, the user may call information about his or her skin characteristics. The skin characteristics of the user may include information about ingredients of cosmetics that are not suitable for the user and information about the user's current or future skin troubles. When the user's skin characteristics are loaded, the user can select a type of cosmetics for which a customized cosmetic recommendation service is desired. According to the type of cosmetics selected by the user, the customized cosmetics recommendation service system may recommend customized cosmetics.

When the input of information in the first area 120 , the second area 122 , and the third area 124 is completed, the user uses the user terminal 110 to access the first area 120 and the second area 122 . ), a fourth area 130 for guiding the user's skin condition based on the information input in ), and a third area for receiving a cosmetic recommendation based on the information input in the third area 124 while considering the user's skin condition. Five regions 132 may be provided. In an embodiment, the user may select 'View Details' of the fourth area 130 to check his/her skin type (dry, oily, combination), skin trouble types and conditions, allergy-causing ingredients, and the like. Also, when the user selects 'moisture cream' in the third area 124 , the user may check information on at least one 'moisture cream' in the fifth area 132 . In this case, the information on the recommended cosmetics that can be checked in the fifth area 132 is evaluation information on the corresponding cosmetics. The evaluation information may be evaluation information about the corresponding cosmetic crawled on the network, and may include the skin type of the user who used the cosmetic, the rating, the skin type of the user who searched for the corresponding cosmetic, keywords related to the cosmetic, etc. there is.

When the user accesses the artificial intelligence-based user-customized cosmetic recommendation service system, the user-customized cosmetic recommendation system receives information about the cosmetics the user is currently using from the user and provides the user's ingredient compatibility with the corresponding cosmetics. there is. In one embodiment, the user may input information about a particular cosmetic into the system. For example, the user may input the brand name and product name of cosmetics that are currently being used or will be used into the system. In another example, a user may provide an image of a particular cosmetic to the system. In this case, the system may recognize text related to cosmetics included in the input image (eg, text written on cosmetics or cosmetics case, etc.) and extract information about cosmetics in the image. Then, the system may use information about the cosmetics and the skin characteristics of the user to determine a component compatibility level for determining whether the corresponding cosmetic product is suitable for the user, and provide the determined component compatibility level to the user. For example, the system compares the cosmetic ingredient information input by the user with the user's skin type, calculates a compatibility degree of whether the cosmetic ingredient is suitable for the user's skin type, and provides it to the user.

In addition, the system may provide the user with a makeup method suitable for a particular weather (eg, today or tomorrow weather, etc.). In one embodiment, when the user inputs the weather, the system may provide the user with a makeup method suitable for the corresponding weather. In another embodiment, if the user sets a desired date to receive the makeup method without directly inputting the weather, a makeup method suitable for weather information predicted on the set date may be provided to the user.

2 is a diagram illustrating a system for providing user-customized cosmetics based on artificial intelligence according to an embodiment of the present disclosure. The system for providing user-customized cosmetics based on artificial intelligence includes an imaging device 210 , a genetic analysis server 220 , and a customized cosmetics providing server 240 .

The imaging device 210 is a device capable of photographing a user, and may include, for example, a camera module provided in the user terminal 110 , a camera provided separately from the user terminal 110 , and the like. According to an embodiment, the imaging device 210 may photograph the user's facial skin by the user's manipulation, and may store and store one or more images of the user's facial skin by itself. Also, the imaging device 210 may transmit one or more images of the user's facial skin to the customized cosmetic providing server 240 through the network 230 .

The user may photograph the user's face skin at a predetermined time through the imaging device 210 . For example, a user may capture an image of the facial skin before washing, an image of the facial skin after washing, an image of the facial skin before applying cosmetics, an image of the facial skin after applying cosmetics, and an image taken immediately after waking up. An image or the like may be generated through the imaging device 210 .

The genetic analysis server 220 may generate information on the skin characteristics of the user determined by analyzing the user's genes. According to one embodiment, the gene analysis server 220 analyzes the user's genes obtained through the genetic test performed by the user, and photoaging, sugar aging, skin elasticity, pigmentation, skin moisture, sebum control, acne, skin sensitivity , information on the user's skin characteristics such as atopy can be generated.

Here, photoaging occurs when a mutation is found in at least one of STXBP5L, AGER, and MMP1 genes as a result of genetic analysis, and refers to the ability to maintain skin elasticity when exposed to UV rays. Glucose aging occurs when a mutation is found in the AGER gene as a result of genetic analysis, and refers to the ability to maintain skin elasticity after sugar intake. Skin elasticity occurs when a mutation is found in at least one of the COL3A1 and MMP1 genes as a result of genetic analysis, and refers to the ability to regulate collagen synthesis and degradation. Pigmentation occurs when a mutation is found in at least one of the BNC2, OCA2, and MC1R genes as a result of genetic analysis, indicating the possibility of pigmentation. Skin moisture appears when a mutation is found in at least one of FLG, CLDN1, and FCN1 genes as a result of genetic analysis, and refers to the skin's ability to retain moisture. Sebum regulation occurs when a mutation is found in at least one of TGFB2, SRD5A1, and IL2RA genes as a result of genetic analysis, and refers to the ability to regulate sebum secretion. Acne occurs when a mutation is found in at least one gene among TGFB2, SELL, and DDB2 as a result of genetic analysis, and refers to the ability to maintain healthy skin without acne. Skin sensitivity occurs when a mutation is found in at least one of FLG, TSLP, and BTNL2 genes as a result of genetic analysis, and refers to the skin's ability to protect against external stimuli. Atopy appears when a mutation is found in at least one gene among FLG, GATA3, and SCAPER as a result of genetic analysis, and refers to the sensitivity to atopy.

According to one embodiment, the gene analysis server 220 analyzes the gene and analyzes the above-mentioned photoaging, sugar aging, skin elasticity, pigmentation, skin moisture, sebum control, acne, skin sensitivity, atopy, etc. of the user's skin characteristics. In generating the information, scoring for each skin characteristic and grading (high, slightly high, normal, low, etc.) of the genetic analysis result based on the scoring may be performed. In addition, the gene analysis server 220 generates information about at least one of ingredients of cosmetics that are not suitable for the user and the user's current or future skin troubles as information about the user's skin characteristics through the user's genetic analysis. can In addition, the gene analysis server 220 may analyze the user's gene through the network 230 and transmit information on the user's skin characteristics determined to the customized cosmetics providing server 240 .

The customized cosmetics providing server 240 receives information about one or more images of the user's facial skin from the imaging device 210, and analyzes the user's genes from the genetic analysis server 220 to determine the user's skin characteristics. receive information about Then, the customized cosmetics providing server 240 determines one or more customized cosmetics based on the information on the one or more images and the information on the skin characteristics of the user.

In addition, the customized cosmetic providing server 240 may receive crawled cosmetic evaluation information on the network 230 . Here, the evaluation information on the cosmetics may include a skin type of a user who has used the corresponding cosmetics, a rating, a skin type of a user who has searched for the corresponding cosmetics, keywords related to the corresponding cosmetics, and the like. The customized cosmetic providing server 240 that has received the evaluation information on the cosmetics may determine a plurality of user-customized cosmetic candidates based on information on one or more images and information on the user's skin characteristics, and the determined plurality of customized cosmetics From among the candidates, one or more user-customized cosmetics may be selected based on evaluation information on the cosmetics. At this time, the customized cosmetic providing server 240 inputs information about one or more images, user's skin characteristics, and evaluation information on cosmetics to an artificial neural network model trained to provide customized cosmetics, and outputs customized cosmetics. can

The artificial neural network model here is a statistical learning algorithm implemented based on the structure of a biological neural network or a structure for executing the algorithm in machine learning technology and cognitive science. According to an embodiment, in the artificial neural network model, as in a biological neural network, nodes that are artificial neurons that form a network by combining synapses repeatedly adjust the weights of synapses to obtain correct output and reasoning in response to a specific input. By learning to reduce the error between the outputs, it is possible to represent a machine learning model with problem-solving ability. For example, the artificial neural network model may include arbitrary probabilistic models, neural network models, etc. used in artificial intelligence learning methods such as machine learning and deep learning. According to an embodiment, the artificial neural network model may include an artificial neural network model configured to input target data and output label information. This artificial neural network model can be trained for each user. That is, the artificial neural network model is generated for each user terminal or each user, and may be used for learning or inference for each user terminal or each user.

The artificial neural network model is implemented as a multilayer perceptron (MLP) consisting of multiple layers of nodes and connections between them. The artificial neural network model according to the present embodiment may be implemented using one of various artificial neural network model structures including MLP. The artificial neural network model is located between an input layer that receives an input signal or data from the outside, an output layer that outputs an output signal or data corresponding to input data, and an output layer that receives a signal from the input layer and extracts characteristics. It consists of n hidden layers, where n is a positive integer. Here, the output layer receives a signal from the hidden layer and outputs it to the outside.

In the learning method of the artificial neural network model, there are a supervised learning method that learns to be optimized to solve a problem by input of a teacher signal (correct answer), and an unsupervised learning method that does not require a teacher signal. there is. The customized cosmetics providing server 240 may train the artificial neural network model through information on a plurality of reference images, skin characteristics of a plurality of users, and evaluation information on a plurality of cosmetics. The artificial neural network model learned in this way may be stored in a storage system, and at least one user-customized cosmetic product may be output to the user terminal.

3 is a schematic diagram illustrating a configuration in which a customized cosmetic providing server 240 is connected to communicate with a plurality of user terminals 110_1 to 110_n in order to provide artificial intelligence-based customized cosmetics according to an embodiment of the present disclosure. . The customized cosmetic providing server 240 may provide a user-customized cosmetic recommendation service based on artificial intelligence to the plurality of user terminals 110_1 to 110_n through the network 230 . According to an embodiment, the customized cosmetic providing server 240 may store, provide, and execute computer-executable programs (eg, downloadable applications) and data related to the artificial intelligence-based customized cosmetic recommendation service. It may include a server device and/or a database, or one or more distributed computing devices and/or a distributed database based on a cloud computing service. The artificial intelligence-based user-customized cosmetic recommendation service provided by the customized cosmetic providing server 240 may be provided to the user through an application for service provision installed in each of the plurality of user terminals 110_1 to 110_n.

The plurality of user terminals 110_1 to 110_n may communicate with the customized cosmetic providing server 240 through the network 230 . The network 230 may be configured to enable communication between the plurality of user terminals 110_1 to 110_n and the customized cosmetic providing server 240 . Network 230 according to the installation environment, for example, Ethernet (Ethernet), wired home network (Power Line Communication), telephone line communication device and wired networks such as RS-serial communication, mobile communication network, WLAN (Wireless LAN), Wi -Can be composed of wireless networks such as Fi, Bluetooth and Zigbee, or a combination thereof. The communication method is not limited, and a plurality of user terminals 110_1 to 110_n as well as a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, etc.) that the network 220 may include ) may also include short-range wireless communication between

The user terminal 110 may be any computing device capable of voice and/or wireless communication and in which an application for service provision may be installed and executed. For example, the user terminal 110 is a smart phone, a mobile phone, a navigation system, a computer, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, a game console (game console), It may include a wearable device, an Internet of things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, and the like.

According to an embodiment, the customized cosmetic providing server 240 may receive a customized cosmetic recommendation request from the user terminal 110 through an application for service provision. In addition, the customized cosmetic providing server 240 may receive information on one or more images of the user's facial skin and information on the user's skin characteristics determined by analyzing the user's genes, and the received one or more images One or more user-customized cosmetics may be determined based on the information on the user's skin and the user's skin characteristics, and the determined customized cosmetics may be provided to the user terminal 110 .

According to an embodiment, the customized cosmetic providing server 240 generates a user's facial skin image through the imaging device 210 so that a plurality of images of the user's facial skin can be captured at a plurality of predetermined times. The request may be made to the user terminal 110 . That is, the customized cosmetic providing server 240 may provide a plurality of instructions for a plurality of predetermined times to the user terminal 110 . For example, the plurality of instructions may be an instruction to image the user's facial skin before applying one or more cosmetics, and an instruction to image the user's facial skin after applying one or more cosmetics. In addition, the plurality of instructions may be an instruction to image the user's facial skin before washing face, an instruction to image the user's facial skin after washing face, and an instruction to image the user's facial skin immediately after waking up.

4 is a block diagram illustrating an internal configuration of a customized cosmetic providing server 240 according to an embodiment of the present disclosure. 5 is a diagram illustrating a system for performing crawling according to an embodiment of the present disclosure, and FIG. 6 is an exemplary view showing evaluation information on cosmetics received by crawling according to an embodiment of the present disclosure. 7 is an exemplary view showing information on the skin image possessed by each user according to an embodiment of the present disclosure, and FIG. 8 is information about the user's skin characteristics determined by the result of genetic analysis according to an embodiment of the present disclosure. It is an example diagram showing.

The customized cosmetic providing server 240 may include a communication module 242 , a processing module 244 , and a database 246 .

The communication module 242 receives information about one or more images of the user's facial skin from the imaging device 210 through the network 230 , and analyzes the user's genes from the genetic analysis server 220 to determine the user Receive information about the skin characteristics of In addition, the communication module 242 transmits the information on the user-customized cosmetics determined by the processing module 244 to the user terminal 110 through the network 230 .

The processing module 244 determines one or more user-customized cosmetics based on the information on the one or more images and the information on the skin characteristics of the user. In addition, the processing module 244 may be based on information on one or more images and information on skin characteristics of the user, as well as evaluation information on cosmetics, when determining the user-customized cosmetics.

The database 246 may store information corresponding to each database through the functional and structural combination of the customized cosmetic providing server 240 and software and hardware, and may search, store, and manage information stored in the database. In addition, it can be implemented in various ways such as a linked-list, a tree, and a relational database, and includes all data storage media and data structures that can store information corresponding to the database. According to an embodiment, the database 246 includes various information and information received by the customized cosmetic providing server 240 from the user terminal 110 , the imaging device 210 , the genetic analysis server 220 , the external server 510 , and the like. You can store, search, and manage data.

For example, as shown in FIG. 5 , the database 246 may be connected to the first external servers 510_1 to nth external servers 510_n through the network 230 . The database 246 may collect evaluation information on various cosmetics stored in the first external server 510_1 to the n-th external server 510_n through web crawling. The evaluation information on cosmetics received from the first external server 510_1 to the n-th external server 510_n and stored in the database 246 is, as shown in FIG. 6 , the brand name of the cosmetics and the user who used the cosmetics. of the skin type (dry, oily, combination), the user's rating, the user's skin type who searched for the corresponding cosmetic, and keywords related to the corresponding cosmetic may be included. In this case, the user's rating may be stored in association with the user's rating.

Also, as shown in FIG. 7 , the database 246 may classify and store one or more images of the user's facial skin received from the imaging device 210 for each user. At this time, the image to be stored is an image of the facial skin before washing, an image of the facial skin after washing, an image of the facial skin before applying cosmetics, an image of the facial skin after applying cosmetics, and an image immediately after waking up It may be at least one image, and when a plurality of images are stored for each user, they may be stored together with a time stamp.

Also, as shown in FIG. 8 , the database 246 may store information on the user's skin characteristics determined by analyzing the user's gene received from the genetic analysis server 220 . The user's skin characteristics stored in the database 246 may be classified into photoaging, sugar aging, skin elasticity, pigmentation, skin moisture, sebum control, acne, skin sensitivity, atopy, etc., and scores and grades for each skin characteristic (high, slightly high, normal, low, etc.), mutated genes can be stored together.

9 is a diagram illustrating an example of an artificial neural network model according to an embodiment of the present disclosure.

The artificial neural network 900 is a statistical learning algorithm implemented based on the structure of a biological neural network or a structure for executing the algorithm in machine learning technology and cognitive science. That is, in the artificial neural network 900 , as in the biological neural network, nodes that are artificial neurons that form a network by combining synapses repeatedly adjust the weights of synapses to obtain correct output and inferred output corresponding to a specific input. It represents a machine learning model with problem-solving ability by learning to reduce the error between them.

In general, an artificial neural network is implemented as a multilayer perceptron (MLP) consisting of multilayer nodes and connections between them. The artificial neural network 900 according to the present embodiment may be implemented using one of various artificial neural network structures including MLP. As shown in FIG. 9 , the artificial neural network 900 includes an input layer 920 that receives an input signal or data 910_1 to 910_n from the outside, and an output layer 940 that outputs an output signal or data corresponding to the input data. ), which is located between the input layer 920 and the output layer 940 , receives a signal from the input layer 920 , extracts characteristics, and transmits it to the output layer 940 . Here, the output layer 940 receives signals from the hidden layers 930_1 to 930_n and outputs them to the outside.

In general, the learning method of the artificial neural network 900 includes a supervised learning method that learns to be optimized to solve a problem by input of a teacher signal (correct answer), and an unsupervised learning method that does not require a teacher signal ( There is an unsupervised learning method. The artificial intelligence-based user-customized cosmetic recommendation service system according to the present disclosure uses supervised learning to provide an individualized customized cosmetic recommendation service to a user (or learner), information on a user's face image and a user An artificial neural network 900 capable of inferring user-customized cosmetics is trained by analyzing the skin characteristics of The artificial neural network 900 learned in this way can provide customized cosmetics suitable for the user's skin image and skin characteristics by inferring and generating user-customized cosmetics.

In an embodiment, the artificial neural network 900 may be trained to provide or infer customized cosmetics by receiving information on one or more images, skin characteristics of a user, and evaluation information on cosmetics. Here, the information on the one or more images may include an image in which the user's skin is rubbed.

In an embodiment, when receiving the user's skin characteristics and cosmetic ingredients, the artificial neural network 900 may be trained to output the user's skin and the input cosmetic ingredient compatibility. In this case, the information on the user's skin characteristics may be determined from the image provided by the user and/or the user's genetic information. For example, when a user's skin image is input, a separate artificial neural network trained to infer the user's skin characteristics may be generated. In addition, the cosmetic component may be extracted from the cosmetics input by the user, and each of the plurality of cosmetics and the cosmetic component may be obtained from a corresponding database. In another embodiment, when receiving the user's skin image and cosmetic ingredients, the artificial neural network 900 may be trained to output the user's skin and cosmetic ingredient compatibility. In another embodiment, when receiving information about weather, the artificial neural network 900 may be trained to infer a makeup method suitable for the weather. In this case, not only information about the weather but also skin characteristics of the user may be input to the artificial neural network 900 , and the artificial neural network 900 may be trained to infer a makeup method suitable for the user's skin characteristics and weather.

According to an embodiment, the input variable of the artificial neural network 900 may include information on one or more user skin images, user skin characteristics, and evaluation information on cosmetics. In this case, the input variables input to the input layer 920 of the artificial neural network 900 are vectors 910_1 to 910_n composed of data elements including information on one or more user skin images, user skin characteristics, and evaluation information on cosmetics. ; where n is a natural number). As such, when information on one or more user skin images, user skin characteristics, and evaluation information on cosmetics are input to the artificial neural network 900, the output variables output from the output layer 940 of the artificial neural network 900 are applied to customized cosmetics. It may be a vector (950_1 to 950_n; where n is a natural number) composed of data elements.

According to an embodiment, the input variable of the artificial neural network 900 may include the user's skin characteristics and cosmetic ingredients. In this case, the input variable input to the input layer 920 of the artificial neural network 900 may be a vector (910_1 to 910_n; where n is a natural number) comprising each of the user's skin characteristics and cosmetic components as data elements. there is. As such, when information on each of the user's skin characteristics and cosmetic ingredients is input to the artificial neural network 900, the output variable output from the output layer 940 of the artificial neural network 900 is a vector 950_1 composed of compatibility data elements. to 950_n; where n is a natural number). Alternatively, in order to output such a degree of compatibility, the artificial neural network 900 may be configured to receive information about the user's skin image instead of receiving information (eg, a vector) about the user's skin characteristics. .

According to another embodiment, the input variable of the artificial neural network 900 may include information on weather and skin characteristics of the user. In this case, the input variable input to the input layer 920 of the artificial neural network 900 is to be a vector (910_1 to 910_n; where n is a natural number) consisting of weather information and each of the user's skin characteristics as data elements. can As such, when information on weather and information on each of the user's skin characteristics are input to the artificial neural network 900 , the output variable output from the output layer 940 of the artificial neural network 900 is a vector composed of a makeup method as data elements. (950_1 to 950_n; where n is a natural number).

In this way, the input layer 920 and the output layer 940 of the artificial neural network 900 are matched with a plurality of input variables and corresponding output variables, respectively, so that the input layer 920, the hidden layers 930_1 to 930_n and the output layer 940 are matched. By adjusting the synapse values between nodes included in , it is possible to learn to infer the correct output corresponding to a specific input. Through this learning process, characteristics hidden in the input variable of the artificial neural network 900 can be identified, and the error between the output variable calculated based on the input variable and the target output is reduced between nodes of the artificial neural network 900 . You can adjust the synapse value (or weight). Using the artificial neural network 900 learned in this way, it is possible to extract customized cosmetics, ingredient compatibility, and/or makeup method.

10 is a flowchart illustrating a method for providing user-customized cosmetics based on artificial intelligence according to an embodiment of the present disclosure, and FIG. 11 is an exemplary view showing a genetic analysis result screen according to an embodiment of the present disclosure.

As shown in FIGS. 2 to 4 , the artificial intelligence-based method for providing customized cosmetics according to an embodiment includes an imaging device 210 , a genetic analysis server 220 , a communication module 242 , and a processing module 244 . ) may be performed by the customized cosmetic providing server 240 including a customized cosmetic providing system including the user terminals 110_1 to 110_n. As shown in FIG. 10 , in the method for providing user-customized cosmetics based on artificial intelligence according to an embodiment, the communication module 242 receives information about one or more images of the user's facial skin from the imaging device 210 . It may start with the step of receiving (S1010). Then, in step S1020 , the communication module 242 may receive information about the user's skin characteristics determined by analyzing the user's gene from the genetic analysis server 220 . Thereafter, in step S1030 , the processing module 244 may determine one or more user-customized cosmetics based on information on one or more images and information on a user's skin characteristics. Finally, the communication module 242 may transmit the determined information on the customized cosmetics to the user terminals 110_1 to 110_n.

Before step S1020, the genetic analysis process of the user may be performed in the gene analysis server 220, and the gene analysis server 220 may transmit the analyzed result of the user's gene analysis to the user terminals 110_1 to 110_n. . The results of gene analysis according to an embodiment are shown in FIG. 11 . The genetic analysis result may include a comprehensive result 1110 that comprehensively shows information on the user's skin characteristics such as photoaging, sugar aging, skin elasticity, pigmentation, skin moisture, sebum control, acne, skin sensitivity, atopic dermatitis, etc. there is. The synthesis result 1110 may include grade information of the user's various skin characteristics and diagram information about the user's various skin characteristics. In addition, the genetic analysis result may include a detailed result 1120 that appears when any one 1112 is selected from information about the user's various skin characteristics. The detailed result 1120 may include a score for the selected skin characteristic, a mutated gene for the selected skin characteristic, and the like.

The above-described artificial intelligence-based user-customized cosmetic providing method may be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium is distributed in a computer system connected through a network, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described embodiments can be easily inferred by programmers in the art to which the present invention pertains.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those of ordinary skill in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logic blocks, modules, and circuits described in connection with this disclosure are suitable for use in general purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the present disclosure. It may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementations, the techniques may include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on computer-readable media such as programmable read-only memory), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. It may be implemented as stored instructions. The instructions may be executable by one or more processors, and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

If implemented in software, the techniques may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer-readable medium may contain RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or desired program code in the form of instructions or data structures. may include any other medium that can be used for transport or storage to a computer and can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the Software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, wireless, and microwave, coaxial cable; Fiber optic cables, twisted pair, digital subscriber lines, or wireless technologies such as infrared, wireless, and microwave are included within the definition of a medium. As used herein, disk and disk include CD, laser disk, optical disk, digital versatile disk (DVD), floppy disk, and Blu-ray disk, where disks are usually magnetic Data is reproduced optically, while discs optically reproduce data using a laser. Combinations of the above should also be included within the scope of computer-readable media.

A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, or write information to, the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may reside within the ASIC. The ASIC may exist in the user terminal. Alternatively, the processor and the storage medium may exist as separate components in the user terminal.

Although the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in connection with any computing environment, such as a network or distributed computing environment. . Still further, aspects of the subject matter in this disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across the plurality of devices. Such devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described in connection with some embodiments herein, various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those skilled in the art to which the present disclosure pertains. Further, such modifications and variations are intended to fall within the scope of the claims appended hereto.

110: user terminal
210: imaging device
220: gene analysis server
230: network
240: server for providing customized cosmetics
242: communication module
244: processing module
246: database
510: external server

Claims (10)

In the artificial intelligence-based user-customized cosmetic providing method,
Receiving information about one or more images of the user's facial skin;
receiving information on the user's skin characteristics determined by analyzing the user's genes; and
Determining one or more user-customized cosmetics based on the information on the one or more images and the information on the user's skin characteristics
A method of providing user-customized cosmetics based on artificial intelligence, including.
According to claim 1,
Further comprising the step of receiving evaluation information on the crawled, cosmetics on the network,
A method of providing customized cosmetics based on artificial intelligence.
3. The method of claim 2,
The step of determining the user-customized cosmetics,
determining a plurality of user-customized cosmetic candidates based on the information on the one or more images and the information on the user's skin characteristics; and
selecting one or more user-customized cosmetics from among the determined plurality of user-customized cosmetics candidates based on evaluation information on the cosmetics;
A method of providing user-customized cosmetics based on artificial intelligence, including.
3. The method of claim 2,
The step of determining the user-customized cosmetics,
Including the step of inputting the information on the one or more images, the user's skin characteristics, and the evaluation information on the cosmetics into an artificial neural network model trained to provide customized cosmetics, and outputting the user-customized cosmetics,
A method of providing customized cosmetics based on artificial intelligence.
According to claim 1,
The receiving of the information on one or more images of the user's facial skin includes receiving information on a plurality of images of the user's facial skin captured at a plurality of predetermined times,
A method of providing customized cosmetics based on artificial intelligence.
6. The method of claim 5,
the plurality of images are captured at the plurality of predetermined times according to each of the plurality of instructions;
The plurality of instructions include an instruction to image the user's facial skin before applying one or more cosmetics and an instruction to image the user's facial skin after applying one or more cosmetics,
A method of providing customized cosmetics based on artificial intelligence.
According to claim 1,
The user's skin characteristics include information about at least one of ingredients of cosmetics that are not suitable for the user and current or future skin troubles of the user,
A method of providing customized cosmetics based on artificial intelligence.
According to claim 1,
receiving information about a specific cosmetic product; and,
inputting the user's skin characteristics and information on the specific cosmetics into an artificial neural network model trained to provide a degree of compatibility when receiving information about skin characteristics and cosmetics, and outputting the degree of compatibility, artificial intelligence A method of providing customized cosmetics based on the user.
According to claim 1,
receiving information about the weather; and
outputting a makeup method by inputting the user's skin characteristics and the received weather information into an artificial neural network model trained to provide a makeup method upon receiving the weather information and skin characteristics
Further comprising, artificial intelligence-based user-customized cosmetic providing method.
In the artificial intelligence-based user-customized cosmetic providing system,
a communication module configured to receive information about one or more images of the user's facial skin from the imaging device, and receive information about the user's skin characteristics determined by analyzing the user's genes from the genetic analysis server; and
and a processing module for determining one or more user-customized cosmetics based on the information on the one or more images and the information on the user's skin characteristics,
The communication module is configured to transmit the determined information on the user-customized cosmetics to the user terminal, the system for providing customized cosmetics.

Images