KR102314167B1 - Method of curating beauty care products by analyzing user's facial contour and skin type - Google Patents

Abstract

The present invention provides a beauty care product curating method wherein the facial contour of a user based on three-dimensional depth data and the skin type based on multiple imaging cameras are sensed with the sensor of a camera module, etc. using the technology of the internet of things, and the same are processed using software algorithm to curate beauty care products and using methods thereof based on correct data sets, thereby improving user convenience and the accuracy of curating. The beauty care product curating method comprises the steps of: a device generating the facial shape data and face photographing data of a user by sensing the face of the user, and transmitting the same to a main server; the main server setting and recommending beauty care products and the using methods of the beauty care products matching the facial shape data and face photographing data of the user; and the device playing the facial contour image and the face photographing image of the user and displaying the beauty care products and the using methods of the beauty care products recommended by the main server.

Description

Method of curating beauty care products by analyzing user's face contour and skin type {METHOD OF CURATING BEAUTY CARE PRODUCTS BY ANALYZING USER'S FACIAL CONTOUR AND SKIN TYPE

The present invention relates to a method of curating a beauty care product by analyzing a user's face contour and skin type.

Human desire for beauty is endless, and interest in beauty care products (cosmetics, skin care packs, functional creams, lipsticks, etc.) is increasing day by day.

In particular, healthy and elastic skin with an appropriate skin tone is preferred by people of all ages and genders, and a makeup method that covers their shortcomings and emphasizes their strengths can change their face into an attractive and aesthetically pleasing face. They are constantly researching to improve, maintain, and find a makeup method that suits them.

In the past, a professional curator analyzed the face contour and skin type of consumers and recommended appropriate beauty care products and how to use beauty care products to customers.

However, the method recommended by a professional curator depends on the curator's experience and knowledge, and therefore does not provide accurate curation, and communication between the curator and the customer is often not smooth. there was a problem with In addition, in order to have a professional curator resident in the store, there was a problem in that the labor cost accordingly increased.

On the other hand, Internet of Things (IoT) refers to a technology or environment technology that attaches sensors to objects and exchanges data over the Internet in real time. It means providing a new service that individual objects could not provide.

In addition, artificial intelligence (AI) is a field of computer engineering and information technology that realizes human learning ability, reasoning ability, perceptual ability, and natural language understanding ability through computer programs. In particular, supervised learning, which is used in the most fields of artificial intelligence, is a method of predicting a future value using training data including a label.

If, in the curating field of beauty care products, IoT technology is used to sense the consumer's face, based on this, various beauty care products and how to use beauty care products are automatically recommended, and the effect of curating products is evaluated using artificial intelligence technology. If it is possible to predict and give feedback, it will be possible to provide accurate curation while providing convenient access to consumers.

The problem to be solved by the present invention is to detect a user's face contour (based on three-dimensional depth data) and skin type (based on a multi-image camera) with a sensor such as a camera module using IoT technology, and process it with a software algorithm for beauty It is to provide a method of curating beauty care products that can improve customer convenience and curating accuracy by curating care products and their usage based on an accurate data set.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A method of curating a beauty care product of the present invention for solving the above-described problem includes the steps of sensing a user's face in a device, generating face shape data and facial image data of the user, and transmitting the data to a main server; setting and recommending beauty care products and beauty care products that match the user's face shape data and face image data in the main server; and reproducing the user's face contour image and the captured face image from the device, and displaying beauty care products and usage of beauty care products recommended by the main server.

In the main server, a plurality of face shape data and face image data, and corresponding beauty care products and usage of beauty care products are mapped, and the main server is a beauty care product that matches the user's face shape data and face image data It may be characterized by setting and recommending how to use and how to use beauty care products.

In the main server, the user's face area is determined according to the user's face shape data and face image data, and beauty care products and how to use the beauty care products for each area are set and recommended.

transmitting an order signal for a beauty care product recommended by the main server from at least one of a user terminal or the device to the main server; storing the order signal as user's purchase history data in the main server; transmitting, from the main server to the user terminal, an alarm of the arrival of the repurchase time of the beauty care product by using the user's purchase history data; re-sensing the user's face in the device, regenerating the user's face shape data and facial image data, and transmitting it to a main server; The main server compares the facial imaging data of the previous user with the facial imaging data of the current user to generate efficacy data of the beauty care product, and a beauty care product that matches the facial shape data and facial imaging data of the current user; setting and recommending how to use beauty care products; The method may further include a step in which the current user's face contour image and the captured face image are reproduced from the device, and the beauty care product and usage of the beauty care product newly recommended by the main server are displayed.

The main server sets the facial imaging data of the previous user and the beauty care product used by the user as the feature value, and sets the efficacy data of the beauty care product as the label value to generate an artificial intelligence prediction algorithm through supervised learning, It may be characterized in that the beauty care product matching the current user's facial image data is updated using the intelligent prediction algorithm.

In the method of curating the beauty care product of the present invention, the user's face contour (based on three-dimensional depth data) and skin type (based on multi-image camera) are sensed with a sensor such as a camera module using IoT technology, and this is used as a software algorithm. It has the advantage of improving customer convenience and curating accuracy by curating beauty care products and their usage based on an accurate data set.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a conceptual diagram showing a beauty care total solution device of the present invention.
2 is a perspective view showing a beauty care total solution device of the present invention.
3 is a side view showing the movement of the display module and the rotation and movement of the chair module in the beauty care total solution device of the present invention.
4 is a conceptual diagram illustrating a graphic user interface expressed in a display module of a beauty care total solution device of the present invention.
5 is a schematic diagram showing a system in which a method of curating a beauty care product of the present invention is implemented.
6 is a flowchart illustrating a process in which a method of curating a beauty care product of the present invention is implemented.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, a beauty care total solution device 1000 of the present invention will be described with reference to the drawings. 1 is a conceptual diagram showing a beauty care total solution device of the present invention, FIG. 2 is a perspective view showing a beauty care total solution device of the present invention, and FIG. 3 is a movement and chair of the display module in the beauty care total solution device of the present invention It is a side view showing the rotation and movement of the module, and FIG. 4 is a conceptual diagram illustrating a graphic user interface expressed in the display module of the beauty care total solution device of the present invention.

The beauty care total solution device 1000 of the present invention includes a case 100, a display module 200, a chair module 300, a sensor module 400, a first driving module (not shown), and a second driving module (not shown). city) and an electronic control module (not shown).

The case 100 is an exterior member for supporting the display module 200, and may be manufactured by molding and injection of a metal material, and may have various shapes. Of the case body 110 and the case body 110, the first support leg 120 bent or curved from the lower end of one side toward the rear and the second support leg 130 that is bent or curved toward the rear from the lower end of the other side. can have a form.

The display module 200 may be disposed in the case 100 (in detail, mounted on the case body), and may be a component that generates a graphic interface for a user. Various types of displays may be used for the display module 200 , and for example, a mirror LCD/OLED display or a transparent OLED display may be used, but is not limited thereto.

Based on the data sensed by the sensor module 400, the user's "face contour image" and "face captured image" may be reproduced in the display module 200, and recommended (cured) by the electronic control module (not shown) "Beauty care products" and "How to use beauty care products" may be displayed.

The display module 200 may be provided in the form of a touch screen for the convenience of the user, and thus, the user can display "face contour image", "face captured image" and "beauty care product icon" by a simple touch operation. You can reduce or enlarge the image area of "How to use beauty care products icons", display or delete some of the categories in full screen, and zoom in or out some or all of those categories.

In this case, the "face contour image" is a three-dimensional representation of the user's face shape and may be generated based on three-dimensional death data sensed by the sensor module 400 (first sensor). Since the "face contour image" is an image obtained by analyzing a user's face shape in three dimensions, a beauty care product for shading cosmetics suitable for the user's face shape can be recommended on the display module 200 based on the image, The usage of the beauty care product may be recommended for the amount of the cosmetic, the application area, and the application time, but is not limited thereto.

Also, the "captured face image" expresses the color tone of a user's face as a multi-face captured image, and may be generated based on captured face data sensed by the sensor module 400 (second sensor). "Face image" is a multi-image image of a user's face with light in the visible wavelength band (normal light), ultraviolet light (UV light), and polarized light. Through the image, it is possible to analyze the user's eye wrinkles and the pores around the cheek, melanin and sebum of the facial skin can be analyzed through the image captured with the light in the ultraviolet wavelength band, and the cheek area through the image captured with the polarized light can analyze the pigmentation, and based on the image, beauty care products for functional cosmetics and cover cosmetics suitable for the user's skin type can be recommended on the display module 200. The usage of the beauty care product for the application time may be recommended, but is not limited thereto.

As described above, the display module 200 may be provided in the form of a mirror LCD/OLED display or a transparent OLED display, and accordingly, "face contour image", "face captured image", "beauty care product icon" and " The icon for usage of beauty care products may be displayed on a transparent display or on a display used as a mirror, so that the image area can be expressed on the transparent screen, and the image can be displayed while the user is looking at the mirror. A region can also be represented on a mirror image.

In addition, the display module 200 may have a built-in sensor module 400, and accordingly, the display module 200 is controlled by an electronic control module (not shown) so that the user can be positioned in the correct sensing area. It may move in a first direction (vertical direction) by a first driving module (not shown).

The chair module 300 may be a component mounted on the case 100 and seated by a user. Meanwhile, the chair module 300 may be disposed to be spaced apart from the display module 200 , and accordingly, the user may face the display module 200 while securing a predetermined distance.

The chair module 300 includes a chair body 310 in the form of a saddle on which a user sits, a first rod 320 extending downward from the chair body 310, and a case 100 in the first rod 320 . It may include a second rod 330 extending in the direction in which it is located.

A hinge 321 may be provided on the first rod 320 of the chair module 300 to rotate toward the chair body 310 and the case 100 . Accordingly, the manager may prepare the chair module 300 so that the user can sit by vertically standing the first rod 320 when using the beauty care total solution device 1000 of the present invention, and when not in use, the first rod It can be fixed by laying the 320 horizontally to bring the chair body 310 into contact with the case 100 .

In addition, the second rod 330 of the chair module 300 is moved by the second driving module (not shown) in the second direction (horizontal direction; the direction toward or away from the chair body 310 and the case 200 ). ) to slide. Accordingly, when the user deviates from the correct sensing area, the user can be automatically moved to be located in the correct sensing area.

The sensor module 400 may be a component embedded in the display module 200 to sense a user's state. The sensor module 400 may sense and generate "face shape data", "face image data", and "position data".

To this end, the sensor module 400 includes a first sensor that senses the user's "face shape data", a second sensor that senses the user's "face image data", and a third sensor that senses the user's "location data" It may include a sensor. The first sensor, the second sensor, and the third sensor may be mounted on the same printed circuit board (PCB) and positioned adjacent to each other, or may be mounted on separate printed circuit boards and positioned apart from each other.

Various sensing devices may be used as the first sensor, for example, may be dual cameras spaced apart from each other, and may generate 3D depth data for the user's face by binocular disparity of the dual cameras, but this It is not limited.

The user's "face shape data" is data representing the user's three-dimensional face contour, and may be in the form of three-dimensional depth data. "Face contour image" reproduced in the display module 200 can be generated using the user's "face shape data", and "beauty care products" and " “How to use beauty care products” can be curated.

Various sensing devices may be used as the second sensor. For example, light in the visible light wavelength band, light in the ultraviolet wavelength band, and polarized light are irradiated to the user's face, and the returned light is received to obtain a multi-face image. A multiple imaging camera to generate may be used, but is not limited thereto.

"Face imaging data" of a user is data obtained by capturing a user's face in two dimensions independently or by combining it with light of multiple wavelength bands in two dimensions. It may be in the form of data for an image. It is possible to analyze the user's eye wrinkles and pores around the cheeks through the image captured with light in the visible wavelength band, and melanin and sebum in the facial skin can be analyzed through the image captured by the light in the ultraviolet wavelength band. The pigmentation of the cheek area can be analyzed through the image captured with light, and thus the user's skin type can be comprehensively set. The user's "facial image data" may be used to generate a "face captured image (multiple captured image captured with various lights)" reproduced on the display module 200, and the user's skin type is determined by the electronic control module (not shown). You can curate "beauty care products" and "how to use beauty care products" that fit your needs.

Various sensing devices can be used as the third sensor. For example, the time when the light emitted from the light emitting diode is reflected from the user's face and then recursively is received by the light receiving sensor. A non-contact sensor such as a lidar to generate may be used, but is not limited thereto. Furthermore, the third sensor may be omitted, and in this case, at least one of the first sensor and the second sensor may generate a coordinate value for the position of the user's face through a focusing reference distance or the like.

The "location data" of the user may be data representing the position of the user's face as a coordinate value. It is possible to determine whether the user's face is within the effective sensing area of the first sensor and the second sensor of the sensor module 400 by using the user's "location data", and accordingly, to move the user's face into the effective sensing area After sorting, you can start sensing to get accurate sensing data.

The sensor module 400 may be disposed in various places, for example, may be embedded in the display module 200, but is not limited thereto. Meanwhile, when the sensor module 400 is embedded in the display module 200 , it may be disposed on the bezel of the display module 200 or disposed inside the screen of the display module 200 . In addition, when the sensor module 400 is embedded in the screen of the display module 200, the area corresponding to the sensor module 400 among the screen of the display module 200 is provided in the form of a transparent OLED display, and when sensing, the sensor module 400 It may be provided so that light is transmitted by transparently controlling the screen of the area corresponding to .

The first driving module (not shown) may be a component that provides a driving force for moving the display module 200 in a first direction (vertical direction; up and down direction), and the second driving module (not shown) is a chair module ( 300) in the second direction (horizontal direction; front-rear direction) may be a component providing a driving force.

In the first driving module and the second driving module, an electronic control module (not shown) detects the user's face by the third sensor of the sensor module 400 , and the effective sensing area of the first sensor and the second sensor of the sensor module 400 . When it is determined that the user has departed from , the display module 200 and the chair module 300 may be moved so that the user's face falls within the effective sensing area by driving under the control of the electronic control module.

Various driving devices may be used as the first driving module and the second driving module. For example, each of the first driving module and the second driving module is in the form of a rack-and-pinion gear and is driven to mesh with the driven rack gear and the driven rack gear. The pinion gear and the driving pinion gear may be configured to include, but are not limited to, a stepper motor for providing driving force in stages.

On the other hand, when the first driving module and the second driving module have a rack-and-pinion gear shape, the rack gear of the first driving module may be formed on both sides of the display module 200, and the leg gear of the second driving module is It may be formed on the second rod 330 of the chair module 300 . Accordingly, the second driving module may slide the second rod 330 so that the chair body 310 moves away from or closer to the case 100 .

The electronic control module (not shown) may receive and process a signal from the sensor module 400 to control the display module 200, the first driving module (not shown), and the second driving module (not shown).

In detail, the electronic control module receives the user's "face shape data" and the captured face data from the sensor module 400, processes them, and displays the user's "face contour image" and "face captured image" on the display module 200 can be reproduced, and "beauty care products" and "how to use beauty care products" can be displayed on the recommended display module 200 .

Furthermore, the electronic control module receives and processes the user's "location data" from the sensor module 400 , and drives the first driving module and the second driving module so that the user's face falls within the effective sensing area of the sensor module 400 . can do it

The electronic control module may be an electric component embedded in the case 100, and communicates with the main server 20 used in a method 2000 for curating a beauty care product of the present invention to be described later to receive various information from the main server 20. By receiving the data, "beauty care products" and "how to use beauty care products" processed by the program of the main server 20 may be displayed on the display module 200 .

Hereinafter, a method 2000 of curating the beauty care product of the present invention will be described with reference to the drawings. 5 is a schematic diagram illustrating a system in which a method of curating a beauty care product of the present invention is implemented, and FIG. 6 is a flowchart illustrating a process in which a method of curating a beauty care product of the present invention is implemented.

The method 2000 of curating a beauty care product of the present invention may be performed in a system including the beauty care total solution device 1000 , the user terminal 10 , the network 20 , and the main server 30 .

In this case, the "beauty care total solution device 1000" is owned by the administrator and may be a device installed on an offline store, and the "user terminal 10" is owned by the user and may be a smart device carried by the user, but limited thereto it's not going to be

Since the beauty care total solution device 1000 has been described above, a description thereof will be omitted. Hereinafter, “beauty care total solution device 1000” will be abbreviated as “device 1000”.

The user terminal 10 may be a terminal in which the method 2000 of curating the beauty care product of the present invention and the related web page, app page (app list), program or application are provided to the user.

The user terminal 10 may be implemented as a computer that can access the remote main server 30 through the network 20, for example, a laptop computer equipped with a navigation system, a web browser, or a desktop computer. , a laptop, and the like.

In detail, the user terminal 10 is a wireless communication device that guarantees portability and mobility, and includes navigation, Personal Communication System (PCS), Global System for Mobile communications (GSM), Personal Digital Cellular (PDC), and Personal Handyphone System (PHS). ), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone All types of handheld-based wireless communication devices such as (smartphone), smart pad (smartpad), tablet PC (Tablet PC), etc. may be included, but are not limited thereto. The user terminal 10 may be connected to the main server 30 through the network 20 .

The network 20 means a connection structure capable of exchanging information between each node, such as the user terminal 10 and the main server 30 , and an example of such a network is an RF, 3rd Generation Partnership Project (3GPP) network. , LTE (Long Term Evolution) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, NFC network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. include, but are not limited thereto.

The main server 30 may be a server that provides a web page, an app page, a program or an application related to the method 2000 of curating a beauty care product of the present invention.

The method 2000 of curating the beauty care product of the present invention includes the steps of sensing the user's face in the device 1000 to generate the user's "face shape data" and "facial image data" and transmitting them to the main server (s1) ; setting and recommending "beauty care products" and "how to use beauty care products" matching the user's "face shape data" and "facial image data" in the main server 30 (s2); Step (s3) in which the user's "face contour image" and "facial captured image" are reproduced in the device 1000, and "beauty care products" and "how to use beauty care products" recommended by the main server 30 are displayed (s3) ; transmitting an "order signal" for "beauty care products" recommended by the main server 30 from at least one of the user terminal 10 or the device 1000 to the main server 30 (s4); Storing the "order signal" as the user's "purchase history data" in the main server 30 (s5); transmitting, from the main server 30 to the user terminal 10, an "alarm of the arrival of the repurchase time of the beauty care product" using the user's "purchase history data" (s6); re-sensing the user's face in the device 1000, regenerating the user's "face shape data" and "facial image capturing data" and transmitting them to the main server 30 (s7); The main server 30 compares the previous user's "facial image data" with the current user's "facial image data" to generate "beauty care product efficacy data", and the current "user's face shape data" and setting and recommending “beauty care products” and “how to use beauty care products” that match “face image data” (s8); A step in which the current "user's face contour image" and "face captured image" are reproduced in the device 1000, and "beauty care products" and "how to use beauty care products" newly recommended by the main server 30 are displayed (s9) may be included.

The device 1000 senses the user's face to generate the user's "face shape data" and "face image data" and transmits them to the main server (s1) . The user's "face shape data" and "facial image data" are generated by sensing the face, and the user's "face shape data" and "face image data" generated by the sensor module 400 in the electronic control module of the device 1000 are generated. may be a step of transmitting to the main server 30 .

In the step (s2) of setting and recommending "beauty care products" and "how to use beauty care products" matching the user's "face shape data" and "facial image data" in the main server 30, the main server 30 It may be a step of processing user data and setting "beauty care products" and "how to use beauty care products" suitable for users.

In this case, the "face shape data" of the user may be data used to express the user's face in a three-dimensional form, and the "facial image data" is a multi-imaged image for setting the skin type of the user. It may be data used for representation.

The main server 30 may process the user's "face shape data" so that the user's "face contour image" is reproduced in the device 1000 to be described later. In this case, the "face contour image" is a three-dimensional representation of the user's face shape and may be generated based on three-dimensional death data sensed by the sensor module 400 (first sensor) of the device 1000 .

In addition, the main server 30 may process the user's "captured face data" so that the user's "captured face image" is reproduced in the device 1000 to be described later. In this case, the "captured face image" represents the color tone of the user's face as a multiple-face captured image, and may be generated based on face captured data sensed by the sensor module 400 (second sensor) of the device 1000 .

Furthermore, the main server 30 processes the user's "face shape data" and "facial image data" to select "beauty care products" and "how to use beauty care products" that match each user's face shape and skin type. can

In detail, the main server 30 stores "a plurality of face shape data" and "facial image data" reflecting a variety of individual data, and how to use the corresponding "beauty care products" and "beauty care products" This mapping makes it possible to set and recommend "beauty care products" and "how to use beauty care products" that match the user's "face shape data" and "facial image data".

As an example, since the "face contour image" is an image obtained by analyzing the user's face shape in three dimensions, the main server 30 recommends a beauty care product for shading cosmetics suitable for the user's face shape based on the image. In addition, the usage of the beauty care product for the application amount, application area, and application time of the corresponding cosmetic product may be recommended, but is not limited thereto.

As an example, the "face captured image" is a multi-image image of a user's face with light in the visible wavelength band (normal light), ultraviolet light (UV light), and polarized light, and the visible light wavelength band It is possible to analyze the user's wrinkles around the eyes and pores around the cheeks through the image captured by The pigmentation of the cheeks can be analyzed through the main server 30, and beauty care products for functional cosmetics and cover cosmetics suitable for the user's skin type can be recommended based on the image, and the application amount of the product and the The usage of the beauty care product for the application area and application time may be recommended, but is not limited thereto.

Furthermore, the main server 30 determines the user's face area according to the user's "face shape data" and "facial image data", and sets and recommends beauty care products and how to use beauty care products for each area. .

This is in consideration of the fact that the application area and amount of special-purpose cosmetics such as shading cosmetics are different depending on the face shape in each area of the face, and the application area and amount of special-purpose cosmetics such as acne care products are different depending on the skin type. , for example, in the main server 30, the user's face is generally applied to the first area (the entire area) where the basic cosmetics are applied, the second area (the cheeks or forehead area) to which the shading cosmetics are applied, and the acne care products are applied. It is classified into the third area (areas with severe acne such as under the chin), and according to each area, "beauty care products" and "how to use beauty care products (application amount and application period (cycle) and application method, etc.; application amount depends on the product "Set according to the area of the face area to be applied and the skin type (for example, set according to the severity of acne-prone skin)" may be set. Meanwhile, each region of the user's face may have overlapping regions as described above.

Step (s3) in which the user's "face contour image" and "facial captured image" are reproduced in the device 1000, and "beauty care products" and "how to use beauty care products" recommended by the main server 30 are displayed (s3) of the “face contour image”, “face captured image”, “beauty care product” and “beauty care product” output based on user data from the main server 30 in the electronic control module (not shown) of the device 1000 It may be a step of receiving "how to use" and controlling the display module 200 of the device 1000 to express the image, product, and usage on the graphic user interface.

The step (s4) of transmitting the "order signal" for the "beauty care product" recommended by the main server 30 from at least one of the user terminal 10 or the device 1000 to the main server 30 (s4) is that the user It may be a step of ordering a product curated by the main server 30 using the terminal 10 or the device 1000 .

The step (s5) of storing the "order signal" as the user's "purchase history data" in the main server 30 is stored in the form of "purchase history data" by databaseizing the user's purchased products by date in the main server 30 It may be a step to

The step (s6) of transmitting the “alarm of the arrival of the repurchase time of the beauty care product” by using the user’s “purchase history data” from the main server 30 to the user terminal 10 (s6) is performed by the program of the main server 30 itself. It may be a step of notifying the user that the time to repurchase the product has arrived by predicting when the product is consumed from the date the user purchases the product.

In this case, the main server 30 specifies the type and amount of the product purchased by the user from the "purchase history data", and selects beauty in each area of the user's face in the "how to use beauty care products" previously recommended to the user. By specifying the amount and period in which the care product is applied, when the total exceeds a certain level of the amount of the product, the user may be notified that the repurchase period has arrived.

In the step (s7) of re-sensing the user's face in the device 1000, regenerating the user's "face shape data" and "facial imaging data" and transmitting the data to the main server 30 (s7) , the user obtains the product and then goes back to the offline shop It may be a step of visiting and receiving an examination on the device 1000 . In this case, the user's data may be sensed by the sensor module 400 of the device 1000 and transmitted to the main server 30 by an electronic control module (not shown) of the device 1000 .

The main server 30 compares the previous user's "facial image data" and the current user's "facial image data" to generate "beauty care product efficacy data", and the current user's "face shape data" and In the step s8 of setting and recommending "how to use beauty care products" and "beauty care products" matching the "face image data" , the main server 30 compares the "facial image data" of the previous user and the current user by comparing the “facial image data” of the user to identify the areas where the user's skin condition has improved or deteriorated, output the efficacy index for the beauty care products used in the area, and set the products and usage that match the current user data It may be a step to In this case, since the matching product is different for each zone of the user's face, each zone in which the user's face is zoned is divided, and the previous imaging data and the current imaging data are compared to set the efficacy index of the product.

A step in which the current "user's face contour image" and "face captured image" are reproduced in the device 1000, and "beauty care products" and "how to use beauty care products" newly recommended by the main server 30 are displayed In (s9) , the electronic control module (not shown) of the device 1000 receives the image of the user's face, the recommended product, and usage from the main server 30, and controls the display module of the device 1000 to display the image and It may be a step of controlling the product and usage to be displayed on the graphic user interface.

Furthermore, in the method 2000 of curating beauty care products of the present invention, the list of beauty care products recommended to the user may be updated by using artificial intelligence.

For reference, artificial intelligence can be defined as a methodology that gives a computer the ability to learn on its own without being explicitly programmed, and this is a method that allows a program to learn patterns from data on its own based on data. Machine learning is divided into supervised learning and unsupervised learning depending on whether the correct answer is specified in the data required for learning, and classification by dividing data into finite categories according to the purpose of use, continuous It is divided into regression, which maps to values, clustering, which groups similar data, and dimensional reduction, which maps multidimensional data to a representative lower dimension.

Deep learning, one of the machine learning techniques, has dramatically improved the performance of machine learning, which has been stagnant for a while. , as a deep learning structure, a deep neural network (DNN) in which a plurality of hidden layers exist between an input layer and an output layer, and a filter required for factor extraction in front of the hidden layer Convolutional Neural Network (CNN) that learns filters together with Here, the high performance of the deep learning model is explained in two ways. First, the neural network is a universal approximator that can approximate all kinds of functions by superimposing the weighted sum of functions in each layer. Data can be simulated with high accuracy. Second, in order to classify data well, it is important to properly extract the factors representing the data, and it is possible to extract the optimal factors through filter learning using a spiral neural network. In addition, deep learning is a developed form of a model in the field of artificial intelligence called a neural network. Recent deep learning models have many inner layers, so the number of weights (meaning the strength of connection) connecting nodes can be up to billions.

The main server 30 sets the "face image data" of the previous user and the "beauty care product" used by the user as a feature value (independent variable) in order to learn the artificial intelligence prediction algorithm, and sets the "beauty care product's Efficacy data" is set as a label value (dependent variable) to generate an artificial intelligence prediction algorithm through supervised learning, and beauty care that matches the current user's "facial image data" using the generated artificial intelligence prediction algorithm You can update the product.

That is, in the main server 30, "face image data" and actually used "beauty care products (extracted from purchase history data of multiple users)" from a plurality of users in the past are used as feature values according to the efficacy of the products actually used by the user. “Efficacy data of beauty care products (expressed as efficacy index)” as a label value to build a dataset to learn artificial intelligence By inputting "beauty care product" data according to ", according to the degree of "result value (efficacy data of beauty care product)", if the effect of the corresponding "beauty care product" is predicted to be low, it can be removed from the database.

Furthermore, "face area data" of users in the past may be included as feature values for learning the artificial intelligence prediction algorithm. After the prediction algorithm is completed, you can predict the efficacy of the beauty care products used on a specific skin type in that face area by inputting the "corresponding face area" and its "facial imaging data" and "beauty care product". and, accordingly, the list of beauty care products may be updated.

The method 2000 of curating the beauty care product of the present invention described above may be implemented as a program (or application) to be executed in combination with a server, which is hardware, and stored in a medium.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to functions defining functions necessary for executing the methods, etc. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected by a network, and a computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (5)

A device comprising: sensing a user's face to generate face shape data and face image data of the user and transmitting the data to a main server;
setting and recommending beauty care products and beauty care products that match the user's face shape data and face image data in the main server;
reproducing the user's face contour image and the captured face image in the device, and displaying beauty care products and usage of beauty care products recommended by the main server;
transmitting an order signal for a beauty care product recommended by the main server from at least one of a user terminal or the device to the main server;
storing the order signal as user's purchase history data in the main server;
transmitting, from the main server to the user terminal, an alarm of the arrival of a repurchase time of a beauty care product by using the user's purchase history data;
re-sensing the user's face in the device, regenerating the user's face shape data and facial image data, and transmitting it to a main server;
The main server compares the facial imaging data of the previous user with the facial imaging data of the current user to generate efficacy data of the beauty care product, and a beauty care product that matches the facial shape data and facial imaging data of the current user; setting and recommending how to use beauty care products; and
In the device, the current user's face contour image and face captured image are reproduced, and the beauty care product and usage of the beauty care product newly recommended by the main server are displayed,
The device is
case;
a display module mounted on the case and configured to reproduce a graphic interface;
a chair module mounted in the case to be spaced apart from the display module, and on which a user sits;
a sensor module embedded in the display module and sensing a user's state;
an electronic control module receiving a signal from the sensor module and controlling the display module;
a first driving module for vertically moving the display module; and
a second driving module for moving the chair module in a horizontal direction;
The sensor module includes a first sensor for sensing face shape data, a second sensor for sensing face image data, and a third sensor for sensing user location data, wherein the first sensor is a dual camera spaced apart from each other Generates three-dimensional depth data for the user's face by the binocular disparity of , and the second sensor irradiates visible light wavelength band light, ultraviolet light wavelength band light, and polarized light to the user's face, and returns the light A multi-face image is generated by receiving the light, and the third sensor generates a coordinate value for the position of the user's face with the time when the emitted light is received,
The sensor module is embedded in the screen of the display module, and in the display module, an area corresponding to the sensor module of the screen is provided in the form of a transparent OLED display so that the screen of the area corresponding to the sensor module is transparently controlled during sensing. is provided to pass through,
The electronic control module receives and processes the user's location data from the sensor module, drives the first driving module and the second driving module so that the user's face falls within an effective sensing area of the sensor module, and the sensor module and moving the chair module,
In the main server, a plurality of face shape data and face image data, and corresponding beauty care products and usage of beauty care products are mapped. Set and recommend how to use beauty care products,
The main server determines the user's face area according to the user's face shape data and face image data, sets and recommends beauty care products and usage of beauty care products for each area,
The main server receives the order signal for the beauty care product recommended by the main server of the user terminal and stores it as the user's purchase history data, and specifies the type and amount of the product purchased by the user from the user's purchase history data In the usage of beauty care products, specify the amount and period of application of beauty care products in each area of the user's face, and when the amount of purchased products exceeds a certain level, the user terminal is alerted to the arrival of repurchase time of beauty care products. A method of curating beauty care products, characterized in that sending
delete delete delete According to claim 1,
The main server sets the facial imaging data of the previous user and the beauty care product used by the user as the feature value, and sets the efficacy data of the beauty care product as the label value to generate an artificial intelligence prediction algorithm through supervised learning, A method of curating beauty care products, comprising updating beauty care products that match current user's facial image data using an intelligent prediction algorithm.

Images