KR102167230B1 - SYSTEM FOR PROVIDING ARTIFICIAL INTELLIGENCE MAKE-UP SUPPORT SERVICE USING IoT BEAUTY DEVICE - Google Patents

Abstract

What is claimed is: 1. A system for providing an artificial intelligence makeup assistant service using an IoT beauty device, comprising: the IoT beauty device comprising a palette for accommodating one or more cosmetics, interworking with a user terminal, and communicating with the user terminal; And acquiring a facial feature of a user using at least one sensor included in the IoT beauty device or the user terminal, acquiring makeup assistance information corresponding to the acquired facial feature of the user, and obtaining the obtained makeup assistance information. Outputting, the user terminal; A system comprising: is disclosed.

Description

A system that provides artificial intelligence makeup assistance services using IoT beauty devices {SYSTEM FOR PROVIDING ARTIFICIAL INTELLIGENCE MAKE-UP SUPPORT SERVICE USING IoT BEAUTY DEVICE}

The present invention relates to a system for providing an artificial intelligence makeup assistant service using an IoT beauty device.

Recently, various cosmetic techniques have been developed and shared to the extent that a new word called K-beauty has arisen, and the demand to imitate them is also increasing. Accordingly, in order to provide optimized makeup information for each user, there is a need to develop a system that can assist users with makeup using artificial intelligence.

The artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike the existing rule-based smart system, the machine learns, judges, and becomes smarter. As artificial intelligence systems are used, their recognition rate improves and users' tastes can be understood more accurately, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning (deep learning) and component technologies using machine learning.

Machine learning is an algorithm technology that classifies/learns the features of input data by itself, and element technology is a technology that simulates functions such as cognition and judgment of the human brain using machine learning algorithms such as deep learning. It consists of technical fields such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/text, and includes natural language processing, machine translation, dialogue systems, question and answer, and speech recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, and image improvement. Inference prediction is a technique that logically infers and predicts information by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification), knowledge management (data utilization), and the like. Motion control is a technology that controls autonomous driving of a vehicle and movement of a robot, and includes movement control (navigation, collision, driving), operation control (behavior control), and the like.

Patent Publication No. 10-2013-0127278, published on November 22, 2013

The problem to be solved by the present invention is to provide a system for providing an artificial intelligence makeup aid service using an IoT beauty device.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

A system for providing an artificial intelligence makeup assistance service using an IoT beauty device according to an aspect of the present invention for solving the above-described problem includes a palette for receiving one or more cosmetics, and is interlocked with a user terminal, and the user Using the IoT beauty device and at least one sensor included in the IoT beauty device or the user terminal, which can communicate with a terminal, obtains a user's facial features, and a makeup aid corresponding to the acquired facial features of the user And the user terminal, which obtains information and outputs the obtained makeup aid information.

In addition, the IoT beauty device includes a first body and a second body that is tiltably coupled to the first body, and a rear surface of the second body is tiltably coupled to the rear surface of the user terminal, and the second body 2 The main body may be tilted to be configured to support the user terminal.

In addition, the first body may be coupled to the second body to be tilted and swiveled, and one of a mirror, a display, and the pallet may be provided on both surfaces of the first body, respectively.

In addition, the IoT beauty device transmits a notification signal to the user terminal when the first body is tilted from the second body, and the user terminal receives the notification signal from the IoT beauty device, the artificial You can run applications to provide intelligent makeup assistance services.

In addition, the user terminal searches for one or more makeup auxiliary images corresponding to the user's facial features, and evaluates the searched one or more makeup auxiliary images based on one or more items, and the one or more items include the one or more Each of the makeup auxiliary images may include an item related to a degree corresponding to the user's facial features and a popularity of each of the one or more makeup auxiliary images.

In addition, the user terminal sets a weight for each of the one or more items, sums the evaluation results for each of the one or more items based on the set weight, and the one or more makeup auxiliary images based on the sum result. An evaluation result for each may be calculated, a makeup auxiliary image to be output from the user terminal may be determined according to the evaluation result, and the determined makeup auxiliary image may be output.

In addition, the user terminal recognizes the user's movement using the at least one sensor, determines the user's makeup step based on the recognition result, and outputs makeup assistance information corresponding to the user's makeup step. can do.

In addition, the palette includes one or more sensors capable of detecting contact with each of the one or more cosmetics accommodated in the palette, and the IoT beauty device, when a contact with one of the one or more cosmetics is detected, the contact The detected information on the cosmetic may be transmitted to the user terminal, and the user terminal may output information on a method of using the cosmetic in which the contact is detected based on the information received from the IoT beauty device.

In addition, the system further includes a makeup tool capable of communicating with the user terminal, wherein the user terminal determines the position and movement of the makeup tool by communicating with the makeup tool, and the type of the makeup tool, the makeup tool The make-up step can be determined based on the position and movement of.

In addition, the user terminal compares the output makeup assistance information with the recognized movement of the user, and when it is determined that the makeup assistance information and the movement of the user are different, provides a notification to the user, and the The section of the makeup assistance information different from the user's movement may be output again.

Other specific details of the present invention are included in the detailed description and drawings.

According to the disclosed embodiment, a beauty device that is easy to carry in connection with the user terminal is provided, and further, by providing an IoT function to the beauty device, the user terminal is connected to the user terminal, and the user terminal provides makeup aid information based on artificial intelligence. It has the effect of allowing you to easily apply makeup anywhere.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram illustrating a system of an IoT beauty device according to an embodiment.
2 is a diagram illustrating an example in which an IoT beauty device is coupled to a user terminal.
3 is a diagram illustrating an example in which an IoT beauty device mounted on a rear surface of a user terminal supports a user terminal.
4 is a diagram illustrating an example of an unfolded state of the first body in the embodiment shown in FIG. 3.
FIG. 5 is a view showing another example of an unfolded state of the first body in the embodiment shown in FIG. 3.
6 is a view showing another example of a state in which the first body is unfolded in the embodiment shown in FIG. 3.
7 is a diagram illustrating an IoT beauty device including a rotating part according to an exemplary embodiment.
8 is a diagram illustrating an example in which a first body is rotated according to the embodiment shown in FIG. 7.
9A is a diagram illustrating an example of a method of providing an artificial intelligence makeup assistant service using an IoT beauty device and a user terminal according to the disclosed embodiment.
9(b) is a diagram illustrating an example of a method of providing an artificial intelligence makeup assistant service using an IoT beauty device according to the disclosed embodiment.
FIG. 10A is a diagram illustrating a method of providing makeup assistance information corresponding to a user's makeup step using an IoT beauty device and a user terminal according to an embodiment.
10B is a diagram illustrating a method of providing makeup assistance information corresponding to a user's makeup step using an IoT beauty device according to an embodiment.
FIG. 11(a) is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device and a user terminal according to another embodiment.
11(b) is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device according to another embodiment.
12A is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device and a user terminal according to another embodiment.
12(b) is a diagram illustrating a method of providing an artificial intelligence makeup assistance service using an IoT beauty device according to another embodiment.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and “and/or” includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

The term "unit" or "module" used in the specification refers to a hardware component such as software, FPGA or ASIC, and the "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, "sub" or "module" refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, It includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided within "sub" or "module" may be combined into a smaller number of components and "sub" or "modules" or into additional components and "sub" or "modules". Can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc., as shown in the figure It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is turned over, a component described as "below" or "beneath" of another component will be placed "above" the other component. I can. Accordingly, the exemplary term “below” may include both directions below and above. Components may be oriented in other directions, and thus spatially relative terms may be interpreted according to orientation.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a system of an IoT beauty device according to an embodiment.

Referring to FIG. 1, the system includes a server 10, an IoT beauty device 100, and a user terminal 200.

In one embodiment, the server 10 and the user terminal 200 are understood as a concept including all kinds of computing devices including one or more processors.

For example, the user terminal 200 may be understood as one of a smartphone, a tablet PC, a laptop, and a desktop computer, but is not limited thereto.

In one embodiment, the IoT beauty device 100 communicates with the user terminal 200 using Bluetooth. The type of Bluetooth is not limited, and may include Bluetooth Classic, Bluetooth 3.0, BLE, etc., but is not limited thereto.

In an embodiment, the IoT beauty device 100 may communicate with the user terminal 200 through short-range wireless communication or a network.

The network is a wired network such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), or a mobile radio communication network or a satellite communication network. It can be implemented with any type of wireless network.

In addition, the IoT beauty device 100 may communicate with the user terminal 200 through a dedicated IoT network. For example, the dedicated IoT network may include LoRa, but is not limited thereto.

In one embodiment, the IoT beauty device 100 may be detachably coupled to the user terminal 200.

Embodiments in which the IoT beauty device 100 is used in combination with the user terminal 200 will be described in detail below with reference to FIGS. 2 to 8.

In one embodiment, the user terminal 200 communicates with the server 10 via a network. The user terminal 200 receives from the server 10 at least a part of information for providing an artificial intelligence makeup assistant service according to the disclosed embodiment.

In one embodiment, the IoT beauty device 100 also includes an independent network communication module, so that at least a portion of information for providing an artificial intelligence makeup assistance service according to the disclosed embodiment may be received by communicating with the server 10. .

In one embodiment, the IoT beauty device 100 includes a first body 110 and a second body 120. In one embodiment, the first body 110 and the second body 120 are hinge-coupled, and the IoT beauty device 100 can open and close the first body 110 and the second body 120 using a hinge. It can be configured in the form of a

In an embodiment, different components may be included in the first body 110 and the second body 120. For example, the first body 110 and the second body 120 may each include at least one of a mirror, a display, and a palette including one or more cosmetics, but are not limited thereto.

In addition, at least one of the first body 110 and the second body 120 may be configured on both sides, and each may include at least one of a mirror, a display, and a palette including one or more cosmetics on both sides.

In addition, the first body 110 and the second body 120 may be coupled to each other so as to be able to tilt (unfold) and swivel (rotate).

2 is a diagram illustrating an example in which an IoT beauty device is coupled to a user terminal.

Referring to FIG. 2, the IoT beauty device 100 is coupled to the rear of the user terminal 200. The IoT beauty device 100 has a structure in which the first body 110 and the second body 120 are combined with a hinge 130 to be folded and unfolded, and one side of the IoT beauty device 100 is combined with the rear surface of the user terminal 200. I can.

A method of coupling the IoT beauty device 100 to the rear surface of the user terminal 200 is not limited, and at least one adhesive means or at least one fitting means may be used, but is not limited thereto.

3 is a diagram illustrating an example in which an IoT beauty device mounted on a rear surface of a user terminal supports a user terminal.

In one embodiment, the IoT beauty device 100 is coupled to the user terminal 200 using at least one foldable coupling means 140.

For example, the IoT beauty device 100 is coupled to the user terminal 200 using a coupling means 140 including one or more hinges, and the IoT beauty device 100 is opened by opening the coupling means 140 The terminal 200 can be supported.

That is, the IoT beauty device 100 may be collapsible with the user terminal 200, and thus may be used as a cradle supporting the user terminal 200 in an unfolded state.

4 is a diagram illustrating an example of an unfolded state of the first body in the embodiment shown in FIG. 3.

Referring to FIG. 4, in the embodiment shown in FIG. 3, the first body 110 of the IoT beauty device 100 is unfolded at an angle similar to that of the user terminal 200. For example, the second body 120 is used to support the user terminal 200, and a mirror, a pallet, or a display is provided on one side of the first body 110 so that the user It may be configured to use a mirror, a pallet, or a display provided in the first body 110 while viewing the screen.

FIG. 5 is a view showing another example of an unfolded state of the first body in the embodiment shown in FIG. 3.

Referring to FIG. 5, there is shown an example in which the first body 110 is unfolded vertically from the ground or at an angle similar thereto.

In this case, at least one of a mirror, a pallet, and a display may be provided in a direction facing one side of the second body 120 and one side of the second body 120 of the first body 110, respectively. .

In addition, at least one of a mirror, a pallet, and a display may be provided on the other side of the first body 110.

The user may use the screen of the user terminal 200 in the front direction of the user terminal 200 and a mirror, pallet, or display provided in the direction of the user terminal 200 of the first body 110 together. In addition, the user may use a mirror, a pallet, or a display respectively provided on one side of the first body 110 and the second body 120 in the rear direction of the user terminal 200.

6 is a view showing another example of a state in which the first body is unfolded in the embodiment shown in FIG. 3.

Referring to FIG. 6, an example in which the first body 110 is unfolded at an angle similar to that of the second body 120 is shown.

In this case, at least one of a mirror, a pallet, and a display may be provided in a direction facing one side of the second body 120 and one side of the second body 120 of the first body 110, respectively. .

The user may use a mirror, a pallet, or a display respectively provided on one side of the first body 110 and the second body 120 in the rear direction of the user terminal 200.

7 is a diagram illustrating an IoT beauty device including a rotating part according to an exemplary embodiment.

Referring to FIG. 7, the IoT beauty device 100 further includes a rotating part 150 capable of rotating the first body 110.

The rotating part 150 is provided separately from the hinge 130 as shown in FIG. 7, and may be connected to the first body 110 by a rotatable connecting member. As another example, the rotation unit 150 may include a ball-shaped pivot hinge that allows the first body 110 and the second body 120 to be mutually tilted (unfolded) and swiveled (rotated).

Accordingly, mirrors, pallets, or displays provided on both sides including the first side 112 and the second side 114 of the first body 110 are selectively selected from both sides according to the rotation of the first body 110. Can be used.

8 is a diagram illustrating an example in which a first body is rotated according to the embodiment shown in FIG. 7.

As shown in Figure 8, the first body 110 is rotatable through the rotating unit 150, the first surface 112 and the second surface 114, respectively, the front direction and the rear of the user terminal 200 Can be used selectively in the direction.

Hereinafter, an example of providing an artificial intelligence makeup assistant service using the IoT beauty device 100 will be described with reference to FIGS. 9 to 12.

Artificial intelligence technology includes machine learning (machine learning) technology, and among machine learning technology, it includes deep learning technology, which is particularly widely used to analyze images.

Deep learning is a set of machine learning algorithms that attempts high-level abstractions (summarizing key contents or functions in a large amount of data or complex data) through a combination of several nonlinear transducers. Is defined. Deep learning can be seen as a branch of machine learning that teaches computers how people think in a large framework.

When there is any data, it is represented in a form that can be understood by a computer (for example, in the case of an image, pixel information is expressed as a column vector, etc.) and a lot of research (how to do better) to apply it to learning. How to create representation techniques and how to create models to learn them) is in progress. As a result of these efforts, various deep learning techniques have been developed. Examples of deep learning techniques include Deep Neural Networks (DNN), Convolutional deep Neural Networks (CNN), Reccurent Neural Networks (RNN), and Deep Belief Networks (DBN). have.

Deep Neural Networks (DNNs) are artificial neural networks (ANNs) composed of a plurality of hidden layers between an input layer and an output layer.

Hereinafter, a makeup assistance service that provides optimized makeup assistance information to a user using artificial intelligence will be described in detail.

The IoT beauty device 100 described with reference to FIGS. 9 to 12 may be configured in the structure described with respect to FIGS. 2 to 8, but is not limited thereto, and various types of IoT beauty device 100 as needed. ) Can be selected and used.

In addition, the method of providing a makeup assistance service described in connection with FIGS. 9 to 12 may be performed by processing and control of the user terminal 200, but according to another embodiment, the makeup assistance service may include one or more processors and a network. It may be independently performed by the beauty device 100 including a communication module and a display.

Hereinafter, a method of using the beauty device 100 in combination with the user terminal 200 will be described in detail. However, the processor, the network communication module, and the display of the user terminal 200 may be replaced with a processor, a network communication module, and a display independently provided in the beauty device 100.

9A is a diagram illustrating an example of a method of providing an artificial intelligence makeup assistant service using an IoT beauty device and a user terminal according to the disclosed embodiment. 9(a) is a diagram showing that an IoT beauty device and a user terminal can be separated and interlocked with each other.

9(b) is a diagram illustrating an example of a method of providing an artificial intelligence makeup assistant service using an IoT beauty device according to the disclosed embodiment.

Referring to FIG. 9A, a front portion of the user terminal 200 and a first surface 112 of the first body 110 of the IoT beauty device 100 are illustrated.

Referring to FIG. 9B, a second body 120 including a display and a first surface 112 of the first body 110 of the IoT beauty device 100 are illustrated. Hereinafter, the contents described as being performed by the user terminal 200 with respect to FIG. 9(a) may be similarly performed by the second body 120 of FIG. 9(b). That is, the IoT beauty device 100 may be used in connection with the user terminal 200 or may independently perform the disclosed embodiment without the user terminal 200.

In the embodiment illustrated in FIG. 9, a pallet 170 including one or more cosmetics is provided on the first surface 112 of the first body 110. The types of cosmetics included in the palette 170 are not limited, and various color cosmetics and basic cosmetics may be included.

In one embodiment, one or more cameras 210 may be provided on the front side of the user terminal 200. In an embodiment, one or more sensors 160 may be provided on the first surface 112 of the first body 110. The sensor 160 may include an image sensor, and a light emitting unit that emits light under the control of the user terminal 200 may be included.

As described in connection with FIG. 1, the user terminal 200 and the IoT beauty device 100 can communicate using Bluetooth, and in addition, various short-range wireless communication, wired communication, and communication methods through a network are also provided by the user terminal ( 200) and the IoT beauty device 100 may be used for communication.

In one embodiment, at least one of the sensor 160 and the camera 210 is used to collect information about the user.

For example, at least one of the sensor 160 and the camera 210 is used to collect information on the user's face shape and skin condition. For example, the information collected includes the user's appearance and features, the user's skin color (skin tone), skin characteristics (e.g., dry, oily, complex, etc.), and other peculiarities of the user's skin (e.g., trouble or wound). Etc.).

The user terminal 200 outputs information that assists the user's makeup based on the collected information. For example, the user terminal 200 may acquire and reproduce an image related to a makeup method that suits the user's skin color and skin properties.

For example, the user terminal 200 may search and reproduce a YouTube makeup image of a personal makeup broadcast BJ similar to the user's skin color and skin properties. As another example, the user terminal 200 may search and play a YouTube makeup image targeting a person similar to the user's skin color and skin properties.

In one embodiment, the user terminal 200 searches for an image of a BJ similar to the user's skin color and skin properties, or an image that provides information related thereto, and filters or evaluates the searched images according to a predetermined criterion. Video can be provided.

For example, the user terminal 200 is the popularity of the video, such as the number of subscriptions, the number of followers, the number of sympathies, the number of likes, among the videos of the BJ that are similar to the user's skin color and skin characteristics, or videos that provide information related thereto. By evaluating the influence, it is possible to select and provide the video judged to have the most popularity or the highest influence.

In addition, the user terminal 200 assigns a score according to the popularity or influence of each image, and assigns a score according to the similarity between the model appearing in each image and the user's face shape, skin color, and skin condition, so that the total score is the most. Higher images can be provided to users.

When the user terminal 200 evaluates each image, a weight is assigned to the popularity or influence of each image, and the degree of matching between each image and a user, and a score reflecting the weight may be calculated and provided.

The weight may be automatically assigned or may be set differently according to the user's selection. For example, a user may want to see a more famous and popular video even if the matching level with him is slightly lower, or may want to see a video that best matches his/her own even if the level of recognition is relatively low.

Accordingly, the user terminal 200 may automatically recommend an image in consideration of this, or calculate a score by assigning a different weight to each evaluation item according to the user's selection, and recommend the image according to the calculated score.

Examples related to the above personal broadcasting, BJ and YouTube are presented as examples to aid understanding, and the scope of the present invention is of course not limited thereto.

In an embodiment, the user terminal 200 acquires makeup assistance information configured for the user's skin color and skin properties. For example, the makeup aid information includes a type of cosmetic product suitable for a user, a makeup order, and a specific makeup method according to each procedure.

In an embodiment, the user terminal 200 may recognize a user's motion and a tool held by the user, and recognize which makeup the user intends to wear. For example, if the user terminal 200 recognizes a motion of the user holding a lipstick in his hand and moving in the direction of the lips, the user terminal 200 may reproduce a make-up auxiliary image using the lipstick.

In an embodiment, the IoT beauty device 100 may provide notification information to the user terminal 200 when the unfolding motion of the first body 110 is recognized.

The user terminal 200 may recognize the opening of the IoT beauty device 100 based on information received from the IoT beauty device 100 and automatically execute an artificial intelligence makeup assistant service. For example, you can run an application that provides artificial intelligence makeup assistance services.

Accordingly, the user can execute the artificial intelligence makeup assistance service only by opening and opening the IoT beauty device 100 without having to separately execute the artificial intelligence makeup assistance service.

The user may perform makeup using an artificial intelligence makeup assistant service provided through the user terminal 200.

In an embodiment, an image of a user performing makeup may be photographed and stored by the camera 210. In an embodiment, an image of a user performing makeup may be uploaded to the server 10.

In an embodiment, an image of a user performing makeup may be shared to a social network service through the server 10. In another embodiment, an image in which a user performs makeup may be stored in a database of the server 10, and an image in which various users perform makeup may be managed as big data by the server 10.

The server 10 may provide images of users performing make-up, stored in a database, to different users to help make-up. In addition, the server 10 learns makeup methods according to the user's face shape, skin color, and skin condition by using images stored in the database, which users perform makeup, and in the future, according to the user's face shape, skin color, and skin condition. It can provide an optimized cosmetic method.

The IoT beauty device 100 may refill the contents in the pallet. Through the IoT beauty device 100, a user can replace the contents with a new product every time.

The object that can be used for refill may include a specific mark such as a QR code or a barcode, but is not limited thereto and may include a specific mark that can be recognized by a mobile device. When the IoT beauty device 100 recognizes a specific mark, content related to a product to be refilled may be automatically provided on the mobile phone screen. Contents related to products subject to the refill may be continuously updated. That is, the product-related content subject to refill may include both User Created Contents and marketing/advertising content.

In addition, since the above-described specific mark is interlocked with the server 10, only the mark made by the user can be recognized, thereby preventing the sale of similar products and duplicates.

FIG. 10A is a diagram illustrating a method of providing makeup assistance information corresponding to a user's makeup step using an IoT beauty device and a user terminal according to an embodiment. 10(a) is a diagram showing that the IoT beauty device and the user terminal can be separated and interlocked with each other.

10B is a diagram illustrating a method of providing makeup assistance information corresponding to a user's makeup step using an IoT beauty device according to an embodiment.

Referring to FIG. 10(a), as in FIG. 9(a), the user terminal 200 and the pallet 170 provided in the first body 110 are illustrated.

Referring to FIG. 10(b), as in FIG. 9(b), the second body 120 and the pallet 170 provided in the first body 110 are shown.

Hereinafter, the contents described as being performed by the user terminal 200 with respect to FIG. 10(a) may be similarly performed by the second body 120 of FIG. 10(b). That is, the IoT beauty device 100 may be used in connection with the user terminal 200 or may independently perform the disclosed embodiment without the user terminal 200.

In one embodiment, the user terminal 200 determines the user's makeup step through image analysis using the camera 210 and provides makeup assistance information corresponding to the user's makeup step.

For example, when the user holds the brush 30, the user terminal 200 determines that the user will do makeup using the brush 30, and may display a makeup method using the brush 30.

In one embodiment, the image displayed on the user terminal 200 may be an image related to an existing makeup method, or automatically generated by learning of the server 10, the user's face shape, skin color, and skin condition. It may be a video that informs you of an optimized makeup method based on. In an embodiment, an image indicating an optimized makeup method may be generated and displayed through 2D animation or 3D rendering, or may be displayed in the form of augmented reality. In this case, the user terminal 200 may display a virtual brush 32 corresponding to the recognized brush 30 on the screen.

The user terminal 200 may display an image in which the position of the recognized brush 30 is reflected, and may provide a tip for how and in which direction the brush should be moved at the corresponding position.

In one embodiment, the IoT beauty device 100 may detect that the brush 30 contacts a specific cosmetic 172 of the palette 170. For example, one or more sensors may be provided on the palette 170 to obtain information on the cosmetics 172 contacted by the brush 30.

The IoT beauty device 100 transmits information related to the type, characteristic, and color of cosmetics that the brush 30 has contacted to the user terminal 200. Based on the received information, the user terminal 200 determines that cosmetics having properties corresponding to the cosmetics 172 are applied to the virtual brush 32, and applies the cosmetics 172 to the virtual image according to the user's movement. It is possible to display an image painted with cosmetics with corresponding attributes.

In an embodiment, the user terminal 200 may display an image photographed of the user's image and provide a makeup aid image in the form of an augmented reality combined with the image.

When the user terminal 200 acquires the information selected by the cosmetics 172 from the IoT beauty device 100, information related to the appropriate location, direction, intensity, and order to apply the cosmetics according to the color and properties corresponding to the cosmetics 172 Can provide.

For example, the user terminal 200 may display an image in a form in which the cosmetics 172 is already painted on the user's face through augmented reality according to the color and type of the cosmetics 172.

In addition, the user terminal 200 may also provide information on a position, direction, intensity, and order in which the cosmetics are to be painted in order to paint the cosmetics, such as an image.

The user terminal 200 may recognize that the user is actually painting cosmetics, and determine whether or not the user is painting cosmetics according to the provided information.

When the user paints cosmetics differently from the information provided by the user, the user terminal 200 may provide a related notification or warning, and inform a method for correcting this.

In one embodiment, the brush 30 may be an IoT brush provided with a communication means capable of interworking with the IoT beauty device 110 or the user terminal 200. The user terminal 200 may acquire information on how the user is wearing makeup by recognizing the position and movement of the brush, and may provide makeup assistance information corresponding thereto.

FIG. 11(a) is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device and a user terminal according to another embodiment. 11(a) is a diagram showing that an IoT beauty device and a user terminal can be separated and interlocked with each other.

11(b) is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device according to another embodiment.

Hereinafter, the contents described as being performed by the user terminal 200 with respect to FIG. 11(a) may be similarly performed by the second body 120 of FIG. 11(b). That is, the IoT beauty device 100 may be used in connection with the user terminal 200 or may independently perform the disclosed embodiment without the user terminal 200.

Referring to FIG. 11, an example in which the screen of the user terminal 200 is divided into a first screen 220 and a second screen 230 is illustrated. Referring to FIG. 11, the user terminal 200 displays a makeup aid image on a first screen 220 and a user's image captured through the camera 210 on a second screen 230.

The user may use the second screen 230 as a mirror, and may apply makeup while comparing the makeup assistant image displayed on the first screen 220 with his own image displayed on the second screen 230.

In one embodiment, the user terminal 200 compares the first screen 220 and the second screen 230, and the user's motion displayed on the second screen 230 is displayed on the first screen 220. When different from the operation of the makeup auxiliary image, a notification may be provided, and the operation of the makeup auxiliary image displayed on the first screen 220 may be compared and displayed.

When a difference is found between the first screen 220 and the second screen 230, the user terminal 200 stops or rewinds the reproduction of the makeup aid image on the first screen 220, and plays the same section again. Or slow playback, allowing the user to easily repeat the wrong part again.

12A is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device and a user terminal according to another embodiment. 12(a) is a diagram showing that an IoT beauty device and a user terminal can be separated and interlocked with each other.

12(b) is a diagram illustrating a method of providing an artificial intelligence makeup assistant service using an IoT beauty device according to another embodiment.

Contents described below as being performed by the user terminal 200 with respect to FIG. 12(a) may be performed in the same manner by the second body 120 of FIG. 12(b). That is, the IoT beauty device 100 may be used in connection with the user terminal 200 or may independently perform the disclosed embodiment without the user terminal 200.

Referring to FIG. 12, one side of the first body 110 of the IoT beauty device 100 includes a mirror display.

In one embodiment, the mirror display reflects the user's image using a mirror, but a display is provided on the rear of the mirror or a transparent display is provided on the front of the mirror, so that an image may be superimposed on the user's mirror and displayed .

In another embodiment, the mirror display may display an image of a user's image using a camera in a mirror image, and other images may be superimposed thereon.

In an embodiment, the first body 110 may further include a pallet 180 that can be pulled out in a sliding manner. The method in which the pallet 180 is withdrawn from the first body 110 is not limited thereto, and a separate pallet separated from the first body 110 may be used.

In one embodiment, a user's image is displayed on the first body 110 and a makeup assistant image is displayed on the user terminal 200.

In an embodiment, the user terminal 200 may provide makeup assistance information to the IoT beauty device 100, and the first body 110 may display the makeup assistance information superimposed on the user's appearance.

Referring to FIG. 12, according to a user's selection or according to an order automatically set by the user terminal 200, makeup aid information, that is, information indicating a makeup method, on the appearance of the user displayed on the first body 110 Is displayed overlaid.

For example, information indicating how to draw an eyeline may be superimposed on the user's image displayed on the first body 110 and displayed.

In one embodiment, the portion where the makeup auxiliary information is provided may be enlarged and displayed at a separate location 40.

An image of a user performing makeup, an image of performing makeup according to the makeup aid information provided by the user, an evaluation of the user's makeup, feedback on whether the user performs makeup according to the provided makeup information, etc. Information obtained in the process of providing the artificial intelligence makeup assistant service according to the method is stored as big data by the server 10.

The server 10 can learn a method of providing a make-up method so that the user can easily follow the method of providing a make-up method and learn an optimized make-up method according to the characteristics of the user's face using the stored big data. have.

Accordingly, the artificial intelligence makeup assistance service according to the disclosed embodiment is learned in a direction in which artificial intelligence further develops as the service is provided, and thus, gradually improved services can be provided.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. Software modules include Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. You can understand. Therefore, the embodiments described above are illustrative in all respects, and should be understood as non-limiting.

10: server
100: IoT beauty device
200: user terminal

Claims (11)

In a system that provides artificial intelligence makeup assistance services using an IoT beauty device,
The IoT beauty device comprising a palette for accommodating one or more cosmetics, interworking with a user terminal, and capable of communicating with the user terminal; And
Acquires a user's facial features using at least one sensor included in the IoT beauty device or the user terminal, acquires makeup assistance information corresponding to the acquired facial features of the user, and stores the obtained makeup assistance information Outputting, the user terminal;
Including; a control unit for acquiring the makeup assistance information corresponding to the acquired facial features of the user, and outputting the obtained makeup assistance information on a display;
The user terminal recognizes the user's movement using the at least one sensor, determines the user's makeup step based on the recognition result, and outputs makeup assistance information corresponding to the user's makeup step;
The pallet includes one or more sensors capable of sensing contact with each of the one or more cosmetics accommodated in the pallet,
The IoT beauty device, when a contact with one of the one or more cosmetics is detected, transmits information on the cosmetics for which the contact is detected to the user terminal,
The user terminal,
A system for outputting information on a method of using the cosmetic in which the contact is detected based on the information received from the IoT beauty device.
The method of claim 1,
The IoT beauty device is:
A first body; And
A second body that is tiltably coupled to the first body; Including,
The rear surface of the second body is tiltably coupled to the rear surface of the user terminal,
The system, characterized in that the second body is configured to be tilted to support the user terminal. The method of claim 2,
The first body is coupled to the second body to be tilted and swiveled,
The system, wherein one of a mirror, a display and the pallet is provided on both sides of the first body. The method of claim 2,
The IoT beauty device,
When the first body is tilted from the second body, it transmits a notification signal to the user terminal,
The user terminal, when the notification signal is received from the IoT beauty device, executes an application for providing the artificial intelligence makeup assistant service. The method of claim 1,
The user terminal,
Searching for one or more makeup auxiliary images corresponding to the user's facial features, and evaluating the searched one or more makeup auxiliary images based on one or more items,
The one or more items include items related to a degree to which each of the one or more makeup auxiliary images corresponds to a facial feature of the user and a popularity of each of the one or more makeup auxiliary images. The method of claim 5,
The user terminal,
Set a weight for each of the one or more items, sum the evaluation results for each of the one or more items based on the set weight, and calculate an evaluation result for each of the one or more makeup aid images based on the summation result And, according to the evaluation result, determining a makeup auxiliary image to be output from the user terminal, and outputting the determined makeup auxiliary image. delete delete The method of claim 1,
The system,
Further comprising a cosmetic tool capable of communicating with the user terminal,
The user terminal,
A system for determining the position and movement of the makeup tool by communicating with the makeup tool, and determining the makeup step based on the type of the makeup tool and the position and movement of the makeup tool. The method of claim 1,
The user terminal,
Comparing the output makeup assistance information with the recognized movement of the user, and when it is determined that the makeup assistance information and the movement of the user are different, a notification is provided to the user, and the makeup different from the movement of the user A system that outputs the section of auxiliary information again.

delete

Images