KR20120108817A - System and method for virtual make-up service, and device supporting the same - Google Patents

Abstract

PURPOSE: A virtual makeup service system, a method thereof, and a terminal supplying the same are provided to generate model data applying texture corresponding to the type of user skin and to apply texture to the generated model data according to various makeup. CONSTITUTION: A terminal(100) collects a plurality of images related to a subject(50). The terminal transmits the collected image to an external service unit(300). The terminal receives various texture information and model data about the image from a service device. The terminal applies the textual information to the model date. The service apparatus selects texture information and generates the model data. [Reference numerals] (200) Network; (300) Service device; (301) Communication unit; (303) Virtual makeup supplying unit; (305) Model generating unit

Description

System and method for virtual make-up service, and device supporting the same}

The present invention relates to a virtual makeup service, and more particularly, to a virtual makeup service system and a method and a terminal supporting the virtual makeup service that can provide a virtual makeup state applying user characteristics and surroundings based on model data.

Recently, with the remarkable development of information and communication technology and semiconductor technology, the spread and use of various terminals is rapidly increasing. In particular, among recent terminals, portable terminals have reached the stage of mobile convergence, which does not stay in their own traditional areas, but encompasses the range of other terminals. In the case of a mobile communication terminal, in addition to general communication functions such as voice call and message transmission and reception, a TV viewing function (for example, a mobile broadcast such as Digital Multimedia Broadcasting (DMB) or Digital Video Broadcasting (DVB)) and a music playback function (for example, MP3 (MPEG Audio Layer-3), picture taking function, Internet connection, and other features are being added.

Meanwhile, due to the development and expansion of communication networks, the Internet population is rapidly increasing, and with the growth of the Internet, various contents including e-commerce, portals, and the like are appearing. In addition, recently, the boundary between the online and offline domains has been broken, replacing the existing offline service with an online service. As the standard of living improves, the quality of life and personality are becoming more important, and the desire and interest in the management of makeup is greatly increased.

For example, technology such as make-up is rapidly increasing information sharing including various methods and cases through the development of communication network. Accordingly, users who want to make up can access the network and test or apply the makeup according to the makeup information provided by the network.

Makeup technology, on the other hand, is characterized by the characteristics of each user and the surrounding environment. That is, each user basically has a different skin type, a different skin color, and a different purpose of makeup by the user, and as a result, the surrounding environment for completing and performing makeup may be different for each user. However, since the conventional make-up information simply discloses a method of performing a user according to a specific user, there is a problem in that the user does not fit well with a specific user. In addition, a variety of cosmetic products are used to perform makeup. In the case of such cosmetic products, there is a problem that unnecessary purchase may occur when purchasing a product that does not suit the user because it is a method of using after purchase.

Accordingly, the present invention has been proposed to solve the above-mentioned problems of the prior art, and an object of the present invention is to provide a model data for a specific part of a user to which makeup is applied, for example, a head (including a head and a face) of a subject, An object of the present invention is to provide a virtual makeup service system and method capable of applying an appropriate virtual makeup by applying skin characteristic information for each user, and a terminal supporting the same.

It is also an object of the present invention to provide a virtual makeup service system and method for allowing a user to make a desired makeup by observing a degree of makeup change applied to an environment element to model data, and a terminal supporting the makeup.

According to an embodiment of the present invention, a virtual makeup service system collects a plurality of images related to a subject and transmits the collected images to a service device, and transmits the collected images to the service device. A terminal for receiving various texture information to be applied to the model data based on the model data and the user selection information, and applying the received texture information to the model data and outputting it; And a service device for receiving a subject-related image from a terminal, generating model data, providing the model data to the terminal, and selecting and transmitting a plurality of texture information to be applied to the model data based on user selection information to the terminal.

In addition, as another means for solving the above-described problem, the service apparatus for supporting a virtual makeup service according to an embodiment of the present invention, by forming a communication channel with the terminal to receive the subject-related image and texture-related user selection information A communicator configured to transmit model data and texture information generated from the subject-related image to the terminal; A model generator configured to generate model data rendered to have a plurality of polygons based on the subject-related image transmitted by the communication unit; And a virtual makeup providing unit selecting texture information to be applied to the model based on the user selection information received through the communication unit and providing the texture information to the terminal through the communication unit.

In the service apparatus according to the present invention, the model generator delivers the generated model data to the virtual makeup provider, the virtual makeup provider applies a texture according to the user selection information to the model data, and applies the texture model data to the communication unit. Characterized in that provided to the terminal through.

In the service apparatus according to the present invention, the virtual makeup providing unit includes: a database storing various skin type textures, various tone textures, and various lighting textures; At least one skin type texture selected according to the skin type information of the user selection information among the textures stored in the database, at least one texture according to the tones indicated by the user selection information, and at least according to illumination indicated by the user selection information. And a control module for searching for and transmitting at least one of the textures of one illumination.

In the service apparatus according to the present invention, at least one of the communication unit, the virtual makeup provider, and the model generator may be implemented as one or more servers operating on a cloud computing basis.

In addition, as another means for solving the above problems, the terminal supporting the virtual makeup service according to a preferred embodiment of the present invention, the input unit for receiving the user's selection information; A controller configured to select a specific texture provided according to skin type, lighting, and color tone based on user selection information selected according to an input signal of the input unit, and apply the selected texture to model data generated based on a subject-related image and output it; And a display unit configured to output a screen to which texture information is applied to model data under the control of the controller.

In the terminal according to the present invention, the controller comprises: a model generator for generating model data composed of a plurality of polygons based on a subject-related image; A texture processing unit which selects a texture to be applied to model data according to user selection information transmitted from an input unit and transmits the texture to the model reproducing unit; And a model reproducing unit for applying the texture transmitted from the text processing unit to the model data and outputting the texture to the display unit.

In the terminal according to the present invention, the texture may include at least one of a skin type texture divided according to the skin type for each facial region, a texture for each color tone according to various tones, and a texture for each light according to various illuminations.

In the terminal according to the present invention, the texture processing unit displays a makeup editor for allowing a user to select at least one of a skin type texture, a tone texture, and a lighting texture and a portion to which the corresponding texture is applied in the model data. To output to.

In the terminal according to the present invention, when the texture processing unit is selected to apply a plurality of textures to a predetermined portion of the model data, the texture processing unit generates a texture having a color combined with a plurality of textures, and controls to apply the texture to a specified portion do.

The terminal according to the present invention may further include a storage unit which stores one or more of a plurality of textures applicable to the subject-related image, the subject-related model data, and the model data.

The terminal according to the present invention further includes a communication unit for transmitting and receiving data through a network, wherein the control unit receives one or more of a subject-related image, subject-related model data, and a plurality of textures applicable to the model data from an external device through the communication unit. can do.

The terminal according to the present invention may further include a camera for photographing a subject-related image, and the controller may further perform a function of generating model data from the subject-related image photographed by the camera.

In addition, as another means for solving the above problems, the virtual makeup service method of the terminal according to a preferred embodiment of the present invention, a collection step of collecting a plurality of images related to the subject; Generating model data rendered to include a plurality of polygons based on the subject images; An applying step of applying textures according to user selection to model data; And outputting the model data to which the texture is applied.

In the virtual makeup service method according to an embodiment of the present invention, the applying step may select at least one of a skin item texture item, a color tone texture item, and a light texture item classified according to the skin type of the face part, and among the textures. Displaying a makeup editor including an item for selecting a portion of model data to which the selected texture is to be applied; Receiving an input signal for selecting any one of items displayed on the makeup editor; The method may include applying the selected texture to model data according to the input signal.

In the virtual makeup service method according to the present invention, the applying step includes applying a texture for each skin type determined according to user selection to model data; Applying a texture for each color tone determined according to a user selection to model data to which the texture for each skin type is applied; The method may include at least one step of applying a texture for each illumination determined according to a user's selection to the model data to which the texture for each tone is applied.

The present invention also provides a computer readable recording medium having recorded thereon a program for executing the above-described virtual makeup service method.

As described above, according to the virtual makeup service system and method and a terminal supporting the same, the present invention provides a model in which a texture corresponding to a skin type for each user is applied based on model data of a subject on which makeup is performed. By creating data and applying textures to various generated color tones to the generated model data, it is easy to search for makeup techniques and cosmetics that are suitable for real users. We can support you to observe.

1 is a flowchart illustrating a virtual makeup service method in a terminal according to an embodiment of the present invention.
2 is a view showing an example of model data of polygon type according to an embodiment of the present invention.
3 is a diagram illustrating an example of model data to which a texture is applied according to an embodiment of the present invention.
4 is a view showing in more detail the configuration of a terminal according to an embodiment of the present invention.
5 is a view showing in more detail the configuration of the control unit of the terminal configuration of the present invention.
6 is a view schematically showing the configuration of a virtual makeup service system according to an embodiment of the present invention.
7 is a diagram illustrating a signal flow between components of a system for explaining a virtual makeup service method according to an exemplary embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that, in the drawings, the same components are denoted by the same reference numerals as possible. Further, the detailed description of known functions and configurations that may obscure the gist of the present invention will be omitted.

It should be noted that the embodiments of the present invention disclosed in the present specification and drawings are only illustrative of the present invention in order to facilitate the understanding of the present invention and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention can be carried out in addition to the embodiments disclosed herein.

1 is a flowchart illustrating a virtual makeup service method according to an embodiment of the present invention. The virtual makeup service method according to an embodiment of the present invention is based on a terminal, wherein the terminal is all mobile communication terminals operating based on communication protocols corresponding to various communication systems. terminals, as well as portable multimedia players (PMPs), digital broadcasting players, personal digital assistants (PDAs), music players (e.g. MP3 players), mobile game terminals, smart phones, notebooks, PCs, And all kinds of user devices such as all information communication devices such as tablet PCs, multimedia devices, and application devices thereof.

Referring to FIG. 1, when power is supplied to a terminal according to the present invention and an operation is executed, the terminal outputs a standby screen waiting for a user input to execute a function according to a user input as in step 101.

In the standby screen state, when the user's input signal is generated, the terminal branches to step 103 and checks whether the corresponding input signal is an input signal for activating the virtual makeup function. In this step, if the corresponding input signal is not an input signal for activating the virtual makeup function, the terminal branches to step 105 and functions of the terminal corresponding to the input signal, for example, a file play function, a file search function, a broadcast reception function, and a call. Control to perform a function or the like.

On the other hand, when an input signal for activating the virtual makeup function is generated in step 103, the terminal branches to step 107 to generate model data. To this end, the terminal may control to collect a specific image of the user who wants to use the virtual makeup, for example, a subject image including a head and a face. In this case, the terminal may output a guide message through a display unit or audio processing to collect images from various angles with respect to the subject so as to generate appropriate model data based on the collected subject related image. The user may operate to collect a plurality of images from various angles with respect to the subject according to the guide message. When a plurality of images are collected, the terminal may generate model data based on the collected images. In particular, the terminal may generate model data in 3D form rendered in a plurality of polygon types as shown in FIG. 2 using the plurality of images.

Next, the terminal branches to step 109 to control to apply the texture according to the skin type selected by the user to the model data. To this end, the terminal may pre-store textures according to the skin type of each part of the face, for example, dry, neutral, or oily, or may be provided from an external device through a network. Accordingly, when the user selects a skin type for each part, the terminal loads a texture according to the selected skin type and applies it to each part of the model data to generate model data to which the texture for each skin type is applied as shown in FIG. 3. can do.

Next, the terminal branches to step 111 to check whether an input signal for outputting the makeup editor is generated. In this step, when the input signal for the makeup editor output is generated, the terminal may branch to step 113 and control to apply the texture according to the color tone selected by the user to the model data to which the texture for each skin type is applied. To this end, the terminal may store in advance a texture for each color to be applied to model data for each color tone or may be provided from an external device. The texture for each color tone may be output as a value reflecting the color of the texture for each skin type to be located. In addition, the texture for each hue may have values reflecting various hue, contrast, and saturation values, and the value may be changed according to the position of each face part, and may be set to have the same value in all face parts according to a user setting. Can also be. To this end, the user may activate the makeup editor and generate an input signal for selecting a color tone and a part of model data provided by the makeup editor. The makeup editor may then control the selected color to be applied to the output model data.

On the other hand, the terminal branches to step 115 may determine whether there is an input signal generation for the illumination selection. In this step, if there is an input signal generation for light selection, the controller branches to step 117 to apply a texture according to the selected light to the model data. In more detail, the terminal supports to include an item for selecting lighting in the makeup editor, and the user can select a desired surrounding environment by selecting the corresponding item and selecting the type of lighting. For example, the user may select a time, for example, noon, and enter weather information when the makeup is completed. In addition, the user may input information on the inside of the building or the outside of the building as information on the activity place. When the above-described various surrounding environment information is input, the makeup editor may calculate an illumination value according to the surrounding environment information and control the application of a texture according to the illumination. To this end, the makeup editor may provide items that the user can select for various surrounding environments. In addition, the makeup editor can control to change the texture of each applied color tone according to the lighting and output it.

On the other hand, if there is no separate input signal for outputting the makeup editor in step 111, or there is no input signal for light selection in step 115, and when the texture is applied according to the selected light in step 117, the terminal enters step 119. You can check whether an input signal is generated to terminate the function. In this case, when there is no input signal for terminating a separate function, the terminal branches to step 103 and may control to repeatedly perform the following process. In step 119, when an input signal for ending the function occurs, the terminal may return to step 101 and perform the following procedure again.

As described above, the virtual makeup service method of the terminal according to the present invention regenerates the input model data with a texture applied differently according to the skin type of each part to the model data rendered with a plurality of polygons, and distinguishes them by hue and lighting thereon. By applying the textured texture, it is possible to provide a makeup application environment close to the real life.

4 is a block diagram schematically illustrating a configuration of a terminal 100 supporting a virtual makeup service according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the terminal 100 of the present invention includes a communication unit 110, an input unit 120, an audio processor 130, a display unit 140, a storage unit 150, a camera 170, and a controller 160. It may include. As described above, the terminal 100 includes all mobile communication terminals operating in accordance with communication protocols corresponding to various communication systems, including a portable multimedia player (PMP), Digital communication player, PDA (Personal Digital Assistant), music player (e.g. MP3 player), portable game terminal, smart phone, notebook, notebook, PC and tablet PC It may include an application for.

The terminal 100 having the above-described configuration collects images of various angles with respect to a subject by operating the camera 170 and generates model data 153 rendered to have a plurality of polygons based on the collected images. Can be. The model data 153 includes, for example, model data implemented on a 3D basis. When the model data 153 is generated, the terminal 100 generates the model data to which the texture for each skin type is applied by applying a texture according to the skin type for each part of the face according to the user's selection to the model data 153. According to the present invention, the texture of the skin type may be supported to generate the model data with the texture of the tone applied. In addition, the terminal 100 of the present invention may apply to change the color, brightness and saturation of the texture by applying the lighting effect selected by the user to the model data to which the texture for each color tone is applied. Hereinafter, each configuration of the terminal 100 for supporting the virtual makeup service of the present invention will be described in detail.

The communication unit 110 performs the formation of a communication channel for transmitting a voice signal and / or data of the terminal 100 and data transmission and reception under the control of the controller 160. That is, the communication unit 110 may form a voice call channel, a data communication channel, a video call channel, and the like with various network devices through a network. In particular, the communication unit 110 may receive a texture for each skin type, a texture for each color tone, a light effect calculation algorithm, etc. to be applied to the model data 153 generated with a plurality of polygon types from an external device through a network. In addition, the communication unit 110 may receive an image related to a subject through another terminal or a network.

The input unit 120 may receive numeric or various character information, and may include input keys and function keys for setting various functions and controlling functions of the terminal 100. For example, the input unit 120 may output a call key for a voice call request, a video call key for a video call request, an end key for requesting an end of a voice call or a video call according to the function support of the terminal 100, and output of an audio signal. It may include a volume key and a direction key to adjust the volume. In particular, the input unit 120 stores an input signal for activating a virtual makeup service function according to an embodiment of the present invention, an input signal for controlling the camera 170 to generate model data 153, and model data 153. An input signal for generating, an input signal for activating the makeup editor, and various input signals for operating the makeup editor may be generated according to user control. The generated input signal may be transferred to the controller 160 and converted into a command for supporting the virtual makeup service function of the present invention. The input unit 120 may include various input devices such as a touch pad, a touch screen, a button keypad of a general key arrangement, a button keypad of a qwerty method, etc. according to the form of the terminal 100. It may be formed of any one or a combination thereof.

The audio processor 130 controls the speaker SPK and the microphone MIC in order to output various audio data generated during the operation of the terminal 100 and to collect audio data necessary for supporting the call function of the terminal 100. It may include. The audio processor 130 may output an effect sound necessary for the operation of the camera 170 through the speaker SPK, and output an effect sound related to the application of the model data 153 and the texture for each color tone. The sound effect output function may be omitted according to a user's selection.

The display unit 140 displays information input by the user or information provided to the user, including various menus of the terminal 100. That is, the display unit 140 may provide various screens according to the use of the terminal 100, for example, a standby screen, a menu screen, a message writing screen, a call screen, and the like. In particular, the display unit 140 according to an exemplary embodiment of the present invention may output a related screen of the camera 170 activated for collecting an image of a subject, for example, a preview image output screen of the subject. In addition, the display unit 140 may output a screen to which various textures are applied to the model data 153 and the model data 153 generated based on the subject image. For example, the display unit 140 may apply a plurality of polygon type model data 153 and model data 153 generated from a subject related image to skin type texture applied model data to which a skin type texture for each facial region selected by a user is applied. For example, the user may output the texture-specific model data for each color tone applied with the texture selected by the user and the lighting model data with the light effect selected by the user. In addition, the display unit 140 of the present invention provides a makeup editor having an environment in which various textures or lights can be selected to support the above-described virtual makeup service function. The makeup editor may be activated after the model data 153 is generated or may be pre-activated so that the corresponding model data 153 supports texture application upon generation of the model data 153. The display unit 140 may be formed of a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AMOLED), or the like. In addition, when the display unit 140 is formed as a touch screen, the function of the input unit 120 may be performed.

The storage unit 150 includes an operating system (OS) of the terminal 100, an application program required for other option functions such as a sound play function, an image or a video play function, and the like, and transmits and receives user data. Data can be stored. In particular, the storage unit 150 according to the present invention may store the virtual makeup program 151 supporting the virtual makeup service of the present invention, and may store the generated model data 153 and the texture DB 155. Here, the texture DB 155 may be received and stored from an external network device through the communication unit 110 or the like, and may be received and stored and updated through various interfaces, and may be produced or modified by a user and reproduced in a new form. Can be.

The virtual makeup program 151 controls the camera 170 and the display unit 140 for generating the model data 153 for supporting the virtual makeup service of the present invention, and displays the input unit 120 or the touch screen function. According to an input signal input from the 140, various textures are applied to the model data 153 to support the output. In particular, the virtual makeup program 151 is a routine for activating the camera 170 when an input signal for activating the virtual makeup service function is generated, and guides the collection of a plurality of images of a subject based on the activated camera 170. An image collection routine to perform a routine to generate model data 153 by rendering a plurality of polygons based on the collected object-related image, and a makeup editor to apply various textures to the generated model data 153. It may include one or more of the routines that support. The routine supporting the makeup editor may support to automatically activate the makeup editor upon generation of the model data 153 or to activate the makeup editor according to a user's request. As described above, the makeup editor supports applying textures according to various skin types to the model data 153. When a command for indicating a specific color tone is generated based on the model data 153 to which the texture of the skin type is applied, According to the command, it supports to apply a specific color tone to a texture of a specific area and output it. Accordingly, the user can proactively observe what color can be seen in what manner for each skin type. In addition, the user may apply the makeup through the selection of lighting and acquire information that enables the makeup to be optimally performed in the surrounding environment to be active.

The model data 153 is a component that is generated by the virtual makeup program 151 and stored in the storage 150. The model data 153 may be output to the display unit 140 under the control of the controller 160. In this case, under the control of the controller 160, the model data 153 may be applied with various textures included in the texture DB 155 pre-stored in the storage 150 or received from an external network device.

The texture DB 155 is a component that stores various textures. Here, the various textures may include a texture group, a texture group for each color tone, and a texture group for each light according to the skin type of each part of the face. And the texture DB 155 may further include textures according to the combination of each group. Substantially, the texture DB 155 may include a calculation algorithm that calculates the color, contrast, and saturation of the texture that is changed when each group is combined, and when the specific textures overlap each other, the texture DB 155 may use the calculation algorithm to apply to the model data 153. You can determine the state of the texture to be output.

The camera 170 is a device for photographing a subject. The camera 170 may include an image sensor and an image processor. The image sensor is a sensor that converts a subject into an electrical image signal. The image sensor may be a complementary metal oxide semiconductor (CMOS) type or a charge coupled device (CCD) type image sensor. The image processor is a device for processing and outputting an image input from an image sensor. That is, the image processing unit resizes the image input from the image sensor, moves between memories, converts RGB signals, converts A / D, removes noise, digital zooms, rotates, encodes, etc. The image processing function of the can be performed. In particular, the camera 170 according to an embodiment of the present invention may be automatically activated when an input signal for activating the virtual makeup service function is generated, and then a plurality of images of a subject may be photographed according to user control. The captured image may be transmitted to the controller 160.

The controller 160 may control an overall operation of the terminal 100 and a signal flow between internal blocks of the terminal 100, and may perform a data processing function for processing data. In particular, the controller 160 according to the present invention may include a configuration as shown in FIG. 3 to support a virtual makeup service function.

Meanwhile, although the terminal 100 configured as described above has disclosed an example of storing object-related model data and texture information to be applied to the model data in the storage unit 150, the terminal 100 is not limited thereto. For example, the terminal 100 may store the object-related model data or texture information to be applied to the model data in an external device connected through a network, or may receive data from an external device. 5 is a diagram illustrating in more detail the configuration of the controller 160 of the terminal 100 according to an embodiment of the present invention.

Referring to FIG. 5, the controller 160 of the present invention may include a configuration of the model generator 161, the model reproducer 163, and the texture processor 165.

The model generator 161 generates a model based on the subject images collected and transmitted by the camera 170. The model generator 161 generates a model data 153 by controlling the output of a guide message or the like to perform image capturing of various angles with respect to a subject, and performing a rendering process having a plurality of polygons based on the captured image. do. The generated model data 153 may be stored in the storage 150.

The model reproducing unit 163 is configured to output the model data 153 stored in the storage unit 150 to the display unit 140. The model reproducing unit 163 may operate a high speed graphics engine to output rendered model data 153. In particular, the model reproducing unit 163 may support to output the makeup editor under the control of the texture processing unit 165 to the display unit 140. The model reproducing unit 163 may control to apply and output a texture for each position of the model data 153 based on various textures transmitted by the texture processing unit 165 and position information on which the corresponding texture is output.

The texture processor 165 supports to output the makeup editor through the model reproducing unit 163. In addition, the texture processing unit 165 supports the texture DB 155 stored in the storage 150 to call the texture required for the current state, that is, the texture according to the user's request, and apply it to the model data 153 for output. To this end, the texture processor 165 may select a texture group for each skin type in the makeup editor, an item for selecting a texture group for each color tone, an item for selecting a texture group for each light, and a predetermined portion of the model data 153. You can control to include an item from which you can select. In addition, when a specific texture is selected from each item, the texture processor 165 may collect position information on the model data 153 to which the texture is to be applied, and transmit the texture information and the position information to the model reproducing unit 163. Here, when a specific area of the model data 153 overlaps a plurality of textures, that is, textures by skin type, color tone, and lighting, the texture processing unit 165 generates a texture having a color corresponding to the corresponding area by using an algorithm. The generated texture may be transferred to the model reproducing unit 163. Here, the texture processing unit 165 may support to allow the texture group for each color tone to be the same group but to allow a plurality of different textures to be applied, and may support different thicknesses to be applied even for the same color tone. Accordingly, the user may acquire clearer makeup information by controlling various color tones included in the texture group for each color tone, or controlling the thickness of each color tone differently. Here, the thickness of the color tone provides a basic thickness, but may be applied in a form in which the thickness becomes thicker as the number of application is increased, and the color change according to the change in thickness is obtained based on experimental data of applying cosmetics of actual color using a brush or the like. Could be. In addition, the texture processing unit 165 of the present invention may control to download various textures from an external device according to user control, at regular intervals or in real time, and control to store the downloaded textures in the storage unit 150. Can be.

As described above, the virtual makeup service according to an exemplary embodiment of the present invention supports a model generation based on the terminal 100 and various application of textures for each skin type, color tone, and lighting according to user selection. .

In the above, the configuration of the terminal 100 performing the virtual makeup service function support of the present invention and the role of each component have been described.

Meanwhile, the virtual makeup service according to the present invention may be provided as an online service made through a network.

Hereinafter, a system configuration for supporting the virtual makeup service function of the present invention and the role of each component will be described.

6 is a view schematically showing the configuration of the virtual makeup service system of the present invention.

Referring to FIG. 6, the virtual makeup service system 10 according to an exemplary embodiment of the present invention may include a configuration of a subject 50, a terminal 100, a network 200, and a service device 300.

The virtual makeup service system 10 of the present invention having such a configuration uses the terminal 100 to generate a plurality of images of the head of the subject 50 including a specific part of the subject 50, for example, a face and a head. And collect the collected image and provide the collected image to the service device 300 through the network 200. Then, the service device 300 generates model data consisting of a plurality of polygons based on the related image of the subject 50, and provides the terminal 100 with various texture information to be applied to the model data along with the model data. do. The terminal 100 synthesizes and outputs the model data and the texture information transmitted by the service apparatus 300 according to a user's selection so that the user can check what textures, particularly textures related to makeup, are well matched. In another embodiment of the present disclosure, the service device 300 may synthesize model data and texture information and provide the synthesized data to the terminal 100.

Hereinafter, each configuration of the virtual makeup service system 10 will be described in more detail.

The subject 50 may be a head including a face and a head of a human body. The subject 50 may be disposed spaced apart from the camera 170 mounted on the terminal 100 under a user's control, and may provide a plurality of images of the head under the control of the camera 170. have. The plurality of images of the subject 50 may be transferred to the service device 300 and converted into model data having a plurality of polygons.

The terminal 100 may collect an image of the subject 50 for generating a model by using the mounted camera 170 and transmit the collected image to the service device 300. In addition, the terminal 100 may receive related model data of the subject 50 transmitted from the service device 300. In addition, the terminal 100 may download and output various textures constituting the model data from the service device 300 supporting the virtual makeup service of the present invention and apply the selected texture when outputting the model data according to a user's selection. . In this case, the terminal 100 may establish a communication channel with the service device 300, and output a web page to the display unit 140 to select various textures provided by the service device 300. In addition, the texture selected according to the user's selection may be received from the service device 300 to be temporarily stored in the storage unit 150 and applied to the corresponding model data when the model data is output. In addition, the terminal 100 may download a texture group for each skin type, a texture group for each color tone, a texture group for each illumination, and a texture complex algorithm from the service device 300, and store the downloaded information in the storage 150. . The terminal 100 may provide a screen interface for selecting various textures according to the makeup editor function support.

The network 200 is a configuration that enables data transmission and reception between the terminal 100 and the service device 300 by connecting the terminal 100 and the service device 300. The network 200 may be any one of a network supporting various communication methods. For example, the network 200 may be an IP network that provides a data transmission / reception service and a data service without disconnection through a large amount of data through an Internet protocol (IP). , 3G mobile communication network including IP network, wired network, Wibro (Wireless Broadband) network, WCDMA, HSDPA (High Speed Downlink Packet Access) network and LTE network It may include at least one of a 3.5G mobile communication network, including a 4G mobile communication network including LTE advanced, a satellite communication network and a wireless LAN including a Wi-Fi (Wi-Fi) network.

The service device 300 includes a variety of devices to support the virtual makeup service system 10 requested by the terminal 100 through a process of internally transmitting and sharing information, correcting, deleting, and discarding information, respectively. The service device 300 may include a communication unit 301, a virtual makeup provider 303, and a model generator 305 to support a virtual makeup service.

The communication unit 301 performs data transmission and reception with the terminal 100, storage and history management of the terminal 100 to support the virtual makeup service. In addition, the communication unit 301 may transmit the data received from the terminal 100 to the model generator 305 and the virtual makeup provider 303. In more detail, when the terminal 100 attempts to access the communication unit 301, the communication unit 301 may check whether the terminal 100 may use a virtual makeup service by checking authentication information. When the terminal 100 is a valid terminal, the communication unit 301 may transmit a related image of the subject 50 transmitted by the terminal 100 to the model generator 305 and the virtual makeup provider 303. In this case, the communication unit 301 may transmit only the information of the terminal 100 to the virtual makeup provider 303. In addition, the communication unit 301 may transmit the model data delivered by the model generator 305 and the texture group delivered by the virtual makeup provider 303 to the terminal 100. In this case, the communication unit 301 may store model data transmitted to the corresponding terminal 100 and store specific texture information selected by the terminal 100. In addition, when the terminal 100 is connected later, the communication unit 301 may control to transmit previously stored model data to the corresponding terminal 100 without generating new model data. In addition, the communicator 301 may support the texture information to be transmitted preferentially to the terminal 100 of textures similar to texture information selected by the user of the previous terminal 100.

The virtual makeup provider 303 classifies and stores various textures and provides specific textures or texture groups requested by the terminal 100 through the communication unit 301. To this end, the virtual makeup provider 303 may provide the terminal 100 with a web page for viewing various textures. The virtual makeup provider 303 may also support texture composite algorithms or texture integrators that can process color representations according to combinations of provided texture groups, as well as support of texture groups. The virtual makeup provider 303 may support to register a specific texture integrated by the terminal 100 as a new texture. The virtual makeup provider 303 may include a database for storing various types of textures updated by an administrator, and a control module for controlling to provide the stored textures to the terminal 100. Meanwhile, the virtual makeup provider 303 may receive model data from the model generator 305, apply the default or selected textures to the corresponding model data, and then transmit the model data to which the textures are applied to the terminal 100. have. The virtual makeup providing unit 303 receives information about the skin type for each facial region from the terminal 100, applies a skin type texture corresponding to the information to model data, and then applies model data to which the skin type texture is applied. May be transmitted to the terminal 100. In addition, the virtual makeup provider 303 supports selecting textures by color and lighting by the terminal 100 through the web page, and generates model data to which corresponding textures are applied according to the input of the terminal 100. This may be transmitted to the terminal 100.

The model generator 305 generates a model based on the related image of the subject 50 transmitted by the terminal 100 through the communication unit 301. The model generator 305 may extract depth information or distance information based on contrast information or feature point extraction of an image from a related image of the subject 50. The model generator 305 may generate model data of a plurality of polygons based on the corresponding information. Model data generated by the model generator 305 may be transmitted to the terminal 100 through the communication unit 301. In addition, the model generator 305 may transfer the model data to the virtual makeup provider 303.

As described above, the virtual makeup service system 10 of the present invention transmits a related image of the subject 50 captured by the terminal 100 to the service device 300 to apply various texture groups to be applied to the model data and the model data. It supports receiving and outputting textures by user selection to model data. Accordingly, the virtual makeup service system 10 of the present invention can reduce the burden of the terminal 100 performing model generation by processing the corresponding process without requiring a hardware equipped with a high-speed engine required for model generation, and a service device. Various texture groups may be received from 300 and applied to model data.

Meanwhile, the service device 300 configured as described above may be implemented as one or more servers operating in a server-based computing method or a cloud computing method. In more detail, the service device 300 may be implemented as one or more servers in which the communication unit 301, the model generation unit 305, and the virtual makeup provider 303 are each independently, or the communication unit 301 and the model generation unit ( 305 and the virtual makeup provider 303 may be implemented as one or more servers. In addition, the service device 300 may provide a service to the terminal 100 in a cloud computing manner.

Here, cloud computing refers to on-demand virtualized IT resources, for example, hardware (server, storage, network, etc.), software (database, security, web server, etc.), services, data, etc. using Internet technology. ) Means technology that serves in a way. That is, the service device 300 provides the terminal 100 with computing resources including at least one of hardware, software, services, and data required to provide a virtual makeup service according to the present invention. For example, the service device 300 provides a virtual computing space in which model data and texture information are distributed to the terminal 100, and provides more texture information by exchanging and sharing the texture information with the computing space. It can service the terminal 100. In addition, the service device 300 may provide a storage space for storing data and / or software related to model data, and perform the above-described function through the storage space.

In the above, the respective components of the virtual makeup service system 10 and the roles of the components have been described. Hereinafter, a signal flow between system components for providing a virtual makeup service of the above-described virtual makeup service system 10 will be described in more detail with reference to the accompanying drawings.

FIG. 7 schematically illustrates a signal flow between representative components of the virtual makeup service system 10 for explaining a virtual makeup service operating method according to an exemplary embodiment of the present invention.

Referring to FIG. 7, in the method for operating a virtual makeup service of the present invention, first, the terminal 100 collects a plurality of images related to the subject 50 in step 701. To this end, the user generates an input signal for activating the virtual makeup function provided as a menu item by using the input unit 120 of the terminal 100 or selects an icon allocated to support the corresponding function and being displayed on the display unit 140. Can be controlled. When an input signal for activating the virtual makeup function is generated, the terminal 100 may activate the camera 170 to collect an image of the subject 50.

Next, the terminal 100 may control to transmit the image related to the subject 50 collected in step 703 to the service device 300. To this end, the terminal 100 may access the service device 300, in particular, the communication unit 301 through the network 200. In addition, the terminal 100 performs a user authentication process and transmits an image related to the subject 50 to the model generator 305 through the communication unit 301 when the effective user authentication is completed. In this case, the communication unit 301 may transmit the terminal 100 connection information and the user selection information to the virtual makeup provider 303. Here, the user selection information provides a reference for texture selection that the user of the terminal 100 wants to apply to the model data. The user selection information includes information on skin type for each face part of the user, color information desired by the user, and lighting information desired by the user. It may include.

In operation 705, the service device 300 may generate model data based on the received image related to the subject 50. In particular, the model generator 305 of the service device 300 may generate model data rendered by a plurality of polygons by using an image related to the subject 50 transmitted by the terminal 100.

Next, the generated model data may be transmitted to the terminal 100 in step 707. That is, when the model data is generated, the model generator 305 may transmit the model data to the terminal 100 that has transmitted the image of the subject 50 through the communication unit 301.

The terminal 100 may output the received model data and support the user to select a texture to be applied to the model data. In particular, the terminal 100 collects at least one of information on the skin type for each facial region to be basically applied to the model data, color tone selection information that the user wants to make up, and lighting selection information that the user expects, and corresponds to each piece of information. The texture information may be requested to the service device 300 in step 709.

In operation 711, the virtual makeup provider 303 that receives at least one of skin type related information, color tone selection information, and lighting selection information may search for texture information corresponding to the received information. In this case, the virtual makeup provider 303 may receive model data from the model generator 305 and search for texture information corresponding to the model data. When the texture information search is completed, the virtual makeup provider 303 may transmit the texture information searched in step 713 to the terminal 100.

Then, the terminal 100 may control to output the display data to the display unit 140 by applying the model data and the texture information received in operation 715. The terminal 100 may configure the model data in consideration of the integrated state of each texture information when outputting the model data. That is, the terminal 100 may integrate a plurality of texture information to be applied to a specific part of the face part, convert the specific texture into a specific texture, and apply the converted texture to the specific part. Meanwhile, the terminal 100 may control to store the received model data and texture information in the storage 150.

Meanwhile, in the above description, the model generator 305 transmits model data to the terminal 100, but the present invention is not limited thereto. That is, the model generator 305 may transfer the model data to the virtual makeup provider 303. Then, the virtual makeup provider 303 applies the corresponding textures to the model data based on the user skin type information, the color tone selection information, and the light selection information transmitted from the terminal 100, and then applies the applied model data to the terminal 100. You can also send to.

As described above, the virtual makeup service operating method of the present invention may generate model data based on the service device 300, and may receive the texture information applied to the model data and output the received model information from the terminal 100. In the virtual makeup service operating method of the present invention, the service device 300 receives at least one of user skin type information, color tone selection information, and lighting selection information from the terminal 100 and corresponds to the above-described information in the model data. The texture-applied model data may be generated and provided to the terminal 100. Accordingly, the virtual makeup service operating method of the present invention can operate various textures while reducing the burden of generating a model of the terminal 100, thereby supporting the makeup environment more conveniently and variously.

On the other hand, the virtual makeup service method according to an embodiment of the present invention described above may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable recording medium. In this case, the computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. On the other hand, the program instructions recorded on the recording medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.

Computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, and magnetic disks such as floppy disks. Magnetic-Optical Media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. In addition, program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention.

In the above description of the preferred embodiments of the virtual makeup service according to an embodiment of the present invention through the specification and the drawings, although specific terms are used, it merely describes the technical details of the present invention and help the understanding of the invention. It is only used in a general sense for this purpose, the present invention is not limited to the above-described embodiment. That is, it is apparent to those skilled in the art that various embodiments based on the technical idea of the present invention are possible.

The present invention relates to a virtual makeup service system and method and a terminal supporting the same, and the present invention provides a user with a variety of makeup states based on the model data associated with the user at a desired time and environment without restriction of time and space. By ensuring that you have the proper makeup conditions, you can minimize the waste of time spent on makeup and help you maintain a satisfactory makeup.

100: terminal 110: communication unit 120: input unit
130: audio processing unit 140: display unit 150: storage unit
151: virtual makeup program 153: model data 155: texture DB
160: control unit 161: model generation unit 163: model reproduction unit
165: texture processing unit 170: camera 200: network
300: service device 301: communication unit
303: virtual makeup provider 305: model generator
10: virtual makeup service system

Claims (17)

After collecting a plurality of images related to the head of the head including the head and face, and transmits the collected image to the external service device, based on the model data and the user selection information for the image from the service device to the model data A terminal receiving various texture information to be applied and applying the texture information to the model data and outputting the texture data; And
A service device configured to receive an image related to a subject from the terminal, generate and provide the model data to the terminal, and select and transmit a plurality of texture information to be applied to the model data based on the user selection information to the terminal;
Virtual makeup service system comprising a. A communication unit configured to form a communication channel with the terminal to receive the subject-related image and the texture-related user selection information, and to transmit model data and texture information generated from the subject-related image to the terminal;
A model generator configured to generate model data rendered to have a plurality of polygons based on the subject-related image transmitted by the communication unit; And
A virtual makeup providing unit selecting texture information to be applied to the model data based on the user selection information received through the communication unit and providing the texture information to the terminal through the communication unit;
Service device for supporting a virtual makeup service comprising a. The method of claim 2,
The model generation unit
The service device for supporting a virtual makeup service, characterized in that for passing the generated model data to the virtual makeup provider. The method of claim 3,
The virtual makeup provider
And applying the texture according to the user selection information to the model data and providing the model data to which the texture is applied to the terminal through the communication unit. The method of claim 4, wherein
The virtual makeup provider
A database that stores various skin type textures, various tone textures, and various lighting textures for each makeup product;
At least one skin type texture selected according to skin type information of user selection information among textures stored in the database, at least one texture by tone according to a tone indicated by user selection information, and lighting according to user selection information. A control module for searching for and transmitting at least one of at least one texture for each illumination to the terminal;
Service device for supporting a virtual makeup service comprising a. 6. The method according to any one of claims 2 to 5,
And at least one of the communication unit, the virtual makeup provider, and the model generator is implemented as at least one server operating on a cloud computing basis. An input unit for receiving selection information of a user;
Based on the user selection information selected according to the input signal of the input unit, a specific texture prepared according to a skin type, lighting, and color tone is selected in relation to a specific makeup product, and the selected texture is applied to model data generated based on an image related to a subject. A control unit which controls to apply and output the same; And
A display unit configured to output a screen to which the texture information is applied to the model data under control of the controller;
Terminal for supporting a virtual makeup service comprising a. The method of claim 7, wherein
The control unit
A model generator configured to generate model data composed of a plurality of polygons based on the subject related image;
A texture processing unit which selects a texture to be applied to the model data according to user selection information from the input unit and transmits the texture to the model reproducing unit; And
A model reproducing unit for applying the texture transmitted from the text processing unit to the model data and outputting the texture to the display unit;
Terminal for supporting a virtual makeup service comprising a. 9. The method of claim 8,
The texture is
Skin type textures classified according to skin types of face parts;
Tonal textures according to various shades;
Lighting star texture according to various lights;
Terminal for supporting a virtual makeup service, characterized in that it comprises at least one of. 10. The method of claim 9,
The texture processing unit
And a makeup editor for allowing a user to select at least one of the skin type texture, the tone texture, and the illumination texture, and a portion to which the corresponding texture is applied in the model data. Terminal that supports virtual makeup service. The method of claim 10,
The texture processing unit
When a plurality of textures are selected to be applied to a predetermined portion of the model data, a texture having a color integrated with a plurality of textures may be generated, and the virtual texture service may be controlled to apply the texture to a designated portion. terminal. The method of claim 7, wherein
A storage unit for storing one or more of the subject-related image, the subject-related model data, and a plurality of textures applicable to the model data;
The terminal supporting the virtual makeup service, characterized in that it further comprises. The method of claim 7, wherein
Further comprising a communication unit for transmitting and receiving data through a network,
The control unit
And at least one of a subject-related image, subject-related model data, and a plurality of textures applicable to the model data from an external device through the communication unit. The method of claim 7, wherein
The camera further includes a camera for shooting a subject-related image,
The controller supports the virtual makeup service, characterized in that for generating the model data from the subject-related image taken by the camera. A collection step of collecting, by the terminal, a plurality of images related to the subject;
Generating model data rendered to include a plurality of polygons based on the subject images;
An application step of applying textures according to a user selection to the model data; And
Outputting model data to which the texture is applied;
Virtual makeup service method comprising a. 16. The method of claim 15,
The application step
At least one of a skin type texture item, a tone color texture item, and a light texture item may be selected according to a skin type of a face part, and a selected part of the model data to which the selected texture is applied among the textures may be selected. Displaying a makeup editor comprising the item;
Receiving an input signal for selecting any one of items displayed in the makeup editor;
Applying a texture selected in accordance with the input signal to the model data;
Virtual makeup service method comprising a. A computer-readable recording medium having recorded thereon a program for executing the virtual makeup service method according to claim 15.

Images