KR101748371B1 - Method and apparatus for virtual make-up service - Google Patents

Abstract

The present invention relates to a virtual make-up service system and method, and a terminal supporting the virtual make-up service system. The virtual make-up service system provides user-specific characteristics and a virtual makeup state based on model data. The virtual makeup service system of the present invention includes a terminal and a service device. The terminal collects a plurality of images related to the subject of the head including the head and the face, transmits the collected image to the external service device, and transmits the model data and the model data Receives texture information to be applied to the texture data, and outputs texture data to the model data. The service device receives the subject-related image from the terminal and generates model data, and selects a plurality of texture information to be applied to the model data based on the user selection information and transmits the selected texture information to the terminal.

Description

TECHNICAL FIELD The present invention relates to a virtual make-up service method,

The present invention relates to a virtual make-up service, and more particularly, to a virtual make-up service system and method capable of providing a virtual make-up state applying user characteristics and surrounding environment based on model data, and a terminal supporting the same.

Recently, due to the remarkable development of information communication technology and semiconductor technology, the spread and use of various terminals are rapidly increasing. In particular, mobile terminals of recent mobile terminals are reaching a mobile convergence stage that does not stay in a conventional unique area but covers other mobile terminals. Typically, in the case of a mobile communication terminal, in addition to general communication functions such as voice communication and message transmission / reception, a TV broadcasting function (for example, a mobile broadcasting such as a DMB (Digital Multimedia Broadcasting) or a DVB (Digital Video Broadcasting) MP3 (MPEG Audio Layer-3), a picture taking function, and an Internet access function.

On the other hand, the Internet population is growing at a rapid pace due to the development and expansion of communication networks. Various contents including electronic commerce and portals are emerging along with the growth of the Internet. In addition, the boundaries of online and offline areas have recently been destroyed, replacing existing offline services with online services. As the level of living improves, the quality of life and personality are important, and the need for and interest in makeup management is greatly increasing.

For example, the technology such as make-up is rapidly spreading the information sharing including various methods and cases through the development of the communication network. Accordingly, the users who want to make makeup can test the makeup according to the makeup information provided by the communication network and actually apply the makeup.

On the other hand, make-up technology has characteristics that are greatly influenced by each user's characteristics and surrounding environment. That is, the skin type, which is basically different for each user, is different from that of the skin color, and the user has different purpose of make-up, so that the environment in which the make-up is completed and active may be different for each user. However, since conventional make-up information simply discloses a method of performing the method according to a specific user, there is a problem in that it is not suitable for a user having different characteristics from a specific user. In addition, various cosmetic products are used to perform makeup. In the case of such cosmetic products, there is a problem in that unnecessary purchasing may occur when purchasing a product which does not fit the user because it is used after purchasing it first.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is an object of the present invention to provide an image processing apparatus, A virtual makeup service system and method capable of applying appropriate virtual makeup application by applying skin characteristic information for each user, and a terminal supporting the same.

It is another object of the present invention to provide a virtual make-up service system and method, and a terminal supporting the virtual make-up service system, in which a user can observe the degree of make-up change applied to the model data by surrounding elements.

According to an aspect of the present invention, there is provided a virtual makeup service system for collecting a plurality of images related to a subject, transmitting the collected images to a service device, A terminal receiving various texture information to be applied to the model data based on the model data and the user selection information, applying the received texture information to the model data, and outputting the received texture information; And a service device receiving the subject-related image from the terminal, generating model data and providing the model data to the terminal, and selecting a plurality of texture information to be applied to the model data based on the user selection information and transmitting the selected texture information to the terminal.

According to another aspect of the present invention, there is provided a service providing apparatus for supporting a virtual makeup service according to a preferred embodiment of the present invention includes a communication channel with a terminal to receive a subject-related image and texture-related user selection information A communication unit for transmitting model data and texture information generated from the subject-related image to the terminal; A model generation unit for generating model data rendered so as to have a plurality of polygons based on a subject-related image transmitted from the communication unit; And a virtual makeup provider for selecting texture information to be applied to the model based on the user selection information received through the communication unit and providing the selected texture information to the terminal through the communication unit.

In the service apparatus according to the present invention, the model generating unit transmits the generated model data to the virtual makeup providing unit, the virtual makeup providing unit applies the texture corresponding to the user selection information to the model data, To the terminal.

In the service device according to the present invention, the virtual makeup provider may include: a database for storing various skin type textures, textures for various tones, and textures for various lights; At least one skin type texture selected according to the skin type information of the user's selection information among the textures stored in the database, at least one texture according to the color tone indicated in the user selection information, And a control module for searching at least one of the illumination-specific textures and transmitting the search result to the terminal.

In the service apparatus according to the present invention, at least one of the communication unit, the virtual makeup service providing unit, and the model generating unit may be implemented as one or more servers operating on a cloud computing basis.

According to another aspect of the present invention, there is provided a terminal for supporting a virtual makeup service, comprising: an input unit for receiving selection information of a user; A controller for selecting a specific texture prepared according to skin type, illumination, and color tone based on the selected user selection information according to an input signal of an input unit, and applying the selected texture to model data generated based on a subject-related image and outputting the selected texture; And a display unit for outputting a screen to which texture information is applied to the model data under the control of the control unit.

In the terminal according to the present invention, the control unit may include: a model generating unit that generates model data composed of a plurality of polygons based on the subject-related image; A texture processor for selecting a texture to be applied to the model data according to the user selection information transmitted from the input unit and transmitting the selected texture to the model reproduction unit; And a model reproducing unit for applying the texture transferred from the texture processing unit to the model data and outputting the texture to the display unit.

In the terminal according to the present invention, the texture may include at least one of a skin type for each skin type, a texture for each hue according to various hues, and a texture for illumination according to various lights.

In the terminal according to the present invention, the texture processor may include a makeup editor that supports the user to select at least one of the texture of the skin type, the texture of each tone, the texture of each tone, .

In the terminal according to the present invention, when a plurality of textures are selected to be applied to a certain portion of the model data, the texture processor generates a texture having a color combining the plurality of textures, and controls the texture do.

The terminal according to the present invention may further include a storage unit for storing at least one of a plurality of textures applicable to a subject-related image, subject-related model data, and model data.

The terminal according to the present invention further includes a communication unit for transmitting and receiving data through a network, and the control unit receives one or more of a plurality of textures applicable to the subject-related image, the subject-related model data, can do.

The terminal according to the present invention may further include a camera for photographing a subject-related image, and the control unit may further perform a function of generating model data from a subject-related image photographed by the camera.

According to another aspect of the present invention, there is provided a virtual makeup service method for a terminal, comprising: a collection step of collecting a plurality of images related to a subject; Generating rendered model data to include a plurality of polygons based on subject images; An application step of applying the textures according to the user selection to the model data; And outputting texture-applied model data.

In the virtual makeup service method according to the present invention, at least one of a texture item per skin type, a texture item per color tone, and a texture item per illumination may be selected according to a skin type of a face region, Displaying a makeup editor including an item for selecting a portion of model data to which the selected texture is to be applied; Receiving an input signal for selecting any one of the items output to the makeup editor; And applying the selected texture to the model data according to the input signal.

In the virtual make-up service method according to the present invention, the application step may include: applying a skin type-specific texture determined according to user selection to model data; Applying texture to the model data to which the texture of each skin type is applied according to the user's selection; And applying the illumination-specific texture determined according to the user's selection to the model data to which the hue texture is applied.

The present invention also provides a computer-readable recording medium on which a program for executing the virtual makeup service method described above is recorded.

As described above, according to the virtual make-up service system and method and the terminal supporting the virtual make-up service according to the embodiment of the present invention, the present invention can be applied to a virtual make-up service system, a model in which a texture corresponding to a skin type per user is applied based on model data of a subject, By creating data and applying the texture according to various hue makeup to the generated model data, it is possible to easily search for makeup technology suitable for actual users and cosmetics, and makeup state adaptively changed according to the surrounding environment Can be supported.

1 is a flowchart illustrating a virtual makeup service method in a terminal according to an embodiment of the present invention;
2 is a diagram showing an example of model data of a polygon type according to an embodiment of the present invention;
3 is a diagram illustrating an example of model data to which a texture is applied according to an embodiment of the present invention;
4 is a detailed diagram illustrating a configuration of a terminal according to an embodiment of the present invention.
5 is a block diagram illustrating a configuration of a control unit in the terminal configuration of the present invention in more detail;
FIG. 6 is a view schematically showing a configuration of a virtual makeup service system according to an embodiment of the present invention; FIG.
FIG. 7 is a diagram illustrating signal flow between system configurations for explaining a virtual makeup service method according to an embodiment of the present invention; FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that, in the drawings, the same components are denoted by the same reference numerals as possible. Further, the detailed description of known functions and configurations that may obscure the gist of the present invention will be omitted.

It should be noted that the embodiments of the present invention disclosed in the present specification and drawings are only illustrative of the present invention in order to facilitate the understanding of the present invention and are not intended to limit the scope of the present invention. It is to be understood by those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

1 is a flowchart illustrating a virtual makeup service method according to an embodiment of the present invention. A virtual makeup service method according to an embodiment of the present invention is based on a terminal, wherein the terminal is a mobile communication terminal that operates based on communication protocols corresponding to various communication systems, terminals, portable multimedia players (PMPs), digital broadcasting players, PDAs (personal digital assistants), music players (e.g. MP3 players), portable game terminals, smart phones, And a tablet PC, and all kinds of user devices such as multimedia devices and application devices therefor.

Referring to FIG. 1, when power is supplied to the terminal according to the present invention, the terminal outputs an idle screen to wait for a user input to perform a function according to a user input,

If the input signal of the user is generated in the idle screen state, the terminal may branch to step 103 and check whether the input signal is an input signal for activating the virtual make-up function. If the input signal is not an input signal for activating the virtual make-up function, the terminal branches to step 105 and displays the function of the terminal corresponding to the input signal, for example, a file playback function, a file search function, Function and so on.

On the other hand, if an input signal for activating the virtual make-up function is generated in step 103, the terminal may branch to step 107 and generate model data. For this purpose, the terminal can control to collect an image of a specific area of a user who wants to use the virtual makeup, for example, a subject image including a head and a face. At this time, the terminal may output a guide message through a display unit or audio processing so as to collect images at various angles with respect to the subject, so as to generate appropriate model data based on the collected subject-related images. The user can operate to collect a plurality of images at various angles with respect to the subject in accordance with this guide message. When a plurality of images are collected, the terminal can generate model data based on the collected images. In particular, the terminal can generate 3D model data rendered in a plurality of polygon types as shown in FIG. 2 using a plurality of images.

Next, the terminal branches to step 109 and controls to apply the texture according to the skin type selected by the user to the model data. For this purpose, the terminal may store textures corresponding to skin types such as dryness, neutrality and intelligibility of each part of the face in advance, or may be provided from an external device through a network. Accordingly, when the user selects a skin type for each site, the terminal loads the texture according to the selected skin type and applies the skin to each part of the model data to generate model data having a skin type-specific texture as shown in FIG. 3 can do.

Next, the terminal branches to step 111 to check whether or not an input signal for outputting a makeup editor is generated. In this step, when the input signal for the output of the make-up editor is generated, the terminal branches to step 113 and controls the texture according to the color tone selected by the user to be applied to the model data of the skin type-specific texture. In order to do this, the terminal may pre-store a texture for each color tone to be applied to the model data or may be provided from an external device. Here, the texture of the hue can be output as a value reflecting the color of the texture of the skin type to be located. The texture of each color tone can have various color, contrast, and saturation values. Depending on the location of each face, the value can be changed and it is set to have the same value at all the faces according to the user setting There is also water. To this end, the user can activate the makeup editor and generate an input signal for selection of the hue provided by the makeup editor and for selecting the part of the model data. Then, the makeup editor can control to output the selected color tone to the model data being output.

On the other hand, the terminal branches to step 115 to check whether or not an input signal for lighting selection is generated. If there is an input signal for lighting selection at this stage, the control unit branches to step 117 and applies a texture corresponding to the selected illumination to the model data. In more detail, the terminal supports the makeup editor to include an item for selecting a light, and the user can select a desired surrounding environment by selecting the item and selecting a type of illumination. For example, the user can select a time when the makeup is completed, for example, noon, and input weather information. The user can input information about the place of activity or the outside of the building as information on the activity place. The makeup editor can calculate an illumination value according to the surrounding environment information and control the application of the texture according to illumination when the various surrounding environment information is input. To this end, the makeup editor can provide items that the user can select for various surrounding environments. And the makeup editor can control to change already applied hue texture according to illumination and output it.

On the other hand, if there is no separate input signal for the makeup editor output in step 111, no input signal is generated for illumination selection in step 115, and the texture application according to the selected illumination is completed in step 117, the terminal enters step 119 It is possible to confirm whether an input signal for function termination is generated or not. If there is no input signal for another function termination, the terminal branches to step 103 and controls to repeatedly perform the following procedure. Step 119 When an input signal for function termination is generated, the terminal returns to step 101 and re-executes the following procedure.

As described above, the virtual makeup service method of a terminal according to the present invention regenerates input model data that is differently applied to model data rendered by a plurality of polygons according to a skin type per part, Applying the texture can provide a close-up makeup application environment.

4 is a block diagram schematically showing the configuration of a terminal 100 supporting a virtual makeup service according to an embodiment of the present invention.

4, the terminal 100 of the present invention includes a communication unit 110, an input unit 120, an audio processing unit 130, a display unit 140, a storage unit 150, a camera 170, . ≪ / RTI > As described above, the terminal 100 includes all mobile communication terminals operating based on communication protocols corresponding to various communication systems, a portable multimedia player (PMP) A mobile phone, a digital broadcasting player, a PDA (Personal Digital Assistant), a music player (for example, an MP3 player), a portable game terminal, a smart phone, a notebook, And may include an application device for.

The terminal 100 having the above-described configuration operates the camera 170 to collect images at various angles with respect to the subject, and generates the rendered model data 153 having a plurality of polygons based on the collected images . The model data 153 includes, for example, 3D-based model data. When the model data 153 is generated, the terminal 100 applies the texture corresponding to the skin type of the face part to the model data 153 according to the user's selection to generate model data to which the skin type texture is applied, Can be used to generate model data with a texture of the hue to the texture of the skin type. In addition, the terminal 100 of the present invention can support changing the color, brightness, and saturation of the texture by applying a lighting effect selected by the user to the model data to which the texture of each color tone is applied. Hereinafter, each configuration of the terminal 100 for supporting the virtual makeup service of the present invention will be described in detail.

The communication unit 110 performs formation of a communication channel for data transmission and / or data transmission / reception of the terminal 100 under the control of the control unit 160. That is, the communication unit 110 may form a voice communication channel, a data communication channel, a video communication channel, and the like with various network devices through a network. In particular, the communication unit 110 can receive texture data for each skin type, texture for each hue, a lighting effect calculation algorithm, and the like to be applied to the model data 153 generated by a plurality of polygon types from an external device through a network. Also, the communication unit 110 can receive a subject-related image through another terminal or a network.

The input unit 120 may include input keys and function keys for receiving numeric or various character information, setting various functions, and controlling functions of the terminal 100. For example, the input unit 120 may include a call key for a voice call request, a video call key for a video call request, an end key for requesting a voice call or a video call end, A volume key and a direction key for adjusting the volume, and the like. In particular, the input unit 120 stores an input signal for controlling the camera 170, model data 153 for generating an input signal, model data 153 for activating the virtual makeup service function according to an embodiment of the present invention, An input signal for activating the makeup editor, various input signals for operation of the makeup editor, etc. can be generated according to the user's control. The input signal thus generated is transferred to the control unit 160 and converted into a command for supporting the virtual makeup service function of the present invention. The input unit 120 may include various input devices such as a touch pad, a touch screen, a general key arrangement type button-type keypad, a QWERTY type button-type keypad, Or a combination thereof.

The audio processing unit 130 includes a speaker SPK and a microphone MIC for outputting various audio data generated during the operation of the terminal 100 and for collecting audio data necessary for supporting a call function of the terminal 100 . The audio processing unit 130 can output sound effects necessary for the operation of the camera 170 through the speaker SPK and output the sound data related to the model data 153 and the application of the texture for each specific tone. The effect sound output function may be omitted depending on the user's selection.

The display unit 140 displays various menus of the terminal 100 as well as information input by the user or information provided to the user. That is, the display unit 140 can provide various screens such as an idle screen, a menu screen, a message creation screen, a call screen, and the like depending on the use of the terminal 100. In particular, the display unit 140 according to an exemplary embodiment of the present invention may output a preview image output screen for a related screen of the camera 170 activated for capturing an image of a subject, for example, a subject. Also, the display unit 140 may output a screen having various textures applied to the model data 153 and the model data 153 generated based on the subject image. For example, the display unit 140 may include a plurality of polygon type model data 153 generated from a subject-related image, a skin-type texture model data 153 for applying a skin type texture for each face part selected by the user to the model data 153, , Texture application model data for each hue to which the texture of the hue selected by the user is applied, and illumination application model data to which the lighting effect selected by the user is applied, respectively. The display unit 140 of the present invention provides a makeup editor having an environment in which various textures and lights can be selected for supporting the virtual makeup service function. The makeup editor may be activated after the model data 153 is created, or may be pre-activated so that the model data 153 may support texture application when the model data 153 is created. The display unit 140 may be a liquid crystal display (LCD), an organic light emitting diode (OLED), or an active matrix organic light emitting diode (AMOLED). In addition, when the display unit 140 is formed as a touch screen, the display unit 140 may perform the function of the input unit 120.

The storage unit 150 stores application programs necessary for the operating system (OS) of the terminal 100 and other options functions such as a sound playback function, an image or moving picture playback function, Data and the like can be stored. In particular, the storage unit 150 according to the present invention may store the virtual makeup program 151 supporting the virtual makeup service of the present invention, and may store the generated model data 153 and the texture DB 155. [ Here, the texture DB 155 may be received and stored from an external network device through the communication unit 110 or the like, or received and stored via various interfaces, and may be stored and updated. Also, the texture DB 155 may be modified by a user, .

The virtual makeup program 151 controls the camera 170 and the display unit 140 for generating the model data 153 for supporting the virtual makeup service of the present invention and controls the display unit 140 and the input unit 120, And outputs various textures to the model data 153 in accordance with an input signal input from the input unit 140. FIG. In particular, the virtual makeup program 151 is a routine for controlling activation of the camera 170 when an input signal for activating the virtual makeup service function is generated, a plurality of image acquisitions for the subject based on the activated camera 170 A routine for generating the model data 153 by rendering the polygons to have a plurality of polygons based on the collected subject-related images, a makeup editor 155 for applying various textures to the generated model data 153, Or < / RTI > The routine that supports the makeup editor may support activating the makeup editor automatically upon creation of the model data 153 or activating the makeup editor upon user request. As described above, the makeup editor supports applying a texture according to various skin types to the model data 153. When a command for designating a specific color tone is generated based on the model data 153 to which the texture of the skin type is applied, It supports to apply a specific hue to a texture of a specific part according to an instruction and output it. Accordingly, the user can preliminarily observe how the color tone of each skin type can be seen. The user can also obtain information that allows the makeup to be applied and the makeup optimized for the surrounding environment to be performed proactively through the lighting selection.

The model data 153 is generated by the virtual makeup program 151 and stored in the storage unit 150. The model data 153 may be output to the display unit 140 under the control of the controller 160. At this time, the model data 153 may be output by applying various textures stored in the texture database 155 stored in the storage unit 150 or received from the external network device under the control of the control unit 160.

The texture DB 155 is a structure for storing various textures. Here, the various textures may include a texture group according to the skin type of the face part, a texture group according to the hue, and a texture group for the illumination. The texture DB 155 may further include textures according to combinations of groups. In practice, the texture DB 155 includes an operation algorithm for calculating color, contrast, saturation, etc. of a texture to be changed when each group is combined. In the case where specific textures overlap each other, You can determine the state of the texture to be output.

The camera 170 is a device for photographing a subject. Such a camera 170 may include an image sensor and an image processing unit. An image sensor is a sensor that converts an object into an electrical image signal. The image sensor may be a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type image sensor. The image processing unit is a device that processes an image input from the image sensor and outputs the processed image. That is, the image processing unit converts an image input from the image sensor into a digital image by resizing, moving between memories, converting RGB signals, A / D conversion, noise elimination, digital zoom, rotation, The image processing function of the image processing apparatus can be performed. In particular, the camera 170 according to the embodiment of the present invention can be automatically activated when an input signal for activating the virtual makeup service function is generated, and then can photograph a plurality of images of the subject according to user control. The photographed image may be transmitted to the controller 160.

The control unit 160 may control the overall operation of the terminal 100 and the signal flow between the internal blocks of the terminal 100 and may perform a data processing function for processing the data. In particular, the controller 160 according to the present invention may include a configuration as shown in FIG. 3 to support a virtual makeup service function.

Meanwhile, the terminal 100 configured as described above has shown an example of storing the object-related model data and the texture information to be applied to the model data in the storage unit 150, but the present invention is not limited thereto. For example, the terminal 100 may store texture-related model data or texture information to be applied to the model data in an external device connected through a network, or receive the external information from the external device. 5 is a detailed block diagram of the controller 160 of the terminal 100 according to an embodiment of the present invention.

Referring to FIG. 5, the controller 160 of the present invention may include a model generator 161, a model generator 163, and a texture processor 165.

The model generation unit 161 generates a model based on the subject images collected and transmitted by the camera 170. [ The model generation unit 161 controls the output of guidance messages and the like so as to perform image capturing at various angles with respect to the subject and performs a rendering process with a plurality of polygons based on the captured image to generate model data 153 do. The generated model data 153 may be stored in the storage unit 150.

The model reproducing unit 163 is configured to support the output of the model data 153 stored in the storage unit 150 to the display unit 140. [ The model reproducing unit 163 can operate a high-speed graphic engine for outputting the rendered model data 153. [ In particular, the model reproducing unit 163 can support a makeup editor controlled by the texture processing unit 165 to be output to the display unit 140. The model reproducing unit 163 may control to output the textures of the respective positions of the model data 153 based on the various textures transmitted from the texture processor 165 and the position information on which the textures are output.

The texture processor 165 supports the makeup editor through the model reproducer 163. Then, the texture processor 165 calls the textures according to the user request required for the current state in the texture DB 155 stored in the storage unit 150, and applies them to the model data 153 for output. To this end, the texture processor 165 includes an item for selecting a texture group for each skin type, an item for selecting a texture group for each hue, an item for selecting a texture group for each illumination, Can be controlled so as to include an item that can be selected. When a specific texture is selected in each item, the texture processor 165 collects position information on the model data 153 to which the texture is to be applied, and transmits the texture information and the position information to the model reproducing unit 163. In this case, when a specific part of the model data 153 is overlapped with a plurality of textures, that is, skin type, color tone, and lighting texture, the texture processor 165 generates a texture having a color matching And transmit the generated texture to the model reproducing unit 163. [ Here, the texture processor 165 can support a plurality of different texture applications in the same group, and can support different thicknesses to apply even if they are the same color tone. Accordingly, the user can acquire clear make-up information by controlling the mixing of the various tones included in the texture group for each tincture or controlling the thickness of each tincture to be different. Here, the thickness of the color tone provides a basic thickness, but it can be applied in a form of thickening as the number of times of application is increased, and the change in color tone according to the thickness change can be obtained based on empirical data of a real color cosmetic using a brush or the like . In addition, the texture processor 165 of the present invention can control various textures to be downloaded from an external device according to a user's control, periodically or in real time, and controls the downloaded texture to be stored in the storage unit 150 .

As described above, the virtual makeup service according to the embodiment of the present invention supports model creation based on the terminal 100, various textures for each skin type, color tone, and illumination depending on the user's selection .

The configuration of the terminal 100 for supporting the virtual makeup service function of the present invention and the roles of the respective components have been described above.

Meanwhile, the virtual makeup service according to the present invention may be provided as an online service through a network.

Hereinafter, the system configuration and the roles of the respective components for supporting the virtual makeup service function of the present invention will be described.

6 is a diagram schematically showing a configuration of a virtual makeup service system according to the present invention.

6, a virtual makeup service system 10 according to an embodiment of the present invention may include a configuration of a subject 50, a terminal 100, a network 200, and a service device 300. FIG.

The virtual makeup service system 10 of the present invention having the above-described configuration is configured to use the terminal 100 to acquire a plurality of images of a head part 50 of a subject 50 including a face and a head, And provides the collected image to the service apparatus 300 through the network 200. [ Then, the service apparatus 300 generates model data composed of a plurality of polygons based on the related image of the subject 50, and provides various texture information to be applied to the model data to the terminal 100 together with the model data do. The terminal 100 synthesizes and outputs the model data and the texture information transmitted by the service device 300 according to the user's selection, thereby enabling the user to check which texture, particularly the texture related to the makeup, is well suited to the user. In another embodiment of the present invention, the service device 300 may synthesize the model data and the texture information and provide the combined data to the terminal 100.

Hereinafter, each configuration of the virtual make-up service system 10 will be described in more detail.

The subject 50 may be a head including a face and a head of a human body. The subject 50 may be disposed at a predetermined distance from the camera 170 mounted on the terminal 100 under the control of the user and may provide a plurality of images of the tofu under the control of the camera 170 have. A plurality of images of the subject 50 may be transmitted to the service device 300 and converted to model data having a plurality of polygons.

The terminal 100 may collect the image of the subject 50 for model generation using the camera 170 and transmit the collected image to the service device 300. [ The terminal 100 may receive the related model data of the subject 50 transmitted from the service device 300. [ In addition, the terminal 100 may download various textures constituting the model data from the service device 300 supporting the virtual makeup service of the present invention, and then apply the selected texture upon outputting the model data according to the user selection, . At this time, the terminal 100 forms a communication channel with the service device 300, and may output a web page to the display unit 140 from which various textures to be provided by the service device 300 can be selected. The selected texture may be received from the service device 300 and temporarily stored in the storage unit 150 according to the user's selection, and may be applied to the corresponding model data when the model data is output. In addition, the terminal 100 can download the skin type texture group, the texture group for each hue, the texture group for each illumination, and the texture complex algorithm from the service device 300, and store the downloaded information in the storage unit 150 . The terminal 100 can provide a screen interface for selecting various textures according to the makeup editor function.

The network 200 is a configuration that enables data transmission / reception between the terminal 100 and the service device 300 by connecting the terminal 100 and the service device 300. The network 200 may be any one of networks that support various communication schemes. For example, the network 200 may be an i-pimp network that provides a large-capacity data transmission / reception service through an internet protocol (IP) , A third generation mobile communication network including an All IP network, a wired network, a Wibro (Wireless Broadband) network, a WCDMA, a High Speed Downlink Packet Access (HSDPA) network, and an LTE network, And a wireless LAN including a 3.5G mobile communication network, a fourth generation mobile communication network including LTE advanced, a satellite communication network, and a Wi-Fi network.

The service device 300 is a configuration for supporting the virtual makeup service system 10 requested by the terminal 100 through a process of transferring, sharing, correcting, deleting and discarding information internally including various devices. The service device 300 may include a communication unit 301, a virtual makeup provider 303, and a model generation unit 305 for supporting a virtual makeup service.

The communication unit 301 performs data transmission / reception with the terminal 100, data storage, and history management to support the virtual makeup service. The communication unit 301 may transmit the data received from the terminal 100 to the model generating unit 305 and the virtual makeup providing unit 303. In more detail, when the terminal 100 attempts to access the communication unit 301, the communication unit 301 can confirm whether the terminal 100 can use the virtual makeup service by checking the authentication information. The communication unit 301 may transmit the related image of the subject 50 transmitted by the terminal 100 to the model generating unit 305 and the virtual makeup providing unit 303. [ At this time, the communication unit 301 may transmit only the information of the terminal 100 to the virtual makeup provider 303. The communication unit 301 can transmit the model data transferred by the model generating unit 305 and the texture group delivered by the virtual makeup provider 303 to the terminal 100. At this time, the communication unit 301 may store the model data transmitted to the terminal 100, and may store the specific texture information selected by the terminal 100. The communication unit 301 can control to transmit the stored model data to the corresponding terminal 100 without newly generating the model data when the terminal 100 is connected later. In addition, the communication unit 301 can support textures of a type similar to the texture information selected by the user of the previous terminal 100 to be transmitted to the terminal 100 in preference to the texture information.

The virtual makeup provider 303 classifies and stores various textures and provides specific texture or texture groups requested by the terminal 100 through the communication unit 301. [ To this end, the virtual makeup provider 303 may provide the terminal 100 with a web page capable of browsing various textures. The virtual makeup provider 303 may also support texture combination algorithms or texture integrators that can handle color representations according to combinations of provided texture groups, as well as support for texture groups. The virtual makeup provider 303 can support registration of a specific texture integrated by the terminal 100 as a new texture. The virtual makeup provider 303 may include a database for storing various types of textures updated by an administrator and a control module for controlling the stored textures to be provided to the terminal 100. [ Meanwhile, the virtual makeup provider 303 may receive the model data from the model generation unit 305, apply the textures set to the default or the selected textures to the corresponding model data, and then transmit the model data to which the textures are applied to the terminal 100 have. Then, the virtual makeup provider 303 receives the information on the skin type per face part from the terminal 100, applies the skin type-specific texture corresponding to the information to the model data, To the terminal (100). Also, the virtual makeup provider 303 supports the selection of a texture for each color tone and a texture for each illumination from the terminal 100 through a web page, generates model data to which the corresponding textures are applied according to the input of the terminal 100 , And transmits it to the terminal 100.

The model generation unit 305 generates a model based on the related image of the subject 50 transmitted by the terminal 100 through the communication unit 301. [ The model generation unit 305 may extract depth information or distance information based on the contrast information or the feature point extraction of the image in the related image of the subject 50. The model generating unit 305 may generate a plurality of polygon-type model data based on the information. The model data generated by the model generation unit 305 can be transmitted to the terminal 100 via the communication unit 301. [ The model generating unit 305 can also transmit the model data to the virtual makeup provider 303. [

As described above, the virtual makeup service system 10 of the present invention transmits the related image of the subject 50 photographed by the terminal 100 to the service device 300 and transmits various texture groups to be applied to the model data and the model data And applying the user-selected textures to the model data for output. Accordingly, the virtual makeup service system 10 of the present invention can process the corresponding process without requiring the hardware equipped with the high-speed engine necessary for model creation, thereby reducing the burden of the terminal 100 performing model creation, It is possible to receive various texture groups from the texture group 300 and apply them to the model data.

Meanwhile, the service device 300 configured as described above may be implemented as one or more servers operating in a server-based computing mode or a cloud computing mode. More specifically, the service apparatus 300 includes a communication unit 301, a model generating unit 305, and a virtual makeup providing unit 303, each of which is implemented as one or more independent servers, 305 and a virtual makeup providing unit 303. The virtual makeup providing unit 303 may be implemented as one or more servers. In addition, the service device 300 can provide services to the terminal 100 in a cloud computing manner.

Here, cloud computing refers to the use of Internet technology to connect virtualized IT resources such as hardware (server, storage, network), software (database, security, web server, etc.) ) Technology. That is, the service apparatus 300 provides the terminal 100 with computing resources including at least one of hardware, software, services, and data necessary for providing the virtual makeup service according to the present invention. For example, the service device 300 may provide a virtual computing space in which model data and texture information are distributed to the terminal 100, and may exchange various types of texture information according to exchange and sharing of various texture information. So that the terminal 100 can be serviced. In addition, the service apparatus 300 may provide a storage space for storing data and / or software related to the model data, and may perform the functions described above through the storage space.

In the foregoing, the respective components of the virtual makeup service system 10 according to the embodiment of the present invention and the roles of the respective components have been described. Hereinafter, the signal flow between the system configurations for providing the virtual makeup service of the virtual makeup service system 10 will be described in detail with reference to the drawings.

FIG. 7 is a schematic diagram illustrating a signal flow between representative configurations of a virtual makeup service system 10 for explaining a virtual makeup service operation method according to an embodiment of the present invention.

Referring to FIG. 7, in operation 701, the terminal 100 collects a plurality of images related to a subject 50 in the method of operating the virtual makeup service of the present invention. For this purpose, the user may use the input unit 120 of the terminal 100 to generate an input signal for activating a virtual makeup function provided as a menu item, or to select an icon to be allocated to the corresponding function and output to the display unit 140 . When the input signal for activating the virtual make-up function is generated, the terminal 100 can activate the camera 170 to control the acquisition of the image of the subject 50. [

Next, the terminal 100 may control to transmit the image of the subject 50 collected in step 703 to the service device 300. To this end, the terminal 100 can access the service apparatus 300, in particular, the communication unit 301 through the network 200. [ When the valid user authentication is completed, the terminal 100 can transmit the image related to the subject 50 to the model generating unit 305 through the communication unit 301. [ At this time, the communication unit 301 can transmit the access information and the user selection information of the terminal 100 to the virtual makeup provider 303. Here, the user selection information provides a criterion of texture selection that the user of the terminal 100 wants to apply to the model data. The user selection information includes information on the skin type of the user's face part, hue information desired by the user, .

Then, the service device 300 can generate the model data based on the image related to the subject 50 received in step 705. Particularly, the model generation unit 305 of the service device 300 can generate model data rendered with a plurality of polygons by using an image related to the subject 50 transmitted by the terminal 100. [

The generated model data may be transmitted to the terminal 100 in step 707. FIG. That is, when the model data is generated, the model generating unit 305 can transmit the model data to the terminal 100 that has transmitted the image of the object 50 through the communication unit 301. [

The terminal 100 can output the model data that has been used and support the user to select a texture to be applied to the model data. In particular, the terminal 100 collects at least one of skin type information per face part to be basically applied to the model data, hue selection information that the user wants to make up, and illumination selection information expected by the user, In step 709, the texture information may be requested to the service apparatus 300.

Skin type-related information, color tone selection information, and illumination selection information, the virtual makeup provider 303 may search for texture information corresponding to the information received in step 711. FIG. At this time, the virtual makeup provider 303 receives the model data from the model generating unit 305, and can search each texture information corresponding to the model data. When the texture information search is completed, the virtual makeup provider 303 can transmit the texture information retrieved in step 713 to the terminal 100. [

Then, the terminal 100 may control the display unit 140 to apply the model data and the texture information received in step 715 to each other. The terminal 100 may construct the model data in consideration of the integrated state of each texture information when outputting the model data. That is, the terminal 100 can support a plurality of pieces of texture information to be applied to specific parts of the face part, integrating them into specific textures, and applying the converted textures to specific parts. Meanwhile, the terminal 100 may control the storage unit 150 to store the received model data and texture information.

In the above description, the model generation unit 305 transmits model data to the terminal 100, but the present invention is not limited thereto. That is, the model generation unit 305 can deliver the model data to the virtual makeup provider 303. Then, the virtual makeup supplier 303 applies the corresponding texture to the model data based on the user skin type information, the color tone selection information, and the illumination selection information transmitted from the terminal 100, and then applies the applied model data to the terminal 100, As shown in FIG.

As described above, the virtual makeup service operating method of the present invention can generate model data based on the service device 300, provide texture information applied to the model data, and support the output of the texture information. Also, the virtual makeup service operating method of the present invention is a method of operating the virtual makeup service, wherein the service device 300 receives at least one of the user's skin type information, color tone selection information, and illumination selection information from the terminal 100, It is possible to generate model data to which the texture with the texture is applied and to provide the model data to the terminal 100. Accordingly, the virtual makeup service operating method of the present invention can operate the various textures while reducing the burden of model creation of the terminal 100, thereby making it possible to more conveniently and variously support the makeup environment.

Meanwhile, the virtual makeup service method according to the embodiment of the present invention described above can be implemented in the form of a program command which can be executed through various computer means and recorded in a computer-readable recording medium. At this time, the computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. On the other hand, the program instructions recorded on the recording medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.

The computer-readable recording medium includes a magnetic recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical medium such as a CD-ROM and a DVD, a magnetic disk such as a floppy disk, A magneto-optical media, and a hardware device specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The program instructions also include machine language code, such as those generated by the compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. And the present invention is not limited to the above-described embodiments. That is, it is apparent to those skilled in the art that various embodiments based on the technical idea of the present invention are possible.

The present invention relates to a virtual make-up service system and method, and a terminal supporting the virtual make-up service system. The present invention relates to a virtual make-up service system and method, It is possible to support an appropriate make-up state so as to minimize the time-consuming time for make-up while maintaining a satisfactory make-up state.

100: terminal 110: communication unit 120: input unit
130: audio processing unit 140: display unit 150:
151: virtual makeup program 153: model data 155: texture DB
160: control unit 161: model generation unit 163: model reproduction unit
165: texture processor 170: camera 200: network
300: Service device 301:
303: Virtual makeup provider 305: Model creation unit
10: Virtual make-up service system

Claims (17)

delete A communication unit for forming a communication channel with the terminal and receiving user-selected information related to a subject and an image related to the subject of the head including a head and a face, and transmitting model data and texture information generated from the subject related image to the terminal;
A model generating unit for generating model data rendered to have a plurality of polygons based on the subject-related image transmitted by the communication unit; And
A skin texture for each skin type, a texture for each illumination according to various lights, and a texture for each color tone according to skin types to be applied to the model data based on the user selection information received through the communication unit, Selecting at least one texture, applying a texture according to the user selection information to the model data, and providing model data to the terminal through the communication unit;
Lt; / RTI >
The virtual makeup provider,
A database for storing various skin type textures, various hue textures, various illumination textures, and operation algorithms for determining the state of textures to be output when certain textures overlap for each makeup product;
At least one texture for each skin type selected according to the skin type information of the user's selection information among the textures stored in the database, at least one texture for each color tone according to the color tone indicated in the user selection information, A control module for searching at least one of at least one illumination-specific texture and transmitting the search to the terminal;
Wherein the virtual make-up service is a virtual service. 3. The method of claim 2,
The model generation unit
And transmits the generated model data to the virtual makeup providing unit. delete delete 4. The method according to any one of claims 2 and 3,
Wherein at least one of the communication unit, the virtual makeup providing unit, and the model creating unit is implemented as one or more servers operating on a cloud computing basis. An input unit for receiving user selection information;
A texture of a skin type classified according to a skin type according to a face part, a texture of an illumination according to various lights, a texture of a color tone according to various hues, A control unit for selecting at least one texture and applying the selected texture to model data generated based on a subject-related image and outputting the selected texture; And
A display unit for outputting a screen to which the texture is applied to the model data under the control of the control unit;
Lt; / RTI >
Wherein,
A model generation unit for generating model data composed of a plurality of polygons based on the subject-related image;
A texture to be applied to the model data according to the user selection information from the input unit is selected and transmitted to the model reproducing unit, and when a plurality of textures are selected to be applied, A texture processor for determining the texture of the object; And
A model reproducing unit for applying the texture transferred from the texture processor to the model data and outputting the texture to the display unit;
Wherein the virtual make-up service is a virtual camera. delete delete Claim 10 has been abandoned due to the setting registration fee. 8. The method of claim 7,
The texture processor
And a makeup editor that supports a user to select at least one of the texture for each skin type, the texture for each color tone, and the illumination-specific textures and the region to which the texture is to be applied in the model data, to the display unit A terminal that supports virtual makeup services. Claim 11 has been abandoned due to the set registration fee. 11. The method of claim 10,
The texture processor
Wherein a plurality of textures are selected to be applied to a certain region of the model data, and when a specific texture overlaps with each other, a texture having a plurality of textures is integrated and a texture is applied to a designated region A terminal that supports makeup service. Claim 12 is abandoned in setting registration fee. 8. The method of claim 7,
A storage unit for storing at least one of a subject-related image, subject-related model data, and a plurality of textures applicable to the model data;
Further comprising: a terminal for receiving the virtual makeup service. Claim 13 has been abandoned due to the set registration fee. 8. The method of claim 7,
And a communication unit for transmitting and receiving data through the network,
The control unit
Wherein the communication unit receives at least one of a subject-related image, subject-related model data, and a plurality of textures applicable to the model data from an external device via the communication unit. Claim 14 has been abandoned due to the setting registration fee. 8. The method of claim 7,
Further comprising a camera for photographing a subject-related image,
Wherein the controller generates the model data from a subject-related image photographed by the camera. A terminal comprising: a collection step of collecting a plurality of images related to a subject;
Generating rendered model data to include a plurality of polygons based on the subject images;
An application step of applying the user-selected textures to the model data; And
Outputting model data to which the texture is applied;
Lt; / RTI >
Wherein the application step comprises:
At least one of texture items for skin type, texture items for color tone, and texture for illumination can be selected according to the skin type of the face region, and when a plurality of textures are selected to be overlapped with each other, Displaying a makeup editor including an item for selecting a certain portion of the model data to which a selected texture among the textures is to be applied,
Receiving an input signal for selecting any one of the items output to the makeup editor;
Applying a texture selected in accordance with the input signal to the model data;
Wherein the virtual make-up service method comprises: delete A computer-readable recording medium storing a program for executing the virtual makeup service method according to claim 15.

Images