KR20180065951A - Apparatus for virtual plastic surgery based on subjective preference and method for the same - Google Patents

Abstract

Disclosed are an apparatus and a method for virtual face plastic surgery. According to an embodiment of the present invention, the apparatus for virtual face plastic surgery may include an attribute-specific preference evaluation processing unit for receiving an attribute-specific preference for at least one predetermined example face model from at least one experiment subject and storing the attribute-specific preference; an experiment subject tendency learning processing unit for classifying and managing the at least one experiment subject into a plurality of predetermined tendency groups on the basis of the attribute-specific preference of the at least one experiment subject; a user tendency classification processing unit for receiving an attribute-specific preference for the at least one predetermined example face model from the user, and determining a tendency group of the at least one user on the basis of the received attribute-specific preference; and a virtual plastic surgery information processing unit for creating a preferred face model on the basis of the attribute-specific preference of the user and the attribute-specific preference of the experiment subject corresponding to the tendency group of the user, and creating a virtual plastic surgery face model by applying the preferred face model to a face model of the user. According to the present disclosure, provided are an apparatus and a method in which a user without having knowledge about face plastic surgery may estimate a face model after a harmonized plastic surgery of his or her face only by inputting simple information.

Description

[0001] APPARATUS FOR VIRTUAL PLASTIC SURGERY BASED ON SUBJECTIVE PREFERENCE AND METHOD FOR THE SAME [0002]

The present disclosure relates to a technique for generating three-dimensional information, and more particularly, to a method and apparatus for generating 3D face model information by confirming and reflecting a user's preference.

In recent years, interest in cosmetic surgery has been increasing in accordance with the culture of modern society, which emphasizes personal appearance. Accordingly, a virtual molding system capable of easily predicting the shape of the user after molding is being developed with low cost and time.

Conventional virtual facial shaping apparatuses constitute a model for individual objects (e.g., eyes, nose, mouth, jaw, etc.) constituting a face, and combine local objects by using a model for each object, And can be modified or modified using a user interface such as a control point or a slide bar.

Such a conventional virtual facial shaping apparatus has an advantage that it is easy to selectively select individuals constituting a face to be formed. However, without expert knowledge of forming, many attempts are made to obtain a user's idealized face for a long time It is possible to cause problems such as repeatedly making a mistake or failing to harmonize the individuals constituting the face and outputting results that are not similar to the user's actual face model.

An object of the present invention is to provide a device and a method for a user who is not knowledgeable about face shaping to predict his or her harmoniously formed face model by simply inputting information.

It is another object of the present invention to provide an apparatus and method for predicting and providing a post-forming face model with a high degree of user satisfaction even if the user only inputs preference for a predetermined example face model.

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to an aspect of the present disclosure, a virtual facial shaping apparatus can be provided. Wherein the virtual face forming apparatus comprises: a preference-by-attribute evaluation processing unit for receiving and storing preference-by-attribute preferences for at least one predetermined example face model from at least one subject; A subject preference learning processing unit for classifying and managing the at least one subject in a predetermined plurality of preference groups, and a user preference input unit for inputting a preference for each attribute of the predetermined at least one example face model from the user, A user preference classifying processor for determining the preference group of at least one user based on the preference and a preference face model based on the preference for each attribute of the user and the preference for each attribute of the user corresponding to the user's preference group And the preferred face model Applied to a group of users face model may include forming a virtual information processing unit for generating a virtual model molding face.

The features briefly summarized above for this disclosure are only exemplary aspects of the detailed description of the disclosure which follow, and are not intended to limit the scope of the disclosure.

According to the present disclosure, it is possible to provide an apparatus and a method which allow a user who is not knowledgeable of face shaping to predict his or her harmonized post-model with only the input of information.

In addition, according to the present disclosure, an apparatus and a method for predicting a post-molding face model having a high degree of satisfaction of a user can be provided even if the user inputs only the pre-determined preference for the sample face model.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

1 is a block diagram showing a configuration of an apparatus for generating virtual formation information according to an embodiment of the present disclosure.
2 is a diagram showing a detailed configuration of a preference evaluation processing unit for each attribute provided in the virtual formed information generating apparatus according to the embodiment of the present disclosure.
3 is a diagram illustrating a data structure of an example face model of an example face model DB provided in the virtual forming information generating apparatus according to an embodiment of the present disclosure.
FIG. 4 is a diagram illustrating a preference data structure for each subject of a subject of a subject's preference DB provided in the virtual formed information generating apparatus according to an embodiment of the present disclosure; FIG.
FIG. 5 is a diagram showing a detailed configuration of a subject tendency learning processing unit provided in the virtual formed information generating apparatus according to an embodiment of the present disclosure. FIG.
6 is a diagram illustrating a propensity classifier parameter data structure of the propensity classifier parameter DB included in the virtual forming information generating apparatus according to an embodiment of the present disclosure.
FIG. 7 is a diagram illustrating a structure of a subject preference assistance DB provided in the virtual formation information generation apparatus according to an embodiment of the present disclosure.
8 is a diagram showing a detailed configuration of a user propensity classification processing unit provided in the virtual formed information generation apparatus according to an embodiment of the present disclosure.
9 is a diagram illustrating a data structure of user information of a user information DB included in the virtual forming information generating apparatus according to an embodiment of the present disclosure.
10 is a diagram showing a detailed configuration of a virtual forming information processing unit provided in the virtual forming information generating apparatus according to an embodiment of the present disclosure.
11 is a block diagram illustrating a computing system that executes a virtual forming information generating apparatus according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

In the present disclosure, when an element is referred to as being "connected", "coupled", or "connected" to another element, it is understood that not only a direct connection relationship but also an indirect connection relationship May also be included. Also, when an element is referred to as " comprising "or" having "another element, it is meant to include not only excluding another element but also another element .

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

In the present disclosure, the components that are distinguished from each other are intended to clearly illustrate each feature and do not necessarily mean that components are separate. That is, a plurality of components may be integrated into one hardware or software unit, or a single component may be distributed into a plurality of hardware or software units. Thus, unless otherwise noted, such integrated or distributed embodiments are also included within the scope of this disclosure.

In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some may be optional components. Thus, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Also, embodiments that include other elements in addition to the elements described in the various embodiments are also included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a block diagram showing a configuration of an apparatus for generating virtual formation information according to an embodiment of the present disclosure.

1, the virtual forming information generating apparatus 10 includes an input / output interface 11, an input / output device 12, a property preference evaluation processing unit 13, a subject's subject learning processing unit 14, a user propensity classification processing unit 15 ), And a virtual forming information processing unit 16. The virtual molding information generation apparatus 10 includes an example face model DB 100, a subject preference DB 200, a propensity classifier parameterizer 300, a subject preference aiding DB 400, and a user information DB 500, . ≪ / RTI >

In the embodiment of the present disclosure, both the user and the user may be a user who uses the virtual forming information generating apparatus. However, in order to distinguish the virtual forming information generating apparatus according to the characteristics of the user, The user participating in the inclination learning is referred to as the subject, and the user participating in the generation of the virtual forming information is referred to as the user.

The input / output interface 11 is a device for inputting data provided by a function provided in the virtual forming information generating apparatus 10 according to an embodiment of the present disclosure, or various services provided by a function unit provided in the virtual forming information generating apparatus 10 And can provide the corresponding user interface to the input / output device 12. In response, the input / output device 12 can output the data provided by the function unit provided in the virtual molding information generation apparatus 10, or the user interface through an output device (e.g., a display, a touch screen device) or the like.

The input / output device 12 may include an input device capable of receiving a key input from a user or a subject, and may receive input from an input device (e.g., a keyboard device, a mouse device) via the input / output interface 11 It is possible to provide a function section provided in the virtual molding information generation apparatus 10 provided in the virtual molding information generation apparatus 10. [ The input device included in the input / output device 12 may include a physical keypad or a virtual keypad displayed on the touch screen device.

The attribute-based preference evaluation processing unit 13 can operate in conjunction with the example face model DB 100 and the subject's preference DB 200. The attribute-based preference evaluation processing unit 13 displays an example face stored in the example face model DB 100 through the input / output device 12 and displays an environment (e.g., a user Interface). For example, the attribute preference evaluation processing unit 13 displays attributes (e.g., cuteness, sexyness, elegance, cleanliness, friendliness, etc.) of at least one example face model displayed through the input / output device 12, It is possible to provide a user interface for receiving the preference of each attribute. The attribute-based preference evaluation processing unit 13 receives preferences for each of the attributes (e.g., cuteness, sexyness, elegance, cleanliness, friendliness, etc.) of at least one example face model from at least one subject , And stores it in the subject's preference DB 200.

On the other hand, various example face models can be stored in the example face model DB (100) used for property preference evaluation.

Example face models can include models for face models that are likely to have high scores in terms of preference attributes of subjects or users, such as cuteness, sexyness, elegance, cleanliness, friendliness. Furthermore, the exemplary face model may include shape information and texture information, and further, shape information and texture information may be stored separately from each other.

In one embodiment of the present disclosure, the example face model may include a 3D face model. In addition, the 3D face model may be a real face model generated by photographing a face of a real person, a virtual face model obtained by manually transforming a reference face template through a professional 3D designer, or a face model generated by photographing a face of a real person And may include a mixed face model generated by modifying by hand.

Meanwhile, the subject's subject propensity learning processing unit 14 may operate in conjunction with the subject's preference DB 200, the propensity classifier parameter DB 300, and the subject's preference aiding DB 400.

The subject preference DB 200 may store a plurality of preference preferences inputted from a plurality of subjects as the evaluation result of at least one face model. The subject subject preference learning processing unit 14 detects a plurality of preference preference characteristics And can classify the subjects' tendencies into a plurality of groups on the basis thereof.

At this time, the subject's propensity learning processing unit 14 can separate the characteristics of preference by attribute in the classified group unit and store the characteristics of the preference in the propensity classifier parameter DB 300. Also, the subject's preference evaluation result is classified into different DB Can be stored in the constituent participant preference sub-DB 400. For example, when the subject tendency group is classified into M (M is a natural number) group, M subjects DBs may be provided in the subject's preference aiding DB 400, each group is assigned to each DB, The subject's preference evaluation result may be stored.

The user propensity classification processing unit 15 can operate in conjunction with the example face model DB 100 and the propensity classifier parser DB 300.

The user propensity classification processing unit 15 displays an example face stored in the example face model DB 100 through the input / output device 12 and displays an environment in which the preference for the attribute of the displayed example face model can be inputted ). ≪ / RTI > For example, the user propensity classification processing unit 15 displays attributes (e.g., cuteness, sexyness, elegance, cleanliness, friendliness, etc.) of at least one example face model displayed through the input / output device 12, It is possible to provide a user interface for receiving the preference of the attribute. The user propensity classification processing unit 15 may receive the preferences for each of the attributes (e.g., cuteness, sexyness, elegance, cleanliness, friendliness, etc.) of at least one example face model from the user.

In addition, the user propensity classification processing unit 15 may detect the user characteristics from the preferences inputted from the user, and use the propensity classifier parser DB 300 to determine the user tendency groups corresponding to the detected user characteristics.

Furthermore, the user propensity classification processing unit 15 may operate in conjunction with the user information DB 500. [ Specifically, the user propensity classification processing unit 15 provides an environment for managing user accounts, and can store and manage user preference groups and user preference groups in units of user accounts.

For example, the user preference classification processing unit 15 may provide an environment for creating a user account, so that information necessary for user account management (e.g., ID, password, personal identification number, personal authentication information Etc.) to create a user account. The user propensity classification processing unit 15 receives information (e.g., ID, password, and the like) necessary for accessing the user account, and only when the input information is valid information, , And may generate and provide information about user preference groups.

Meanwhile, the virtual molding information processing unit 16 can operate in conjunction with the example face model DB 100, the subject preference aiding DB 400, and the user information DB 500.

The virtual molding information processing unit 16 can confirm the preference for each attribute corresponding to the user account and the information about the user tendency group through the user information DB 500. [ That is, the virtual molding information processing unit 16 can confirm the preference (hereinafter, referred to as first preference information) for each attribute of the face model corresponding to the user account. In addition, the first preference information corresponding to the user account can be displayed while the first preference information can be modified. Thus, the virtual forming information processing unit 16 can confirm the preference of each attribute (hereinafter, referred to as 'preference for each second attribute') modified by the user. Furthermore, the virtual molding information processing unit 16 can check the preference (hereinafter referred to as third preference information) of each face model stored in the user's preference group. Then, the virtual forming information processing unit 16 can generate weight information by combining at least one of the first, second, and third preference information.

Further, the virtual formation information processing unit 16 can generate the user's preferred face model based on the weight information generated through the above-described operation.

On the other hand, the virtual molding information processing unit 16 can generate the user's face model using the image of the user's face. Then, the virtual formation information processing unit 16 can apply the preferred face model to the user's face model to generate a virtual formed face model.

Further, the virtual forming information processing unit 16 may provide an environment (e.g., a preferred face model providing interface, a virtual forming face model providing interface, and the like) that provides the generated preferred face model or the virtual formed face model.

Further, an environment (e.g., a preferred face model providing interface, a virtual molded face model providing interface, and the like) that provides a generated preferred face model or a virtual molded face model may include feedback information on the generated preferred face model or virtual formed face model, And an environment in which preference correction information can be input. For example, the virtual molding information processing unit 16 can re-input the preference for each attribute of the face model from the user through the button for inputting the preference correction information, and the second preference Weight information, a preferred face model, and a virtual formed face model can be regenerated.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the configuration of each component included in the virtual formed information generating apparatus according to an embodiment of the present disclosure will be described in detail with reference to the above description and drawings and the accompanying drawings.

2 is a diagram showing a detailed configuration of a preference evaluation processing unit for each attribute included in the virtual formed information generating apparatus according to an embodiment of the present disclosure.

The attribute-based preference evaluation processing unit 20 may include an example face playback unit 21, an attribute-based preference evaluation unit 22, and a preference storage unit 23.

The example face reproducing section 21 provides the example face models stored in the example face model DB 100 to the input / output device 12 through the input / output interface 11 sequentially or simultaneously at the same time so that the subject can evaluate the preference of the example face by attribute can do.

The example face model stored in the example face model DB 100 can be illustrated as shown in FIG. Further, the example face model 101 may include morphological information 102 and texture information 103, and further, morphological information 102 and texture information 103 may be stored separately from each other.

On the other hand, the property-based preference evaluating unit 22 is synchronized with at least one example face model provided by the example face reproducing unit 21, and at least one of the at least one example face model provided by the example face reproducing unit 21 (E.g., a user interface) that displays attributes (e.g., cuteness, sexyness, elegance, cleanliness, friendliness, etc.) and can input preferences for each attribute.

The attribute-based preference evaluating unit 22 may provide the preference storage unit 23 with the preference for each attribute of the exemplary face model input from the subject via the environment (e.g., user interface) described above.

The preference storage unit 23 may store the preference value for each attribute of the example face model evaluated by the subject in the subject's preference DB 200. [

For example, the preference value for each attribute stored in the subject's preference DB 200 can be illustrated as shown in FIG.

The subject's preference DB 200 may store a preference value by attribute for a plurality of subjects. The preference value 201 by attribute of each subject may include a preference value by attribute for a plurality of example face models . The attribute preference value 202 for the example face model may include an attribute 203 for each example model and a preference value 204 for each attribute.

In the embodiment of the present disclosure, the attribute-based preference evaluation processing unit 20 is provided in the virtual molding information generation apparatus, but the present disclosure is not limited thereto. As another example, the attribute-based preference evaluation processing unit 20 includes an example face playback unit 21, an attribute preference evaluating unit 22, a preference storage unit 23, an example face model DB 100, and a subject preference DB Output interface 11, an input / output device 12, and a communication interface (not shown) together with the input / output interface 200, as shown in FIG. In this case, the attribute-based preference evaluating device can transmit / receive data stored in the example face model DB 100 and the subject's preference DB 200 through wired / wireless communication with other electronic devices.

As another example, the attribute-based preference evaluating apparatus includes an example face reproducing unit 21, an attribute preference evaluating unit 22, a preference storing unit 23, an input / output interface 11, an input / output device 12, (Not shown), and can transmit and receive data corresponding to the face model DB 100 and the subject's preference DB 200 provided in other electronic devices.

FIG. 5 is a diagram showing a detailed configuration of a subject tendency learning processing unit provided in the virtual formed information generating apparatus according to an embodiment of the present disclosure. FIG.

The subject tendency learning processing unit 50 according to an embodiment of the present disclosure includes a preference characteristic generation unit 51, a propensity classifier learning unit 52, a propensity classifier parameter storage unit 53, a propensity classifying unit 54, And a star preference storage unit 55.

The preference characteristic generation unit 51 may convert the preference for each attribute of the example face models into one characteristic value for each subject. For example, the preference characteristic generation unit 51 may convert the preference per attribute of N sample face models for each subject into one vector. That is, the preference characteristic generation unit 51 may convert the preference for each of the N example face models for the i-th subject into the vector X _i illustrated in Equation (1) below.

는 각각 k번째 예제 얼굴 모델에 대한 귀여움 속성의 선호도 수치, 섹시함 속성의 선호도 수치, 우아함 속성의 선호도 수치, 청순함 속성의 선호도 수치, 친근함 속성의 선호도 수치를 나타낸다. In Equation (1)

Represents the preference value of the cuteness attribute for the k-th example face model, the preference value of the sexy attribute, the preference value of the elegance attribute, the preference value of the innocent attribute, and the preference value of the affinity attribute.

Although embodiments of the present disclosure illustrate preference attributes as cuteness, sexyness, elegance, cleanliness, friendliness, etc., the present disclosure is not limited thereto, and may be variously changed.

The hunting classifier learning unit 52 generates a k-means algorithm, a hierarchical clustering algorithm, and the like, which are generally used for non-background learning using a set of preference vectors of each subject converted into a vector, (For example, M, M is a natural number) of the subject's tendency through a mean-shift algorithm, for example.

The propensity classifier parameter storage unit 53 may store, in the propensity classifier parameter DB 300, a parameter 601 (see FIG. 6) necessary for the operation of the configured propensity classifier.

Here, the propensity classifier parameter is a parameter required when the classifier classifies by the various non-geometry learning classifier algorithms. For example, in the case of a classifier using a k-means algorithm, the propensity classifier parameter may include a predetermined number of groups, an average value of each group, and the like.

The propensity classifying unit 54 classifies each subject's preference characteristic (e.g., vector) transformed through the preference characteristic generating unit 51 into the propensity for each subject using the propensity classifier that reflects the parameters stored in the propensity classifier parameter DB 300 It is categorized as one of the M subject-oriented groups.

Further, it is preferable that the tendency of each subject is classified into one group among the M subject incentive groups, but it is also possible to belong to a plurality of groups if they are likely to belong to one or more groups.

The preference preference storage unit 55 may store the preference score stored in the subject preference DB 200 according to the subject's preference classification result according to the subject's preference group and store the preference preference value in the subject's preference assistance DB 400.

For example, a plurality of DBs 400-1, 400-2, ..., 400-n may be provided in the subject's affinity support DB 400, and each of the subject's affinity preferences DB 400-1, ..., 400-n may correspond to the subject's tendency group, respectively. For example, the preferences of the subjects belonging to the first subject's tendency group can be stored in the first subject's preference assistance DB 400-1, and the preference of the subjects belonging to the second subject's preference group can be stored in the second subject's preference assistance And may be stored in the DB 400-2.

In the embodiment of the present disclosure, the subject-oriented learning processing unit 50 is provided in the virtual forming information generating apparatus, but the present disclosure is not limited thereto. As another example, the subject tendency learning processing unit 50 includes a preference characteristic generation unit 51, a propensity classifier learning unit 52, a propensity classifier parameter storage unit 53, a propensity classifying unit 54, Output interface 11, an input / output device 12, and a communication interface (not shown) together with the input / output interface 55, and may be provided as a separate subject's subject learning processing device. In this case, the subject's propensity learning processing apparatus can transmit and receive data stored in the example face model DB 100, the propensity classifier parameter DB 300, and the subject's preference aiding DB 400 via wired / wireless communication with other electronic devices have.

As another example, the subject's propensity learning processing apparatus includes a preference characteristic generation unit 51, a propensity classifier learning unit 52, a propensity classifier parameter storage unit 53, a propensity classifying unit 54, a propensity preference storage unit 55 (Not shown), and an example facial model DB 100, a propensity classifier parameter DB 300, and a facial classifier parameter database 300 provided in other electronic devices, and an input / output interface 11, an input / output device 12, Data corresponding to the subject's preference sub-DB 400 can be transmitted and received.

8 is a diagram showing a detailed configuration of a user propensity classification processing unit provided in the virtual formed information generation apparatus according to an embodiment of the present disclosure.

The user preference classification processing unit 80 includes a user preference classifying unit 83, a preference storing unit 84, a preference characteristic generating unit 85, a user preference classifying unit 84, Unit 86, and a user propensity lowering unit 87. [

The user account management unit 81 provides an environment for managing user accounts, and can store and manage user preference groups and user preference groups in units of user accounts. For example, the user account management unit 81 provides an environment for creating a user account, and provides information necessary for user account management (e.g., ID, password, personal identification number, personal authentication information ) To create a user account. The user account management unit 81 can store the generated user account in the user information DB 500 and manage the same. The user account management unit 81 receives information (e.g., ID, password, etc.) necessary for accessing the user account, and can access the user account only when the received information is valid information.

The example face reproducer 82 sequentially provides the example face models stored in the example face model DB 100 to the input / output device 12 through the input / output interface 11 so that the user can evaluate the preference for each attribute of the example face can do.

The attribute-based preference evaluating unit 83 is synchronized with at least one example face model provided by the example face reproducing unit 82 and is provided with at least one attribute (for example, (E.g., a user interface) that can display the preferences for each attribute and display preferences for each attribute (e.g., cuteness, sexyness, elegance, purity, friendliness, etc.).

The attribute-based preference evaluating unit 83 may provide the preference storage unit 23 with the preference for each attribute of the example face model input from the subject via the above-described environment (e.g., user interface).

The preference storage unit 84 can store and manage the preference value for each attribute of the sample face model evaluated by use. For example, the preference storage unit 84 may store the preference value by attribute for the sample face model evaluated by use in the user information DB 500. [

The preference property generation unit 85 may convert the preference value by attribute of the sample face model evaluated by the user into at least one property value. For example, the preference characteristic generation unit 85 generates a preference characteristic value for each of the N sample face models for the user (e.g., cute, sexy, elegant, clean, friendliness, etc.) Can be converted.

The propensity classifying section 86 classifies the preference characteristic (e.g., vector) generated by the preference characteristic generating section 85 as the propensity of the user by using the propensity classifier that reflects the parameters stored in the propensity classifier parameter DB 300 as M It can be categorized into one of groups of tendency.

The user propensity storage unit 87 may receive the propensity classification result of the user from the propensity classifying unit 86 and may store it in the user information DB 500.

9, the user information DB 500 stores and manages the user account information 901, the user preference group information 902, and the attribute preference value 903 for the example face model can do.

Furthermore, the example face regenerating unit 82 and the property-specific preference evaluating unit 83 are the same as the example face regenerating unit 21 and the property-specific preference evaluating unit 22 provided in the above- It is possible to perform the reproduction of the example face model and the preference evaluation operation for each attribute. The preference characteristic generating unit 85 and the user propensity classifying unit 86 classify preference characteristics by the same method as the preference characteristic generating unit 51 and the propensity classifying unit 54 provided in the above- You can create properties and classify user preferences.

On the other hand, in the embodiment of the present disclosure, the user propensity classification processing unit 80 is provided in the virtual forming information generating apparatus, but the present disclosure is not limited thereto. As another example, the user propensity classification processing unit 80 includes a user's account management unit 81, an example face playback unit 82, a property preference evaluation unit 83, a preference storage unit 84, a preference property generation unit 85, An input / output interface 11, an input / output interface 11, an input / output interface 11, an input / output interface 11, an input / output interface 11, an input / output interface 11, An apparatus 12, and a communication interface (not shown), and may be provided as a separate user propensity classifying apparatus. In this case, the user propensity classifying device can transmit and receive data stored in the example face model DB 100, the propensity classifier parameter DB 300, and the user information DB 500 through wired / wireless communication with other electronic devices.

As another example, the user propensity classification apparatus may include a user account management unit 81, an example face playback unit 82, a property preference evaluation unit 83, a preference storage unit 84, a preference property generation unit 85, And may include a preference classifying unit 86, a user preference lowering unit 87, an input / output interface 11, an input / output device 12, and a communication interface (not shown) The user data DB 100, the propensity classifier parameter DB 300, and the user information DB 500. [

10 is a diagram showing a detailed configuration of a virtual forming information processing unit provided in the virtual forming information generating apparatus according to an embodiment of the present disclosure.

10, the virtual molding information processing unit 1000 includes a user's face photographing unit 1001, a user's face model generating unit 1002, a user's face model storage unit 1003, a property preference adjusting unit 1004, The preferred embodiment of the present invention includes the overlap weight generation unit 1005, the preferred face model generation unit 1006, the virtual molded face model generation unit 1007, the virtual molded face model reproduction unit 1008, and the virtual molded face model storage unit 1009 .

The user's face photographing unit 1001 may include a three-dimensional scanner that generates three-dimensional data of an object using at least one camera, a pattern light, a depth sensor, a laser, and the like. In particular, the user's face photographing unit 1001 can perform three-dimensional scanning on the face region of the user to generate user face image data.

The user's face model generation unit 1002 can generate the user's three-dimensional face data using the user's face image data generated by the user's face photographing unit 1001. [ For example, the user's face model generation unit 1002 generates three-dimensional face data by applying user face image data to a reference face template, or generates three-dimensional face data from a stereo camera or a plurality of cameras provided in the user face photographing unit 1001 Dimensional face data can be restored by restoring the three-dimensional face data from two or more user face image data obtained by the user face photographing unit 1001 or applying the depth image acquired through the depth sensor provided in the user face photographing unit 1001 to the reference face template have.

Further, the user's face model generation unit 1002 may be configured to generate the user's face using the Blanz 1999 method (V. Blanz and T. Vetter, "A morphable model for the synthesis of 3D faces," In Proc. ACM SIGGRAPH, Los Angeles, (Sumner 2004, RW Sumner, J. Popovic, Deformation transfer for triangle meshes, In Proc. ACM SIGGRAPH, Los Angeles, CA, 2004); Beeler 2010 system (T. Beeler, B. Bickel, P. Beardsley, R. J. Roth, Y. Tong, X. Liu, " Adaptive < / RTI > 3D Face Reconstruction from Unconstrained Photo Collections, Proc. IEEE Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, Jun. 2016), Kazemi2014 method (V. Kazemi, C. Keskin, J. Taylor, P. Kohli , and S. Izadi, "Real-time Face Reconstruction from a Single Depth Image," In Proc. International Conference on 3D Vision (3DV 2014), Tokyo, Japan, Dec. 2014.) On Seconds may process the three-dimensional face data restoring.

The user's face model storage unit 1003 can store the three-dimensional face data generated by the user's face model generation unit 1002 in the user information DB 500.

The attribute preference adjusting unit 1004 displays the user preference for each attribute stored in the user information DB 500 through the input / output interface 11 and the input / output device 12 , It is possible to provide an environment in which the preference of each user can be modified. Accordingly, the preference-by-attribution adjusting unit 1004 can confirm the preference for each attribute (hereinafter, referred to as 'preference for each second attribute') modified by the user.

The preference overlap weight generation unit 1005 may receive the preference for each first attribute and the preference for each second attribute from the per attribute preference adjuster 1004. The preference overlap weight generation unit 1005 can check the preference of each participant (hereinafter referred to as 'preference for each third property') from the subject's preference aiding DB 400 corresponding to the user preference group.

Furthermore, the preference overlap weight generation unit 1005 may generate preference weight information by combining at least one of the first, second, and third preference preferences.

The preference (E _u ) for each first attribute can be illustrated as the following Equation (2), the preference for each second attribute (b) can be exemplified as the following Equation (3) (4) " (4) "

In Equation (4), E _i represents the preference of each i- th subject for each attribute. The preference by the third attribute is preference by attribute for a plurality of subjects included in the subject's preference sub-DB 400 corresponding to the user preference group, so that the preference for each attribute by the number P of the plurality of subjects is included And can be represented as Equation (5) below.

As another example, the preference overlap weight generation unit 1005 may be configured as shown in Equation (6) by combining preferences for the first and third attributes.

As another example, the preference overlap weight generation unit 1005 may be configured as shown in Equation (7) using only the preference for each first attribute, excluding the preference for each third preference.

On the other hand, the preference weighting weight generator 1005 may generate preference weighting information by combining at least one of preferences of the first, second, and third attributes. The preference weighting information may be weighted by the first, 3 property and the following equation (8).

In addition, the preference weight information is used for constructing the preferred face model by superimposing the example face models, and may include the weight values for the respective example face models. The preference weight information can be expressed by Equation (9) below.

이고,

의 조건을 만족하여야만 왜곡 없이 선호 얼굴 모델의 생성이 가능하다. 따라서, 선호도 중첩 가중치 생성부(1005)는 전술한 수학식 8과 선호도 가중치 정보(W)의 제약 조건(

,

)을 동시에 만족하는 가중치 정보를 Quadratic programming 기법을 통해 산출할 수 있다. In Equation (9), W _l can represent a weight for the l-th (l is a natural number) face model. Further, the preference weight information W

ego,

It is possible to generate the preferred face model without distortion. Therefore, the preference overlap weight generation unit 1005 generates the preference weight weight value W based on the above-described Equation (8) and constraint conditions of the preference weight information W

,

) Can be calculated through the quadratic programming technique.

,

)을 전술한 수학식 8에 부드러운 조건(soft constraints)으로 변형 및 반영하여 가중치 정보를 산출할 수도 있다. As another example, the preference overlapping weight generator 1005 may generate hard constraints for the weight information W (e.g.,

,

) May be transformed into soft constraints in Equation (8) to reflect the weight information.

On the other hand, the preferred face model generation unit 1006 can apply the weight information W to the vertex data of the type information constituting each corresponding example face model, and generate the preferred face model by combining the applied example face models .

The virtual formed face model generation unit 1007 applies deformation of the favorite face model generated in the preferred face model generation unit 1006 to the user's face model stored in the user information DB 500 based on the Deformation Gradient method, It is possible to generate a virtual formed face model in which the position of a vertex of the face model is changed. For example, the virtual formed face model generation unit 1007 can generate a virtual formed face model through the calculation of Equation (10).

은 전체 삼각 메쉬의 수, j는 삼각 메쉬의 인덱스,

와

는 각각 선호 얼굴 모델과 사용자 얼굴의 j번째 메쉬의 3x3 변환 함수를 나타낸다. 그리고, 각 정점이 한 점에서 만나는 조건(

)에서 각 메쉬의 변환 함수의 frobenius 거리를 최소화하는 정점들(

)을 산출할 수 있다. In Equation (10)

Is the total number of triangular meshes, j is the triangular mesh index,

Wow

Represent the 3x3 transform functions of the preferred face model and the jth mesh of the user's face, respectively. Then, the condition that each vertex meets at one point (

) Vertices minimizing the frobenius distance of each mesh's transform function (

) Can be calculated.

Further, in order to reflect the user's taste, the virtual molded face model generation unit 1007 reflects the user's face model to the user's face model, and stores the user's face model in which the weight is reflected, The final virtual molded face model can be generated by linear superposition processing with the model.

The virtual molded face model reproduction unit 1008 receives the virtual molded face model (or the final virtual molded face model) from the virtual molded face model generation unit 1007 and receives the virtual molded face model A molded face model (or a final virtual molded face model) can be displayed.

On the other hand, the virtual molded face model storage unit 1009 can store and manage the virtual molded face model (or the last virtual molded face model) generated by the virtual molded face model generation unit 1007 in the user information DB 500 .

In the embodiment of the present disclosure, the virtual forming information processing unit 1000 is provided in the virtual forming information generating apparatus, but the present disclosure is not limited thereto. As another example, the virtual molding information processing unit 1000 includes a user's face photographing unit 1001, a user's face model generating unit 1002, a user's face model storage unit 1003, a preference degree adjusting unit 1004, A virtual formed face model storage unit 1009, a sample facial model DB 1002, a virtual face model storage unit 1004, Input interface 11, input / output device 12, and communication interface (not shown) together with the user's personal preference information processing device 100, subject's affinity assist DB 400 and user DB 500, As shown in FIG. In this case, the virtual molding information processing device can transmit and receive data stored in the example face model DB 100, the subject's preference aiding DB 400, and the user DB 500 via wired / wireless communication with other electronic devices.

As another example, the virtual forming information processing apparatus 1000 includes a user's face photographing section 1001, a user's face model generating section 1002, a user's face model storing section 1003, a property preference adjusting section 1004, A virtual molding face model storage unit 1009, a virtual molding face model storage unit 1009, an input / output interface (I / O) interface unit 1005, a preferred face model generation unit 1006, An object model DB 100, a subject's preference aiding DB 400, and a user DB 500, which are provided in other electronic devices, can be included in the input / output device 11, the input / output device 12, and the communication interface (not shown) And can transmit and receive data corresponding to the data.

11 is a block diagram illustrating a computing system that executes a virtual forming information generating apparatus according to an embodiment of the present disclosure.

11, a computing system includes at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a storage 1600, , And a network interface 1700. [

The processor 1100 may be a central processing unit (CPU) or a memory device 1300 and / or a semiconductor device that performs processing for instructions stored in the storage 1600. Memory 1300 and storage 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) and a RAM (Random Access Memory).

Thus, the steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by processor 1100, or in a combination of the two. The software module may reside in a storage medium (i.e., memory 1300 and / or storage 1600) such as a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, You may. An exemplary storage medium is coupled to the processor 1100, which can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor 1100. [ The processor and the storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (1)

A preference-based preference evaluation processing unit for receiving and storing preference-by-attribute preferences for at least one predetermined example face model from at least one subject,
A subject tendency learning processing unit for classifying and managing the at least one subject to a predetermined plurality of tendency groups on the basis of the preference for each attribute of the at least one subject,
A user propensity classifying processor for receiving a preference for each attribute of the predetermined at least one example face model from a user and determining a propensity group of the at least one user based on the preference for each attribute,
A preferred face model is generated based on the user's preference for each attribute and the user's preference for the attribute of the user corresponding to the user's preference group and the preferred face model is applied to the user's face model And a virtual molding information processing unit for performing virtual molding of the virtual face.

Images