KR100608840B1 - Method for synthesis of 3d avata model of handset - Google Patents

Abstract

The present invention proposes a method of synthesizing a 3D avatar based on a portrait, and since there is no separate photo editing or image processing process and no skilled work is required, the general user can easily and easily create an avatar. In addition, the present invention has the advantage that the user can create a three-dimensional avatar model on the fly by using a picture stored in the mobile terminal or a picture taken by the camera of the mobile terminal. The present invention includes the steps of photographing a target person, selecting a three-dimensional face model similar to the target person, detecting a keypoint in the face of the captured person, and converting the detected keypoint coordinates into the three-dimensional face model. The mapping process includes applying the three-dimensional face model to the avatar to complete the three-dimensional avatar model.

3d avatar

Description

3D avatar synthesis method of mobile terminal {METHOD FOR SYNTHESIS OF 3D AVATA MODEL OF HANDSET}

1 is a flowchart illustrating a three-dimensional avatar face generation procedure according to a first embodiment of the present invention.

2 is a diagram showing focusing criteria displayed on the portable terminal of the present invention;

3 shows keypoints displayed on a portable terminal of the present invention;

4 is a diagram representing a multidimensional morphing function according to the present invention.

5 is a flowchart illustrating a three-dimensional avatar face generation procedure according to a second embodiment of the present invention.

The present invention relates to an avatar synthesis method, and more particularly, to a 3D avatar synthesis method using a photo image.

An avatar representing an alter ego or avatar is an animated character that takes the role of a user in cyberspace, and refers to a graphic icon representing a user in a 3D or virtual reality game or a web chat.

Avatars can be referred to as virtual bodies representing users in cyberspace, and the fields in which avatars are used are being expanded to cyber shopping malls, virtual education, and virtual offices as well as chatting and online games.

 Three-dimensional avatars have appeared to improve the lack of realism by implementing the existing avatar in two-dimensional graphics. However, although the three-dimensional avatar has a three-dimensional and realistic feeling, it does not fully satisfy the user's needs in that it uses a graphic character provided by a service provider different from the actual user.

In order to solve this problem, a method of synthesizing an avatar model using a user's photo has appeared, but in the case of using a user's photo, a face obtained through editing a photo image by using a separate graphic program such as Photoshop The skilled work is required to synthesize an image in a three-dimensional model generator such as 3DS Max.

Therefore, the avatar model synthesizing method using such a photograph also has a problem that a general user must purchase an expensive program and be able to perform professional graphic work.

The present invention is to solve the problems described above, the object is,

The present invention provides a method for synthesizing a three-dimensional avatar of a mobile terminal for automatically generating a three-dimensional avatar using a portrait photograph taken by the portable terminal.

In order to achieve the above object, the present invention, a three-dimensional avatar synthesis method of a mobile terminal

Photographing the target person; Selecting a three-dimensional face model similar to the target person, detecting a keypoint in the face of the photographed person, mapping the detected keypoint coordinates to the three-dimensional face model, and selecting the three-dimensional face model. Applied to the avatar, a three-dimensional avatar model is completed.

Preferably, the photographing process includes the steps of switching the portable terminal to a photographing mode, displaying a predetermined focusing criterion on a screen, aiming a face of a target person at the focusing criterion, and driving a mounted camera. Characterized in that the process of photographing the target person.

Preferably, the predetermined focusing criterion is characterized in that it has an H shape.

Preferably, the keypoint detection process is detected by displaying a plurality of keypoints on a screen, arranging the keypoints to match the outline of the photographed face, and obtaining coordinates of the keypoints. .

Preferably, the method of synthesizing the 3D avatar according to the present invention comprises comparing the similarity between the face of the photographed person and the 3D model with a threshold, and when the similarity is less than or equal to the threshold, avatar the 3D face model. It further includes the process of completing the three-dimensional Avatar model by applying to.

In order to achieve the above object, the three-dimensional avatar face synthesis method of the mobile terminal according to the present invention,

Refer to the process of photographing the target person, the process of detecting the facial features of the person photographed, the process of obtaining a three-dimensional face shape similar to the photographed face, the features of the photographed face and the features of the three-dimensional face shape In this case, a three-dimensional face model of the target person is generated.

In order to achieve the above object, the 3D avatar face synthesis method of a mobile terminal according to the present invention,

Importing an image stored in a memory, selecting a 3D face model similar to the target person shown in the image, detecting a keypoint on the face of the target person, and converting the keypoint of the detected face into the 3D face model The process of mapping to the process consists of applying the three-dimensional face model to the avatar to complete the three-dimensional avatar model.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

1 is a flowchart illustrating a three-dimensional avatar face generation procedure according to a first embodiment of the present invention. Referring to Figure 1, a three-dimensional avatar synthesis process according to the present invention will be described in detail.

In order to create an avatar, the user first selects the avatar synthesis menu of the mobile terminal. (S10) When the avatar synthesis menu is selected, the mobile terminal switches to the shooting mode, and the H-type focusing criteria (H- Style Fucusing Reference) is displayed (S20). As shown in FIG. 2, the user aims the H-type focusing criteria on the face of the person to be the avatar. At this time, the eyes of the subject and the face width of the subject are matched with the horizontal bar (-) and the vertical bar (|) of H, respectively.

When the aiming of the target person is completed based on the H-type focusing criteria, the camera mounted on the portable terminal is photographed to capture the target person. (S30) And as shown in FIG. (E.g., eight) is displayed on the terminal screen (S40).

The N keypoints are for detecting features of the face shape of the target person, and are displayed at a designated position (coordinate) of the screen in accordance with the completion of shooting of the target person.

As shown in FIG. 3, when the keypoints are displayed on the screen, the user rearranges the keypoints to match the outline of the photographed face using the keypad of the terminal. That is, the keypoint to be adjusted is selected by pressing the numeric keys of the keypad, and the selected keypoint is moved to the desired point by pressing the direction key of the keypad.

In this way, when relocation of the keypoints is completed, the user instructs to generate a three-dimensional avatar face by pressing a predetermined command key. The mobile terminal stores a facial feature (coordinates of keypoints) of the photographed person in a memory (S40), and according to the user's instruction, a 3D to be applied to the avatar based on the facial feature of the photographed person. Make up your face. The avatar synthesis algorithm according to the present invention preferably generates a simple three-dimensional face model that can effectively utilize the resources of the mobile terminal, rather than pursuing a high-quality three-dimensional face model.

The construction method of the three-dimensional face according to the present invention is as follows.

When the relocation of the keypoints is completed according to the user's operation, the user combines a predetermined number of face shapes (hereinafter, referred to as basic face shapes) that are already made, and has a three-dimensional face shape similar to the face shape of a photographed person (hereinafter, (S50) and a multidimensional morphing function is used in the process of obtaining the original face shape.

4 is that the image of the multi-dimensional morphing function (Multidimensional Morphing Function), multi-dimensional morphing function of Figure 4 by a linear combination (Linear Combination) the basic face shape, the original face type (f _K) similar to the photographed face type It is a function to find. W in FIG. 4 is a control parameter vector w ₁ , w ₂ ,..., W _N , N is the number of control parameters, and also means the number of the basic face shapes. The control parameter w _j is represented by any real value between 0 and 1 (w _{j ∈} [0, 1], j = 1, 2, ..., N), and the sum of all control parameters is 1. (Σw _j = 1)

The basic face shape (a _K , b _K , ..., e _K ) is a human face in the shape (shape) such as round face, square face shape, triangular face shape, inverted triangular face shape, etc. It is classified according to. The basic face shapes are created by the 3D modeler manually, so they are independent of each other.

Since mobile terminals are generally developed for the purpose of a call, a graphics processing unit (Graphics Processing Unit, GPU). Considering the limited resources of such a mobile terminal, the basic face type is appropriately about 5 (N = 5), and the number of key points K is about 8 appropriately. Of course, such set values (N = 5, K = 8) are illustrated for the purpose of explanation of the invention.

If necessary, the user adjusts the control parameter vector (W) value of the multidimensional morphing function to correct the generated original facial shape. The 3D face model of the photographed face is obtained based on the original face shape (S60).

Equation 1 shows a procedure for obtaining a three-dimensional face model (key points q of a three-dimensional face shape), and t in Equation 1 is a difference between key points of each basic face shape. Is a fixed constant given, w is the control parameter and o is the keypoint of the original facial shape.

[Equation 1]

q = t × w + o, (w = (p-o) / t)

In Equation 1, t x w represents a distance difference between the key point of the photographed face and the key point of the original face shape. The control parameter w becomes large in the part where the difference of a basic face shape is large (for example, the coordinate value of a jaw part), and becomes small in the part where the difference of a basic face shape is small (for example, the coordinate value of the center of a forehead).

According to the present invention, as shown in Equation 1, by adding or subtracting the distance difference between the key point p of the photographed face and the key point of the original face shape to or from the key point o of the original face shape, the key point of the three-dimensional face shape. Find (q). In this process, two-dimensional coordinate values are mapped to three-dimensional coordinate values.

For the calculation of Equation 1, a coordinate value corresponding to the Z axis of o and q should be set to 0 (zero). That is, since o and q are represented by three-dimensional coordinate values of the X, Y, and Z axes, and p is represented by two-dimensional coordinate values of the X and Y axes, a coordinate value corresponding to the Z axis of o and q is 0 (zero). ), O (X, Y coordinate), q (X, Y coordinate) and p (X, Y coordinate) can be compared or calculated.

Then, when the three-dimensional face model of the photographed person is generated, the similarity between the actual face of the photographed person and the three-dimensional face model is measured. The present invention uses Euclidian distance as the similarity measurement function. Calculation formula of the similarity measurement function is shown in Equation 2 below.

[Equation 2]

E = Σ || p _i -q _i || (i = 1, 2, ..., K)

Here, K denotes the number of keypoints and p _i denotes keypoints of the photographed face. And q _i denotes key points of the 3D face model.

The present invention compares the measured similarity with a predetermined threshold, and if the similarity is greater than the threshold, adjusts the control parameter w value of the morphing function to compare the similarity with the threshold again. This comparison between the similarity and the threshold is repeated until the similarity is less than or equal to the predetermined threshold, and by adjusting the control parameters each time, a three-dimensional face model similar to the face of the photographed person is obtained.

After obtaining the three-dimensional face model, measuring the similarity with the photographed face is at the user's choice. If the user determines that it is unnecessary, the present invention does not perform the similarity measuring process.

Then, the present invention is to cut the center portion (part including eyes, nose, mouth) of the face taken in accordance with the connection line of the key points of the picture, and according to the planar mapping method, the cut out the three-dimensional face generated Synthesize with the model (S70) and smooth the curved surface and corners of the synthesized three-dimensional face using Gouraud shading method.

5 is a flowchart illustrating a three-dimensional avatar face generation procedure according to a second embodiment of the present invention. If the first embodiment of the present invention uses a photographed image, the second embodiment of the present invention to be described below relates to a three-dimensional avatar generation procedure using a stored image, that is, a previously generated image.

Referring to Figure 5, it will be described in detail the three-dimensional avatar synthesis process according to the present invention.

First, the user selects the avatar synthesis menu of the mobile terminal, and loads an image of the person required for avatar synthesis from the memory (S110 & S120). By combining basic face shapes in the same manner as in the first embodiment of the present invention, An original face shape similar to that of the target person is obtained. (S130).

Thereafter, the mobile terminal displays an H-style focusing reference in the center of the screen, and the user aims the "focusing criteria" on the face of the target person so as to match the face of the target person. That is, the "focusing criterion" is moved by using the keypad of the terminal so that the horizontal bar (-) of the H-type focusing standard and the vertical bar (| |) coincide with the eyes of the target person and the face width of the target person.

When the focusing criterion and the face of the target person match, the mobile terminal displays N key points (for example, 8) at a designated position (coordinate) of the screen. The user rearranges the keypoints to match the outline of the face of the subject and detects coordinate values (face feature of the subject) of the relocated keypoints p _i . (S150)

Thereafter, a three-dimensional face model of the target person is generated based on the face feature of the target person (coordinate values of keypoints) and the original face shape. (S160)

The procedure for obtaining the three-dimensional face model of the subject is the same as that of the first embodiment of the present invention. When the three-dimensional face model is generated, the present invention measures the similarity between the key points p _i of the face of the subject and the key points q _i of the three-dimensional face model according to Equation 2 above.

The measured similarity is compared with a predetermined threshold, and if the similarity is greater than the threshold, the control parameter w of the morphing function is adjusted to compare the similarity with the threshold again. The comparison between the similarity and the threshold is repeated until the similarity is less than or equal to the predetermined threshold, and by adjusting the control parameters each time, a three-dimensional face model similar to the face of the subject is obtained. (S170)

After obtaining the three-dimensional face model of the target person, measuring the similarity between the key point (p) of the face of the target person and the key point (q) of the three-dimensional face model is at the user's choice and the user determines that it is unnecessary. In the present invention, the similarity measurement process is not performed.

Then, the present invention is to cut the center portion (part including eyes, nose, mouth) of the face taken in accordance with the connection line of the key points of the picture, and according to the planar mapping method, the cut out the three-dimensional face generated Synthesize with the model (S180) and smooth the curved surface and the edge portion of the synthesized three-dimensional face using the Gouraud shading (Gouraud shading) method.

As described above, according to the first and second embodiments of the present invention, when the three-dimensional face to be applied to the avatar is completed, the head style, the hair color, and the style of the beard (mustache, beard, etc.) of the avatar are handheld. Is displayed on the screen, the user selects the desired one according to taste.

When the setting of the avatar's head and beard is completed, a side view of the avatar is displayed on the screen of the mobile terminal, and a menu window for adjusting the nose length, the chin weight, and the forehead inclination is displayed. Again, the user decides the shape of the nose, chin and forehead according to his or her taste.

Subsequently, when the style setting for the side view of the avatar is completed, a menu window for setting the body shape of the avatar, the surrounding background of the avatar, and the animation motion (dance, emotion expression) of the avatar is displayed on the screen. The user selects the desired one according to his or her taste. When all the settings for the avatar's face, body shape, motion, and the like are completed, the synthesis process of the 3D avatar is completed.

As described above, the present invention proposes a method for synthesizing a 3D avatar based on a portrait, and there is no separate photo editing or image processing process, and no skilled work is required so that a general user can easily and conveniently You can create an avatar.

In addition, the present invention has the advantage that the user can create a three-dimensional avatar model on the fly by using a picture stored in the mobile terminal or a picture taken by the camera of the mobile terminal.

Although the present invention has been described with reference to the embodiments illustrated in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (18)

Photographing the target person; Selecting a 3D face model similar to the target person; Detecting a keypoint from the face of the photographed person; Mapping the detected keypoint coordinates to the three-dimensional face model; 3. A method of synthesizing a 3D avatar of a mobile terminal, comprising: applying the 3D face model to an avatar to complete a 3D avatar model. The method of claim 1, wherein the photographing process Switching the mobile terminal to a shooting mode; Displaying predetermined focusing criteria on the screen of the terminal; Aiming the face of the target person against the focusing criterion; 3. A method of synthesizing a 3D avatar of a mobile terminal, comprising: driving a camera of a terminal to photograph the target person; 3. The method of claim 2, wherein the predetermined focusing criteria is 3D avatar synthesis method of a mobile terminal, characterized in that the H-shape. The method of claim 1, wherein the keypoint detection process is performed. Displaying a plurality of key points on the screen; Arranging the keypoints to match the outline of the photographed face; When the arrangement of the key point is completed, a method of synthesizing the 3D avatar of a mobile terminal, characterized in that the process of obtaining the coordinates of the key points. The method of claim 4, wherein the keypoint placement process is performed. When a plurality of keypoints are displayed on the screen, using a predetermined button on the keypad, selecting a desired keypoint; And moving the selected keypoint to coincide with the outline of the photographed face by using a direction key of the keypad. The method of claim 1, further comprising: comparing a similarity between the face of the person photographed with the three-dimensional model with a threshold; And if the similarity is less than or equal to a threshold value, applying the three-dimensional face model to the avatar to complete the three-dimensional avatar model. Photographing the target person; Detecting facial features of the photographed person; Obtaining a three-dimensional face shape similar to the photographed face; 3. A method of synthesizing a 3D avatar of a mobile terminal, comprising: generating a 3D face model of the target person with reference to the feature of the photographed face and the 3D face type. The method of claim 7, wherein the photographing process Switching the mobile terminal to a shooting mode; Displaying predetermined focusing criteria on the screen; Aiming the face of the target person against the focusing criterion; 3. A method of synthesizing a 3D avatar of a portable terminal, comprising: driving a mounted camera to photograph the target person. The method of claim 8, wherein the predetermined focusing criterion is 3D avatar synthesis method of a mobile terminal, characterized in that the H-shape. The method of claim 7, wherein the detecting of the facial feature comprises Displaying a plurality of key points on the screen; Arranging the keypoints to match the outline of the photographed face; When the arrangement of the key point is completed, a method of synthesizing the three-dimensional avatar of the mobile terminal, characterized in that the process of obtaining the coordinates of the key points. The method of claim 10, wherein the keypoint placement process is performed. When a plurality of keypoints are displayed on the screen, using a predetermined button on the keypad, selecting a desired keypoint; And moving the selected keypoint to coincide with the outline of the photographed face by using a direction key of the keypad. The method of claim 7, wherein the three-dimensional face shape 3D avatar synthesis method of a mobile terminal, characterized in that obtained by linear combination of a number of basic face types. The method of claim 7, further comprising: synthesizing the generated three-dimensional face model and the ear of the photographed face. Loading an image stored in the memory; Selecting a three-dimensional face model similar to the subject person shown in the image; Detecting a keypoint on the face of the target person; Mapping keypoints of the detected face to the three-dimensional face model; 3. A method of synthesizing a 3D avatar of a mobile terminal, comprising: applying the 3D face model to an avatar to complete a 3D avatar model. The method of claim 14, wherein the keypoint detection process is performed. Displaying the image and predetermined focusing criteria on a screen; Moving the focusing criterion to aim the face of the target person; When the aiming is completed, displaying a plurality of key points on the screen; Arranging the keypoints to match the outline of the face; When the arrangement of the key point is completed, a method of synthesizing the three-dimensional avatar of the mobile terminal, characterized in that the process of obtaining the coordinates of the key points. 16. The method of claim 15, wherein the keypoint placement process is When a plurality of keypoints are displayed on the screen, using a predetermined button on the keypad, selecting a desired keypoint; And moving the selected keypoint to coincide with the outline of the photographed face by using a direction key of the keypad. 16. The method of claim 15, wherein the predetermined focusing criteria is 3D avatar synthesis method of a mobile terminal, characterized in that the H-shape. 15. The method of claim 14, further comprising: synthesizing the generated three-dimensional face model and the object of the target person.

Images