KR100700872B1 - Method for displaying 3 dimension private character image of mobile terminal and the mobile terminal thereof - Google Patents

Abstract

A method and a mobile terminal for displaying the facial expression of a three-dimensional personal character are provided to acquire facial form information, select a three-dimensional facial model, map the acquired facial form information to the facial mode, and control an expression control point so that the facial expression of the character can be variously changed. A method for displaying a facial expression of a three-dimensional personal character comprises the following steps. A position matching image for selecting a reference point is overlaid on a two-dimensional facial image(S3). Facial form information is acquired and a three-dimensional facial model approaching the facial form information is selected(S6,S7). A three-dimensional personal character is generated by mapping the facial form information to the three-dimensional facial model(S8). The facial expression of the character is set and displayed by processing an expression control point(S11).

Description

TECHNICAL FOR DISPLAYING 3 DIMENSION PRIVATE CHARACTER IMAGE OF MOBILE TERMINAL AND THE MOBILE TERMINAL THEREOF}

1 is a block diagram of a mobile communication terminal equipped with a three-dimensional personal character expression display function according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a guide line screen for performing photographing using a guideline technique in the mobile communication terminal of FIG. 1. FIG.

FIG. 3 is an exemplary view of a screen for position registration for aiming a face of a person to be photographed in FIG. 2; FIG.

4 is a detailed block diagram of the expression processing unit in FIG. 1;

5 is a diagram illustrating a reference line of a face set to select an expression control point in a 2D face image according to an embodiment of the present invention.

6 is a view showing a muscle vector according to a selected facial expression control point in a two-dimensional face image according to an embodiment of the present invention.

7 is a flowchart illustrating an operation of generating a 3D personal character and generating an expression in the mobile communication terminal according to an embodiment of the present invention shown in FIG. 1.

8 is a flowchart illustrating an operation of generating and displaying an expression on a 3D personal character in a mobile communication terminal according to an embodiment of the present invention shown in FIG. 1.

<Description of the symbols for the main parts of the drawings>

100: mobile communication terminal 110: memory

120: control unit 121: recording processing unit

122: screen display processing unit 123: face information acquisition unit

124: model selection unit 125: character generator

126: facial expression processing unit 126a: facial expression control point selection unit

126b: facial expression selection processor 126c: calculator

126d: facial expression generator 127: download processing unit

130: RF unit 140: voice processing unit

150: speaker 160: microphone

170: key input unit 180: display unit

190 camera

The present invention relates to a three-dimensional personal character service of a mobile communication terminal, and more particularly, to generate a personal character such as a three-dimensional avatar reflecting an individual's face characteristics in a mobile communication terminal and varying facial expressions of the generated personal character. The present invention relates to a 3D personal character facial expression display method of a mobile communication terminal for changing and displaying the same, and a mobile communication terminal for the same.

The avatar represents another self existing in the Internet or a mobile communication environment, which is a virtual space, and is capable of being transformed into any form that can be expressed, such as an animal or a plant, as well as a human form.

The production and use of such an avatar is similar to the user's own appearance, and may represent the user's characteristics, and may include meanings such as user's curiosity, surrogate satisfaction, and the ideal person required by the individual. Accordingly, the user has become very interested in the production and use of avatars representing individuals.

In order to induce various additional services and interests of the mobile terminal, various techniques of such avatars have been introduced.

For example, a three-dimensional avatar capable of giving a three-dimensional appearance as well as a two-dimensional avatar is introduced to a mobile communication terminal, causing user interest.

However, since the 3D avatar used in the conventional mobile communication terminal is uniformly provided by the mobile communication provider or the content provider, it is difficult to provide an avatar that meets various needs of numerous users.

For example, a user wants an avatar that has the same face shape as the user and makes various expressions, but the conventional technology cannot satisfy the needs of such a user.

The technical problem to be achieved by the present invention is to generate a personal character such as a three-dimensional avatar reflecting the personal characteristics of the person in the mobile communication terminal, and to change the facial expression of the generated personal character in various ways.

Another technical problem to be solved by the present invention is to improve the user interface of a mobile communication terminal to be more friendly to a user by applying a facial expression change of a 3D personal character.

According to an aspect of the present invention for achieving the technical problem, the first step of obtaining the face shape information from the two-dimensional face image and selecting the three-dimensional face model close to the acquired face shape information, and the selected three-dimensional The second step of generating the 3D personal character by mapping the acquired face shape information to the face model, and selecting an expression control point from the acquired face shape information and converting the selected expression control point to express the expression of the 3D personal character. It provides a three-dimensional personal character facial expression display method of a mobile communication terminal comprising a third step of setting and displaying on the screen.

Preferably, the 3D personal character facial expression display method of the mobile terminal according to the present invention may further include a photographing step of generating a two-dimensional face image by photographing a human face through a camera before performing the first step.

Preferably, the photographing step may be characterized by overlaying and displaying a guide line screen showing a guide line of a face on an image of a photographing target input through a camera in order to be able to aim the object to be photographed.

Preferably, in the first step, when performing a calculation for acquiring face shape information from a two-dimensional face image, a two-dimensional face for position registration screen for preselecting a reference point for acquiring information on each element of each face is obtained. And overlay the image on the image.

Preferably, the position matching screen is made of sculpture images corresponding to each part of the face having a face shape, and each sculpture image is selected and moved by manipulation of a key input unit.

Preferably, the three-dimensional personal character expression display method of the mobile communication terminal according to the present invention comprises at least one or more three-dimensional face model information to be used to map the face shape information obtained from any server through the network before performing the first step. The method may further include downloading and storing.

Preferably, the third step includes selecting an expression control point from the acquired face shape information, selecting a type of expression to be set for the 3D personal character, and calculating a conversion value of the expression control point for setting the selected type of expression. And applying the converted value of the expression control point calculated to the three-dimensional personal character to display on the screen.

According to another aspect of the present invention, a memory for storing at least one or more three-dimensional face model information to map the acquired face shape information, and obtains the face shape information from the two-dimensional face image and is close to the acquired face shape information Control unit for generating a 3D personal character by mapping the acquired face shape information to the 3D face model, and setting the expression of the 3D personal character through the process of converting the selected facial expression information from the acquired face shape information to the screen. It provides a mobile communication terminal having a three-dimensional personal character expression display function comprising a.

Preferably, the control unit includes a face information acquisition unit for acquiring face shape information from a two-dimensional face image, a model selection unit for selecting a three-dimensional face model in proximity to the face shape information acquired by the face information acquisition unit, and the selected three The character generator generates a 3D personal character by mapping the acquired face shape information to the 3D face model, and selects an expression control point from the acquired face shape information and expresses the expression on the 3D personal character through the conversion process of the selected expression control point. And a screen display processing unit configured to display a 3D personal character having a facial expression set by the facial expression processing unit on the screen.

Preferably, the expression processing unit includes an expression control point selecting unit for selecting an expression control point from the acquired face shape information, an expression selection processing unit for selecting the type of expression to be set for the 3D personal character, and an expression control point for setting the selected type of expression. Provided is a mobile communication terminal equipped with a three-dimensional personal character facial expression display function comprising a calculation unit for calculating a transform value and an expression generator for generating a facial expression by applying the transform value of the facial expression control point calculated to the three-dimensional personal character.

Preferably, the control unit may further include a photographing processor that generates a two-dimensional face image by photographing a human face through a camera.

Preferably, the 2D face image is human face data stored in a memory or downloaded through a network.

Preferably, the control unit further includes a download processing unit for downloading at least one or more three-dimensional face model information to be used to map the face shape information obtained from any server through the network and to store in the memory.

According to another aspect of the present invention, the first step of setting the type of expression to be assigned to the three-dimensional personal character for each service menu, and if the execution of any service menu is requested to set to the three-dimensional character corresponding to the service menu 3D personal character facial expression display of the mobile communication terminal comprising a second step of determining the type of facial expression, and a third step of setting the expression of the determined type to the three-dimensional personal character through the conversion of the expression control point and display on the screen Provide a method.

Preferably, the method for displaying a 3D personal character expression of a mobile communication terminal according to the present invention obtains face shape information from a 2D face image and selects a 3D face model close to the acquired face shape information before performing the first step. The method may further include mapping the acquired face shape information to the selected 3D face model to generate a 3D personal character to be displayed when the service menu is executed.

Preferably, the facial expression control point is at least one vector coordinate selected from a two-dimensional face image to generate an arbitrary facial expression through vector transformation of the facial expression control point.

Preferably, the service menu may include at least one service menu of a photo collage service menu, a phone book service menu, a video call service menu, a sound setting service menu, a color setting service menu, an emoticon service menu, and a customer care service menu.

According to another aspect of the present invention, a memory for storing a three-dimensional personal character to be displayed when the service menu is executed, and a type corresponding to the corresponding service menu through conversion of the expression control point when an execution of an arbitrary service menu is requested. It provides a mobile communication terminal having a three-dimensional personal character facial expression display function comprising a control unit for setting the expression of the expression to the three-dimensional character stored in the memory to display on the screen.

Preferably, the control unit is a setting unit for setting the type of expression to be assigned to the three-dimensional personal character for each service menu, and if the execution of any service menu is requested to determine the type of expression to be set for the three-dimensional character corresponding to the service menu A determination unit, an expression processing unit for setting the type of expression judged on the three-dimensional personal character through transformation of the expression control point, and a screen display processing unit for displaying the 3D personal character having the expression set by the expression processing unit on the screen; do.

More preferably, the control unit includes a face information acquisition unit for acquiring face shape information from a two-dimensional face image, a model selection unit for selecting a three-dimensional face model in proximity to the face shape information acquired by the face information acquisition unit, and the selected three. The apparatus may further include a character generator configured to map the acquired face shape information to the dimensional face model to generate a 3D personal character to be displayed when the service menu is executed.

Preferably, the facial expression control point is at least one vector coordinate selected from a two-dimensional face image to generate an arbitrary facial expression through vector transformation of the facial expression control point.

Preferably, the service menu may include at least one service menu of a photo collage service menu, a phone book service menu, a video call service menu, a sound setting service menu, a color setting service menu, an emoticon service menu, and a customer care service menu.

Hereinafter, a personal character service method and a mobile communication terminal of a mobile communication terminal according to an embodiment of the present invention with reference to the accompanying drawings.

1 is a block diagram illustrating a mobile communication terminal equipped with a 3D personal character expression display function according to an embodiment of the present invention.

Referring to FIG. 1, a mobile communication terminal having a 3D personal character expression display function according to an embodiment of the present invention may include a memory configured to store a program in which a procedure for performing a 3D personal character expression display service is programmed ( 110 and a controller for reading a program stored in the memory 110 to perform a procedure programmed in the program to generate a 3D personal character from the photographed face image, and to vary the facial expression of the generated personal character. 120, an RF unit 130, a voice processing unit 140, a speaker 150, a microphone 160, a key input unit 170, a display unit 180, and a camera 190 It is configured by.

The memory 110 stores an operation program and a system program of the controller 120, and the operation program and the system program are stored in a ROM region which is normally included and can be erased as necessary. Electrically erasable ROMs include EEPROM and flash memory. The memory 110 also includes a RAM that temporarily stores data generated during execution of various operation programs.

In addition, the memory 110 stores a program for generating a 3D personal character and at least one 3D face model information to map the acquired face shape information.

The control unit 120 controls the overall operation of the mobile communication terminal, and performs control of the overall procedure when performing a personal character service in the mobile communication terminal according to the present invention. Then, control of various components provided is achieved.

The controller 120 acquires face shape information from a two-dimensional face image photographed by the camera 190, stored in the memory 110, or downloaded through a network, and is applied to a three-dimensional face model close to the acquired face shape information. The 3D personal character is generated by mapping the acquired face shape information, and the expression of the 3D personal character is set and displayed on the screen through the transformation process of the expression control point selected in the 2D face image.

To this end, the controller 120 downloads the photographing processor 121, the screen display processor 122, the face information acquirer 123, the model selector 124, the character generator 125, and the facial expression processor 126. It is configured to include a processing unit 127.

The photographing processor 121 stores the human face input through the camera 190 as a two-dimensional face image for generating a three-dimensional personal character according to a photographing command through the key input unit 170 and stores it in the memory 110. .

At this time, the photographing processing unit 121 stores the image displayed on the display unit 180 as a two-dimensional face image when the manipulation of the key input unit 170 for commanding photographing in a state of being aimed at the face of the photographing subject by the user is performed.

The screen display processor 121 displays various screens necessary for a series of procedures for performing an expression service of a 3D personal character on the display unit 180.

For example, when the photographing mode is executed by the photographing processing unit 121, as shown in FIG. 2, a guideline of the face is shown to aim the object to be photographed so that photographing by the guideline technique can be performed. The guide line screen is input through the camera 190 to be overlaid and displayed on the image displayed on the display unit 180.

The outline of the face is shown in the guide line displayed on the display unit 180 by the screen display processor 123. That is, the display unit 180 is overlaid with guide lines indicating the areas of the eyebrows, eyes, nose, and mouth while the face image input through the camera 190 is displayed.

Accordingly, the user may adjust the position of the mobile communication terminal to perform focusing on the photographing target so that the corresponding portion of the photographing target is matched to the displayed guide line by being overlaid on the display unit 180.

In addition, the screen display processing unit 122 is photographed by the camera 190 to more accurately obtain the shape information about the face when the face information acquisition unit 123 acquires the face image information from the two-dimensional image memory 110 In the state in which the face image stored in the display on the display unit 180 is displayed by overlaying the screen for position matching.

Here, the position matching screen is a screen in which sculpture images corresponding to each part of the face have a face shape as shown in FIG. 3.

As shown, fragment images of the left eye, the right eye, the left ear, the right ear, the nose, the mouth, and the chin are displayed while the face image is displayed on the display unit 180.

Each of the fragment images may be selected by manipulation of the key input unit 170, and the selected fragment images may be moved up, down, left, and right through manipulation of a direction key or a number key provided in the key input unit 170.

That is, the user manipulates the direction key or the number key of the key input unit 170 while the face image is displayed on the display unit 180 and the fragment images are overlaid thereon. The image is moved to be almost exactly overlaid on the eye, and the fragment image corresponding to the right eye is overlaid on the right eye of the face image.

The user moves the pieces of the left ear, right ear, nose, mouth and chin in the same manner so that the image is overlaid on the corresponding part of the face image and then presses the confirmation key provided in the key input unit 170 to complete the position matching operation. .

The position matching operation corresponds to a task of preselecting a reference point for acquiring information on each element of each face when the face information acquisition unit 123 performs an operation for acquiring face information from the face image.

Therefore, as the position matching operation is performed properly, the possibility of generating a more realistic three-dimensional personal character is increased.

In addition, the screen display processor 122 displays the 3D personal character having the facial expression set by the facial expression processor 126 on the screen.

The face information acquirer 123 acquires various face information from the two-dimensional face image photographed by the photographing processor 121 and stored in the memory 110.

The face information obtaining unit 123 is selected by the position matching operation when the position matching operation is performed by the screen display processing unit 122 on the 2D face image photographed by the photographing processing unit 121 and stored in the memory 110. Face shape information is obtained based on the coordinate positions of the fragment images of each face element.

For example, the face information acquirer 123 obtains face shape information based on coordinate positions of pieces of images of each face element selected by the position matching operation by the screen display processor 122.

For example, the left and right length of the forehead, the left and right length of the chin, the length from the forehead to the chin, the left jaw line, the right jaw line, the width of the face passing through the eyes, the length from the face width to the left jaw line, the length from the face width to the right jaw line Is measured and output as face information on the corresponding two-dimensional face.

Accordingly, the face information acquisition unit 123 stores face shape information acquired based on the fragment image coordinates of each face element in the memory 110.

Here, the face shape information includes color images of each face element.

The model selector 124 selects the 3D face model close to the face shape information acquired by the face information acquirer 123 from among the 3D face models stored in the memory 110.

The model selector 124 is acquired by the face information acquirer 123 and selects the face shape information in the memory 110 in order to select a 3D face model that is close to the face shape information acquired by the face information acquirer 123. Analyze and select the nearest 3D model according to the analyzed result.

That is, the model selecting unit 124 analyzes coordinate values of pieces of images of each face element and, for example, the left and right lengths of the forehead and left and right jaws measured by the face information obtaining unit 123 from the corresponding coordinate values. 3D model stored in the memory 110 using the length, the length from the forehead to the chin, the left jaw line, the right jaw line, the width of the face passing through the eyes, the length from the face width to the left jaw line, and the length from the face width to the right jaw line Select the 3D model with the closest shape among them.

The character generator 125 maps face shape information, such as coordinate positions, for each face element obtained by the face information acquirer 123 to a three-dimensional face model selected by the model selector 124. Create a dimension personal character. At this time, color images for each face element are also mapped.

In more detail, the character generator 125 cuts out the central portion (including the eyes, nose, and mouth) of the face from the two-dimensional face image photographed and stored by the photographing processor 121, and plans planar mapping. According to the mapping), the cut portion is mapped with the three-dimensional face model selected by the model selecting unit 124. The Gouraud shading method is used to smooth the curved surfaces and edges of the mapped 3D face.

In this case, the character generator 125 maps the 2D face image by using mapping information of the 3D face model selected by the model selector 124. The mapping information of the 3D face model includes information on the normal vector of each area by dividing the face of the selected 3D face model into an arbitrary number of regions (for example, 64 regions).

The character generator 125 performs mapping with the 2D face image through linear interpolation on each normal vector included in the mapping information of the 3D face model.

The facial expression processor 126 selects a predetermined number of facial expression control points based on the position coordinates of the pieces of images for each facial element obtained by the position matching operation by the screen display processor 122, and selects the facial expression control points. The conversion process sets the expression for the three-dimensional personal character.

The download processor 127 downloads and stores at least one or more three-dimensional face model information to be used to map the face shape information obtained from an arbitrary server through a network and stores it in the memory 110.

The RF unit 130 is controlled by the controller 120. When the RF unit 130 receives a signal of a frequency received through a radio channel of a frequency band through the antenna ANT, it converts the band down and distinguishes the type of data and outputs it to the control unit 120 or the voice processing unit 140. )

Data output from the RF unit 130 to the control unit 120 becomes paging signals and signaling signals received through text data or a paging channel. The data output to the voice processor 140 becomes voice data received when a voice call is set.

In addition, the RF unit 130 performs band-up conversion to transmit the data received from the control unit 120 or the coded voice data received from the voice processing unit 140, and converts the signal into a radio signal of a frequency band through the antenna ANT. Output

The speech processing unit 140 is generally composed of a VOCODER, etc., and is driven by the control of the controller 120. The voice processor 140 modulates the electrical signal input by the microphone (MIC) 160 to convert the voice signal into voice data and outputs the voice signal to the RF unit 130. In addition, the speech processing unit 140 demodulates the encoded speech data input from the RF unit 130 into an electrical signal and outputs the result to the speaker (SPK) 150, and the speaker (SPK) 150 converts it into an audible sound. Output

The key input unit 170 generally has a key matrix structure. The key input unit 170 includes a plurality of numeric keys, direction keys, and function keys for performing various functions, and outputs an electrical signal of key data to the controller 120 by a user's key input.

The display unit 180 displays the current state of the terminal or changes in the progress of the program. It also displays incoming short messages. Such a display device may be configured as a display device such as a liquid crystal display (LCD).

The camera 190 includes a semiconductor image sensor such as a solid state imaging device (CCD) or a CMOS image sensor (CIS), a component of a digital camera such as a flexible printed circuit board (FPCB), a lens for a camera, at least one flash, and the like, and includes a control unit. An external object is photographed according to the control signal of 120 to acquire image data.

4 is a detailed block diagram of the facial expression processor of FIG. 1.

Referring to FIG. 4, the expression processing unit 126 includes an expression control point selecting unit 126a, an expression selection processing unit 126b, a calculating unit 126c, and an expression generating unit 126d.

The facial expression control point selecting unit 126a selects the facial expression control point from the acquired facial shape information photographed and stored by the photographing processor 121 based on the facial shape information acquired by the facial information obtaining unit 123.

Here, the expression control point is at least one vector coordinate selected from the face image to generate an arbitrary expression through vector transformation of the expression control point. The expression control point refers to a change point of each end point selected corresponding to each element of the face for generating an expression for changing the expression of the face.

Therefore, a muscle vector that can change the facial expression control point is designated corresponding to the facial expression control point. The muscle vector refers to a vector capable of changing the facial expression control point.

The generation of various expressions is made possible by changing the position of the muscle vector of the specified type for each function.

For example, muscle vectors of representative facial expressions of facial expressions such as happy facial expressions, sad facial expressions, anger, vanity, cynicism, and surprise may be specified and used.

The facial expression control point is photographed by the photographing processor 121 and the pieces of images of the elements of each face are arranged and aligned through the screen for the position matching by the screen display processor 122 with respect to the image stored in the memory 110. The face information acquisition unit 123 is selected to reflect the face characteristics of the individual by reflecting the coordinate position of each face element.

That is, as shown in FIG. 5, the eyebrow 1, the eye 2, the nose 3, the nose 4, the mouth 5, and the center of each face element, such as the jaw 6, may be referred to as a reference. Muscle vectors set as shown in 6 can be used. At this time, the muscle vector is designated to have a position, a direction, and a size for generating an arbitrary facial expression based on the center point at six control positions.

Accordingly, the muscle vector (a) having an upward direction is designated to three positions of the eyebrow (1), and by adjusting the size of the muscle vector, it is possible to produce a facial expression change to widen or narrow the forehead. .

Further, a muscle vector b1 having an upward direction and an oblique direction toward the outside of the face is designated for the right eye, and a muscle vector b2 having an upward direction and an oblique direction toward the outside of the face is specified for the left eye. have.

In addition, muscle vectors c1 and c2 having upward and oblique directions toward the outside of the face are designated for the left and right cheeks, and muscle vectors d1 and d2 having the outward direction of the face are specified for the nose. .

For the lip region, muscle vectors (e1, e2) having three directions, symmetrically outward, upward, and downward, are designated. Vector f is specified.

The expression selection processing unit 126b selects the type of expression to be set for the three-dimensional personal character. The types of facial expressions selected by the facial expression selection processing unit 126b include all kinds of facial expressions that can be generated by changing the positions of the muscle vectors of the kinds designated for each function for generating each facial expression.

For example, it can include facial expressions that people can have, such as happy expressions, sad expressions, anger, vanity, cynicism, and surprise.

When a certain facial expression is selected by the facial expression selection processor 126b, a facial expression is generated by reading a designated motion vector to perform facial expression change by muscle vectors for generating the facial expression.

The calculating part 126c calculates the conversion value of the facial expression control point for setting the facial expression of the kind selected by the facial expression selection processing part 125b.

Vectors generally have size and direction. Therefore, to change the motion vector to match the target model, the size and direction must be changed.

The calculating unit 126c calculates the size and direction of the motion vector corresponding to each expression by calculating the normal vector of the three-dimensional face model selected by the model selecting unit 124 and the normal vector of the muscle vectors for each function. You can set facial expressions for individual characters.

Here, the motion vector is a vector representing the transformation of the control point of the gauge from the vector before the facial expression change to the final vector after the facial expression change.

The facial expression generator 126d generates the facial expression by applying the converted value of the facial expression control point calculated to the three-dimensional personal character.

The facial expression generator 126d may generate various facial expressions by selecting the size and direction of the facial expression vector for each facial element set as the type of facial expression to be set for the 3D personal character.

That is, the expression of the ball in the vector size and direction of raising and lowering the position of the eyebrow set to a, raising the left and right eyes of b1 and b2 set to b, raising and lowering the left and right right of the upper ball set to c, and narrowing the spread apart. I can make it.

In addition, in the case of the nose set to d, the nose widening and narrowing may be adjusted to the control points set at the right and left sides, respectively.

In addition, the mouth set to e1, e2 may be selected according to the size and direction of spreading the left and right, spreading up and down the set vector.

FIG. 7 is a flowchart illustrating an operation of generating a 3D personal character and generating an expression in the mobile communication terminal according to an embodiment of the present invention shown in FIG. 1.

Referring to FIG. 7, the download execution unit 126 downloads 3D face model information used to perform 3D face modeling from a corresponding server through a network, and stores the 3D face model information in the memory 110 ( S1).

In the state where the 3D face model information is stored in the memory 110, when the 3D personal character generation menu is executed by the user, the photographing processor 121 drives the camera 190 to execute the photographing mode (S2).

When the capturing mode is executed by the capturing processor 121, the screen display processor 126 may display the face image of the photographing target input through the camera 190 on the display 180 and a guide line screen for aiming the face of the person to be photographed. The image is overlaid on the face image (S3).

Accordingly, the display unit 180 displays a face image input through the camera 190 in a state in which guide lines indicating the areas of the eyebrows, eyes, nose, and mouth are displayed.

Accordingly, the user adjusts the position of the mobile communication terminal to perform focusing on the photographing target so that the corresponding portion of the photographing target matches the guide line for each part of the face displayed on the display unit 180.

When the manipulation of the key input unit 170 for commanding photographing while the face of the subject to be photographed is performed, the photographing processing unit 121 stores the image displayed on the display unit 180 as a 2D face image in the memory 110 ( S4).

The screen display processing unit 122 is a fragment image of each face element along with the 2D face image on the display unit 190 for the position registration necessary to obtain the face shape information from the 2D face image photographed by the photographing processing unit 121. The made position matching screen is displayed on the display unit 180 (S5).

When the screen for processing the position registration is displayed on the display unit 180 by the screen display processing unit 122, the user uses the key input unit 170 to rearrange the fragment images of each face element to match the face contour of the 2D face image. . For example, a fragment image of a face element to be adjusted is selected by pressing a numeric key or a direction key of the key input unit 170, and the selected fragment image is moved to a desired point by pressing a numeric key or a direction key of the key input unit 170.

When the user's confirmation signal is input through the key input unit 170 while the position matching operation is completed, the face information obtaining unit 123 stores face shape information of the 2D face image in the memory 110 (S6).

When face shape information is acquired by the face information acquirer 123 and stored in the memory 110, the model selector 124 may use the face information acquirer 123 among three-dimensional face models stored in the memory 110. A three-dimensional face model close to the face shape information obtained by the step S9 is selected.

When the 3D face model is selected by the model selector 124, the character generator 125 acquires the face obtained by the face information acquirer 123 on the 3D face model selected by the model selector 124. The shape information is mapped to generate a three-dimensional personal character (S8).

When the three-dimensional personal character is generated by the character generator 125, the facial expression control point selector 126a of the facial expression processor 126 selects the facial expression control point (S9).

The expression selection processing unit 126b selects the type of expression to be set for the three-dimensional personal character (S10).

The calculating unit 126c is configured to determine the normal vector of the three-dimensional face model selected by the model selecting unit 124 and the normal vector of the muscle vectors of the specified type to set the expression of the type selected by the facial expression selection processing unit 125b. By calculating the size and direction of the motion vector for the muscle vector corresponding to the facial expression through the calculation to calculate the transformed value of the facial expression control point (S11).

The facial expression selection processing unit 126b determines whether there are more facial expressions to be set (S12), and if there are more facial expressions, the calculation unit 126c calculates a converted value of the facial expression control point for each facial expression and stores them in the memory 110.

FIG. 8 is a flowchart illustrating an operation of generating and displaying an expression on a 3D personal character in a mobile communication terminal according to an embodiment of the present invention shown in FIG. 1.

Referring to FIG. 8, the facial expression selection processor 126b selects a type of facial expression to be set for the 3D personal character to be displayed on the screen as the 3D character service provided by the mobile communication terminal is performed (S21).

When the facial expression is selected by the facial expression selection processor 126b, the facial expression generator 126d reads the converted value of the facial expression control point corresponding to the selected facial expression from the memory 110 (S22) and applies it to the three-dimensional personal character. The expression is generated by changing the position of the muscle vector of the type (S23).

The screen display processing unit 122 displays the three-dimensional personal character having the facial expression generated by the facial expression generating unit 126d on the screen of the display unit 180 (S24).

The present invention has been described with reference to preferred embodiments. However, those skilled in the art will understand that many other embodiments other than those specifically described also fall within the spirit and scope of the invention.

For example, although not shown in the drawings, a mobile communication terminal equipped with a 3D personal character facial expression display function according to another embodiment of the present invention includes a memory for storing a 3D personal character to be displayed when the service menu is executed, and When the execution of the service menu is requested, the control unit may include a control unit for setting a three-dimensional character stored in the memory on the screen by setting the expression corresponding to the service menu through the transformation of the expression control point.

The control unit may be configured to include a setting unit, a determination unit, an expression processing unit, and a screen display processing unit.

The setting unit sets the type of expression to be assigned to the 3D personal character for each service menu.

Here, the service menu is a menu of various services that may be provided or provided in the mobile communication terminal, for example, a photo collage service menu, a phone book service menu, a video call service menu, a sound setting service menu, a color setting service menu, and an emoticon. Service menus, customer care service menus may be included.

The determination unit determines the type of expression to be set for the 3D character according to the service menu when an execution of an arbitrary service menu is requested.

Here, the types of facial expressions include all kinds of facial expressions that can be generated by changing a position of a muscle vector of a type designated for each function for generating each facial expression as described in one embodiment of the present invention.

For example, it can include facial expressions that people can have, such as happy expressions, sad expressions, anger, vanity, cynicism, and surprise.

The facial expression processor sets the facial expression of the type judged on the 3D personal character through the transformation of the facial expression control point. Generating the facial expression through the transformation of the facial expression control point has been described fully in one embodiment of the present invention.

The screen display processing unit displays the 3D personal character having the facial expression set by the expression processing unit on the screen.

The control unit may further include a face information acquisition unit, a model selection unit, and a character generation unit in addition to the setting unit, the determination unit, the expression processing unit, the screen display processing unit.

The face information acquisition unit acquires face shape information from a 2D face image and has been described fully in an embodiment of the present invention.

The model selecting unit selects a 3D face model that is close to the face shape information obtained by the face information obtaining unit, and has been described in detail in one embodiment of the present invention.

The character generator generates a three-dimensional personal character to be displayed when the service menu is executed by mapping the face shape information acquired to the three-dimensional face model selected by the model setting unit, which has been fully described in an embodiment of the present invention.

The mobile communication terminal equipped with the 3D personal character expression display function configured as described above performs a first step of setting the type of expression to be assigned to the 3D personal character for each service menu in order to perform the 3D personal character expression display method.

When the execution of any service menu is requested while the first step is executed, the second step of determining the type of expression to be set for the 3D character according to the corresponding service menu is performed.

When the type of the facial expression to be set is determined by the second step, the third step of setting the facial expression of the determined type to the 3D personal character through the transformation of the facial expression control point is performed.

In this case, before performing the first step, acquiring face shape information from the two-dimensional face image and selecting a three-dimensional face model close to the acquired face shape information, and obtaining the acquired face shape information on the selected three-dimensional face model. Mapping to generate a 3D personal character to be displayed when the service menu is executed.

Here, the facial expression control point is at least one or more vector coordinates selected from the facial shape information to generate an arbitrary facial expression through the vector transformation of the facial expression control point, which has been fully described in an embodiment of the present invention.

As described above, in the mobile communication terminal according to another embodiment of the present invention, various photo collage service menus, phone book service menus, video call service menus, sound setting service menus, color setting service menus, and emoticon services provided by the mobile communication terminals are provided. When the menu and the customer care service menu are executed, the user-friendly 3D personal characters are displayed in conjunction with each other, so that the user can perform the service more interestingly.

According to the present invention, in generating a personal character in a mobile communication terminal, directly photographing an object to obtain face shape information from a captured image, and selecting a three-dimensional face model close to the face information obtained from the acquired face shape information. The 3D personal character is generated by mapping the acquired face shape information to the selected face model, selecting an expression control point from the acquired face shape information, and controlling the selected expression control point to variously change the expression of the 3D personal character. Can be.

Accordingly, it is possible to effectively generate a three-dimensional personal character that reflects personal characteristics well from the two-dimensional face image, and various expressions can be easily set to one generated three-dimensional personal character, thereby providing a variety of fun to the mobile terminal user. can do.

In addition, by applying various expression changes of various three-dimensional personal characters to various service menus displayed on the mobile communication terminal, a user-friendly user interface can be provided.

Claims (22)

Overlaying and displaying a position matching screen on the two-dimensional face image to preselect a reference point for acquiring information on each element of the face from an arbitrary two-dimensional face image; Acquiring face shape information from the two-dimensional face image through the position matching screen and selecting a three-dimensional face model close to the acquired face shape information; Generating a 3D personal character by mapping the acquired face shape information to a selected 3D face model; Selecting a facial expression control point from the acquired face shape information and setting a facial expression of the three-dimensional personal character through a process of converting the selected facial expression point and displaying the expression on the screen; . The method according to claim 1, And a photographing step of generating a two-dimensional face image by photographing a human face through a camera before performing the first step. The method of claim 2, wherein the photographing step, 3. The method of displaying a 3D personal character expression of a mobile communication terminal, wherein a guide line screen showing a guide line of a face is overlaid on an image of a shooting target input through a camera so that the target to be photographed can be aimed at. delete The method of claim 1, wherein the position registration screen, Sculpture images corresponding to each part of the face has a face shape, each sculpture image is selected by the operation of the key input unit is movable, characterized in that the 3D personal character facial expression of the mobile terminal. The method according to claim 1, Downloading and storing at least one or more three-dimensional face model information to be used to map the obtained face shape information from any server through the network before performing the selection step, the three-dimensional personal character of the mobile communication terminal How to show facial expressions. The method of claim 1, wherein the expression setting step, Selecting an expression control point from the obtained face shape information; Selecting a kind of expression to be set to the 3D personal character; Calculating a conversion value of the facial expression control point for setting the facial expression of the selected kind; 3. The method of displaying a 3D personal character expression of a mobile communication terminal, comprising: applying the calculated value of the calculated control point to the 3D personal character and displaying the converted value on the screen. A memory for storing at least one piece of 3D face model information to map the acquired face shape information; A face information acquisition unit for acquiring face shape information from a two-dimensional face image; A model selecting unit which selects a three-dimensional face model proximate to the face shape information obtained by the face information obtaining unit; A character generator for generating a 3D personal character by mapping the acquired face shape information to a selected 3D face model; An expression processing unit which selects an expression control point from the acquired face shape information and sets an expression on the 3D personal character through a conversion process of the selected expression control point; A mobile communication terminal provided with a three-dimensional personal character facial expression display function comprising a screen display processing unit for displaying on the screen the third-dimensional personal character having an expression set by the expression processing unit. delete The method according to claim 8, wherein the facial expression processing unit, An expression control point selecting unit which selects an expression control point from the obtained face shape information; An expression selection processing unit for selecting a type of expression to be set for the three-dimensional personal character; A calculation unit for calculating a conversion value of the facial expression control point for setting the facial expression of the selected kind; And a facial expression generator for generating a facial expression by applying the calculated transform value of the facial expression control point to the three-dimensional personal character. The method according to claim 8, 3. The mobile communication terminal of claim 3, further comprising a photographing processor configured to photograph a human face through a camera to generate the two-dimensional face image. The method of claim 8, wherein the two-dimensional face image, 3. The mobile communication terminal having a 3D personal character expression display function, wherein the face data is stored in the memory or downloaded from a network. The method according to claim 8, The mobile device includes a 3D personal character facial expression display function further comprising a download processor for downloading and storing at least one or more 3D face model information to be used for mapping the acquired face shape information from a server through a network in the memory. Communication terminal. A first step of setting the type of expression to be assigned to the three-dimensional personal character for each service menu; A second step of determining a type of expression to be set for the 3D character according to the service menu when an execution of an arbitrary service menu is requested; 3. The method of displaying a 3D personal character facial expression of a mobile communication terminal, comprising the step of setting the determined type of expression on the 3D personal character through a transformation of an expression control point and displaying the expression on the screen. The method according to claim 14, Acquiring face shape information from a two-dimensional face image and selecting a three-dimensional face model proximate to the acquired face shape information before performing the first step; And mapping the acquired face shape information to a selected three-dimensional face model to generate a three-dimensional personal character to be displayed when the service menu is executed. The method of claim 15, wherein the facial expression control point, And at least one or more vector coordinates selected from the two-dimensional face image to generate an arbitrary expression through vector transformation of the expression control point. The method of claim 14, wherein the service menu, 3D personal character of a mobile communication terminal including at least one service menu of a photo collage service menu, phone book service menu, video call service menu, sound setting service menu, color setting service menu, emoticon service menu, customer care service menu How to show facial expressions. A memory for storing a three-dimensional personal character to be displayed when the service menu is executed; A three-dimensional control unit including a control unit for setting a three-dimensional character stored in the memory on the screen by setting a facial expression corresponding to the corresponding service menu when the execution of an arbitrary service menu is requested; A mobile communication terminal equipped with a personal character expression display function. The method of claim 18, wherein the control unit, A setting unit for setting the type of expression to be assigned to the three-dimensional personal character for each service menu; Determination unit for determining the type of expression to be set to the three-dimensional character corresponding to the service menu when the execution request for any service menu, An expression processor configured to set the determined type of expression on the three-dimensional personal character by converting an expression control point; A mobile communication terminal provided with a three-dimensional personal character facial expression display function comprising a screen display processing unit for displaying on the screen the third-dimensional personal character having an expression set by the expression processing unit. The method of claim 19, wherein the control unit, A face information acquisition unit for acquiring face shape information from a two-dimensional face image; A model selecting unit which selects a three-dimensional face model proximate to the face shape information obtained by the face information obtaining unit; Mobile communication with a three-dimensional personal character expression display function further comprises a character generator for generating a three-dimensional personal character to be displayed at the time of execution of the service menu by mapping the acquired face shape information to a selected three-dimensional face model terminal. The method of claim 20, wherein the facial expression control point, And at least one or more vector coordinates selected from the two-dimensional face image to generate an arbitrary expression through vector transformation of the facial expression control point. The method of claim 18, wherein the service menu, 3D personal character expression display function including at least one service menu of photo collage service menu, phone book service menu, video call service menu, sound setting service menu, color setting service menu, emoticon service menu, customer care service menu. Mobile communication terminal provided.

Images