KR20030091306A - The dynamic character and image character making system and the method using face components relationship on face image. - Google Patents

Abstract

PURPOSE: A system and method for producing a character and an emotion-expressing dynamic character using correlation of contours of portions of a face image are provided to generate a character image based on shape information of an input face image. CONSTITUTION: A face image is transmitted from a character producing system and corrected(P10). Positions and regions of portions of a face are selected according to reference points(P20). Basic contours of the portions of the face are determined(P30). Three-dimensional effect of the face is processed and design data of a portion to which a design is applied is corrected using correlation between the determined contours and the portions of the face(P40). The contours are gradually changed to generate new contours(P50). A make-up portion of the face is decided and colors are applied to the make-up portion(P70). Moving portions of the face are scheduled and a dynamic image is gradually generated(P60). A costume is applied to a generated character face(P80). The picture quality of the generated character image is improved(P90).

Description

A character and emotion expression dynamic character production apparatus using the interrelation of the outer curve of each part in the face image and its production method. {The dynamic character and image character making system and the method using face components relationship on face image.}

The present invention relates to an apparatus and method for generating a character composite image based on the shape information of a face image input as a character production system, generating a composite image such as molding and makeup, and generating a dynamic character image representing an emotional expression. .

The face of a person is a representative character of the person. Especially in the modern society where non-face-to-face communication is common on the Internet, such as the Internet, face expression methods such as caricatures and avatars can be replaced with related industries. Together, its importance is increasing. In addition, his own means of expression are required to express only his characteristics rather than direct methods. In order to replace yourself, you have to resemble your face, and when you want to shape a part of your face composite image, you must be able to freely shape it, and depending on the purpose, you should be able to exaggerate or reduce it. In addition, if a natural dynamic expression change according to emotion is possible, it will be used in various fields.

According to this demand, various character representations are being serviced. Typical methods for creating a character include a method in which a designer draws a character by looking at a picture of a user or a user, selects a character that a user wants from a given character, and constructs a character by combining each item stored in a database. The most similar design application method is known in the database by extracting feature points for each part from face image data. However, it is mainly aimed at synthesizing face images.

Designers can produce characters that are close to the user's appearance, but they are expensive, take a long time, are inadequate for real-time online service, and facial motion effects are impossible. It is also difficult to deal with the user's modification requirements immediately, which is expensive and takes a long time.

The method of letting the user select from among the provided characters is to select from a limited number of characters, so that the face has the result of that face and loses its ugly self. In addition, a large storage device is essential to provide various character shapes. Facial motion effects are also limited. The existing character's movement shows disconnected movements (ie, eyes blinking continuously and completely closed images). Since the combined character providing method is selectively combined by placing them in a database for each face part, an imbalance between parts may occur and results are different from those of a face. The method using similarity results in similarity detection in a limited database, and as a result, the similarity decreases, and feature point extraction by image processing requires strict constraints (lighting conditions, background conditions, camera conditions, etc.) in the image acquisition stage. There is a limit to feature point extraction. There is a problem that the state of the image is changed according to the illumination for image acquisition or various environmental changes, and an appropriate method for supplementing the image is not provided. In addition, to increase the similarity, a large database due to the use of a plurality of reference images is required. In addition, the above methods are limited to generating natural facial expression gestures in common.

The present invention was created in order to solve and supplement the above problems, and produced by simple positioning and movement only if the outline of each part of the face can be visually identified by the face of the front face without any special limitation condition of the input face image. The user can obtain more natural and sophisticated character image, and can freely shape and make up according to the user's intention, and use the contour curves for each part of the face to automatically generate the natural emotional expression of the face The present invention provides an apparatus for producing a character, an emotion expression character, and a method thereof and a service method using the same.

1 is a block diagram showing a functional configuration of an embodiment of a character production service using a communication network according to the present invention.

2 is a block diagram showing a functional configuration of an embodiment of a self character production service according to the present invention.

3 is a block diagram illustrating an apparatus configuration of a computer system in which the embodiments of FIGS. 1 and 2 of the present invention are implemented.

4 is an internal configuration diagram of a character production system according to the present invention.

5 is a basic flowchart of a character generation process of the character production system according to the present invention.

6 is a flowchart illustrating a face image input and correction step of FIG. 4 and a location and area setting step for each part.

FIG. 7 is a flowchart illustrating a step of determining an outline curve for each part of FIG. 4.

FIG. 8 is a flowchart illustrating a three-dimensional and design application step of FIG. 4.

9 is a flow chart showing the forming step of FIG.

FIG. 10 is a flowchart illustrating a facial motion expression step of FIG. 4.

11 is a flowchart illustrating a makeup step of FIG. 4.

12 is a flowchart illustrating a clothes application step of FIG. 4.

FIG. 13 is a flowchart illustrating steps of improving image quality and adjusting image size of FIG. 4.

14 is a face image and a caricature image.

15 is an example of a blinking caricature generated image.

16 is an example of an open mouth caricature type image.

The present invention was created to solve the above problems and to enable new functions. The present invention can be easily configured with the outer curves of the face components in the actual face image, and utilizes the three-dimensional effect and the design for each part of the face by using the user's face. It is a dynamic character automatic production system that can express natural emotions by automatically creating a realistic character by automatically applying molding and makeup according to the intention, and automatically generating a dynamic facial composite image according to the change of facial expression. It can be optionally supported as an avatar type.

The character production service system can be divided into a character production service system through a communication network and its own character production service system capable of producing itself.

1 is a block diagram showing a practical example of a character production service system through a communication network according to the present invention.

As shown, the character production system according to the present invention is configured as a connection of the communication network 20 between the character server 30 and the user PC 10, to be connected to the plurality of user PCs 10 through the communication network (20). Can be. The communication network 20 is representative of the Internet, and various types of LAN, WAN, PSTN (Public Service Telephone Network), PSDN (Public Service Data Network), cable TV network, and wireless communication network.

The user PC 10 includes an image acquisition device 11 capable of acquiring a face image of a digital camera, a scanner, an input device 13 for inputting points or characters such as a mouse or a keyboard, and an image display or print. An output device 12 such as a monitor or a printer for printing, and a communication processing device 14 such as a LAN card and a modem for communication through the communication network 20.

The character server 30 includes a character production system 32 for performing character production, a user information database D1 for user management, and a user character database D2 having a database of information used for user character construction. A design database (D3) where design information is stored;

And a communication processing device 31 for controlling communication with a plurality of user terminals 3 connected to the communication network 20.

In the user information database (D1), information for user management, including user registration information, is formed into a database, and in the user character database (D2), the user's character-related products include outline curve information, character images, and motion for each face part. Images are databased, and the design database (D3) contains design data used for character generation, and design data applied to the design of hair, eyes, and ears used in the character's face and character costume.

After the user registers and logs on to the character server, the user generates an image of his face using the image acquisition device 11 and transmits it to the character server 30 through the communication network 20, and the character production system 32. According to the character production step (FIG. 4), the character is generated, and the character image is stored in the user character database D2 to be used.

The character production system of FIG. 2 refers to a system capable of various services such as business card production or advertisement production using a character by generating a character in the character production system itself and serving the user without using a communication network.

The user PC, the character production system, etc. are made of a computer, the configuration of which is as shown in FIG.

3 is a computer comprising a central processing unit (C2) in charge of all operations and processing, a storage device (C1) storing data, and input / output devices (C3, C4) for inputting and outputting data.

The present invention is obtained as a result of the execution of the character production system 32 under such an environment, and solves the above-mentioned problems and produces a high quality, highly realistic dynamic character.

The quality of the service provided varies depending on how realistically the character can express the shape of the face, how he or she can modify it according to the user's intention, and additionally, how naturally the various facial expressions can be expressed. In addition, various services that could not be provided by the existing method are possible.

The present invention allows the user to easily create the outer curves of the face, eyes, nose, mouth, face shape, etc. in his face, and can use them as they are to create a realistic face shape by applying the facial three-dimensional and design elements Make sure The present invention utilizes the interrelationship of facial components based on this information using the outline curve information as it is, instead of selecting from the database through the similarity of shapes. In other words, it analyzes the position, shape, size, etc. of eyes, nose, mouth, ear, and face shape to apply the three-dimensional effect and design of the face to compose the character and change according to the mutual movement of each facial component according to the expression change. By analyzing and reconstructing the elements, a natural expression is created. Such a form may be provided in two forms, a caricature type and an avatar type, according to a user's selection. The motion elements in the face generate natural changes by creating changes in facial stereoscopic motion associated with each motion, as well as up / down eye couple movements, eye movements, lip movements, and jaw movements. In addition, various types of facial three-dimensional effects are provided for selection so that characters of various moods can be generated from the same facial component.

In the face shaping, the shaping of the intended part of the current face is shown step by step, instead of being converted into a completely different face, so that the user can determine the shaping shape. In this way, eyelids, eye zooming, nose raising, and jaw line shaping are possible.

Functions of the present invention are made in the character production system and its internal configuration is shown in Figure 4, the image input and correction step (P10), the location and region selection step (P20) for each part, the outline curve determination step (P30), face It consists of a three-dimensional and design application step (P40), makeup step (P50), face shaping step (P60), motion application step (P70), costume application step (P80), image quality improvement and image size adjustment step (P90).

5 is a basic flowchart of a character generation process. The detailed technical details for each step are as follows.

6 is a flowchart illustrating a position and region selection for each part in image input.

The face image input and correction step (P10) of the character production system is performed by the user using his / her PC 10 to display his / her front face image using an image acquisition device 11 such as a scanner or a digital camera. After generating 14), the image file is transmitted through the communication network 20 and transmitted to the character server by selecting the design type, the caricature type, and the avatar type of the hair, ears, and eyes (P101). (* .Bmp, * .dib, * .gif, * .jpg ...) The character server sends the transferred face image to the commercial PC and the user enters the center of the eyes. (P102) the face image is aligned in size, horizontally and vertically using the distance and tilt between two center points to correct the tilt of the face image and the size of the face image (P103).

In the step of selecting the location and area of each face, the user provides the user with reference points for the head end point, nose end point, mouth center point, chin point point, and upper and lower points of both ears on the basis of the distance between the eyes on the image corrected in step 1 of FIG. 4. (P104) and provide a correction function for the user to correct (P105) and to determine the final reference points. The area for each part is provided to the user based on the selected reference points, the user can correct (P106), and the final reference points and the area for the last part can be stored (P107).

7 is a flowchart showing an outline curve determination step. In the outer curve step, basic contour curves for each part are provided by transforming the data, which are already determined, with a proportional size considering the position of the reference point and the center of the eye center point (P301). The user may determine correct information of the shape of the outline curve by correcting control points constituting the outline curve of each region (P302) to match the outline shape of each region of the face image. The reference point and the area determined in steps 1 and 2 of FIG. 4 are used. The outline for determining the face shape is made by using the outline curve connecting the center end point of the head with the top and bottom reference points of the both ears and the end point of the jaw. The eyebrow curve is the centerline of the eyebrow shape, the eye shape is the curve forming the upper and lower eyelids, and the outer curve of the nose is the curve forming the outer curve of the nose end point and the nose hole and the outer curve of the left and right nose wings. The outer curve of the mouth is the outer curve of the upper lip, the outer curve of the lower lip, and the centerline curve of the mouth. The outer curve minimizes the number of curve control points to achieve the shape of each part. The modification process is finalized so that the user can save the file with the design information already selected (P303).

8 is a flowchart illustrating a facial three-dimensional effect and design application step. The facial three-dimensional effect and design application step generate a three-dimensional effect of the face by using the interrelationship of the face parts based on the outer curves of the facial parts stored in step 3 of FIG. 4. This is done by virtually setting the position of the light, for example, by referring to the up and down curves of the nose, eyebrows, and eyelids to create areas of light and dark areas and coloring them with the light and dark colors of the face. It is a method of generating artificially creating a three-dimensional effect by bending (P402). In addition, design data such as hair, ears, eyebrows, eyelashes, and pupils provided in the design database selected by the user are naturally converted by converting the position, direction, and size through the transducer (P404) according to the outline curve and reference position for each face part. Allow the shape to be applied (P405). The same applies when face accessories, such as glasses, earrings, and hats, are selected. If the three-dimensional and design application is confirmed in this step, the three-dimensional application information and the application design information is stored (P406) (Fig. 14).

11 is a flowchart showing a makeup step. The face makeup step is a step to be applied to make up according to the user's intention, so that the makeup area for each facial part can be generated by the interrelationship between parts (P603), and the user can make up by selecting a desired color (P604). . The type of makeup may be selected such as eye shadow, foundation, lipstick, and a face background (P601), and the selected color is applied to the makeup area (P605) by the interrelationship for each region in the same manner as the three-dimensional application. In the case of eyelashes, it selects the configuration data already stored according to the type of eyelashes and applies them according to the shape of the upper curve of the upper eyelid. The selected information in this step is changed or added to a file and stored (P606).

9 is a flowchart showing a molding step. The face shaping step is a step in which the user selects a part to be repaired (P501) from the synthetic face data generated through steps 3, 4, and 5 of FIG. (P503) and the user decides the final shaping shape. At this time, the three-dimensional appearance and the design already applied are automatically reconfigured (P504) according to the shape of the contour curve for each part where the shaping is confirmed. Information changed in this step is stored in a file (P505).

10 is a flowchart showing a motion expression applying step. The motion expression applying step is a step of applying the motion for each facial region using the face synthesis data determined in one of steps 4, 5, and 6 of FIG. 4 while deforming the outer curve according to the characteristics of each motion. To be able. Looking at the types of movements of the face parts, eyebrow movement, eye movement, nose movement, lip movement, chin movement, etc. can be mentioned. The eyebrow movements include the movement of the eyebrows, the movement of the eyebrows, the movement of the eyebrows, the movement of the eyebrows, and the movement of the eyebrows.The eyelids blink, the eyelids blink like a surprised eye, the eyelids move when laughing, and the eyelids when crying Eye movements include the movement of the pupils up, down, left, and right. (FIG. 15) The lip movement corresponds to various types of mouth opening operation. The jaw motion correlates with the open mouth motion to achieve relative movement according to the open mouth. Each movement involves related movements according to its operation state. In the case of mouth opening, components such as teeth and tongue inside the mouth appear. This is a shape of a mouth which is already stored and its size and position are modified and applied. (Fig. 16)

Each action can be created step by step by the user's action schedule. For example, the blinking of the eye is composed by transforming the outer curve of the outer curve forming the upper eyelids step by step, and creating motions by applying the eye-related parts such as eyelashes and pupil design accordingly. Serve The movements of the lips, eyebrows, and jaws are also generated in the same way. The user may select an operation to apply above, and the movement for each part may be scheduled by the user (P701) and may modify the operation time interval. When the operation schedule is determined, the three-dimensional effect and the design of the parts that correlate with the operation of each time are changed to generate motion images (P703 and P704), and the final selection is determined and stored (P705) to be reproduced in continuous operation. The unit continuous operation can be repeatedly played. In this step, the information generated for the continuous operation is stored in a file (P706).

13 is a flowchart illustrating steps of improving image quality and adjusting image size. In the image quality improvement and the image size adjustment step, since the image image generated in each step of FIG. 4 utilizes an outline curve for each face part, an image quality is degraded due to an aliasing phenomenon. In particular, when the size of the image is small, it is necessary to compensate for the deterioration of the image. In order to compensate for this, in the present invention, the blurring process (P903) of the images generated at each step is generated as a small image, thereby compensating for the quality deterioration due to the freezing phenomenon and the size reduction (P904).

Finally, the design data applied to the design for each face part is stored in the design database D3, and the files used for generation in the above steps are stored in the user character database D2 (P905).

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited to the drawings.

The present invention not only synthesizes a face image similar to a real user's face, but also freely molds and deforms, and automatically generates a synthesized image according to a change in facial expression, thereby providing a higher quality, higher reality, and motion application. It will produce the result of giving vitality, and can be widely used for online and offline services, and it can be used for various businesses related to character business.

In addition, it is different from the one-off uniform character that ignores the user's characteristics, and a large amount of database for face composition is unnecessary.

The automatic generation of facial motion, which is the strength of the present invention, is also a feature of the present invention that is difficult to generate by the conventional method. Various emotion expressions can be added, so that motions similar to actual motions are provided.

In addition, by giving basic depth information to the outer curve of the present invention can be used to generate a three-dimensional character.

The field using the present invention can be usefully used in various and wide fields such as caricature and avatar generation, business card production, various advertisement production, internet games, computer wallpaper service, e-mail application, mobile service, fancy industry, etc. have.

Claims (12)

An image input and correction step of receiving a face image from the character production system and correcting it using a reference point; A location and region selection step for each facial region according to the determined reference point, The outline curve determination step of presenting and modifying the basic outline curve for each face part, A three-dimensional effect and design application step of modifying and applying the three-dimensional treatment of the face and the design data of the design application area by using the determined boundary curve information and the interrelationship between the face parts; Forming step of transforming and shaping the outer curve gradually according to the user's intention from the outer curve to create a new outer curve and A make-up step of determining a makeup part by mutual relationship between parts and applying a tone, A facial motion expression step of generating a motion image by time-scheduling a part that operates for each facial part for expression of emotion by time zone; The step of applying the costume to the created face character face A method and apparatus for manufacturing a character production system comprising a quality improvement and an image size adjustment step of improving the image quality of the character images generated in each of the above steps. The method and apparatus for providing a service to a multiparty user through a communication network using a character production system. The method and apparatus of claim 1, wherein the service is provided to commercial users as a standalone system using a character production system. The method of claim 1, wherein in the image input and correction step and the location and area selection step for each part, Receiving a center point of both eyes from the user, A method of correcting image size / horizontal / vertical according to the distance and tilt of both eyes based on the input eyes; Based on the distance between the eyes, the nose point, the lip center point, the chin point, the center point of the head, the upper and lower points of both ears are provided and the method can be corrected. A method of providing and modifying the area of each site based on established reference points. The method of claim 1, wherein A method of modifying and providing basic outline curve information through a modifier using the reference point and the reference region; Modifying the outer curved components for each part to match each part of the face image; Storing the modified result. According to claim 1, In the three-dimensional and design application step By virtually setting the position of the light, the area of the light and dark areas is created by referring to the up / down curve information of the nose, eyebrows, and eyelids, and the light and dark areas of the face are colored according to the curvature of the face. How to artificially create a three-dimensional effect, Design data such as hair, ears, eyebrows, eyelashes, and pupils provided in the design database selected by the user are converted to position, direction, and size according to the outer curve and reference position of each face part through a transducer to naturally apply a design form. How to, The same method is applied to eyeglasses, earrings, hats and the like, which are accessories related to the face. How to save this result. The method of claim 1, wherein in the forming step A method of selecting the parts to be repaired by the user and forming them step by step with a proportional difference from the molding reference data for each part; The three-dimensional effect and design of the parts other than the molding are automatically reconstructed according to the shape of the contour curve for each part determined by molding. How to save the result. The method of claim 1, wherein in the makeup step A method of generating the makeup area by mutual relations for each part and selecting a desired color for makeup; A method of changing other parts to be changed according to the makeup part, How to save the result. The method of claim 1, wherein A method of modifying an outer curve according to the characteristics of each part of the motion; A method of generating each of the operations step by step by setting a user's operation schedule; Selecting an operation to be applied by the user and scheduling and modifying an operation time interval by the user for movement of each part; A method of generating motion images, determining a final selection, and storing the same by reproducing the continuous motion by changing the three-dimensionality and the design of the parts correlated with the motions of each time; How to save the result. According to claim 1, wherein in the step of applying the costume How to choose a costume design, convert it through a translator, configure it in the character's proper position, and save the result. 11. The method of claim 6, 7, 8, 9 and 10, the method of storing the results of the image blurring (Blurring) generated in each step to improve the image quality and the image quality of the image adjustment step (Blurring) How to handle, The result is a method that saves the resized image. The method of claim 1, further comprising: storing contour curve information, character images, and dynamic character files for each of the face parts of the user in the user character database; In addition, the design database is used to create a face without designing the design of all the face parts of the character in the database, the method of database only the hair, eyes, ears and accessories related to the face, A method of database only clothing design data used for the application of the clothes.