WO2008078897A1

WO2008078897A1 - Face modification of game character in game system

Info

Publication number: WO2008078897A1
Application number: PCT/KR2007/006611
Authority: WO
Inventors: Jai Hie Kim; Sang Youn Lee; Kang Ryoung Park; Jong Sun Kim
Original assignee: Industry-Academic Cooperation Foundation, Yonsei University
Priority date: 2006-12-26
Filing date: 2007-12-18
Publication date: 2008-07-03
Also published as: KR20080059865A; KR100859315B1

Abstract

Provided is a method of modifying a face of a game character in a game system. The method includes extracting a facial region and facial features from a selected facial image, and directly mapping extracted 2D facial data (and facial features) onto a 2D face of a game character selected by a game user, mapping the generated 2D caricature onto the 2D face of the game character selected by the game user, mapping the generated 3D face onto the 3D face of the game character selected by the game user, or texture mapping the generated 2D face onto the 3D face of the game character selected by the game user, to modify a 2D or 3D face of the game character selected by the game user. Accordingly, the level of user interest in a game can be increased, particularly through immersing the game user in the game by providing an illusion that it is the user moving within a virtual space defined by the game.

Description

FACE MODIFICATION OF GAME CHARACTER IN GAME SYSTEM

TECHNICAL FIELD

The present invention relates to a game system, and more particularly, to a face modification of a game character in a game system.

BACKGROUND ART

Typically, for game systems, which allow a wide range of game genres to be enjoyed, including online games, video games, arcade games, 1st or 3rd person shooting games, racing games, strategic simulation games, and dancing games, the documents described below representatively disclose research that has been conducted for allowing game users in a game system to detect faces that can be applied to creating 2D or 3D characters, extract facial features, make 2D characters, and generate 3D characters from 2D facial data.

First, a face detection method is disclosed by Rein- Lien Hsu et al. in (Rein-Lien Hsu, Abdel-Mottaleb, M. Jain, A. K., "Face detection in color images", Pattern Analysis and Machine Intelligence, IEEE Transactions on Volume 24, Issue 5, Page (s): 696 - 706, 2002), which is hereby incorporated in its entirety by reference. The method uses colors in the facial region for detection. Because in related art red- green-blue (RGB) color charts, the facial region is difficult to define with skin color distribution, the proposed method converts data into alternate color space grouping formats such as the format with luminance, blue chroma, and red chroma components (YCbCr) for detection. However, it is difficult to detect intricate facial regions. Second, a method for extracting facial features is proposed by Rikert, T. D. in (Rikert, T. D., Jones, M. J., Viola, P., "A cluster-based statistical model for object detection", Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on Volume 2, Page (s): 1046 - 1053, 1999) , which is hereby incorporated in its entirety by reference. This method generates and extracts facial features using a Gaussian combining method to combine multi-scale and multi-resolution filters, based on statistical data. Because this method is inclined to rely heavily on knowledge-based images, its processing time is extensive and its algorithms are complex.

Third, a method for 2D characterization is proposed by J. Lui in (J. Liu, Y. Chen, W. Gao, R. Fu, and R. Zhou, "Creative cartoon face synthesis system for mobile entertainment," LNCS 3768, Page (s) : 1027-1038, 2005), which is hereby incorporated in its entirety by reference. However, because this method only represents caricature images as simple binary images with respect to real pictures, user interest can diminish.

Fourth, a method of generating a 3D face from 2D facial data is proposed by V. Blanz and T. Vetter in (V. Blanz and T. Vetter, "A Morphable model for the synthesis of 3D faces," Computer Graphics Proc. SIGGRAPH, Page (s) : 187-194, 1999), which is hereby incorporated in its entirety by reference. In this method, when 2D facial data is input, a 3D morphable model is used to generate a 3D facial image. Because the positions of the initial characterizing points of a facial image cannot be determined during the face detecting stage, a separate manually performed phase is required. Thus, much time is expended for fitting 2D images to 3D facial images, which presents obvious drawbacks for applications in realtime gεime systems.

On one hand, a game user, in a related art game system that employs the above-described face detecting methods, face feature extracting methods, 2D characterizing methods, and 3D face generating from 2D facial data methods, may select a game character that resembles the user's own unique 2D or 3D game character from a selection of game characters with distinguishing traits offered by a game manufacturer.

In this case, because the number of game characters selectable by a game user is inherently limited, users can easily lose interest and be less involved in the game, leading to reduction of gaming hour, and ultimately, to declining profits from the game system for the game manufacturer.

DISCLOSURE OF THE INVENTION TECHNICAL PROBLEM Embodiments provide An object of the present invention is to provide a method of 2D or 3D face modification of a game character in a game system by extracting a facial region and facial features from a selected image of a face obtained through a camera-captured photograph, etc., and either directly mapping the extracted 2D face (and facial features) data onto a 2D face of a game character, mapping the face of a game character after performing added characterizing, or generating 3D facial data from the extracted 2D face (and facial feature) data and mapping the same onto a 3D game character face. TECHNICAL SOLUTION

In a first embodiment of the present invention, there is provided a method of modifying a face of a game character in a game system, including: extracting a facial region and facial features from a selected facial image; and modifying a 2D face of a game character selected by a game user through directly mapping extracted 2D facial data and facial feature data onto the 2D face of the game character selected by the game user.

In a second embodiment of the present invention, there is provided a method of modifying a face of a game character in a game system, including: pre-storing facial feature data of game characters in a DB (database) ; and generating the 2D caricature through matching and integrating the 2D facial features extracted from the selected facial image with the pre-stored facial feature data of the game characters, based on similarities and shape data.

In a third embodiment of the present invention, there is provided a method of modifying a face of a game character in a game system, including: extracting a facial region and facial features from a selected facial image; generating a 3D face from 2D facial data and facial feature data; and modifying a 3D face of a game character selected by a game user through mapping the generated 3D face onto the 3D face of the game character selected by the game user.

In a fourth embodiment of the present invention, there is provided a method of modifying a face of a game character in a game system, including: extracting a facial region and facial features from a selected facial image; generating a 2D face corresponding to an average 3D model from extracted 2D facial data and facial feature data; and modifying a 3D face of a game character selected by a game user through texture mapping the generated 2D face onto the 3D face of the game character selected by the game user.

The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.

ADVANTAGEOUS EFFECTS

As described above, the present invention modifies the face of a game character based on user-selected facial images (for example, an image of a game user's own face or an image of another person's face). Thus, the level of user interest in a game can be increased, particularly through immersing a game user in a game by providing an illusion that it is the user moving within a virtual space defined by the game. Accordingly, an increase in the cumulative number of user gaming hours will translate into increased revenue from a game system.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a flowchart of a method for modifying a face of a game character in a game system according to the present invention.

Fig. 2 is a diagram showing a process of applying a method of modifying a face of a game character according to an embodiment of the present invention to a game system. Fig. 3 is a diagram showing a process of characterizing a face of a game character with a 2D facial image according to a first embodiment of the present invention, based on a method of modifying a face of a game character in a game system according to the present invention. Fig. 4 is a diagram showing a process of characterizing a face of a game character with a 2D facial image according to a second embodiment of the present invention, based on a method of modifying a face of a game character in a game system according to the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Below, the first, second, third, and fourth embodiments of the present invention will be described in detail, with reference to the drawings. Fig. 1 is a flowchart of a method for modifying a face of a game character in a game system according to the present invention, and operations SlO, S20, and S30 in the first, second, third, and fourth embodiments are performed in the same manner in each of the four embodiments. Fig. 2 is a diagram showing a process of applying a method of modifying a face of a game character according to an embodiment of the present invention to a game system, and shows the processes of obtaining and selecting facial images

(i.e. - an image of a game user's own face or that of another person's face) through a camera-captured photograph, etc., and using the same to modify a 2D or 3D face of a game character used in a game system.

Referring to Fig. 1, in all of the first, second, third, and fourth embodiments of the present invention, the foremost processes are to obtain (in operation SlO) from a camera- captured photograph, etc. and select a facial image for modifying the face of a game character, and extract a facial region and facial features from the selected facial image in operations S20 and S30. Here, the camera used may be a typical charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) personal computer (PC) camera, a surveillance-oriented closed circuit television (CCTV) camera, a digital camera, or a mobile phone camera. Also, with quick and accurate face detection by generating templates through training using brightness contrast by region for a face, an AdaBoost Face Detection method, a Principle Component Analysis (PCA) method, an Independent Component Analysis (ICA) method, etc. may be used to detect a facial region from the facial image, and lighting normalization for face detection (such as homomorphic filtering, etc.) may be performed to increase the accuracy rate of facial region detection.

A training-based facial feature extraction method, using an Active Appearance Model (AAM) extracting facial features in a detected facial region quickly and precisely, may extract features including all or a portion of both eye ends, both eyes, nose, mouth line, facial contour, and hairline. As described above, with a facial region and facial features extracted from a selected facial image obtained through a camera-captured photograph, etc., the method of modifying the face of a game character in respective game systems according to the first, second, third, and fourth embodiments, will be described in detail. First, in the first embodiment of the present invention, after a facial region and facial features are extracted from a selected facial image obtained from a camera-captured photograph, etc., in operations SlO to S30, and in operations S40 and S42, 2D facial data and facial features extracted from the selected facial image for game character modification are directly mapped onto a 2D face of a game character selected by a game user, and the 2D face of the game character selected by the game user is modified. Second, in the second embodiment of the present invention, after a facial image is obtained through a camera- captured photo, etc. and the facial region and the facial features are extracted from the selected facial image in operations SlO to S30 as above, in order to modify a game character, 2D facial data and facial feature data extracted from the selected facial image are characterized to generate a 2D caricature in operation S50.

Here, as shown in Fig. 3, a game character designer may store the facial features of the game character in a database (DB) beforehand, and perform matching and integration of the 2D facial features extracted from the selected facial image (for modifying the game character) with pre-stored facial feature data of game characters, on the basis of similarities (e.g., Euclidean distances, etc.), to generate a 2D caricature.

As shown in Fig. 4, a game character designer may store the facial features of the game character in a database (DB) beforehand. After converting 2D facial feature data of a game user's face extracted from the selected facial image (for modifying the game character) to facial feature regions, a 2D caricature may be generated by matching and integrating the conversion coefficients of the facial feature regions in terms of similarity (e.g. Euclidean distances, etc.) with pre-stored facial feature data of game characters. In particular, a Principal Component Analysis (PCA) based facial feature region modification method, an Independent Component Analysis (ICA) based facial feature region modification method, and a Local Feature Analysis (LFA) based facial feature region modification method may be used to extract the facial feature regions.

Next, the generated 2D caricature is mapped onto a 2D face of a certain game character selected by the game user, and the 2D face of the game character selected by the game user is modified, in operations S52 and S54. Third, in the third embodiment of the present invention, after a facial region and facial features are extracted from a selected facial image obtained from a camera-captured photograph, etc., in operations SlO to S30, a 3D face is generated in operation S60 from 2D facial data and facial feature data extracted from the selected facial image for modifying the game character.

Here, the 3D face may be generated using, for example, an Active Appearance Model (AAM) or image-based 3D face modeling technology. Next, in operations S62 and S64, the generated 3D face is mapped onto the 3D face of the game character selected by the game user, and the 3D face of the game character selected by the game user is modified.

Fourth, in the fourth embodiment of the present invention, when compared to the third embodiment, in order to increase the processing speed of the game by replacing the face of a 3D character with a 2D face, after a facial region and facial features are extracted from a selected facial image obtained from a camera-captured photograph, etc. in operations SlO to S30, a 2D face is generated in operation S70 corresponding to an average 3D face model from the 2D facial data and the facial feature data extracted from the facial image selected for modifying the game character.

Next, in operations S72 and S74, the generated 2D face is texture mapped onto the 3D face of the game character selected by the game user, and the 3D face of the game character selected by the game user is modified.

The above-described methods of modifying game character faces game systems according to the present invention are not limited to the above embodiments, and it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Although embodiments have been described with reference to a number of illustrative embodiments thereof, it should be understood that numerous other modifications and embodiments can be devised by those skilled in the art that will fall within the spirit and scope of the principles of this disclosure. More particularly, various variations and modifications are possible in the component parts and/or arrangements of the subject combination arrangement within the scope of the disclosure, the drawings and the appended claims., In addition to variations and modifications in the component parts and/or arrangements, alternative uses will also be apparent to those skilled in the art.

Claims

1. A method of modifying a face of a game character in a game system, the method comprising: extracting a facial region and facial features from a selected facial image; and modifying a 2D face of a game character selected by a game user through directly mapping extracted 2D facial data and facial feature data onto the 2D face of the game character selected by the game user.

2. A method of modifying a face of a game character in a game system, the method comprising: extracting a facial region and facial features from a selected facial image; generating a 2D caricature through characterizing extracted 2D facial data and facial feature data; and modifying a 2D face of a game character selected by a game user through mapping the generated 2D caricature onto the 2D face of the game character selected by the game user.

3. The method of claim 2, wherein the generating of the 2D caricature through characterizing the extracted 2D facial data and facial feature data comprises: pre-storing facial feature data of game characters in a DB (database) ; and generating the 2D caricature through matching and integrating the 2D facial features extracted from the selected facial image with the pre-stored facial feature data of the game characters, based on similarities and shape data.

4. The method of claim 2, wherein the generating of the 2D caricature through characterizing the extracted 2D facial data and facial feature data comprises: pre-storing facial feature region data of game characters in a DB, and converting the 2D facial feature data extracted from the selected facial image to facial feature regions; and generating the 2D caricature through matching and integrating the 2D facial feature data extracted from the selected facial image with the pre-stored facial feature data of the game characters, based on similarity data of conversion coefficients of the facial feature regions.

5. The method of claim 4, wherein the pre-storing facial feature region data of game characters in a DB, and converting of the extracted 2D facial feature data to facial feature regions comprises using one of a PCA (Principal Component Analysis) method, an ICA (Independent Component Analysis) method, LFA (Local Feature Analysis) method to extract facial feature regions.

6. A method of modifying a face of a game character in a game system, the method comprising: extracting a facial region and facial features from a selected facial image; generating a 3D face from 2D facial data and facial feature data; and modifying a 3D face of a game character selected by a game user through mapping the generated 3D face onto the 3D face of the game character selected by the game user.

7. The method of claim 6, wherein the generating of the 3D face from the 2D facial data and facial feature data comprises employing an AAM (Active Appearance Model) or an image-based 3D face modeling technology to generate the 3D face.

8. A method of modifying a face of a game character in a game system, the method comprising: extracting a facial region and facial features from a selected facial image; generating a 2D face corresponding to an average 3D face model from extracted 2D facial data and facial feature data; and modifying a 3D face of a game character selected by a game user through texture mapping the generated 2D face onto the 3D face of the game character selected by the game user.

9. The method of any one of claims 1, 2, 6, and 8, wherein the extracting of the facial region and facial features from the selected facial image comprises using one of an AdaBoost Face Detection method, a PCA (Principal Component Analysis) method, an ICA (Independent Component Analysis) method to extract facial regions of a user.

10. The method of any one of claims 1, 2, 6, and 8, wherein the extracting of the facial region and facial features from the selected facial image comprises using a training-based facial feature detection method employing an AAM (Active Appearance Model) to extract all or a portion of both ends of an eye, both eyes, a nose, a mouth line, a facial contour, and a hairline as facial features.