KR20120092960A - System and method for controlling virtual character - Google Patents

Abstract

PURPOSE: A virtual character control system and a method thereof are provided to control movement of a character on a virtual space through human body movement degree of a user inputted through a camera without additional controllers. CONSTITUTION: An input management module(100) includes a camera unit(110) and a feature extraction unit(120). The camera unit photographs a human body of a user. The feature extraction unit extracts human body information including a human body feature through the photographed input image. A virtual character management module(200) generates a virtual character. The virtual character management module moves the virtual character according to the movement of the human body by matching the processed human body information with the virtual character. A rendering module(300) stores the movement of the virtual character in an image form.

Description

SYSTEM AND METHOD FOR CONTROLLING VIRTUAL CHARACTER}

The present invention relates to a virtual character control system and method, in various external spaces such as leisure space and education space, to control the movement of the character in the virtual space using the movement information of the user's body parts coming through the camera, the character movement image The present invention relates to a virtual character control system and method that can share with others.

Conventionally, the entertainment apparatus displays contents, music, and images previously stored on the screen under the control of a user, and recognizes the displayed contents, music, and images by visual or audio. The user has been acquainted with this by following an operation suitable for the displayed information. That is, their behavior is controlled according to the behavior of the character provided in the virtual space, not the control of the character provided in the virtual space.

Alternatively, the motion of the person is captured as it is, and the digitized motion information is input to the system to move the character using the input motion information. Of course, this is called motion capture technology. Of course, such motion capture technology is widely used for movies, games, and broadcast animations. Conventional motion capture technology has a disadvantage that the information stored in the existing one-way has a one-way nature.

As mentioned above, the expression method of the media used for the existing entertainment is not diverse and there is a problem that the fun is halved. Furthermore, in the virtual space, the character's control technology that follows the user's movement improves the fun and immersion through the character of the virtual space that has the same movement as the actual user's movement, and the real life (ie daily life, not limited space). There is an urgent need for entertainment devices that can be utilized in free space and education space.

Therefore, the present invention can control the movement of the character in the virtual space through the degree of movement of the body part of the user coming through the camera without a separate controller to solve the above problems, the movement of the character in the virtual space moving according to the user It provides a virtual character control system and method that can be stored in the form of animation, and can be distributed or transmitted through the web.

An input management module including a camera unit for photographing a user's body, a feature extractor for extracting body information including body features from the photographed input image, and a virtual character to be generated and processed. It provides a virtual character control system including a virtual character management module for matching the information and the virtual character to move the character in the same manner as the movement of the body and a rendering module for storing the movement of the virtual character as an image.

The feature extractor may include a preprocessing module for preprocessing an input image, a face detection module for detecting a face portion from the preprocessed image, and a body region extraction module for extracting positions of body parts such as arms and legs from the preprocessed image. It is characterized by including.

The preprocessing module may include an image scale changing unit for changing the scale of the input image, a color changing unit for changing the color of the image, and a histogram equalizing unit for equalizing the histogram.

The face detection module may include a face area detection unit detecting a face area by applying a CBCH algorithm to a preprocessed image, and a face selection unit detecting a participant's face among the detected face areas.

The body part extraction module detects main body parts such as arms, legs, and torso through the CBCH algorithm on the preprocessed image, extracts a hues-saturation-value color histogram of the body area, and To obtain a color probability distribution, each area region is tracked, and a camshift is performed to the detected body part region.

The virtual character management module may include a character generation module for generating a virtual space and a virtual character, a virtual character setting module for matching the extracted face and body part region location information with the generated virtual character, and the face and body region movement information. Character control module for controlling the movement of the virtual character, and a display module for synthesizing and displaying the background of the virtual character and the virtual space.

In addition, the step of generating a human-shaped 3D character according to the present invention, receiving a user's image through a camera, primary editing of the image through pre-processing of the provided user image, pre-processed image Extracting body region information such as a user's face, arms, legs and torso, matching the body region region information to the body region of the 3D character, and moving the body region information from a preprocessed image. It provides a virtual character control method comprising the step of extracting and controlling the movement of the 3D character using the extracted movement information.

The preprocessing of the image may include downsizing the image to reduce its capacity, changing the color image to a contrast image, and performing equalization of the histogram.

The body region information is detected using the CBCH algorithm in the preprocessed region, the body region movement information is extracted from the HSV color histogram in the body region obtained through the CBCH algorithm, and the position of each region region through CAMshift. And extracting and tracking the movement information thereof.

After controlling the movement of the 3D character, characterized in that for storing the movement of the 3D character as an image.

In addition, a computer having a 3D virtual character control function of the virtual character control system of any one of claims 1 to 6 according to the present invention, a display on which the 3D virtual character is displayed and a camera for photographing the user It provides a game device characterized in that.

As described above, the present invention can control the movement of the character in the virtual space through the degree of movement of the body part of the user coming through the camera without a separate controller. In addition, the preprocessing process may shorten the image processing time and perform fast image processing. In addition, the movement of the character on the virtual space moving according to the user can be stored in the form of animation and distributed or transmitted through the web.

1 is a block diagram of a virtual character control system according to an embodiment of the present invention.
2 is a block diagram of a feature extractor of an input management module in one embodiment;
3 is a block diagram of a virtual character management module of one embodiment.
4 is a flowchart illustrating a virtual character control method according to an embodiment of the present invention.
5 is a conceptual diagram of a game device to which the virtual character control system according to an embodiment of the present invention is applied.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, It is provided to let you know. Wherein like reference numerals refer to like elements throughout.

1 is a block diagram of a virtual character control system according to an embodiment of the present invention.

2 is a block diagram of a feature extractor of an input management module according to an embodiment. 3 is a block diagram of a virtual character management module according to one embodiment.

1 to 3, the virtual character control system according to the present embodiment generates an virtual character and an input management module 100 that receives and processes the user's body information, and matches the processed body information and the virtual character. And a virtual character management module 200 for moving the character in the same manner as the movement of the body, and a rendering module 300 for storing the movement of the virtual character as an image and sharing or transmitting the image through the web.

In addition, the virtual character control system of the present embodiment further includes a music management module for providing music or sound information to a user and a storage module for storing data and information.

The input management module 100 of the present embodiment includes a camera unit 110 for capturing a user's body, and a feature extractor 120 for extracting a body feature from an image captured by the camera unit 110. .

The camera unit 110 includes at least one camera and a camera controller for controlling an operation and an input image thereof. The camera used in the present embodiment is not limited to a specific camera, and various cameras capable of capturing an image may be used. That is, as a mall camera, a camera attached to a small electronic product such as a mobile phone can be used. In this case, a separate wiring for connection with this system is required.

The feature extractor 120 includes a preprocessing module 121 for preprocessing an input image, a face detection module 122 for detecting a face portion from the preprocessed image, an arm from the preprocessed image, as shown in FIG. And a body part extraction module 123 for extracting a position of a body part such as a leg torso.

In general, the image acquired by the camera unit 110 is combined with the image of the surrounding environment as well as the user's face region and the other body of the user. Accordingly, the preprocessing is performed in the preprocessing module 121 to speed up the detection of the face and other body parts and increase the detection rate.

The preprocessing module 121 includes an image scale changing unit for changing the scale of the input image, a color changing unit for changing the color of the image, and a histogram equalizing unit for equalizing the histogram.

Thus, the preprocessing module 121 first changes the scale of the image input from the camera unit 110. That is, when the input image scale is 1, it is effective to change the size to 3/5 to 1/5. At this time, when the size is smaller than the above range, the accuracy in the subsequent processing is reduced, so that it is difficult to extract smooth features. In addition, if it is larger than the above range, it takes a long time to perform the image processing, it is difficult to adjust the character in real time. Preferably, it is effective to change to 1/2 scale of the input image. Subsequently, the preprocessing module 121 converts the downscaled color image into a contrast image. This can also shorten the image processing time as mentioned above. Later, histogram equalization clearly distinguishes black and white in the image.

Each result of the above operation is output by the preprocessing module 121.

The face detection module 122 includes a face area detector for detecting a face area by applying the CBCH algorithm, and a face selector for detecting a participant's face among the detected face areas.

The face detection module 122 uses an object detection algorithm using a cascade of boosted classifiers working haar-like feature (CBCH). In this embodiment, OpenCV 1.0 provided by Intel is used to apply the existing CBCH algorithm to the system.

In addition, the present embodiment detects only the face of the participant by selecting the actual participant and the spectator. That is, the size of the face area for all face areas found by the CBCH algorithm is measured, and only the face of the largest area is detected.

The face detection module 122 uses histogram-equalized image information after being changed to the contrast image by the preprocessing module 121.

The body part extraction module extracts a body region detector that detects a main body region through the CBCH algorithm, an HSV color histogram of the detected body region, and positions and movement information of each region through CAMshift. It includes a body region tracking unit for tracking.

The body part extraction module 123 also detects major body parts such as arms, legs and torso through the CBCH algorithm. That is, at least one of the hand region of the plate, the foot region of the leg, and the belly region of the torso is detected. Effectively, it is effective to detect hand and foot areas. In this case, the body part extraction module 123 uses the downscaled image information by the preprocessing module 121. That is, the contrast level uses the image image of the previous stage of the image change.

Accordingly, the body part extraction module 123 detects a body region using the CBCH algorithm. Subsequently, a Hue-Saturation-Value (HSV) Color Histogram of a body region is extracted, and a color probability distribution is obtained to track each region region.

CAMshift is performed to the detected body part region. When the body part area is detected, the body part may not be effectively found if it is not the front, but the position and rotation information of each body part may be obtained using a camshift algorithm. That is, the camshift algorithm can quickly calculate and track not only the position but also the rotation angle and size using the color information of the object, that is, the color probability distribution.

Therefore, the camshift can continuously track how body parts change in real time. The body part extraction module 123 obtains movement information of the body part by using the position and inclination information of the body part obtained through the camshift, and stores the obtained information.

In the above description, the face detection module 122 and the body part extraction module 123 have been described with a focus on separation. However, the present embodiment is not limited thereto, and the two modules may be integrated and operated as the body extraction module. In addition, the face detection module 122 may also obtain movement information of the face through a cam shift.

The face region module 122 and the body region extraction module 123 described above generate face region position and movement information and body region region position and movement information, respectively. If necessary, the face detection module 122 may provide image information corresponding to the face area.

Next, as shown in FIG. 3, the virtual character management module 200 matches the character generation module 210 that generates the virtual space and the virtual character, and the extracted face and body part region location information with the generated virtual character. The virtual character setting module 220, a character control module 230 for controlling the movement of the virtual character according to the face and body region movement information, and a display module 240 for synthesizing and displaying the background of the virtual character and the virtual space. It is provided.

The character generation module 210 generates a virtual space and a background and a virtual character used in the virtual space. At this time, it is effective that the generated virtual character and virtual space are 3D type. That is, the character generation module 210 generates a 3D virtual character. At this time, it is effective that at least one generated virtual character. In addition, various types of 3D virtual characters may be produced according to a user input. In this case, it is preferable to implement the character motion using motion capture.

The virtual character setting module 220 matches (ie, matches) the face region position information and the body region region information of the feature extractor 120 to the corresponding region of the 3D virtual character. That is, the virtual character setting module 220 extracts coordinate data from the face and body region region location information, matches a coordinate extractor for extracting coordinate data of the face and body region of the 3D virtual character, and the extracted coordinates. A coordinate matching unit is provided.

The virtual character setting module matches the body region of the user and the body region of the 3D character photographed and extracted by the camera.

The character control module 230 controls the movement of the 3D virtual character according to the face region movement information and the body region region movement information of the feature extractor 120.

That is, the character control module 230 moves a corresponding position of the 3D virtual character by a moving position measuring unit measuring a moving position of a face and a body in the face and body region movement information and a relative moving distance of the measured moving position value. It includes a character moving unit. In this case, it is effective that the character control module 230 processes three-dimensional information.

The display module 240 synthesizes the 3D virtual character and the background moving in the virtual space according to the user's body movement information. In addition, the 3D virtual character synthesized with the background is displayed on the screen display device.

The rendering module 300 includes an image storage unit storing the 3D virtual character moving by the virtual character management module 200 as an image, and an image transmitting unit sharing or transmitting the stored image through a network.

Through the above-described system, the 3D virtual character can be moved in the same manner as the movement of the user (ie, the participant) without using a separate controller.

4 is a flowchart illustrating a virtual character control method according to an embodiment of the present invention.

As shown in FIG. 4, a 3D character having a human shape is generated (S110). In this case, a character identical to a human shape may be generated by using a motion capture method.

Subsequently, an image of the user is provided through the camera (S120). In the present embodiment, user information is provided using only a camera without using a separate controller.

Subsequently, the image is first edited through preprocessing of the provided user image (S130).

In other words, the image provided from the camera is downscaled to reduce its capacity, and the color image is changed to the contrast image. In addition, the histogram is equalized.

Subsequently, body region information such as a face, an arm, a leg, a torso, and the like of the user is extracted from the preprocessed image (S140). That is, the body region is detected using the CBCH algorithm in the preprocessed region.

Subsequently, the body part region information is matched with the body part of the 3D character (S150). This matches the coordinates of each piece of information with the coordinates of the body part of the 3D character in the provided body part area information.

Subsequently, movement information of the body part is extracted from the preprocessed image (S160).

That is, the HSV color histogram is extracted from the body region obtained through the CBCH algorithm, and the body region movement information is extracted by tracking the position of each region region and its movement information through CAMshift.

Subsequently, the movement of the 3D character is controlled using the extracted movement information (S170). In the present embodiment, the movement of the 3D character is stored as an image (S180).

The stored image may be stored or transmitted through a network.

In addition, the present invention can provide a game device to which the above-described virtual character control system is applied.

5 is a conceptual diagram of a game device to which a virtual character control system according to an embodiment of the present invention is applied.

As shown in FIG. 5, the game device to which the virtual character control system is applied includes a computer 1000 having a virtual character control function, a display 2000 on which a 3D virtual character is displayed, and a camera 3000 photographing a user. Include.

Through this, the user's image is captured by the camera 3000 and the captured image is transmitted to the computer 1000 having the virtual character control function. In this case, it is effective to digitize and provide the image information. The computer 1000 extracts body region information from the provided image information, generates a 3D virtual character, and displays it on the display 2000.

The computer 1000 then matches the body region information of the user with the body region of the 3D virtual character. Thereafter, the body region movement information is extracted from the user's image and applied to the 3D virtual character to control the movement of the 3D virtual character. In addition, the movement of the 3D virtual character is displayed on the display 2000.

In addition, the above-described operation is performed by a virtual character control module (ie, a control system). Of course, the virtual character control module may be embedded in the computer 1000 in the form of a program, or may be mounted in the computer 1000 as a separate board type.

In addition, the virtual character control system records the movement of the 3D virtual character displayed on the display 2000 and stores it. At the request of the user, the motion image of the recorded 3D virtual character may be shared on the web or transmitted to a specific terminal.

Through this, the user's motion or movement can be observed from outside and corrected or corrected.

In addition, the display 2000 may include a speaker (not shown) that emits a sound such as music in an embedded form. And, a microphone for receiving sound from the outside may be provided. Through this, the game device of the present embodiment can be used as a device such as karaoke. Of course, the game device is not limited to games, but can be used in educational or public interest places.

In the present embodiment, the display 2000 and the computer 1000 are separately arranged, but the present invention is not limited thereto, and the computer 1000 may be embedded in the display 2000.

Although the technical spirit of the present invention described above has been described in detail in a preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention.

100: input management module 110: camera unit
120: feature extraction unit 121: preprocessing module
122: face detection module 123: body part extraction module
200: virtual character management module 210: character generation module
220: virtual character setting module 230: character control module
240: display module 300: rendering module

Claims (11)

An input management module having a camera unit for photographing a user's body and a feature extractor for extracting body information including body features from the photographed input image;
A virtual character management module for generating a virtual character and matching the processed body information with the virtual character to move the character in the same manner as the movement of the body; And
And a rendering module for storing the movement of the virtual character as an image. The method of claim 1,
The feature extractor may include a preprocessing module for preprocessing an input image, a face detection module for detecting a face portion from the preprocessed image, and a body region extraction module for extracting positions of body parts such as arms and legs from the preprocessed image. Virtual character control system comprising a. The method of claim 2,
The preprocessing module includes a video scale changing unit for changing the scale of the input image, a color changing unit for changing the color of the image, and a histogram equalizing unit for equalizing the histogram. The method of claim 2,
The face detection module includes a face region detector for detecting a region of a face by applying a CBCH algorithm to a preprocessed image, and a face selector for detecting a participant's face among the detected face regions. The method of claim 2,
The body part extraction module detects main body parts such as arms, legs, and torso through the CBCH algorithm on the preprocessed image, extracts a hues-saturation-value color histogram of the body area, and Obtaining a color probability distribution, tracing each area region, and performing a CAMshift to the detected body region region. The method of claim 1,
The virtual character management module may include a character generation module for generating a virtual space and a virtual character, a virtual character setting module for matching the extracted face and body part region location information with the generated virtual character, and the face and body region movement information. And a display module for synthesizing and displaying the background of the virtual character and the virtual space.
Generating a 3D character in a human shape;
Receiving an image of a user through a camera;
Primary editing the image through preprocessing of the provided user image;
Extracting body region information such as a face, an arm, a leg and a torso of a user from a preprocessed image;
Matching the body part region information to the body part of the 3D character;
Extracting movement information of the body part from the preprocessed image; And
And controlling the movement of the 3D character using the extracted movement information. The method of claim 7, wherein
The pre-processing of the image comprises scaling down the image to reduce its capacity, changing the color image to the contrast image, and performing an equalization of the histogram. The method of claim 7, wherein
The body region information is detected using the CBCH algorithm in the preprocessed region, the body region movement information is extracted from the HSV color histogram in the body region obtained through the CBCH algorithm, and the position of each region region through CAMshift. And extracting by tracking the movement information thereof. The method of claim 7, wherein
After controlling the movement of the 3D character, the virtual character control method, characterized in that for storing the movement of the 3D character as an image.
A computer having a 3D virtual character control function of the virtual character control system of any one of claims 1 to 6;
A display on which the 3D virtual character is displayed; And
Game device comprising a camera for photographing the user.

Images