KR20080050283A - Corelation extract method for generating 3d motion data and motion capture system and method for easy composition of humanoid character to real background image using as the same - Google Patents

Abstract

A correlation extraction method for generating 3D motion data, a motion capture system for easily synthesizing a body-type character to a real background image by using the same, and a method therefor are provided to capture the 3D motion data capable of being accurately matched to a real background image photographing camera by obtaining a correlation between a fixed camera for motion capture and the real background image photographing camera. A correction frame calibrates positions of a motion capture fixed camera(103) and a real background photographing movable camera(101). The motion capture fixed camera photographs a corner point which exists in the correction frame and a background marker or a background, and captures the motion of operator. The real background photographing movable camera moves the position and photographs a real background. A motion capture processing unit(100) extracts a geometric correlation between the motion capture fixed camera and the real background photographing movable camera, and generates 3D motion data automatically matched to the real background.

Description

Correlation Extraction Method for 3D Motion Data Generation and Motion Capture System and Method for Easy Synthesis of Humanoid Characters in Photorealistic Background Image OF HUMANOID CHARACTER TO REAL BACKGROUND IMAGE USING AS THE SAME}

The present invention relates to a motion capture method for easily synthesizing a humanoid character in a live action background image, and more particularly, the present invention provides a camera for motion capture in a place (outdoor or studio) for shooting a live action background, Before performing motion capture for the humanoid character to be synthesized, obtain a correlation between the fixed camera for motion capture and the live action background image capture camera to capture 3D motion data that can be accurately matched to the live action background image capture camera. It is about how it can be.

The present invention is derived from a study conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication and the Ministry of Information and Telecommunication Research and Development. [Task Management Number: 2005-S-003-02, Title: Diligence-level Digital Video Contents Authoring software development].

In the conventional patent, there is a method of generating motion data for a humanoid character by wearing an infrared reflector or a light emitter, an ultrasonic generator, a magnetic sensor, an optical fiber, and the like on a joint, and then restoring the three-dimensional position of the sensor.

In this method, the motion capture for anthropomorphic characters was attached to the joints to detect the three-dimensional position of the joints so that the motion data of the characters themselves could be captured accurately, but there was no correlation with the synthesized background image. When there is an interaction with a real prop or a third person in the synthesized photorealistic image, a considerable amount of modification must be made to the captured data.

In the method of extracting background geometric information by performing camera tracking on a background shot image, there was a method of extracting a 3D geometric structure of a prop of a live action background, but motion capture data is not generated for a human body. I couldn't.

Therefore, in the past, motion capture and photorealistic background image shooting must be performed separately, which makes it difficult to synthesize a character to which motion captured data is applied to the photorealistic background. In particular, when there is an interaction with a real prop or a third real person, a lot of manual work is required, which causes a problem in that the synthesis process takes a lot of time.

Accordingly, an object of the present invention is to solve the problems of the prior art, an object of the present invention is a method for generating three-dimensional human-like character motion data that can automatically match the actual person in the actual background image To provide.

The motion capture system of the present invention for achieving the above object is a correction frame for calibrating the position of the fixed camera for motion capture and mobile camera for live background shooting, and the corner point existing in the correction frame and background marker or background Geometric correlation between a fixed camera for capturing motion and capturing motion of a motion, a live camera for photographing a live background while moving a position, and a fixed camera for motion capturing and a mobile camera for capturing a live background And a motion capture processing device for generating three-dimensional motion data that automatically extracts the relationship and automatically matches the photorealistic background.

In addition, the motion capture processing apparatus detects a three-dimensional geometric mutual position between the fixed camera for motion capture and the corner marker present in the background marker or background, thereby geometrical correlation between the fixed camera for motion capture and the mobile camera for real background shooting. And a three-dimensional motion data generator for generating a three-dimensional motion data that automatically captures the motion of an actually photographed operator and automatically matches the photorealistic background according to the extracted correlation. It is characterized by.

On the other hand, the correlation extraction method for generating three-dimensional motion data of the present invention is to obtain a geometric relationship between the correction frame and the background marker or the corner point existing in the background, and for the motion capture via the correction frame from the obtained geometric relationship Obtaining a three-dimensional geometric mutual position between the camera and a background marker or a corner point existing in the background; and calculating the three-dimensional geometric mutual position between the motion capturing camera and the mobile camera for photorealistic background shooting from the obtained three-dimensional geometric mutual position. And extracting a correlation between the motion capture camera and the live camera for live background shooting.

Meanwhile, the motion capture method of the present invention is a method of performing motion capture using a fixed camera for motion capture and a mobile camera for live action background shooting, and extracting a geometric correlation between the fixed camera for motion capture and a mobile camera for live action background shooting. And a second step of generating a three-dimensional motion data that automatically matches the photorealistic background by capturing the motion of the actual operator using the extracted correlation.

In addition, the first step is a step 1-1 for obtaining a geometric relationship between the motion capture fixed camera and the correction frame, and the corner markers present in the background marker or background photographed by the correction frame and the live background shooting mobile camera; 3-dimensional geometric mutual position between a fixed camera for capturing motion and a corner marker present in the background marker or background through a correction frame using the geometric relation obtained in the above-described steps 1-2 Obtaining a 3D geometric mutual position between the motion capturing camera and the live background photographing mobile camera from the 3D geometric mutual positions obtained in 1-3, and obtaining the motion capturing fixed camera and the photorealistic background. Characterized in that 1 to 4 steps to extract the correlation between the mobile camera for shooting.

As described above, the motion capture device and method according to the present invention captures by generating three-dimensional motion data that can be automatically matched with the live action background accurately using the correlation between the fixed camera for motion capture and the mobile camera for shooting the live action background image. Even when there is an interaction with a real person such as a third person, a chair, a table, etc. existing in the space, there is an effect that the humanoid character can be easily synthesized on the live action background.

Hereinafter, the motion capture method of the present invention will be described in detail with reference to FIGS. 1 and 2.

1 is a block diagram showing a motion capture device for generating three-dimensional motion data for easy synthesis of a humanoid character in a live action background image according to the present invention.

Referring to FIG. 1, the motion capture processing apparatus 100 according to the present invention captures a corner point existing on a correction frame and a background marker or a background, and captures a motion camera for capturing motion of a motion. And extracts the geometric correlation of the live background photographing mobile camera 101 moving the position and photographing the live background to generate 3D motion data that automatically matches the live background.

The motion capture processing apparatus 100 grasps the three-dimensional geometric mutual position between the fixed camera 103 for motion capture and the corner marker present in the background marker or the background to move the fixed camera 103 for motion capture and live action background shooting. Correlation extraction unit 110 for extracting the geometric correlation between the camera 101 and motion-captured motion of the actually photographed operator according to the extracted correlation to generate three-dimensional motion data that can be automatically matched with the live action background 3D motion data generation unit 120 is configured to include.

The correction frame is for calibrating the positions of the fixed camera 103 for motion capture and the mobile camera 101 for live background shooting.

2 is a flowchart illustrating a three-dimensional motion data generation process using the correlation between the fixed camera 103 and the mobile camera 101 according to the present invention.

The present invention largely consists of a preparation process and an actual operation process in two processes. First, in the preparation process, the correlation between the motion capture fixed camera 103 and the live background photographing mobile camera 101 is extracted and stored.

Specifically, when starting the system and the program (S210), the mobile camera 101 is installed so that the correction frame 102 and the background marker 104 (or the background corner point) are simultaneously observed, and the plurality of fixed cameras ( 103 is installed in a fixed form inserted into a tripod or prop (S211).

Then, using the correction frame 102, each of the fixed camera 103 for motion capture and the mobile camera 101 for live background shooting are calibrated (calculated relative to the calibration frame), and through this, the mobile camera 101 for live action background shooting. ) And the three-dimensional geometric mutual position of the fixed camera 103 for motion capture (S211).

The mobile camera 101 with respect to the correction frame by calculating the geometric position of the background marker 104 (or the background corner point) with respect to the correction frame by means of triangulation using the calibrated two fixed cameras 103, 3D geometric mutual positions of the fixed camera 103 and the background marker 104 (or the background corner point) are determined (S211).

By identifying the mutual position of the fixed camera 103 for motion capture and the mobile camera 101 for live action background shooting with respect to the correction frame, the mutual position (correlation relation) between the fixed camera 103 and the mobile camera 101 can also be determined. (S212).

After extracting and storing the correlation between the fixed camera and the mobile camera through the preparation process as described above, the actual operation process (S213 ~ S214) is performed.

The fixed camera performs motion capture of an actual operator (S213), and generates extracted 3D motion data that is accurately matched to the photorealistic background image captured by the mobile camera by introducing the extracted correlation (S214).

Then, after the preparation process according to the motion capture method as described above, a process of performing a full-scale motion capture will be described with reference to FIGS. 3 and 4.

FIG. 3 is an exemplary diagram for extracting a correlation between a fixed camera for capturing motion according to FIG. 2 and a mobile camera for capturing photorealistic background images.

3 (a) is an exemplary diagram for extracting an initial correlation between a fixed camera for motion capture and a mobile camera for photographing a live action background image, and includes a mobile camera 101 and a correction frame 102 for a live action background image capture. ), A fixed camera 103 for motion capture and a background marker 104 are installed, and the fixed camera 103 is installed in a form inserted into a tripod or a prop.

Thus, the fixed camera 103 for motion capture is installed in the place (outdoor or studio) which photographs the live photo background, and the initial correlation between the fixed camera 103 and the mobile camera 101 is calculated | required.

The initial correlation between the fixed camera 103 for motion capture and the mobile camera 101 for real background image shooting is first obtained through the correction frame 102 for pre-calculating mutual positions between the fixed cameras.

That is, the initial relationship between the fixed camera 103 and the mobile camera 101 is obtained by obtaining the geometric relationship between the correction frame 102 and the live action background information in the mobile camera 101.

In this case, the moving cameras 101, 201, and 301 are positioned to simultaneously observe the correction frame 102 for correcting the fixed camera 103 and the corner points existing in the background or the markers installed in the background.

3 (b) is an exemplary diagram for extracting a correlation between the fixed camera 103 and the mobile camera 201 at another point in time, and FIG. 3 (c) is a mobile camera of another fixed point with the fixed camera 103. This is an exemplary diagram for extracting correlation with 301.

As shown in FIGS. 3B and 3C, the moving camera 201 at another point of view through registration of the background marker 104 (or the background corner point) photographed by the moving cameras 201 and 301 located at another point of view. And the correlation between the moving cameras 201 and 301 and the fixed camera 103 at other points of time are extracted through the relative positions of the background marker 104 and the fixed camera 103.

Accordingly, the motion capture fixed camera 103 and the background marker 104 are extracted through correlation extraction between the mobile cameras 101, 201, 301 and the fixed camera 103 positioned as shown in FIGS. 3A to 3C. Or three-dimensional mutual positional relationship with the corner point which exists in a background is calculated | required.

As described above, the correlation between the fixed camera 103 and the mobile cameras 101, 201, and 301 is extracted in advance, so that the background marker or the corner point of the background is removed even if the correction frame is removed from the capturing area during the actual motion capture. The correlation between the fixed camera for motion capture and the mobile camera for shooting the live background image can be extracted.

In addition, the background correction marker 104 or the corner point of the background used in the present invention is not limited to the marker or the corner point, and any type of singularity, line, face, etc., which is useful for extracting three-dimensional geometric information of the background. It includes.

4 is a diagram illustrating a process of generating three-dimensional motion data that can be automatically matched with the live-action background image according to FIG. 2, wherein (a) represents a first view of a mobile camera for photographing a real-life background image, and (b) 2, the mobile camera viewpoint for the photorealistic background image photographing, (c) shows a camera viewpoint for the third photorealistic background image capturing.

First, as shown in (a) to (c) of FIG. 3, a fixed camera and a mobile camera are installed, and the correlation between the fixed camera and the mobile camera is extracted and stored according to the position of each mobile camera.

Then, when motion capture is performed through the band for acquiring the human body character motion data, the 3D motion capture data generated from the actual motion capture is automatically generated as 3D data linked to the mobile camera for photographing the live background image.

That is, motion data generated by interaction with a real person such as a third person, a chair, a table, etc. existing in the shooting space is generated as 3D motion data that can be accurately matched to a mobile camera for shooting a live background image. Therefore, it is possible to easily synthesize the live action background image and the humanoid character.

Referring to FIG. 4, in FIG. 4A, the mobile camera 401 at the first viewpoint position includes a background marker 104 to photograph the live background. Similarly, in FIG. 4B, the mobile camera 501 at the second viewpoint position includes a background marker 104, and photographs the photorealistic background. In FIG. 4C, the mobile camera 601 at the third viewpoint position is photographed. The background marker 104 is included to photograph the live action background.

Then, for each viewpoint, the fixed camera 103 photographs the actual operator 402 and extracts the movement.

3D motion data that can be automatically matched to the actual background of each viewpoint photographed by the mobile cameras 401 to 601 with respect to the movement of the actual operator extracted according to the correlation between the mobile cameras 401 to 601 and the fixed camera 103. To generate (No. 0507780, "High-speed marker-free motion capture method robust to noise")

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

1 is a block diagram showing a motion capture device for generating three-dimensional motion data for easy synthesis of a humanoid character in a live action background image according to the present invention;

2 is a flowchart illustrating a motion capture process for generating three-dimensional motion data for easy synthesis of a humanoid character in a live-action background image according to the present invention;

3 is an exemplary diagram for extracting a correlation between a fixed camera for motion capture and a mobile camera for photographing a live action background image according to the present invention, and (a) extracts an initial correlation between a fixed camera and a mobile camera; (b) shows the extraction of the correlation between the fixed camera and the mobile camera at another point of view, (c) is an exemplary diagram showing the correlation extraction between the fixed camera and another moving camera of another view

4 is an exemplary diagram of generating 3D motion data that can be automatically matched in the photorealistic background image according to the motion capture process of FIG. 2, wherein (a) is a first moving camera viewpoint, and (b) is a second movement. The viewpoint of the camera, (c) is an exemplary view showing the viewpoint of the third mobile camera.

<Description of Major Symbols in Drawing>

101, 201, 301, 401, 501, 601. Mobile camera for live action background shooting

100. Motion capture processing unit 102. Correction frame

103. Fixed camera for motion capture 104. Background marker

110. Correlation extractor 120. 3D motion data generator

Claims (12)

Obtaining a geometric relationship between a correction frame and a corner marker existing in the background marker or background; Obtaining a three-dimensional geometric mutual position between the motion capture camera and a background marker or a corner point present in the background from the obtained geometric relationship; And Extracting a correlation between the motion capturing camera and the live background photographing mobile camera by obtaining a 3D geometric mutual position between the motion capturing camera and the live background photographing mobile camera from the obtained 3D geometric mutual position; A correlation extraction method for generating three-dimensional motion data, characterized in that. The method of claim 1, And the correction frame is installed to correct the position of the camera for capturing motion. The method of claim 1, The corner marker present in the background marker or the background includes any one of all types of singularities, lines, and planes. A correction frame for calibrating positions of the fixed camera for motion capture and the mobile camera for live background shooting; A fixed motion capture camera for capturing corner points existing in the correction frame and the background marker or in the background, and capturing motion of the motion; A mobile camera for shooting a live action background for moving a position and taking a picture of the live action background; And And a motion capture processing device for extracting geometric correlation between the fixed motion capture camera and the mobile camera for live background shooting to generate 3D motion data that automatically matches the live background. Motion capture system for easy synthesis of characters. The method of claim 4, wherein The background marker or the corner point existing in the background includes any one of all types of singularities, lines, and faces. The apparatus of claim 4, wherein the motion capture processing apparatus Correlation extraction unit for extracting geometric correlation between the motion capture fixed camera and the live camera for real background shooting by grasping the three-dimensional geometric mutual position between the motion capture fixed camera and the background marker or corner point present in the background ; And Easily the humanoid character in the live action background image, characterized in that it comprises a three-dimensional motion data generation unit for generating a three-dimensional motion data to automatically match the live action background by capturing the motion of the actual motion of the operator in accordance with the extracted correlation Motion capture system for one synthesis. The method of claim 6, The three-dimensional geometric mutual position of the motion capture fixed camera and the background marker or corner point existing in the background is obtained from the geometric relationship between the correction frame photographed by the fixed camera and the corner point existing in the background marker or background. Motion capture system for easy synthesis of humanoid characters on a live action background image, characterized in that the loss. In a method of performing motion capture using a fixed camera for motion capture and a mobile camera for live background shooting, Extracting a geometric correlation between a fixed camera for motion capture and a mobile camera for live background shooting; And Motion for easy synthesis of a humanoid character in a live action background image comprising the step of generating a three-dimensional motion data that automatically matches the live action background by performing a motion capture of the actual operator using the extracted correlation Capture method. The method of claim 8, wherein the first step Calculating a geometric relationship between the fixed camera for capturing motion and a correction frame; Obtaining a geometric relationship between the correction frame and a corner marker existing in the background or a background marker photographed by the live background photographing mobile camera; Obtaining a three-dimensional geometric mutual position between the motion capture fixed camera and the background marker or the corner point present in the background by using a geometric frame obtained in steps 1-2; And The correlation between the motion capture fixed camera and the live action background photographing mobile camera is extracted by obtaining the three-dimensional geometric mutual position between the motion capture camera and the live background photographing mobile camera from the three-dimensional geometric mutual positions obtained in the above 1-3. Motion capture method for easy synthesis of a humanoid character on a live action background image, characterized in that it comprises a step 1-4. The method of claim 9, The correction frame is a motion capture method for easy synthesis of the humanoid character on a live action background image, characterized in that installed to correct the position of the camera for motion capture. The method of claim 9, The mobile camera for live action background shooting is a motion capture method for easy synthesis of a humanoid character in a live action background image, characterized in that to locate the correction frame and the background marker installed on the background or a corner point existing in the background at the same time. The method of claim 9, The background marker or the corner point existing in the background includes any one of all types of singularities, lines, and faces.

Images