KR20110117487A - Between real image and cg composed animation authoring method and system by using motion controlled camera - Google Patents

Abstract

The present invention relates to a method for maximizing the efficiency of photorealistic and CG composite animation production, and more particularly, using animating data of CG characters through motion capture, and in order to maintain the texture of the stop motion animation. In the process of completing the content through the compositing of the CG camera, the process of taking pictures of the photorealistic background by controlling the motion of the camera for the photorealistic shooting in conjunction with the movement of the CG camera on the program, and applying the photorealistic lighting to the CG space to make the composition smooth. The present invention relates to a technology and process for producing animations including a process of applying a lighting set, a layer classification, and proper numerical regulation in a process of synthesizing CG and photorealistic images. (a) using motion capture equipment from the movement of an object Steps that generate motion capture data, and a CG character created by the 3D modeling tool by using the motion capture data; (b) a background map is generated by shooting from a background set produced in real space, the background map is included, and a CG lighting set is set that sets the environment of the virtual lighting that illuminates the CG character and the background map. Rendering a character image of the CG character reflecting an environment of virtual lighting; (c) generating a background image of the background set by moving photographing by a control of a camera control unit that is position-controlled according to the movement of the CG character; (d) generating an animation frame by synthesizing the character image with the background image.

Description

{Between Real image and CG Composed Animation authoring method and system by using motion controlled camera}

The present invention relates to a method for producing animation by real-world and CG synthesis, and more specifically, to control and photograph the movement of the camera for shooting in conjunction with the movement of the CG camera on the program during the animation production, according to the lighting production technology and background A process system for producing animation through compositing techniques.

Animation is, as is well known, computer graphic animation (CG animation) that completes the animation by producing a CG character and a background with a computer, stop-motion animation that captures the stop motion of a character doll by frame, There are dozens of papers, including cell animation that draws pictures on paper and silhouette animation that uses shadows.

Stop motion animation uses the stop motion shooting technique, which focuses on the fact that the movie is composed of 24 frames per second, and visualizes individual motions by one frame. The character doll is made of viscous material and It is produced by shooting while deforming posture one frame at a time.

Stop-motion animation is more familiar to the audience than 3D animation because it is made of dolls that children like, but it has a long production period and is difficult to modify because it is shot separately one frame at a time.

Therefore, in order to increase the production efficiency, by eliminating the character animating material, such as dolls that lower the production efficiency, and replacing it with the technology using the animation of 3D CG character using motion capture by significantly reducing the production period and production personnel , The production efficiency can be improved.

At this time, in order to prevent the unique texture of the stop motion animation due to the use of the CG data is reduced, a production process for calculating the final result through CG and photorealistic background synthesis is required.

However, in order to derive the sensibility of the stop motion in the content produced using the motion capture animating data, a miniature photorealistic background may be produced and synthesized with the CG character data. Since the background occupies more than two thirds of the entire screen, the texture of the background reduces the awkwardness of the texture implemented in the CG to the audience, and even makes it understood as the texture of the background. However, to carry out this process effectively, the following skills and processes are required.

For the production of animations through the composition of efficient CG and photorealistic background, CG lighting set implementation technology similar to the photorealistic background lighting and CG camera movement are linked with the motion of the camera for photo shooting. Skills are needed.

The present invention was devised in view of the above-mentioned necessity. For efficient synthesis of CG and photorealistic background, a CG character was produced using a 3D modeling tool, and only a background image was photographed and synthesized. The purpose of the present invention is to implement a method and system for producing a photorealistic and CG composite animation using a motion control camera that enables natural compositing by capturing a background image with a camera whose motion is controlled according to the motion.

In addition, the present invention provides a photorealistic using a motion control camera that renders a character image by rendering a virtual CG lighting set and applying it to a CG character to be identical to a lighting set setting in a background set for a natural synthesis of the CG character and the photorealistic background image. And to implement a method and system for producing CG composite animation.

Generating motion capture data from the movement of the object using motion capture equipment, and generating a CG character by a 3D modeling tool using the motion capture data; (b) a process of generating a background map by photographing from a background set produced in a real space, and a set of CG lighting that includes the background map and sets an environment of virtual lighting that is reflected on the CG character and the background map. A process of setting and rendering a character image of the CG character reflecting an environment of virtual lighting; (c) generating a background image of the background set by moving photographing by a control of a camera control unit that is position-controlled according to the movement of the CG character; (d) generating an animation frame by synthesizing the character image with the background image.

An animation production system using a motion control camera according to an embodiment of the present invention for achieving the above technical problem, CG (computer graphics) using a 3D modeling tool using the motion capture data generated by the motion capture device for the object Create a character, create a CG lighting set including an env map taken from a real background set and a virtual lighting environment setting that is identical to the real lighting environment, and generate the CG character that reflects the virtual lighting environment. A control computer for rendering and extracting camera motion data from a CG camera that captures the CG character; A main controller which communicates with the control computer and receives the camera motion data to drive a motor driver for position control; And a camera control unit 400 which is controlled by a position value of an X-axis, a Y-axis or a Z-axis according to a motor driver driven from the main controller, or controlled by Pan / Tilt, and installed with a camera for background set photographing. The control computer may include generating an animation frame by synthesizing the CG character with the background image photographed from the camera.

According to the present invention, by producing and rendering the direction image of the character in computer graphics, and photographing only the background image and synthesized with the character image, the existing production through the animation production system synthesized with live action and CG to maintain the texture of the stop motion animation Significantly reduce production staff, time and budget than stop-motion animation, and create series content through animating modifications and recycling of motion capture data, reducing the budget and time frame for the series content, and therefore continuously producing The efficiency can be improved.

1 is a flowchart of a method of producing a photorealistic and CG composite animation using a motion control camera according to an embodiment of the present invention. FIG. 2A is a method of manufacturing a photorealistic and CG composite animation using a motion control camera according to an embodiment of the present invention. FIG. 2B is a diagram illustrating a process of setting up a CG lighting set in FIG. 2. FIG. 3 is a schematic diagram of a photorealistic and CG composite animation production system using a motion control camera. FIG. 4 is a block diagram illustrating a system for producing photorealistic and CG composite animation using motion control according to an embodiment of the present invention. FIG. 5 illustrates an occlusion image generation rule and a rendering image layer composition rule. It is a figure explaining.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms, and only the present embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the art to which the present invention pertains. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

1 is a flowchart of a method for producing a photorealistic and CG composite animation using a motion control camera according to an embodiment of the present invention.

First, motion capture data is generated through a motion capture device for an object (S100).

In this case, the object may be a person, a doll of various materials, a robot, or the like.

Acquiring motion capture data using a motion capture device is possible using a magnetic sensor called a marker, and the like, which is a well-known technique, and thus a detailed description thereof will be omitted.

The motion capture data includes skeletal structure and rotation information according to the movement of the object.

This motion capture data can be used to create computer graphics (CG) characters in 3D modeling tools.

In this embodiment, MAYA is used as a 3D modeling tool, and software such as 3D MAX can be used. Therefore, the motion capture data is preferably provided to MAYA software in the .fbx extension format.

In general, motion capture data is generated at 30 frames per second, that is, 30 frames per second (fps). In the embodiment of the present invention, since the stop motion animation is produced, the motion may be simplified rather than smooth. Since there is a need, it converts to 10 fps (S102).

At this time, when it is necessary to deform the produced character-for example, to lift the object or change the movement-to allow the data to be input to each of the forward kinermatics (FK) joint of the character to facilitate modification. In detail, a character set is modified by using an IK joint when two kinds of FK character sets and IK (inverse kinermatics) character sets are connected by constraints and the character's motion needs to be modified.

Next, in order to create a feeling that the produced CG character exists in the live background, a virtual CG lighting set similar to the lighting setting of the background set is set (S104). This process is a process for natural lighting when synthesized with the background image actually photographed afterwards, and a detailed process of setting the CG lighting set will be described later.

Now, the CG character image is rendered in the preset CG lighting set (S106).

In this case, the rendering method renders the image sequence by rendering in the final gathering method of mental ray.

In addition, the rendered CG character image is rendered by dividing the character, shadow, character occlusion, and shadow occlusion by appropriate lighting to match the background image. Character occlusion and shadow occlusion are rendered to create the depth of shadow cast on the character and to create contact shadows, which are the contact surface of the character with the floor.

On the other hand, a background image is required to produce an animation frame, and the background image is generated by shooting from a set of backgrounds of a smaller size than the actual size to give the impression of a stop motion animation.

In this case, since the background image is photographed separately from the CG character, a process of moving and photographing the background image according to the movement of the CG character is required.

Therefore, the movement of the CG character is captured using the CG camera on the program, and camera motion data is extracted from the CG camera at this time (S108).

A camera control unit configured to be position controlled using the extracted camera motion data is program controlled. At this time, it is preferable to set the initial position value of the camera control unit to position control the camera up and down or left and right according to the camera motion data at the initial position (S110).

Now, the camera is installed in the camera control unit in association with the camera motion data of the CG camera to photograph the background set while moving, thereby generating a background image (S112). In other words, the camera installed in the camera control unit becomes a motion control camera, which may be, for example, a digital single-lens reflex (DSLR) camera, and uses an automatic shooting function (timer shooting function). You can also shoot.

Finally, the character image rendered in step S106 and the background image generated in step S114 are synthesized.

5 is a diagram illustrating an occlusion image generation rule and a rendering image layer composition rule.

Specifically, the composition rule will be described with reference to FIG. 5. The composition between the photographed background and the rendered CG character image may include a character occlusion image, a CG character image (RGB), a shadow occlusion, an image shadow, and a background image. Due diligence is synthesized from top to bottom. In this case, as shown in FIG. 5, the character occlusion is multiply in the layer style form, the transparency is 50%, the character image is in the layer style form normal, the transparency is 100%, and the shadow occlusion is multiply in the layer style form. , Transparency has 60%. It is desirable that the shadow has a normal layer style and 60% transparency, and the background image has a normal layer style and 100% transparency. Of course, the layer style shape and transparency can be changed by those skilled in the art as needed.

Furthermore, in the occlusion image, the character occlusion creates a slight depth depending on the shape of the character, and the shadow occlusion creates a strong shadow on the contact area between the character and the floor to express the presence of the character in space strongly. Used to You need a thin, spreading difference between the character occlusion and the shadow occlusion, so you need to change the parameter value of the creation rule occlusion. For example, as shown in FIG. 5, each parameter value is an occlusion parameter in the case of character occlusion. Samples = 32, Bright = 0,0,1 (HSV), Dark = 35,1 , 0.15 (HSV), Spread = 1, Max Distance = 0.8, for Shadow Occlusion, Samples = 32, Bright = 0,0,1 (HSV), Dark = 35,1,0.15 (HSV), Spread = 0.3, Max Distance = 0 can be set, but it can be changed by those skilled in the art as needed.

Through this synthesis process, one animation frame is produced, and an animation is produced by producing a plurality of animation frames.

FIG. 2A is a flowchart illustrating a process of setting a CG light set in a method of producing a real-life and CG composite animation using a motion control camera according to an embodiment of the present invention, and FIG. 2B is a diagram illustrating a process of setting a CG light set. It is a figure shown.

 The CG lighting set to be applied to the CG character must have the same lighting settings on the CG character as the actual lighting on the actual background set. For example, assuming that a character is actually in a set of backgrounds, illuminating the character may cause background reflections to appear in the character's costume or on a particular body part. Therefore, in order to give a feeling in real lighting such as background reflection appearing on a character, first, a background image of a real background set is photographed to create an env map.

In order to produce a background map, a background image is taken by a camera equipped with a fisheye lens from an actual background set (S200). Fisheye lenses are lenses that only have a 180 degree field of view. Since the background of the stop-motion animation requires only a 180-degree section similar to the theater stage, it is efficient to photograph only the front of the stage with this fisheye lens.

Six or more images of the same scene (stage front) are photographed with different exposure, and synthesized by using a fisheye lens to generate a high dynamic ranging imaging (HDRI) image (S202). By controlling the brightness by adjusting the exposure level of the HDRI image, it is possible to create a day and night atmosphere without changing the setting of the CG lighting set, thereby making production more convenient.

In a next process, a process of converting an HDRI image on a sphere into an image on a plane is performed (S204).

After the image is converted into a plane image, the corresponding image is copied and made into an image (background map) of 360 degrees by mirroring (mirroring) (S206). In general, shooting on the back of the stage, which is a dark space where the audience or the director is located, has no meaning as a background map, so the image of the stage front is copied and mirrored and pasted to complete the 360-degree image.

Now, the CG illumination set is set by applying the image (background map) corrected in the 360 degree section (S208).

The CG lighting set consists of a polygon sphere for attaching a background map to 3D space, a polygon floor for extracting shadows and occlusion images of CG characters in 3D, and virtual lights and 3D cubes.

In consideration of the position and size of the actual background set, the reference cube is set as the reference point in the actual background set with a size of 10 cm * 10 cm. By matching the 3D cube of 3D space, which is the reference size, the size of the CG light set and the position of the light are matched by animation scene to maintain a constant size. The cube setting, which is a standard of the space size between the actual background set and the CG lighting set, is necessary to produce a constant lighting for each space.

On the other hand, the 3D virtual light is preferably composed of three key lights, fill light, and bounce light to cast shadows. That is, since the image of the background is reflected and coated on the character when the character renders, a light that is emitted from both sides and a bounce light that prevents shadows on the face from the front of the character may be installed.

That is, the intensity, position, angle, or intensity of the virtual light is set so that the environment of the virtual light is the same setting as the environment of the light of the actual background set.

Through this iterative process, background maps are created from various actual background sets. That is, the background map is produced separately for each episode and scene.

The produced background map stores a background map image for each background set in which various lighting conditions are set, and also stores lighting settings for each background set (S210). Saved background map and lighting settings can easily set CG lighting set by selecting (applying) background map or lighting settings in the program for various episodes and scenes (S212), thereby improving production efficiency. You can.

FIG. 3 is a diagram schematically illustrating a live action and CG composite animation production system using a motion control camera, and FIG. 4 is a block diagram illustrating a live action and CG composite animation production system using a motion control camera according to an embodiment of the present invention. to be.

With reference to FIGS. 3 and 4, the system for producing a photorealistic and CG composite animation using a motion control camera according to an embodiment of the present invention includes a control computer 100, a main controller 200, a camera control unit 400, and a camera control. It may be configured as a camera mounted on the unit 400.

The control computer 100 is provided with MAYA, a 3D modeling tool, and builds an environment in which motion capture data can be acquired in conjunction with a motion capture device.

At this time, the number of frames per second can be converted from the motion capture data, and a three-dimensional (3D) CG character is generated by the 3D modeling tool.

Furthermore, the CG lighting set may be configured using a background map produced by photographing from an actual background set, and the CG character is rendered by applying the CG lighting set reflecting the same lighting environment as the actual background set.

In addition, in order to control the position of the camera control unit 400, a CG camera is generated on a program to extract camera movement data according to the movement of the CG character.

By the command of the control computer 100, this camera motion data is transmitted to the camera control unit 400 via the main controller 200, and the position control of the camera control unit 400 is carried out.

At this time, the communication between the control computer 100 and the main controller 200 may be made by a LAN.

The main controller 200 commands to control the position of the camera control unit 400 by driving a motor driver for position control in order to control the position of the camera control unit 400 from the transmitted camera motion data by receiving a command from the control computer 100. It plays a role.

The main controller 200 includes a computer communication unit 210, a remote communication unit 220, a PLC control unit 230, an LCD controller 240, a position control unit 250, a pan / tilt control unit 260, and a rotation angle control unit 270. It may further include.

The computer communication unit 210 may further include a LAN communication module 211 and a communication interface board 212, and the LAN communication module 211 performs LAN communication with the control computer 100 and a communication interface board 212. ) Configures a communication protocol to enable remote communication with the LAN communication and telecommunications unit 220.

The remote communication unit 220 enables remote control of the control computer 100 and the camera control unit 400 by remote communication with a remote controller 500 to be described later.

The PLC controller 230 performs a function of program controlling the camera control unit 400 by using the camera motion data transmitted from the control computer 100. That is, the position value for controlling the position of the camera control unit 400 is calculated, and the position value is transmitted to the position controller 250 so that the position controller 250 drives the motor drive. In addition, the LCD controller 240 performs a command to display the position value according to the camera movement through the LCD screen display 300.

The LCD controller 240 controls to display the position value on the LCD screen display unit 300 by a command from the PLC controller 230.

The position controller 250 receives the position value from the PLC controller 230 and drives the X-axis motor driver 251, the Y-axis motor driver 252, and Z to drive the servo motor in the X-axis, Y-axis, and Z-axis directions. The shaft motor driver 253 is driven.

The pan / tilt control unit 260 calculates a position value for controlling the pan or tilt of the camera control unit 400 using the camera motion data transmitted from the control computer 100.

The rotation angle controller 270 receives the position value calculated from the Pan / Tilt controller 260 to drive the Pan motor driver 271 and the Tilt motor driver 272.

The camera control unit 400 includes an X-axis servomotor 410 and a Y-axis servomotor 420 respectively driven by the X-axis motor driver 251, the Y-axis motor driver 252, and the Z-axis motor driver 253. , Pan-axis step motor 440, Tilt-axis step motor 450, motion control camera 600, respectively driven by Z-axis servo motor 430, Pan motor driver 271, Tilt motor driver 272. It may be configured to include.

The motion control camera 600 mounted on the camera control unit 400 captures the background image of the background set by the automatic shooting function or the manual shooting.

The remote control 500 remotely commands the position control of the camera control unit 400 to the control computer 100 by remote communication with the remote communication unit 220, and the camera control unit 400 in conjunction with the control computer 100. Remote control.

The remote communication method may be, for example, CAN communication.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, the above-described embodiments are to be understood in all respects as illustrative and not restrictive.

100: control computer 200: main controller
210: computer communication unit 211: LAN communication module
212: communication interface board 220: remote communication unit
230: PLC controller 240: LCD controller
250: position control unit 251: X axis motor driver
252: Y-axis motor driver 253: Z-axis motor driver
260: Pan / Tilt control unit 270: rotation angle control unit
271: Pan motor driver 272: Tilt motor driver
300: LCD screen display unit 400: camera control unit
410: X-axis servo motor 420: Y-axis servo motor
430: Z-axis servo motor 440: Pan-axis step motor
450: Tilt axis step motor 500: Remote control
600: motion control camera

Claims (10)

Generating motion capture data from the movement of the object using motion capture equipment, and generating a CG character by a 3D modeling tool using the motion capture data; A process of generating a background map by shooting from a background set produced in a real space, and a process of setting a CG lighting set that includes the background map and sets an environment of a virtual light that shines on the CG character and the background map. And rendering a character image of the CG character reflecting the environment of the virtual lighting. Generating a background image of the background set by moving photographing under the control of a camera control unit which is position-controlled according to the movement of the CG character; And generating an animation frame by synthesizing the character image and the background image. The method of claim 1, wherein the step (a) comprises: (a-1) converting the number of frames per second of the motion capture data to control the movement of the CG character; How to make a CG composite animation. The method of claim 1, wherein step (b) comprises: (b-1) photographing the background set by a fisheye lens at a 180-degree interval; (b-2) a sphere photographed by the fisheye lens ( (B) converting the distorted image on the plane into an image on the plane; (b-3) copying the converted image by mirroring to generate a background map corrected by a 360 degree section; And (b-4) setting the environment of the virtual lighting to have the same setting as the intensity, position, angle, or size of the actual lighting in the background set, and setting the CG lighting set on which the background map is implemented. How to make photorealistic and CG composite animation using a motion control camera. The motion control camera according to claim 1 or 3, wherein the background map is generated and generated in a separate background set for each episode or scene, and the CG lighting set is further set according to the background map. How to make photorealistic and CG composite animations. The method of claim 1, wherein step (c) comprises: (c-1) extracting camera motion data of a CG camera from a CG camera photographing the CG character in a virtual space of a computer; (c-2) Setting an initial position value of the control unit; And (c-3) positioning the camera control unit left / right or up / down according to the extracted camera movement data, and moving and photographing the background set with a camera installed in the camera control unit to generate a background image. Real-time and CG composite animation production method using a motion control camera further comprising. Create a CG character with the 3D modeling tool using the motion capture data generated by the motion capture device on the object, and create a CG light set that includes a background map taken from the real background set and a virtual lighting environment that is identical to the real lighting environment. A control computer for generating, rendering the CG character reflecting the environment of the virtual lighting, and extracting the camera movement data from the CG camera to shoot the CG character; Communication with the control computer receives the camera movement data to receive the position control motor A main controller for driving a driver; And a camera control unit which is controlled by a position value of an X-axis, a Y-axis, or a Z-axis according to a motor driver driven from the main controller, or which is controlled by a pan or tilt, and which has a background set photographing camera installed therein. Photorealistic and CG composite animation production system using a motion control camera for synthesizing the background image photographed from the camera and the CG character to generate an animation frame. The system of claim 6, wherein the control computer further comprises converting the number of frames per second of the motion capture data to control the movement of the CG character. The motion control camera of claim 6, wherein the background map is generated and generated in a separate background set for each episode or scene, and the CG lighting set is set according to the background map from the control computer. Photorealistic and CG composite animation production system. The apparatus of claim 6, further comprising: a computer communication unit configured to communicate with the control computer to receive the camera motion data; a PLC controller configured to program control the camera control unit based on the camera motion data; to control the camera control unit by the control computer. An LCD controller for instructing display of status information including camera motion data or unit position values; a position controller configured to control the camera control unit by position values of X, Y, or Z axes of an axis driving motor; Rotation angle control unit for controlling the pan or tilt of the control unit of the rotation drive motor; An LCD screen display for displaying status information including camera motion data or unit position values for controlling the camera control unit by the control computer; And a remote communication unit configured to receive camera motion data from the control computer by remote communication from an external device. 2. The motion control camera of claim 9, wherein the animation production system using the motion control camera further comprises a remote control for remotely controlling the camera control unit in cooperation with the control computer by remote communication with the remote communication unit. Animation production system using.

Images