KR20200089826A - Method and system for motion capture - Google Patents

Abstract

The present invention provides a motion capture system. The motion capture system comprises: a camera capturing a motion of a human body with a plurality of circular markers attached; a marker recognition unit recognizing a marker from an image captured by the camera; a marker circle restoration unit determining whether the recognized marker is a circle, and if not, restoring the recognized part to a circular shape according to a radius of the circular shape; and a marker position determination unit determining the center of the circle as a marker position of the center if the recognized marker is a circle and, if the marker is not the circle, determining the center of the circle restored by the marker circle restoration unit as the marker position.

Description

Motion capture system and method{METHOD AND SYSTEM FOR MOTION CAPTURE}

The present invention

Motion capture refers to the process of recording the movement of the human body in digital form by attaching a sensor to the body or using infrared rays. After attaching the sensor to various parts of the body, it is the core of the technology to make the virtual character move in the same motion through the position value of the sensor. The process of storing the motion of a real object as numerical data and passing the motion data to a virtual object made by a computer can be referred to as motion capture.

Motion tracking or motion capture identifies movement by position or angle between markers by the actor wearing a marker near each joint, and the combination of acoustic, inertial, LED, magnetic or reflective markers is at least twice the frequency speed of the desired movement. Tracked. Spatial resolution (resolution) and temporal resolution (resolution) have been developed because motion blur causes almost the same problem as lowering resolution, and most modern systems calculate the actor's joint angle in the background to match the mathematical silhouette. . I can't see the silhouette change when moving. There are hybrid systems in which both markers and silhouettes are available.

The optical system provides a projection that overlaps the 3D position of the object through a triangle between two or more cameras calibrated using data captured by the image sensor. Data acquisition was originally implemented using special markers attached to performers, but recent systems have been able to generate accurate data by tracking each dynamically identified surface function. Tracking a large number of actors or expanding the capture area was solved by increasing the number of cameras. Circular information must be inferred from three or more directions. Modern hybrid systems combine inertial and optical sensors to reduce waste light, increase the number of users, and improve tracking without manual data cleanup.

The passive marker system uses a marker covered with retroreflective material to reflect light generated near the camera lens. The center of the marker is estimated to be the position in the captured 2D image. The black and white value of each pixel can be used to provide partial pixel accuracy by finding the center of the Gaussian. The camera is calibrated using an object with a marker at a known location and the lens deformation of each camera is measured. When two calibrated cameras recognize the marker, a three-dimensional fixation device can be obtained. Generally, the system consists of about 2-48 cameras. With more than 300 camera systems, marker exchange can be reduced. Additional cameras are required to fully capture the object and its surroundings. Since all passive markers appear the same, vendors limit software to reduce marker exchange problems. Unlike active marker systems or magnetic systems, passive marker systems do not require users to wear wires or electronic equipment. Instead, hundreds of rubber balls are attached with reflective tape and replaced periodically. Markers are usually attached directly to the skin or adhered to a full body spandex lycra suit specifically designed for motion capture. The passive marker system can capture a large amount of markers at a frame rate of about 120 to 160 fps, even if it tracks a region of interest that can lower resolution and track down to 10000 fps.

1 is a schematic diagram showing a state in which a marker is attached to a human body in a VICON system widely known among motion catcher systems. As shown in the figure, a marker in the form of a ball is attached to the human body, and the marker is a ball having a diameter of 14 mm for motion of a general size such as walking, and 9 mm or less for motion of a small size such as movement of a face or hand. It is common that a ball of the diameter of is used.

As described above, when a ball-shaped marker is attached and the object moves, the object is captured by a plurality of cameras, and the captured image is edited to represent video data. However, in the case of the ball-shaped marker, a part of the ball is blocked depending on the movement of the object, and the center of the ball that is partially blocked is different from the center of the original ball, so the captured data is different from the actual motion. I have an issue.

US 20040095352 A1 US 20040166954 A1 US 20040155962 A1 JP 17258891 A

In the present invention, when a marker is attached to a human body in a motion capture system, the markers may be partially obscured by the human body according to the movement of the human body. If a part of the marker is hidden and only part of it is recognized, the center (x) is found only with the recognized part because a point with a certain distance from the perimeter is recognized as the center. It aims to solve the problem.

In order to achieve the above object,

In the motion capture system,

A camera that captures motion of a human body to which a plurality of circular markers are attached;

A marker recognition unit that recognizes a marker from the image captured by the camera;

A marker circular restoration unit that determines whether the recognized marker is circular or not, and if it is not circular, restores the circular shape to a recognized portion according to the radius of the circular shape; And

And a marker positioning unit for determining a center of a circle as a marker position when the recognized marker is circular, and a marker positioning unit for determining a center of a circle restored by the marker prototype restoration unit as a marker position if it is not circular. Provide a system.

It provides a motion capture system further comprising a motion capture database for storing the motion capture data calculated by the marker position determining unit marker position.

In order to achieve another object of the present invention,

A step in which the camera photographs the object to be photographed by attaching the markers in the space in which the plurality of cameras 110 are disposed (S11);

Marker recognition unit 120 recognizes a marker from the camera image (S12);

When the shape of the marker recognized in the step S12 is not circular, the marker circular restoration unit 130 restores the remaining circular shape of the recognized shape using the radius of the set marker (S13); And

The marker positioning unit 140 calculates the center of the circle as a marker position when the shape of the marker recognized in step S12 is circular, and when the shape of the marker recognized in step S12 is not circular, in step S13. It provides a motion capture method comprising the step (S14) of calculating the center of the restored circular to the marker position.

When the marker position is calculated in step S14, a motion capture method is further provided, further comprising storing the motion capture data in which the marker position is calculated in a motion capture database.

According to the present invention, even if the marker is partially obscured, the entire circle is restored only with the visible part, and the sensor is found in the restored circle and calculated as a marker position, thereby significantly reducing noise due to an error in the marker position.

1 is a schematic view showing a state in which a marker is attached to a human body.
2 is a block diagram showing the configuration of a motion capture system according to an embodiment of the present invention.
3 is a view showing a marker captured on the camera.
4 is a flowchart illustrating a motion capture method according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted as meanings generally understood by a person having ordinary knowledge in the technical field to which the present invention belongs, unless defined otherwise. It should not be interpreted as a meaning or an excessively reduced meaning. In addition, when the technical term used in this specification is a wrong technical term that does not accurately represent the spirit of the present invention, it should be understood as being replaced by a technical term that can be correctly understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted as defined in the dictionary or in context before and after, and should not be interpreted as an excessively reduced meaning.

Hereinafter, an external force may be applied to the human body model according to various motions, and when capturing motion of the human body, a part of the marker is obscured, so that an error occurs in detection of the center of the marker and noise is prevented. A motion capture system and method according to embodiments of the present invention capable of realizing the motion of the human body by restoring the exact position of the center will be described with reference to FIGS. 1 to 4. The marker may be an infrared marker that reflects infrared rays or an infrared emission marker.

1 shows a state in which markers are attached to the front and rear of the human body. 2 is a block diagram showing the configuration of a motion capture system according to an embodiment of the present invention. As illustrated in FIG. 1, the subject to be moved moves while the markers are attached, and a plurality of cameras 110 disposed around the shooting space captures the movement of the subject. The marker recognition unit 120 recognizes a marker from the captured image. When moving while the markers are attached to the human body as illustrated in FIG. 1, the markers may be partially obscured by the human body according to the movement of the human body. 3 is a view showing the photographed markers. (a) represents the fully recognized state, and (b) to (d) represent the partially recognized state. Shows the hatched and shaded areas. (b) shows a state in which 50% is obscured. In the uncovered state, the center of the circle,'o', is recognized as the center of the marker. When a part of the marker is obscured and only a part is recognized, a point having a constant distance from the perimeter is recognized as the center. Therefore, the center (x) is found in the obscured part, and accordingly, an error occurs in the center position of the marker, and noise is generated in the motion. According to the present invention, when the marker recognition unit 120 recognizes the marker, it has a portion recognized by the marker circular restoration unit 130 and restores the original shape corresponding to the set radius of the marker. The marker position determining unit 140 calculates the restored center of the circle to determine the marker position, and the motion capture database 150 is constructed with the calculated marker position. The motion capture data obtained by calculating the marker position to the center of the restored marker circle may be stored in the motion capture database 150 or displayed in real time for insertion into a movie or video game.

4 shows a motion capture method according to an embodiment of the present invention. When a photographing object with markers in a space in which a plurality of cameras 110 is disposed takes motion, the camera photographs it (S11). The marker recognition unit 120 recognizes a marker from the camera image (S12 ). If the shape of the recognized marker is not circular, the marker circular restoration unit 130 restores the remaining circular shape of the recognized shape using the radius of the set marker (S13). The marker positioning unit 140 calculates the marker position to the restored circular center (S14). The motion capture data for which the marker position has been calculated is stored in the motion capture database as the center of the restored marker circle (S15).

Even if the marker is partially obscured, the entire prototype is restored with only the visible part, and the sensor is found in the restored prototype and calculated as the marker position, which significantly reduces noise due to the error in the marker position.

110: camera 120: marker recognition unit
130: marker circular restoration unit 140: marker positioning unit
150: motion capture database

Claims (4)

In the motion capture system,
A camera that captures motion of a human body to which a plurality of circular markers are attached;
A marker recognition unit that recognizes a marker from the image captured by the camera;
A marker circular restoration unit that determines whether the recognized marker is circular or not, and if it is not circular, restores the circular shape to a recognized portion according to the radius of the circular shape; And
And a marker positioning unit for determining a center of a circle as a marker position when the recognized marker is circular, and a marker positioning unit for determining a center of the circle restored by the marker prototype restoration unit as a marker position if the marker is not circular. system.
According to claim 1,
A motion capture system further comprising a motion capture database that stores motion capture data where the marker position determining unit calculates the marker position.
A step in which the camera photographs the object to be photographed by attaching the markers in the space in which the plurality of cameras 110 are disposed (S11);
Marker recognition unit 120 recognizes a marker from the camera image (S12);
When the shape of the marker recognized in the step S12 is not circular, the marker circular restoration unit 130 restores the remaining original shape of the recognized shape using the radius of the set marker (S13); And
The marker positioning unit 140 calculates the center of the circle as a marker position when the shape of the marker recognized in step S12 is circular, and when the shape of the marker recognized in step S12 is not circular, in step S13. And calculating the restored circular center as a marker position (S14).
The method of claim 3,
When the marker position is calculated in step S14, the motion capture method further comprising the step of storing the motion capture data in which the marker position is calculated in the motion capture database.

Images