JP2003514298A - How to capture motion capture data - Google Patents

Abstract

(57) Abstract: A motion capture data capturing method capable of processing a reduced amount of data by extracting and processing only coordinate values that change according to motion. The present invention relates to a method for capturing motion capture data. In this method, the amount of data indicating movement can be reduced, and natural movements can be easily displayed in video production fields such as game character production and digital animation production. Because there are no connecting cables, the data grantor can act and move without bounds. A plurality of color pointers having different color values are respectively attached to the main joint and the tip of the body of the data giver. Three digital cameras (100, 200 and 300) or four or more cameras are used to capture the action and movement of the data grantor. The image obtained by the digital camera is converted into X, Y and Z coordinate values using a computer. The body movement is extracted as a three-dimensional position coordinate value based on the movement of the color pointer.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to capturing motion capture data, and more particularly, to capturing motion of a human or animal using color markers, and generating 3D motion data based on the captured motion. The capture method of.

[0002]

[Prior art]

With the development of the movie and game industries, three-dimensional modeling, expression, and animation technologies have been developed, and attention has been focused on the technology capable of creating natural movements.

In order to respond to such an interest, conventionally, the movement of a person is directly captured, the captured movement is numerically expressed, the obtained movement data is input to a computer, and the input data is input. A motion capture technology for moving characters based on is proposed.

The most important thing in this motion capture technique is to quickly capture data about natural body movements.

The most common motion capture system is to attach a marker or a sensor to a human body, analyze output data from an image of the sensor or the marker picked up by a camera, and analyze the output data of each joint of the human body. The position and direction of the section are measured.

Such a general motion capture system can be classified into four types. That is, acoustic type, mechanical type, magnetic type, and optical type. These classifications are
It depends on the type of operation of the marker or sensor used. Of these, magnetic and optical systems are currently most widely used.

The various types of known motion capture systems described above have the following characteristics and problems.

An acoustic motion capture system has multiple acoustic sensors and three acoustic receivers. The acoustic sensor is attached to each joint of the performer to continuously generate an acoustic waveform, and the acoustic receiver is attached to a position away from the acoustic sensor to generate an acoustic waveform by the sensor. To receive. The motion capture system uses the time it takes for the receiver to receive the acoustic waveform from the sensor to calculate the distance from the acoustic sensor to the acoustic receiver.
The position of each sensor in the three-dimensional space is obtained based on triangulation using the values respectively calculated by the three receivers. This acoustic system suffers from low sampling frequency and a small number of acoustic generators available at the same time. In addition, the large size of the sensor makes the movement of the performer unnatural. In addition, this acoustic motion capture system, due to the characteristics of its acoustic configuration,
It is significantly affected by acoustic reflection.

A mechanical motion capture system consists of a potentiometer in combination with a slider that measures the movement of the performer's joints. Since this motion capture system has no receiver, it provides an absolute measuring means without environmental disturbances or adverse effects. Thus, compared to acoustic type systems, this mechanical system rarely requires initial calibration and does not require expensive studio equipment. Furthermore, mechanical systems are cheaper than other motion capture systems and have a very high sampling frequency. However, the mechanical motion capture system has a drawback that the movement of the performer becomes unnatural because the mechanical unit is attached to the performer. Also, the measurement accuracy of the mechanical system depends on how accurately the mechanical unit is attached to each joint of the performer.

A magnetic motion capture system attempts to detect a magnetic field by attaching a plurality of sensors to each joint of the performer. As the performer moves around the magnetic field generator, sensors detect changes in the magnetic field. This change in magnetic field is then converted into a spatial value for motion measurement. The sensor, magnetic field generator and system body are interconnected by a cable or, more recently, wirelessly. Magnetic motion capture systems are advantageous in that they are relatively inexpensive, have a good cost-to-performance ratio, and are capable of real-time processing. However, if the sensor is wired, the performer must move only to a limited extent due to the condition of the cable from the sensor to his or her body. As a result, it is difficult for the performer to naturally display his or her complex and rapid movements. Even in the case of wireless systems that use wireless transmitters instead of cables, the performers are not able to exercise naturally. This is due to the large size of the transmitter attached to his or her body. Furthermore, the performer can only move within a limited range, as he or she must be in the area of the magnetic field.

The optical motion capture system has a plurality of infrared-reflecting reflective markers attached to the main joints of the performer and a plurality of cameras each having 3 to 16 infrared filters attached. There is. The camera is adapted to form the two-dimensional coordinates of the reflected marker image. A special program is used to analyze the two-dimensional coordinates captured by the camera, and as a result of the analysis, the coordinates are converted into three-dimensional spatial coordinates. This traditional optical motion
The disadvantage of capture systems is that they focus on the realistic representation of character movements, and as a result require a very large amount of data and a fast computer to process it.

[0012]

[Problems to be Solved by the Invention]

The present invention has been made in view of the above problems, and an object of the present invention is to process a reduced amount of data by extracting and processing only coordinate values that change according to movement. There is a method for capturing various motion capture data.

Another object of the present invention is to provide a method for capturing motion capture data, in which initial calibration for motion capture can be performed very easily and conveniently.

Yet another object of the present invention is to use low cost personal computers, camcorders and USB cameras that process a small amount of data, but without the need for complex or expensive peripherals, thus reducing costs. There is a need to provide a method of capturing motion capture data that can implement a more efficient system.

Another object of the present invention is that there is no cable connection between the performer and the motion data processing system, and thus the motion capture capable of conveniently extracting the motion including the violent action of the performer. -Provides a method for capturing data.

Still another object of the present invention is to provide a method of capturing motion capture data, which can accurately measure the motion of a performer regardless of the surrounding environment such as ambient lighting and radio wave interference. It is in.

[0017]

[Means for Solving the Problems]

The invention of claim 1 is a method of capturing motion capture data, which includes the following steps.

A) Attaching a plurality of color pointers having different color values to the main body part of the performer.

[0019]   b) a step of taking a picture of the movement of the performer by means of a camera.

C) The image data picked up by the camera means is converted into a three-dimensional coordinate value (
X, Y, and Z), detecting the position change of the color pointer in each body part of the performer from the converted three-dimensional coordinate value, and further detecting the position change of the color pointer from the detected position change. A step of extracting three-dimensional position coordinate values based on.

A second aspect of the present invention is the method of capturing motion capture data according to the first aspect, wherein the camera means is at least three cameras, and the cameras are arranged at equal distances from each other. A plurality of cameras arranged equiangularly, the three cameras being a front camera, a left camera and a right camera arranged around the performer,
The step c) includes the following steps.

Vertical movement (Z coordinate) data is captured from all three of the cameras, horizontal movement (X coordinate) data is captured from the front camera, and captured horizontal direction data and left and right cameras are obtained. Capturing forward and backward motion data by performing a predetermined calculation algorithm on the acquired data.

Capturing data for each dead zone of the camera by performing a parallel computing algorithm on complementary data from the camera and creating three-dimensional data based on the captured dead zone data.

According to the present invention, there is provided a method for capturing motion capture data in a motion capture system capable of recording the movement of a person or an object in a form processable by a computer. The purpose can be solved. This method is based on various technical requirements. That is, a color extraction calculation method, a color position tracking calculation method, a technique of tracking the motion of the performer using a plurality of color markers having different color values, and a USB camera for recognizing 24 to 30 frames of video per second. It is a video processing technology and a technology that converts the extracted 2D data into 3D data.

This motion capture technology has a basic interface function of capturing data relating to the movement of the character before realistically expressing the movement of the character.

Given that the motion capture technique is a kind of interface method, the present invention proposes a method that can easily capture data regarding the movement of the performer.

[0027]

BEST MODE FOR CARRYING OUT THE INVENTION

As shown in FIG. 1 and FIG. 2, a plurality of color pointers having different color values are attached to the main joint part and the tip part of the body of the data grantor, and three or four or more cameras are arranged at equal intervals. And are separated from each other, and in this way, the actions and movements of the data grantor are photographed. Using a personal computer, the video data picked up by the camera is converted into three-dimensional coordinate values (X, Y, and Z), and from the converted three-dimensional coordinate values, the body part of each person of the data grantor is converted. The position change of the color pointer is detected, and the detected position change is stored as motion capture data.

The computer receives the video signal from the camera via the buffer, and extracts only the position coordinate value relating to the movement of the color pointer in each part of the body of the data grantor from the received video signal. Since the computer does not extract and store all the video signals but only the position coordinate values that change according to the movement of the color pointer in each body part,
The amount of data to be processed is reduced, resulting in increased processing speed.

Next, a procedure for capturing motion and extracting three-dimensional motion data from the captured image according to the present invention will be described.

The following Table 1 shows an example in which the basic 13 colors of the 256 colors are assigned to the main part of the performer's body.

[0031]

[Table 1] The resolution was set to 160 pixels × 120 pixels, the color depth was set to 16 bits, and the number of frames was set to 30 frames / second. N represents the number of color pointers.

In the above state, the camera extracts the three primary color components, that is, the red, green and blue components from the input image and transfers them to the personal computer (PC).

At this time, the data transferred from the camera to the PC is 160 pixels × 120 pixels × 1
6 bits × 30 frames / second = 9216000 bits / second. Because it can be expressed as the product of resolution, color depth and number of frames). By calculating this value in bytes, the result obtained is 115200.
0 bits / sec. Furthermore, by calculating this in kilobytes,
The result obtained is 1152 Kbytes / sec. In conclusion, 1152Kb per second
The ytes data will be transferred from the camera to the computer.

Since the amount of data calculated as described above is associated with each of the red, green and blue elements, the amount of data captured for all three elements is the calculated amount of data described above. Is three times. That is, 1152000 × 3 = 3456000 bytes /
Seconds. Calculating this for megabytes, the result is 3.456Mbytes /
Seconds. Finally, the total amount of data to be processed by the computer is red,
Obtained by multiplying the amount of data captured for all green and blue components by the number of color pointers.

That is, the total amount of data to be processed in the computer is 3.456 Mbytes / sec when N = 1 and 34.56 Mbytes when N = 10.
/ Sec.

The coordinate system shown in FIGS. 1-9 is an absolute coordinate system that can be used in computer three-dimensional (3D) graphics and animation techniques. In the front view of the coordinate system shown in FIGS. 1 to 9, the Z-axis is arranged in the vertical direction, the X-axis is arranged in the left-right direction, and the Y-axis is arranged in the front-rear direction.

Hereinafter, a method of capturing motion capture data according to the present invention will be described in detail with reference to the drawings.

First, three or four or more cameras capture the actions and movements of the data grantor, and transfer the obtained images to their associated computers. The computer extracts the position changes of the plurality of color pointers respectively attached to the main joint part and the tip part of the data grantor's body from the image transferred from the related camera, and from the position changes extracted in this way. Obtain two-dimensional motion data coordinate values.

Next, the computer transfers the obtained two-dimensional motion data coordinate value to the server computer via the internal network. This server computer
Next, the transferred coordinate values are converted into three-dimensional motion capture data by a combining program based on a three-dimensional combining function engine.

FIG. 1 shows the configuration of a system for executing the method for capturing motion capture data according to the present invention. As shown, three or four or more cameras capture the motion of the data grantor and transfer the resulting images to their associated computers. The computer processes the images transferred from the corresponding cameras by the above-described calculation method, and transfers the processed results to the server computer via the internal network. This server computer obtains three-dimensional motion capture data based on the data transferred from the computer.

FIG. 2 shows a camera configuration for capturing three-dimensional data according to the present invention. As shown in the figure, three cameras, that is, a front camera 100, a left camera 200, and a right camera 300 are arranged around an object or a data giver.

The above-mentioned three cameras are arranged at equal angles as follows. That is, the cameras are spaced apart from each other at a constant angle of 120 °. This 3
A virtual area is set in a predetermined range around the intersection of the fields of view of the cameras.
This is recognized as a coordinate system by the computer.

If three cameras are arranged as shown in FIG. 1, these cameras are
Get the Z coordinate values on all sides of the object. The X and Y coordinate values of the object are obtained as follows. That is, a change in the X coordinate value obtained by the front camera 100 and + X, −Y, + Y and − obtained by the cameras 200 and 300 on the left and right sides.
Obtained by running the appropriate calculation algorithm on the Y data.

FIG. 3 illustrates the detection state of the X and Z coordinate values by the front camera 100 in FIG. All X and Z position coordinate values of the pointer on the front side of the object are detected by the front camera 100 and then given as reference values for comparison with the data detected by the left and right cameras 200 and 300. . The X and Z coordinate values detected by the front camera 100 are also provided as reference values for correction of the data detected by other cameras.

FIG. 4 is a schematic diagram showing a detection state of X (30 °), Y (60 °) and Z coordinate values by the left camera 200 in FIG. The Z coordinate values and the X and Y coordinate values of the pointers on the left side and the back side of the object are detected via the left camera 200. The detected X and Y coordinate values are rotated based on the detected Z coordinate values and then provided as reference values for comparison with the data detected by the front camera 100 and the right camera 300.

FIG. 5 is a schematic diagram showing a detection state of Y (60 °), X (30 °) and Z coordinate values by the right camera 300 in FIG. The Z coordinate values and the X and Y coordinate values of the pointers on the right side and the rear side of the object are detected via the right camera 300. The detected X and Y coordinate values are rotated based on the detected Z coordinate values and then provided as reference values for comparison with the data detected by the front camera 100 and the left camera 200.

FIG. 6 shows a virtual motion area of an object of an object whose movement is to be captured (
It is a figure showing (X and Y coordinate system). This virtual area has three cameras 10
A constant range is set around the intersection of the 0, 200 and 300 fields of view, limiting all movement of the object to within this virtual area. The dead zone of each camera is detected through the intersecting field of view of the other two cameras, and the accurate coordinate values are obtained by proof of the calculation and correction.

[0048]   The dead zones for the three cameras are shown in FIGS. 7-9, respectively.

The coordinate values of the pointer located in the dead zone of the front camera 100 of FIG. 7 are compared with the data detected by the left and right cameras 200 and 300, and detected by the front camera 100 based on the result of the comparison. Determined by proving and correcting the data provided.

The coordinate values of the pointers in the dead zone of the left camera 200 shown in FIG. 8 are compared with the data detected by the front and right cameras 100 and 300, and the left camera is based on the result of the comparison. Determined by proving and correcting the data detected by 200.

The coordinate values of the pointer located in the dead zone of the right camera 300 in FIG. 9 are compared with the data detected by the front and left cameras 100 and 200, and the right camera 300 is based on the result of the comparison. Is determined by proving and correcting the data detected by.

[0052]

【The invention's effect】

As is apparent from the above description, the present invention provides a method of capturing motion capture data that is capable of extracting and processing only color pointer data that is indicative of a performer's movements. Therefore, the amount of data to be processed is extremely small, and as a result, the processing speed is improved as compared with the case of processing the entire video. In addition, the use of low-cost personal computers, camcorders and USB cameras eliminates the need for complex and expensive peripherals, resulting in very easy initial calibration for motion capture anywhere. Further, it is possible to realize a more cost-effective system that can be conveniently performed.

Furthermore, there is no cable connection between the performer and the motion data processing system, only a wireless connection, so there is no limit to the area in which the performer can move, and therefore
It is possible to conveniently extract the movement of the performer, including the violent action.

Furthermore, the overall execution of the system can be easily achieved by simply upgrading the software, compared to the improvement of normal execution based on hardware.

Further, as a result of these features, the present invention can be widely used in entertainment, virtual reality and other video production fields.

While the preferred embodiments of the invention have been described above by way of example, those skilled in the art can make various modifications to the invention without departing from the scope and spirit of the invention described in the appended claims. It will be understood that additions and substitutions can be made.

[Brief description of drawings]

FIG. 1 shows the configuration of a system for executing a method for capturing motion capture data according to the present invention.

[Fig. 2]   3 illustrates a camera configuration for capturing 3D data according to the present invention.

FIG. 3 is a schematic explanatory diagram of a detection state of X and Z coordinate values by the front camera in FIG.

FIG. 4 is a schematic diagram showing a detection state of X, Y, and Z coordinate values by the left camera in FIG.

5 is a schematic diagram showing a detection state of X, Y, and Z coordinate values by the right camera in FIG.

[Figure 6]   3 illustrates a virtual motion region of an object whose movement is to be captured.

FIG. 7 shows a dead zone of the front camera in FIG. 2 and a data detection state in this dead zone.

FIG. 8 shows respective dead zones of the camera on the left side of FIG. 2 and shows a data detection state in the dead zones.

FIG. 9 shows respective dead zones of the camera on the right side of FIG. 2 and shows a data detection state in the dead zones.

[Explanation of symbols]

100 front camera 200 Left camera 300 right camera

─────────────────────────────────────────────────── ─── Continued front page    (81) Designated countries EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, I T, LU, MC, NL, PT, SE), OA (BF, BJ , CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, GM, K E, LS, MW, MZ, SD, SL, SZ, TZ, UG , ZW), EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AE, AG, AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, BZ, C A, CH, CN, CR, CU, CZ, DE, DK, DM , DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, K E, KG, KP, KZ, LC, LK, LR, LS, LT , LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, NO, NZ, PL, PT, RO, RU, S D, SE, SG, SI, SK, SL, TJ, TM, TR , TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW [Continued summary] Extracted.

Claims (2)

[Claims] 1. A method of capturing motion capture data, the method comprising the steps of: a) Attaching multiple color pointers with different color values to the main body part of the performer. b) a step of taking a picture of the movement of the performer by means of camera. c) The image data picked up by the camera means is converted into a three-dimensional coordinate value (
X, Y, and Z), detecting the position change of the color pointer in each body part of the performer from the converted three-dimensional coordinate value, and further detecting the movement of the color pointer from the detected position change A step of extracting three-dimensional position coordinate values based on. 2. The method of capturing motion capture data according to claim 1, wherein the camera means is at least three cameras, and the cameras are equiangularly arranged at equal intervals. A camera having a structure, wherein the three cameras are a front camera, a left camera and a right camera arranged around the performer, and the step c) includes the following steps. Vertical motion (Z coordinate) data is captured from all three cameras, horizontal motion (X coordinate) data is captured from the front camera, captured horizontal data and data obtained from the left and right cameras. Capturing forward and backward motion data by executing a predetermined calculation algorithm for. Capturing data for each dead zone of the camera by performing a parallel computing algorithm on complementary data from the camera and creating three-dimensional data based on the captured dead zone data.