KR20230014581A - Camera-attached image processing and data transmission device for optical motion capture - Google Patents

Abstract

A camera-attached image processing and data transmission device for optical motion capture according to the present invention comprises: an image input device (11) in which each of a plurality of image processing and data transmission devices (1) captures a movement of a plurality of markers (5) attached to a subject (4) and responding to infrared light; a camera-attached FPGA board (12) for post-processing an image captured by the image input device (11); and a wireless LAN (13) for transmitting the post-processed images from the FPGA board (12) to a wireless LAN (21) on a host computer (2) side by wireless communication. The camera-attached FPGA board (12) includes: a first image binarization portion (31) for binarizing and simplifying an image by treating image data related to motion of the subject (4) input from the image input device (11) as data of two colors, black and white; a low-pass filtering portion (32) for low-pass filtering the binarized motion data so as to remove noise due to measurement error and the like from the binarized image data; a second image binarization portion (33) for re-binarizing and simplifying the low-pass filtered image data; a marker labeling portion (32) for labeling and naming the plurality of markers (5), which are attached to the subject (4) and moving; and a marker center point calculation portion (33) for calculating the center points of each marker (5) by generating coordinates of each of the markers (5). According to the present invention, a plurality of cameras can be connected to a host computer by a single line.

Description

Camera-attached image processing and data transmission device for optical motion capture

The present invention relates to an image processing and data transmission device for post-processing motion data input from a camera.

In particular, post-processing of first image binarization, low pass filtering, second image binarization, marker labeling, and marker center point calculation

Camera-mounted image processing and data transmission for optical motion capture with an FPGA (or DSP) board that performs the process

It's about the sending device.

Motion capture means inputting or measuring the motions of objects, including humans, and converting them into computer data.

3D data is obtained by measuring the change over time of each joint position of a multi-joint object.

will compose

In general, in a motion capture system, a plurality of sensors or markers (Makers) for detecting motion are attached to the subject's joints.

and track or calculate the spatial position information and rotation information of the markers through a device capable of recognizing the position of each marker.

The motion capture system can be acoustic, magnetic, or mechanical depending on the operating method or principle of the sensor and recognition device.

It can be divided into mechanical and optical.

Here, the optical motion capture system attaches a plurality of markers that respond to infrared rays to a moving subject.

In addition, after taking images with multiple infrared cameras and generating 2-dimensional coordinates of each marker in each camera, each independent camera

The 2D data of markers in the camera are analyzed by software to calculate coordinates in 3D space.

On the other hand, the motion data received from the camera requires various post-processing, that is, the image input from the camera.

Image binarization (Threshold), which treats data as two-color data of black and white and binarizes it to simplify it, and binary

Low pass filtering, low pass filtering to remove noise caused by measurement errors, etc.

Image binarization that simplifies by re-binarizing the filtered image data, multiple images attached to a moving subject

Labeling of markers to name markers to identify markers, and to create and calculate coordinates of markers

There is one marker center point calculation. This process is typically performed on a work station connected to multiple cameras.

I have been processing it with the software of the Workstation computer.

In addition, the 3D motion data finally processed in the motion capture system is transmitted to the host computer.

Conventionally, this has been transmitted using Ethernet.

The present invention was invented in view of the above, and post-processing of motion data received from the camera is attached to the camera.

In addition to enabling processing by an FPGA (or DSP) board as much as possible, the final processed 3D motion data is token-ringed.

Optical motion capture that can be transmitted to a host computer using a token ring or daisy chain communication method.

Its purpose is to provide a camera-mounted image processing and data transmission device for

A camera-attached image processing and data transmission device for optical motion capture according to the present invention includes a plurality of image processing and data transmission devices.

An image in which each of the data transmission devices 1 captures the movement of a plurality of markers 5 reacting to infrared rays attached to the subject 4.

A camera-attached FPGA board that post-processes the image taken by the image input device 11 and the image input device 11

(12) And, the post-processed image on the FPGA board 12 is transferred to the wireless LAN 21 on the host computer 2 side by wireless communication.

It is equipped with a wireless LAN 13 that transmits data and transmits data, and the FPGA board 12 receives input from the image input device 11.

When simplifying by binarizing the image by treating the image data related to the motion of the corpse (4) as black and white two-color data

The key is the first image binarization unit 31 and the binarized image data in order to remove noise caused by measurement errors, etc.

The low-pass filtering unit 32 performs low-pass filtering on the motion data, and binarizes the low-pass filtered image data again.

In order to distinguish a plurality of moving markers 5 attached to the second image binarization unit 33 and the subject 4, each image binarization unit 33 is simplified by

A marker labeling unit 32 that names each marker 5 by labeling the marker 5 and creates coordinates of each marker 5

Characterized in that it is equipped with a marker center point calculation unit 33 for calculating the center point of the marker

do.

In addition, in the present invention, the FPGA board 12 is made of a DSP board and can perform the same post-processing as the FPGA board 12.

It is characterized by having.

In addition, according to the present invention, the communication method between the camera 11 and the host computer 2 is a token ring communication using a token ring 3

It is characterized by being made by the method.

In addition, according to the present invention, the communication method between the camera 11 and the host computer 2 uses a daisy chain.

It is characterized by being made by a communication method.

According to the present invention, various post-processing of motion data input from the camera, that is, first image binarization, low pass fill

By enabling the camera-attachable FPGA (or DSP) board to process processing, secondary image binarization, marker labeling, and marker center point calculation, it provides remarkable performance, small size, low power, and low cost compared to existing PC-based implementations. effect

be able to perform. In addition, synchronization caused by wired connection of multiple cameras with a host computer

You will be able to solve problems and system complexity.

Moreover, by transmitting the final processed 3D motion data using the token ring method, multiple units can be connected to one LAN line.

The camera and the host computer can be connected, reducing the number of LAN lines and wireless communication by wireless LAN.

use of is possible. In addition, the final processed 3D motion data is transmitted using a daisy chain method.

Thus, multiple cameras and a host computer can be connected with a single line.

1 shows the configuration of an optical motion capture system according to the present invention in which an FPGA board is attached to a motion capture camera.
drawing shown,
2 is a token ring communication method between a motion capture camera and a host computer of an optical motion capture system according to the present invention.
A drawing showing the expression,
3 is daisy chain communication between a motion capture camera and a host computer of an optical motion capture system according to the present invention.
drawing showing how
4 is a conceptual diagram of an optical motion capture system according to the present invention;
5 is a configuration diagram of an image processing and data transmission device 1 of a motion capture system employing an FPGA board according to the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the exemplary drawings.

1 shows the configuration of an optical motion capture system according to the present invention in which an FPGA board is attached to a motion capture camera.

is the drawing shown. The camera-attached image processing and data transmission device 1 for optical motion capture according to the present invention,

Infrared recording the movement of a plurality of markers (5; see FIG. 3) responding to infrared rays attached to each subject (4; see FIG. 3).

A line camera 11, an FPGA board 12 for post-processing an image captured by the infrared camera 11, and the FPGA

The image post-processed by the board 12 is transmitted to the wireless LAN 21 of the host computer 2 by wireless communication.

It is equipped with a wire LAN (13).

That is, the image processing and data transmission apparatus 1 according to the present invention performs various post-processing of motion data input from a camera.

, unlike the conventional method of processing with software of a workstation computer connected to multiple cameras,

It becomes possible to process with the FPGA board 12 that can be attached to (11).

Here, a field programmable gate array (FPGA) is a field programmable gate array, and a programmable logic element

It refers to a semiconductor device that includes a small and programmable internal wire. And, the programmable logic elements are AND, OR,

Programming by replicating the functionality of basic logic gates, such as XOR, NOT, and combinations of more complex decoders or computational functions.

can do. Also, most FPGAs use programmable logic elements as simple flip-flops or more complete blocks of memory.

It can also contain memory elements made of . These logic blocks and internal lines are created after the manufacturing process by the consumer/designer ("on-site program").

loggrammable") is programmable, allowing it to perform any logic function required.

2 is a token ring communication method between a motion capture camera and a host computer of an optical motion capture system according to the present invention.

It is a diagram showing the expression.

As shown in FIG. 2, the three-dimensional model finally processed in the plurality of image processing and data transmission devices 1 according to the present invention

The transaction data is connected to the host computer 2 by using a token ring method using a token ring 3.

all.

That is, the three-dimensional motion data finally processed in the plurality of image processing and data transmission devices 1 is transferred by using the token ring 3.

Several cameras and the host computer 2 can be connected via one LAN line. Therefore, the number of LAN lines

be able to reduce Furthermore, by adding a wireless LAN 13 module, a number of image processing and data transmission devices 1 and calls

Data transmission between the host computers 2 can be performed in a wireless communication manner.

3 is daisy chain communication between a motion capture camera and a host computer of an optical motion capture system according to the present invention.

This is a diagram showing the method.

That is, when there are many devices that generate interrupt requests, the request processing circuits are arranged in series, and the ones closest to the CPU are prioritized.

It is possible to enable communication in a daisy chain (serial connection) method, which is a method of processing in a linear manner.

That is, as shown in FIG. 3, a plurality of image processing and data transmission devices 1 transmit data by interruption,

It can also be used in a daisy chain method that allows communication with one daisy chain line (6), which is an interrupt request signal line.

Communication between multiple cameras and the host computer 2 becomes possible.

4 is a conceptual diagram of an optical motion capture system according to the present invention.

In the state where a plurality of markers 5 that respond to infrared rays are attached to the subject 4, when the subject 4 moves, responding to the movement

After the image is taken by multiple cameras 11 and post-processed, the image is transmitted to the host computer 2 via the wireless LAN module.

It represents the concept of sending.

5 is an image processing and data transmission device 1 of a motion capture system employing an FPGA board according to the present invention.

It is a composition diagram.

When the subject 4 moves, an image corresponding to the movement of the plurality of markers 5 responding to the infrared rays attached to the subject 4 is displayed.

The image input device 11 (e.g., a camera) captures the image, and the captured image is applied to the FPGA board 12

post-processed.

The FPGA board 12 transmits image data related to the motion of the subject 4 received from the image input device 11 in black and white.

A first image binarization unit 31 that simplifies by thresholding the image by treating it as two-color data of

Low-pass binarized motion data to remove noise caused by measurement errors in the same binarized image data

The low pass filtering unit 32 for filtering (low pass filtering) binarizes the low pass filtered image data again to

The second image binarization unit 33 for simplifying, each marker to distinguish a plurality of moving markers 5 attached to the subject 4

The marker labeling unit 32 for designating the name of each marker 5 by labeling (5) and the coordinates of each marker 5

and a marker center point calculator 33 for generating and calculating the center point of the marker.

Subsequently, the motion data post-processed in the FPGA board 12 is transmitted through a host via an image output device 13 (eg, wireless LAN).

It is output and transmitted to the host computer 2 by a wireless communication method.

Meanwhile, in the above description, the FPGA board 12 was described as an example as a post-processing device, but instead of the FPGA board 12, the first image

Functions of binarization, low-pass filtering, second image binarization, marker labeling, and marker center point calculation are equally performed.

Even if a DSP (digital signal processor) board is adopted, the same action and effect can be exhibited.

Claims (4)

Each of the plurality of image processing and data transmission devices 1, the movement of the plurality of markers 5 responding to the infrared rays attached to the subject (4)
An image input device 11 for taking a picture of a job, and a camera for post-processing the image taken by the image input device 11
A wire-attached FPGA board 12, and the image post-processed on the FPGA board 12 is transferred to the wireless LAN 21 on the side of the host computer 2.
It is equipped with a wireless LAN 13 that transmits by wireless communication as
The FPGA board 12,
Image data related to the motion of the subject 4 received from the image input device 11 is converted into black and white two-color data.
A first image binarization unit 31 that binarizes and simplifies the image by processing;
A low-pass filter is applied to the binarized motion data to remove noise caused by measurement errors in the binarized image data.
A low-pass filtering unit 32 for ringing;
a second image binarization unit 33 for binarizing and simplifying the low-pass filtered image data again;
In order to distinguish a plurality of moving markers 5 attached to the subject 4, each marker 5 is labeled so that each marker 5 is identified.
A marker labeling unit 32 for designating a name;
It has a marker center point calculation unit 33 for generating coordinates of each marker 5 and calculating the center point of the marker.
A camera-attached image processing and data transmission device for optical motion capture, characterized in that it is made of Chuo. The method of claim 1, wherein the FPGA board 12 is made of a DSP board and can perform the same post-processing as the FPGA board 12.
A camera-attached image processing and data transmission device for optical motion capture, characterized in that there is. The token ring communication method according to claim 1, wherein the communication method between the camera (11) and the host computer (2) uses a token ring (3).
Camera-mounted image processing and data transmission device for optical motion capture, characterized in that made by the equation. The method of claim 1, wherein the communication method between the camera (11) and the host computer (2) is a daisy chain communication method using a daisy chain.
Camera-attached image processing and data transmission for optical motion capture characterized by being made by a new method
Device.

Images