WO2021062500A1

WO2021062500A1 - System and method for capturing movement

Info

Publication number: WO2021062500A1
Application number: PCT/BR2019/050431
Authority: WO
Inventors: Flavio DINI ALVES COUTINHO
Original assignee: Dini Alves Coutinho Flavio
Priority date: 2019-10-03
Filing date: 2019-10-03
Publication date: 2021-04-08

Abstract

The invention relates to movement tracking for an object or parts of an object in a three-dimensional space, in particular of the human body, formed by two sets of essential elements, the first comprising a group of receivers forming a static base and the second comprising one or more "beacon" or "tag" transmitters associated with the object to be tracked. The invention involves the reception by each receiver in the base of the signals transmitted continuously by each at least one beacon or tag, in which the position of said at least one beacon or tag is determined by trilateration, on the basis of the distance between each such receiver and each such at least one beacon or tag. The determination of each such distance can be based on the intensity of the signal received by each receiver or on the travel time of the signal sent by the tag and the receipt thereof by each of said receivers.

Description

SYSTEM AND METHOD FOR MOVING CAPTURE

Field of invention

[001] Refers to the present invention for tracking the movement of a composite object or parts of an object in a three-dimensional space. In particular, the invention relates to a system designed to track the movements of a human body.

Background of the Invention

[002] The measurement of physical movement with high resolution is important for many medical, sports and ergonomic applications. In addition, in the film and computer games market, there is a great need for motion data for the purpose of advanced animation and special effects. Finally, motion data is required in Virtual Reality (VR) and Augmented Reality (AR) applications for training and simulation.

[003] There are several technologies currently available to track and record data related to movement. Within an overview of the subject, the existing motion capture systems can be classified as external-internal, internal-external and internal-internal systems, names that are indicative of the positioning of the capture sources and sensors.

[004] An external-internal system uses external sensors to collect data from sources placed in the body. Examples of such systems are camera-based tracking devices, in which cameras are the sensors and reflective markers are the sources.

[005] The internal-external systems have sensors placed on the body that collect data from external sources. Electromagnetic systems, whose sensors move in an externally generated electromagnetic field, are examples of internal systems.

[006] The internal-internal systems have their sources and sensors placed on the body. Examples of such devices are electromechanical suits, in which the sensors are potentiometers and the sources are the actual joints within the body.

[007] The main technologies used today that correspond to these categories are optical, electromagnetic and electromechanical tracking systems.

[008] Optical motion capture systems are very accurate to capture certain movements when used in conjunction with state-of-the-art equipment and means. Its operation takes place in real time, and has some limitations, related to the number of markers and the number of artists and cameras.

[009] A typical optical motion capture system is based on a single computer that controls the input of multiple CCD digital cameras. Such cameras contain an array of pixels that typically have resolutions ranging from 128 x 128 image elements to 4096 x 4096 or even higher.

[0010] Higher resolutions result in sharper images. However, this result is obtained at the expense of a longer processing time: in the current technique, a CCD camera with a resolution of 4096 x 4096 produces about one frame per second.

[0011] Such systems generally employ between 4 and 32 cameras, at a sampling rate ranging from 30 to 1000 samples per second. At cameras are usually equipped with their own light sources that create a reflection of the markers, which are usually spheres covered with a reflective material such as Scotch-Brite tape. Infrared light sources are preferred because they create less visual distortions for the user.

[0012] To calibrate the optical system, all cameras must track an object with known dimensions, easily identified by the software, such as a cube or a ruler calibrated with reflective markers. By combining the views of all cameras with the known dimensions of the object, the exact position of each camera in space can be calculated. Every time you change the position of a camera, due, for example, to an accidental impact, a new calibration must be performed.

[0013] Once the camera views have been calibrated, they are digitized. Processing of the data captured through the computer begins. The first step in the process is to obtain a clean reproduction that shows only the markers. Different image processing methods are used to minimize noise and isolate the markers, separating them from the rest of the environment, for example, considering only the groups of pixels that exceed a predetermined limit of luminosity.

[0014] The second step is to determine the two-dimensional coordinates of each marker for each image captured by the camera. These data will later be used in combination with the coordinates of the camera and the rest of the views of the other cameras to obtain the three-dimensional coordinates of each marker.

[0015] The third step is to monitor the variation of the position of each marker along a sequence of images. This step requires the greatest assistance from the operator, since the initial assignment of each bookmark must be registered manually. After this task, the software tries to resolve the rest of the sequence until it loses control of a marker due to blocking the image or crossing with another marker. This requires further intervention by the operator who must reassign the markers in question in order to continue computing. This process continues until the entire sequence is resolved and a file containing positioning data for all markers has been saved. Such a file contains a sequence of global marker positions over time, which means that only the Cartesian coordinates (x, y and z) of each marker are listed by frame and no hierarchy or member rotation is included.

[0016] Electromagnetic motion capture systems are part of the family of six degrees of freedom electromagnetic measurement systems and consist of a series of receivers that measure their spatial relationship with a nearby transmitter. These receivers or sensors are placed on the body and are connected to an electronic control unit, in most cases by individual cables.

[0017] Also called magnetic trackers, these systems emerged from the technology used in military aircraft for helmet mounted displays (HMD - Head Mounted Display). With HMDs, a pilot can acquire a target by visually locating it through a reticle located on the visor. A typical magnetic tracker consists of a transmitter, 11 to 18 sensors, an electronic control unit and software. To take advantage of the real-time capabilities of a magnetic tracker, it must be connected to a powerful computer system that is capable of rendering a large number of polygons in real time. Depending on the needs of the project, the cost of this computer alone may exceed the cost of the magnetic tracker.

[0018] The transmitter generates a low frequency electromagnetic field which is detected by the receivers and entered into an electronic control unit, where it is filtered and amplified. Then, it is sent to a central computer, where the software resolves the position of each sensor in Cartesian x, y and z coordinates and orientation (yaw, tilt and roll). This data is manipulated using another algorithm that converts the orientation and the global position of each sensor into a hierarchical chain with only one position and multiple rotations. Magnetic trackers have a specification called latency, which indicates the amount of time elapsed between data collection and the display of the resulting performance. This specification can vary from a few milliseconds to a few seconds.

[0019] Some transmitters generate direct current (DC) electromagnetic fields, while others use alternating current (AC) fields. Both technologies have different problems associated with the presence of conductive metals in the capture area. AC trackers are very sensitive to aluminum, copper and carbon steel, but not as sensitive to stainless steel or iron, while DC trackers have problems with ferrous metals, such as iron and steel, but not with aluminum and copper.

[0020] Electromechanical garments use potentiometers or angular measurement devices located in the main human joints. These are devices that use technology that has been used for many years in the electronics industry, in applications such as volume controls on radios and amplifiers. Basically they comprise a cursor that moves along a resistance element, producing at the output a variable voltage that depends on the relative position of the cursor. The potentiometers used in motion capture suits and armor are much more complex versions of the old volume controls. They are usually called angular, analog or digital sensors.

[0021] A major disadvantage of electromechanical systems based on in pots is the inability to measure global translations. In most cases, an electromagnetic sensor is added to the set of sensors to solve this problem, which introduces the aforementioned disadvantage of interferences produced by conducting metals in the capture area but which subjects the installation to the same disadvantages as electromagnetic systems, such as sensitivity to nearby metals. In addition, most of these devices only work on simple joints, and are not suitable for sensing complex joints, such as the shoulder.

[0022] There are still motion tracking systems based, at least in part, on radio frequency (RF). These systems typically employ the use of tags that serve as receivers and transmitters. The sensors are placed around a capture area and that sensor also transmits and receives signals. The sensors transmit signals that are picked up by the tags on the objects, producing response signals that allow for individual tracking.

[0023] It is evident, for the technicians in the subject, that the complexity and the cost of such systems increase when the tags and the sensors are transceivers. In addition, known systems require time synchronization between tags and sensors, or at least between multiple sensors, which increases the complexity of the system. In addition, systems that employ radio frequency have an accuracy limited to a few centimeters.

[0024] The patent document US2004017313, entitled Motion Tracking System and Method describes a system and method based on radio frequency in which it seeks to correct the drawbacks of known systems. According to that document and as illustrated in Fig. 1 and in Figures 2-a and 2-b, at least 4 sensors 14 are used delimiting the capture zone 20. Preferably said sensors are positioned at the vertices of a cube, as illustrated in figures 2-a and 2-b, the latter illustrating a preferred configuration with 8 sensors. In addition to the tags 12 that are placed at various points on the object 18 to be tracked, a reference tag (not shown) is placed in said capture zone.

[0025] According to this document, signals are periodically transmitted in the form of bursts, from the reference tag and the object tags 12. These signals are received at the sensors, the identification code and the carrier and code phases, corresponding to each signal, are extracted. The code and carrier phase are processed to determine the position of the tags in relation to the reference tag.

[0026] The system described in this document operates in the range of 5.725-5.850 GHz, with a bandwidth of 125 MHz, with the power transmitted by the tags (ERP) limited to 0.25mW.

[0027] Among the drawbacks of this system is the fact that it requires a relatively high number of sensors: at least 4 and, preferably, 8. Another disadvantage is related to the need for a reference tag within the capture zone.

Objectives of the Invention

[0028] In view of the above, the invention has now been proposed with the objective of providing a movement tracking system that is simpler than the systems known in the current technique.

[0029] Another objective is to dispense with the need for a reference label.

Summary Description of the Invention

[0030] The objectives mentioned, as well as others, are achieved by invention by providing a system comprising two sets of essential elements, the first comprising a group of receivers forming a fixed base, and the second comprising one or more transmitters "beacons" or "tags" associated with the object to be tracked.

[0031] According to another characteristic of the invention, said tags transmit radio frequency signals.

[0032] According to another feature of the invention, said tags transmit ultrasound signals.

[0033] According to another characteristic of the invention, the receivers that form said fixed base are arranged at the vertices of a triangle with known dimensions.

[0034] According to another feature of the invention, the system comprises additional triangular bases, not coplanar with the first fixed base.

[0035] According to another feature of the invention, the system comprises means of data processing provided with a position tracking algorithm.

[0036] According to another characteristic of the invention, the method of the invention comprises the reception, by each receiver of the base, of the signals transmitted continuously by each at least one beacon, where the position of said at least one beacon or tag is determined by means of trilateration based on the distance between each said receiver and each said at least one beacon or tag. [0037] According to another characteristic of the invention, the determination of said each distance is based on the intensity of the signal received by each receiver. [0038] According to another characteristic of the invention, the determination of said each distance is based on the travel time of the signal transmitted by the beacon and its reception by each one of said receivers.

Description of the Figures

[0039] The other characteristics of the invention will be better understood by describing an exemplary non-limiting embodiment of the same, and the figures that refer to it, in which:

[0040] Figure 1 illustrates a system known from the state of the art.

[0041] Figures 2-a and 2-b show the positioning of the sensors delimiting the capture zone in the known system illustrated in the previous figure.

[0042] Figure 3 exemplifies a suit worn by the user, provided with a plurality of beacons positioned at key points.

[0043] Figure 4 illustrates the basic configuration of the proposed system of the invention.

[0044] Figure 5 shows the operation of the position tracking algorithm on which the invention is based.

[0045] Figure 6 is a general flow chart of the system.

[0046] Figure 7 exemplifies the spreadsheet with ID x coordinates.

Detailed Description

[0047] The system is composed of two essential physical parts, one being group of transmitters (beacons or tags), and a group of receivers. As shown in Fig. 3, transmitters 22 are distributed at key points in the user, usually attached to clothing 21, which constitutes a means of distributing transmitters in a standardized and efficient way, however, other methods can be used to said fixation.

[0048] As shown in Fig.4, the A-B-C receptors are arranged on a triangular base, with the sides having 1m. This base must be installed in a fixed position in the space to be used.

[0049] The transmitters 22 or beacons or tags, are radio frequency transmitters, using any type of technology, such as, for example, bluetooth, Wi-Fi, Zig-Bee or any other known and commercially available technology.

[0050] According to the method of the invention, there are two ways to determine the distances between at least one beacon or tag 22 and each of the A-B-C receptors. The first is based on the signal strength captured by each A-B-C receiver, being able to provide results with millimeter accuracy. However, the interposition of obstacles between the signal source and the receiver may impair the accuracy of the result, depending on the frequency range of the RF signal used.

[0051] The second form is based on the signal travel time, and its precision depends on the resolution as the speed of propagation of the electromagnetic wave. Such speed depends on the local atmospheric pressure, temperature and humidity. Its calibration can be done by measuring the travel time between the AB, AC or BC transmitters, since the distance between them is known. Therefore, the accuracy of the measurement of distances 22-A, 22-B and 22-C will depend only on the resolution of the measurement of the propagation time. So, for example, a resolution of 0.1 nanosecond will result in a spatial resolution of 3cm, and 0.01 nano according to a resolution of 3mm.

[0052] If ultrasound signals are used instead of RF signals, it will be possible to determine distances with much greater precision. Thus, for example, a resolution of 1 millisecond in the reception time will provide a spatial resolution, the measurement of the distance, of a fraction of a millimeter. Obviously, the presence of obstructions between the source and the receiver may introduce an error in the aforementioned determination. This can be avoided by using additional bases located in different positions.

[0053] A variant of the second form is based on the determination of the phase of the received signal, where the burst transmitted by the beacon or tag comprises a standardized preamble, which can be, for example, the already known Neuman-Hofman synchronization code, of 16 bits, namely (0000111011101101), in widespread use due to their excellent self-correlation properties.

[0054] Once the distances between the at least one beacon or tag and the A-B-C receivers have been determined, the trilateration method is used to identify the position of the transmission in the x, y, z coordinates.

[0055] Once the real coordinates Rx, y, z are determined, they are communicated to the computer that will calculate the virtual coordinates Vx, y, z that are stored in a database. Such storage uses a spreadsheet with ID x coordinates, which is constantly updated. This spreadsheet is accessed by the program that is using the system, in order to position the virtual markers in their respective positions in Vxz, which are used to build the avatar image. [0056] The detailed operation of the system will be described below, based on the flowchart of Fig. 6.

[0057] When starting, transceivers B and C send signals to the base, which received, having the distance between their points known, makes it possible to calibrate the base in relation to the speed of the wave. With the wave speed calculated for the moment, the position tracking process starts, where at least one beacon or tag sends a signal to the base containing two pieces of information, namely, its identifier (ID) and the time it was sent . Considering the speed of the wave (previously and constantly calculated from the distance between the points of the base) and the time when the signal left the tag and was received by the points, the distances TA, TB and TC are found (these being the distances between the tag and the fixed points of base A, B and C). Such distances are then used in the following system of equations:

R.x = ((Tda * Tda) - (Tdb * Tdb) + (1 * 1)) / 2;

R.z = ((Tda * Tda) - (Tdc * Tdc) + (1 * 1)) / 2;

Where it comes from:

R.y = Mathf.Sqrt ((Tda * Tda) - (T.x * T.x) - (T.z * T.z));

Since the values of T.x and T.z are numerically equal to R.x and R.z, respectively, determined above.

[0058] Since Tda is the relative distance between the tag and point A of the base; Tdb, the relative distance between the tag and point B of the base and Tdc, the relative distance between the tag and point C of the base.

[0059] The equations define the Rxyz points and place them in a vector (array) containing four columns, and lines related to the number of tags being tracked, with these data arranged as follows, for each identifier:

I x I Y I z

T | Tx | Ty | Tz

[0060] The program that uses the system must then update itself in relation to the vector and add Txyz to its respective Vxyz. [0061] All steps performed by the system are described using the following sequence:

1 - the base starts.

2 - points B and C send signal to point A.

3 - the base calculates the relative wave transit time between AB and AC by defining the wave speed.

4 - the base waits for a signal from at least one tag.

5 - if Tda! = NaN, Tdb! = NaN and Tdc! = NaN (that is, if the tag identification signal was received by all points of the base, so that the value of the three distances can be written in numerical form), the base calculates TRxyz; if not, the base waits for a signal.

6 - having TRxyz calculated, the base stores a vector with tag data calculated according to the spreadsheet in Fig. 7.

7 - the receiving program captures vector data and places it in its respective Vxyz.

[0062] For the system to work, a minimum of 3 base points are required, fixed in a fixed way with predefined distances. The base must have sufficient processing capacity to determine the reception time with sufficient resolution (minimum of two nanosecond decimal places for radio frequency, defining an accuracy of up to 3 millimeters) and to perform the calculations related to the tags in at least 90 times a second. The base must have sufficient storage capacity to store all arrays of information regarding the tracked tags.

[0063] A grouping of tags needs sufficient processing capacity to identify the tag in question and its transmission time, with a resolution compatible with the needs of the system (also adding the transit time of the pulse through the conducting medium between processor and tag , this distance being known) taking taking into account that the process is repeated at least 90 times per second. Such repetition rate corresponds to the minimum count of frames per second required so that the virtual reality user does not feel nauseous when using such systems.

Claims

1. MOTION CAPTURE SYSTEM designed to track the movement of a composite object or parts of an object in a three-dimensional space characterized by the fact that it comprises two sets of essential elements, the first comprising a group of receivers forming a fixed base, and the second, comprising one or more transmitters "beacons" or "tags" associated with the object to be tracked.

2. SYSTEM FOR CAPTURE OF MOVEMENTS according to claim 1, characterized by the fact that said tags transmit radio frequency signals.

3. SYSTEM FOR CAPTURE OF MOVEMENTS according to claim 1, characterized by the fact that said tags transmit signals in ultrasound.

4. SYSTEM FOR CAPTURE OF MOVEMENTS according to claim 1, characterized by the fact that the receivers that form the said fixed base are arranged at the vertices of a triangle with known dimensions.

5. MOVEMENT CAPTURE SYSTEM according to claim 4, characterized by the fact that it comprises additional triangular bases, not coplanar with the first fixed base.

6. SYSTEM FOR CAPTURE OF MOVEMENTS according to claim 1, characterized by the fact that it comprises means of data processing provided with a position tracking algorithm.

7. METHOD FOR CAPTURING MOVEMENTS using the system defined in claims 1 to 6, characterized by the fact that it comprises the reception, by each base receiver, of the signals transmitted continuously by each at least one beacon or tag, where the position of said at least one beacon or tag is determined by means of trilateration based on the distance between each said receiver and each said at least one beacon or tag.

8. METHOD FOR CAPTURING MOVEMENTS according to claim 7, characterized by the fact that the determination of each distance is based on the intensity of the signal received by each receiver.

9. METHOD FOR CAPTURE OF MOVEMENTS according to claim 7, characterized by the fact that the determination of each distance is based on the travel time of the signal transmitted by the tag and its reception by each of said receivers.