RU2697942C1 - Method and system for reverse optical tracking of a mobile object - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: invention relates to moving object tracking method and system and can be used in virtual or augmented reality systems. Technical result is achieved by the fact that the tracking method includes a step of automatic adjustment of the tracking zone, on which unique combinations of elementary optical patterns are detected and recorded, and a step of tracking the change in the position and/or orientation of the mobile object, on which unique combinations of elementary optical patterns are detected and compared with those recorded when tuning the tracking system zone. Moving object tracking system comprises at least one tracker arranged on the movable object and having an optical sensor; at least one marker strip containing active markers forming elementary optical patterns on the image obtained from the optical sensor; and central data processing device.

EFFECT: automatic adjustment of the tracking zone during reverse optical or optical-inertial tracking of the mobile object using a tracker mounted on the mobile object and comprising an optical sensor and an inertial sensor.

18 cl, 25 dwg

Description

Technical field

The invention relates to a method and system for tracking a moving object and can be used in virtual or augmented reality systems, in internal logistics systems, in robotic systems, in control systems for unmanned moving objects and in other systems of industrial, scientific, educational and other purposes.

State of the art

Tracking (tracking) of a moving object is used in many systems - virtual reality systems (VR, Virtual Reality) or augmented reality (AR, Augmented Reality), internal logistics systems (for example, warehouse, production or trading), robotic systems, unmanned mobile control systems objects (e.g. vehicles), etc. Further, the main attention is paid to tracking in VR / AR systems, however, the basic principles of tracking are essentially similar in other systems mentioned above.

According to the configuration of the tracking system, direct tracking is distinguished, in which the sensor or several sensors are located in space outside the movable tracked object, and the reference point or several reference points used for tracking are located on the tracked object, and reverse tracking, in which the sensor or several sensors are located on the tracked object, and the reference point or several reference points used for tracking are located in a space outside the moving tracked object.

By the type of sensors, optical tracking is distinguished, in which optical sensors are used, for example, cameras operating in the visible or infrared range of light, and inertial tracking, in which inertial sensors are used, for example, gyroscopes and accelerometers. In addition, tracking systems use magnetic field sensors (magnetometers), altitude sensors (altimeters) and some other types of sensors.

There are also tracking systems using centralized positioning systems - global or local, satellite or ground-based. In addition, there are markerless tracking systems, including those based on the method of simultaneous localization and map building (SLAM, Simultaneous Localization and Mapping).

Most tracking systems designed for use in VR / AR gaming, entertainment, and training systems are combined systems. In particular, optical-inertial tracking systems based on the combined use of optical and inertial sensors complementing each other and compensating for each other’s shortcomings are widespread.

For reverse optical and optical inertial tracking systems in which the video camera is located on a movable tracked object, and optical light-emitting or retroreflective markers, which are reference points for optical tracking, are fixedly fixed in a predetermined manner on the site or in the working area, it is important to quickly and precise definition of the tracking zone.

In the case of a fixed (previously known) tracking zone, the tracking system should provide quick selection of the tracking zone configuration and loading of its parameters. In the case of a variable tracking zone (if the shape and dimensions of the tracking zone are not known in advance), the tracking system should provide the ability to quickly and accurately configure the tracking zone. At the same time, it is highly desirable that calibration of the tracking system before its operation was not required or would be as automated as possible.

Patent applications US2008285854A1, US2013106833A1, US2017086941A1, WO2014199196A1, WO2017050761A1 give a general idea of the state of the art in the field of tracking systems.

Patent applications DE102015013551A1, EP1645241A1 (US2006082789A1), EP2339537A1, US2017168592A1 and US6787750B1 disclose the use of active IR markers in tracking systems. US6787750B1 also discloses automatic determination of a tracking zone.

Patent applications US2011079703A1, US2017358139A1 describe tracking systems using an optical sensor (camera) and an inertial sensor (gyroscope or accelerometer) mounted on a moving object.

Patent applications EP2012170A1, US2004080548A1 disclose the use of active IR markers in tracking systems using an optical sensor (camera) and an inertial sensor (gyroscope or accelerometer) mounted on a moving object.

Patent applications WO2007102857A2, WO2016102721A1 disclose the use of active IR markers in tracking systems using an optical sensor (camera) and an inertial sensor (gyroscope or accelerometer) mounted on a moving object, and recognition of optical patterns formed by fixed markers.

Patent application WO2016187609A1 discloses the recognition of optical patterns formed by fixed markers, and patent application WO2013071981A1 discloses the use of an optical sensor (camera) and an inertial sensor (gyroscope or accelerometer) mounted on a moving object and recognition of optical patterns formed by fixed markers.

The non-patent document [1] describes the identification of a linear pattern formed by four reflective IR markers and a planar pattern using two cameras (direct tracking).

The non-patent document [2] describes the identification of a linear pattern formed by four reflective IR markers and a flat pattern using multiple cameras (direct tracking).

The non-patent document [3] describes the use of complex contrast planar markers in VR systems based on general-purpose portable electronic devices.

The non-patent document [4] describes the identification of various patterns (from single-point to volumetric multi-point) formed by reflective IR markers using multiple cameras (direct tracking).

The non-patent document [5] describes a method for identifying and identifying reflective IR markers by a point cloud using multiple cameras — the point-cloud-based tracking method (direct tracking).

The non-patent document [6] describes the use of Kalman filters for identifying and identifying reflective IR markers using multiple cameras without identifying patterns in the image (direct tracking).

The non-patent document [7] describes the use of color reflective markers for motion capture (motion capture) with the subsequent formation of a 3D model in 3D animation (direct tracking).

The non-patent document [8] describes the use of complex contrast planar markers in a reverse optical tracking system.

The non-patent document [9] describes the use of linear patterns formed by a plurality of light-emitting diodes mounted on the ceiling using a single camera (reverse tracking), indicating the possibility of scaling the tracking zone.

Non-patent document [10] describes the use of a planar pattern with a concentric arrangement of markers for optical tracking of any kind.

Non-patent document [11] describes the use of temporal modulation of the radiation intensity of LED markers for optical tracking of any kind.

The non-patent document [12] describes the use of the Wii controller for direct tracking of wireless headphones equipped with four light-emitting diodes forming a rectangular planar optical pattern for creating a realistic dynamic sound picture.

The technical solution according to non-patent document [9] seems to be closest to this invention in technical essence, it uses LED rulers installed on the ceiling of the room evenly over its area. Rulers are repeating sequences of linear patterns based on a de Bruyne sequence transformed into a cyclic Manchester code.

Nevertheless, the authors of none of the aforementioned patent and non-patent documents failed to solve the problem of automatically adjusting the tracking zone according to the totality of the same type of optical patterns formed by fixed active IR markers installed in a predetermined manner on the site or indoors and limiting the working area, with the opposite optical or optical-inertial tracking of a moving object using a tracker mounted on a moving object and containing an optical sensor and (optionally) inertia cial sensor. It should be noted that such a task in the prior art has not yet been set.

Currently, there are no tracking systems on the market that have this function and are intended for use in virtual or augmented reality systems for gaming, entertainment or educational purposes. The authors of this invention are also not aware of tracking systems having such a function and intended for use in warehouse, production or trade logistics systems, in robotic systems, for controlling manned or unmanned moving objects, etc.

Disclosure of invention

For any systems related to tracking moving objects, in particular for VR / AR systems, it is important to ensure that the system is quickly ready for operation after it is turned on, rebooted, reconfigured or a working scenario. In the case of a fixed (previously known) tracking zone, the tracking system should provide quick selection of the tracking zone configuration and loading of its parameters. In the case of a variable tracking zone (if the shape and dimensions of the tracking zone are not known in advance), the tracking system should provide the ability to quickly and accurately configure the tracking zone. At the same time, it is highly desirable that calibration of the tracking system before its operation was not required or would be as automated as possible.

The task of automatically adjusting the tracking zone during reverse optical or optical-inertial tracking of a moving object using a tracker mounted on a moving object and containing an optical sensor and (optionally) an inertial sensor is solved using the method and system for tracking a moving object according to the invention.

The tracking method of a moving object includes the following steps:

- a step of automatic adjustment of the tracking zone, at which unique combinations of elementary optical patterns are detected and recorded;

- the step of tracking changes in the position and / or orientation of the moving object, on which unique combinations of elementary optical patterns are detected and compared with elementary optical patterns registered when setting up the tracking zone.

Setting up the tracking zone can include the following steps:

(S1) determine a tracking area;

(S2) determine the size of the tracking window and set the tracking window to its initial position;

(S3) determine in the tracking window elementary optical patterns formed by markers;

(S4) form the ID of the "constellation" of the candidate;

(S5) verify the uniqueness of each “constellation” of the candidate and mark the ID of the non-unique “constellation” of the candidate;

(S6) write the ID of the "constellation" of the candidate in the table of "constellations";

(S7) rotate the tracking window by a predetermined angle;

(S8) perform steps S2 – S6 for the rotated tracking window;

(S9) check whether the required number of turns of the tracking window has been completed, and if necessary, repeat steps S7, S8;

(S10) moving the tracking window with a predetermined step;

(S11) perform steps S2 – S9 for the moved tracking window;

(S12) check if the entire tracking area is covered, and if necessary, repeat steps S10, S11;

(S13) remove the non-unique IDs of the “constellations” of the candidates from the table of “constellations”.

Steps S4, S5, S6, and S13 may be performed with respect to each candidate constellation ID and the ID of each of its child candidate constellations. Prior to step S2, it is possible to divide the projection of the tracking zone into cells, and the size of the tracking window in step S2 may be expressed in the number of such cells.

Tracking a change in the position and / or orientation of a moving object may include the following steps:

(S21) read the image from the optical sensor located on the moving object, and perform its preliminary processing;

(S22) detecting elementary optical patterns formed by markers in the tracking window;

(S23) form a constellation ID;

(S24) identify the generated constellation ID from the constellations table;

(S25) determine the position and orientation of the moving object based on the "constellation" identified in step S24.

Step S21 may include the following steps:

(S31) read the image from the optical sensor;

(S32) perform geometric distortion compensation of the read image;

(S33) optionally projecting the image onto the work plane;

(S34) normalize the image;

(S35) highlight the tracking window on the image.

Step S22 may include the following steps:

(S41) three points corresponding to markers are arbitrarily selected in the tracking window;

(S42) from the selected three points, two reference points are determined;

(S43) for the reference points, the nominal position of the third point is determined in accordance with the elementary optical pattern;

(S44) determining a deviation δ of the geometric position of the third point from its nominal position;

(S45) checking the value of δ; if the value of δ does not exceed a predetermined threshold value, then (S46) the three points are added to the list of elementary optical candidate patterns, and if the value of δ exceeds the predetermined threshold value, then (S47) the three points are rejected;

(S48) check whether all combinations of the three points are processed and, if necessary, repeat steps S41 – S47;

(S49) sorting the list of elementary optical candidate patterns by the value of δ;

(S50) from the list of elementary candidate patterns, an elementary optical candidate pattern with a minimum value of δ is selected;

(S51) checking for the presence of used points in the elementary optical candidate pattern; if at least one used point is found in it, (S52) such an elementary optical candidate pattern is discarded, if no used points are found, (S53) it is moved to the list of valid elementary optical patterns, while the points included in this elementary optical pattern, marked as used;

(S54) perform steps S50 – S53 with respect to the remaining elementary optical candidate patterns.

The tracking system of a moving object contains:

- at least one tracker located on a moving object (for example, on a user of a VR / AR system) and containing an optical sensor;

- at least one marker strip containing active markers forming elementary optical patterns in the image obtained from the optical sensor;

- a central data processing device, partially performing a method for tracking a moving object, described in any one of paragraphs. 1-7.

The optical sensor may be a matrix optical sensor. At least a portion of the marker strips may be located on the floor or other base. Marker strips can be integrated with resilient precast flooring. At least a portion of the marker strips may be located on a ceiling, walls, beams, masts, or otherwise secured in a tracking area above a floor or other base.

Active markers can be markers that emit light in the infrared range. The elementary optical pattern may be a linear or non-linear elementary optical pattern. The tracker may comprise a data processing device partially executing the method of tracking a moving object described above.

The tracker and the central data processing device may be connected by a wireless data transmission channel for transmitting tuning data in the step of automatically adjusting the tracking area and the position and / or orientation data of the tracker in the tracking step in the method of tracking a moving object described above.

This system can be configured to automatically adjust the tracking zone by identifying and registering unique combinations of elementary optical patterns in the manner described above. This system can also be configured to track changes in the position and / or orientation of a moving object by identifying unique combinations of elementary optical patterns and matching them with elementary optical patterns registered when setting up the tracking zone using the method described above.

Brief Description of the Drawings

In FIG. 1 shows an example implementation of a VR / AR system in accordance with the invention.

In FIG. 2 shows a block diagram of a tracker.

In FIG. 3 shows an example of a marker strip in expanded form.

In FIG. 4 shows an example of a rolled up marker strip.

In FIG. Figure 5 shows an example of the installation of markers using elements of elastic prefabricated floor coverings (puzzle floor mats).

In FIG. Figure 6 shows an example of the location of marker strips on the floor.

In FIG. 7 shows an example of mounting marker strips on the ceiling.

In FIG. Figure 8 shows the placement of a marker strip in an elastic prefabricated floor covering, each element of which corresponds to a cell of a tracking zone.

In FIG. 9 shows an example of numbering cells in a tracking zone.

In FIG. 10 shows an elementary optical pattern formed by a marker strip, with a circle marked its conditional center, and an arrow shows its conditional direction, which may have a numerical designation (encoding 0, 1, 2, 3).

In FIG. 11 shows an example of the arrangement of elementary optical patterns in a square tracking zone.

In FIG. 12A shows an example of a unique combination of elementary optical patterns.

In FIG. 12B shows an example of a non-unique combination of elementary optical patterns.

In FIG. 13A, B, a flow chart of the registration of unique combinations of elementary optical patterns (“constellations”) is presented.

In FIG. 14 shows an example of marker shading in a multi-user VR / AR system.

In FIG. 15 is a flowchart of a tracking algorithm.

In FIG. 16 shows an example of an image obtained from an optical sensor.

In FIG. 17 shows an example of the image of FIG. 16 after compensation for geometric distortion and projection onto the work plane.

In FIG. 18 shows an example of the image of FIG. 17 after rationing.

In FIG. 19 shows an example of the image of FIG. 18, in which, for clarity, the position of the elementary optical patterns relative to the grid is shown.

In FIG. 20 is a flowchart of an algorithm for reading and preprocessing an image from an optical sensor.

In FIG. 21 is a flowchart of an algorithm for detecting elementary optical patterns in an image from an optical sensor.

In FIG. 22 shows an example of the nominal and actual position of the third point of an elementary optical pattern.

In FIG. 23 shows an example of a non-collinear arrangement of elementary optical patterns.

The implementation of the invention

In VR / AR systems, the tracking zone can undergo changes, in particular, the shape and size of the tracking zone can change during operation of the system in connection with its transfer to another site or other room, with a change in the number of users or with a script implemented in the VR / AR. Changing the shape and size of the tracking zone requires reconfiguring the tracking system. On VR / AR mobile systems, tracking system configuration is required each time the system is deployed. In the tracking system according to this invention, the task of automatically adjusting the tracking zone is solved, the accuracy and reliability of tracking is improved, including in multi-user mode.

In the inverse optical inertial tracking system, the device providing tracking (tracker) is located on the moving tracking object, and the light sources (active markers), which form characteristic combinations on the image captured by the tracker (optical patterns), are stationary and located in the space of the working area of the VR / AR. A VR / AR system may contain one or more trackers, with each tracker allowing you to determine the position and orientation of the corresponding moving object (or part thereof) independently of other trackers.

In FIG. 1 shows an example implementation of a VR / AR system 1 comprising a head display 5 in the form of a helmet or glasses, a tracker 10 located on the head display 5, a central data processing device 6 (host) also located on the head display 5, tracker 20, tracker 30 , and a marker strip 4 containing active markers 40, 41, 42. The head display 5 provides a demonstration to user 7 of the image generated by the system 1 based on tracking data received from trackers 10, 20, 30. The image shown to the user may be 3D virtual image awn or 3D image of augmented reality. The tracking data generated by the trackers 10, 20, 30, contains data on the location of the trackers and, therefore, the corresponding parts of the user's body 7, on their orientation in space, and (optionally) on the parameters of their movement - direction, speed and acceleration.

The tracker 10 (Fig. 2) contains an optical sensor 101 and one or more inertial sensors 102. For processing tracking data, the tracker 10 includes a data processing device 103. The data processing device 103 provides synchronization of all sensors, obtaining data from the optical sensor and from one or more inertial sensors and processing data from the optical sensor to obtain information about the markers contained in the frame and determine the position and orientation of the tracker. The tracker 10 also includes a power source 104, a communication device 105, a storage device 106, and other parts necessary for its functioning as part of the VR / AR system 1. The tracker 10 is connected to the central data processing device 6 by a wireless communication channel, examples of which are WiFi, Bluetooth, etc. channels for transmitting setting data and tracking data.

The tracker 20 and the tracker 30 may be identical to the tracker 10 or may have differences related to the different nature of the movement of their corresponding body parts of the user. In particular, the trackers 20, 30 connected to the game controllers in the hands of the user may be able to track faster and more abrupt movements than the tracker 10 located on the user's head. Nevertheless, the general principle of operation of the tracking system remains essentially unchanged. Various tracker options are described in detail in patent applications PCT / IB2017 / 058068 and US15844967 of the same applicant, the entire contents of which are incorporated herein by reference. The construction of an optical sensor suitable for use in this invention is described in patent applications PCT / RU2014 / 001019 and US15540313 of the same applicant, the entire contents of which are also incorporated herein by reference.

Marker strip 4 (Fig. 3) contains a base, which is a narrow flexible strip of polymer material with flexible conductors that provide power to the active markers 40, 41, 42. The base material can be woven or non-woven. The flexibility of the base and conductors ensures the compactness of the marker strip when folded during storage and transportation (Fig. 4), while the design of the base ensures its sufficient rigidity in the transverse direction (in the plane of its installation) and a constant distance between the markers, which is necessary to ensure geometric stability optical pattern. Conductors can be metal, metallized, oxide, composite, made by thick-film or thin-film technology, providing safe voltage supply at a power consumption of several watts. Active markers 40, 41, 42 can be attached to the base in any suitable way, providing reliable mounting in predetermined locations and supply voltage. In an illustrative embodiment of the invention, tape metal conductors are used, and active markers 40, 41, 42 are attached to the base with magnetic connectors, which allow fast installation of markers on the base and dismantle from it, while the ease of dismantling the markers prevents marker strip 4 from destruction in case Unintentional physical contact of the user with markers. Power to the markers 40, 41, 42 is supplied from the power source 45 through the cable 44 and the electrical connector 43.

In an illustrative embodiment, the active markers 40, 41, 42 are light emitting IR markers. In other embodiments, active markers 40, 41, 42 may operate in other frequency ranges, for example, in the visible light range.

The power source 45 may be any mains or autonomous power source providing the necessary safe voltage (for example, 5, 6, 12, 24, 36, or 48 V) of direct or alternating current. For example, as a stand-alone power supply, it is possible to use a 5 V or 12 V DC PowerBank battery used to power / charge mobile devices, or a 12.6 V DC Booster battery designed to facilitate the starting of car engines with insufficient charge main battery. When using multiple marker bands 4 in system 1, power can be provided centrally from one common power source, or power can be distributed when each marker strip 4 is powered by a separate power source. Centralized power may be preferred in stationary commercial VR / AR systems, especially if marker strip 4 is embedded in the floor or located on the walls or ceiling of the room. Distributed power may be preferred in mobile amateur VR / AR systems, particularly suitable for outdoor use.

Cable 44 and electrical connector 43 may be of any suitable type. Their choice is a trivial task for a specialist, therefore their description is omitted.

Each of the active markers 40, 41, 42 is an infrared LED emitter with a near-hemispherical radiation pattern. The required radiation pattern is provided by the location and orientation of one or more LED crystals, as well as a diffuser, for example, made on the basis of a Fresnel lens. The marker may contain a driver that ensures the health of the LED emitter in a wide range of supply voltages.

The combination of active markers 40, 41, 42 arranged in a predetermined manner forms an elementary optical pattern (hereinafter, for brevity, it can be called an elementary pattern), the presence and location of which in space is determined by host 6 based on data about markers present in the data stream from tracker 10 The use of active (light emitting) markers is determined by the principle of reverse tracking and provides a number of advantages compared to the use of passive (reflective) markers. In particular, when using active markers, additional power is not required for the tracker to provide illumination for passive markers, which is especially important when the tracker is placed on a moving object. The maximum size of the tracking zone is related to the size and brightness of the markers. Powered active markers from network or stand-alone power sources can repeatedly increase the radiation power of markers, thereby increasing the size of the tracking zone. At the same time, the energy consumption of the tracker itself remains unchanged.

An elementary pattern can be a linear pattern, which is identified by the marker data processing algorithm. The linear pattern is characterized by a predetermined ratio of the distances between the markers. In particular, in an embodiment of the invention with 3 markers lying on the same line in the sequence 40–41–42, as shown in FIG. 3, the distance between the markers 40 and 41 is twice as large as the distance between the markers 41 and 42. Of course, the ratio of the distances between the markers may be different, provided that it is possible to reliably identify such an elementary pattern in the system.

The working area can be scaled by increasing the length of the marker strip 4 while maintaining the ratio of distances and providing a sufficient area of the luminous surface of the markers and the brightness of their glow. This makes it possible to significantly increase the working area while maintaining high tracking characteristics while maintaining the angular and brightness parameters of marker images in the frame of the optical sensor.

Of course, that an elementary pattern can be a linear pattern with a large number of markers, for example, with four, five, etc. markers; the elementary pattern can also be a nonlinear pattern, i.e. Markers may not be on the same line.

In addition to the above-described method of increasing the working area, there is the possibility of almost infinite scaling of the working area by adding additional marker strips. In this case, not combinations of individual markers, but combinations of elementary patterns are added to the tracking picture. Thus, the system determines the presence in the frame of previously known elementary patterns and then determines the location of these patterns relative to each other. This makes it possible to design a combination of elementary patterns in such a way that, based on the map of elementary patterns constructed when setting up the tracking zone, the tracking system can uniquely determine the position of the tracker within the working area. Such an approach can be used to build not only a simple working area in one room, but also a complex working area, including one distributed across several adjacent rooms.

Thus, one of the advantages of the invention is the simple scaling of the tracking zone by adding new marker stripes and the automatic adjustment of the tracking zone after such scaling. In direct optical tracking systems with markers on a moving object, to expand the tracking zone, the addition of additional cameras is required, which, in most cases, is technically difficult and expensive. In the claimed system, the cost of marker strips is low and the complexity of their installation is negligible.

Depending on the scenario, marker strips may be on the floor, on the walls or on the ceiling. Combined options are also possible, for example, with marker strips on the walls and on the ceiling or on the floor and walls. In stationary embodiments of the invention, markers can be embedded in the floor, walls or ceiling. In mobile embodiments of the invention, for example, in exhibition samples, markers can be embedded in a collapsible floor, which is part of the exhibition installation kit. It is also possible to fix marker strips on racks, beams, consoles and other supporting structures, including collapsible and mobile ones.

Alternatively, in both stationary and mobile embodiments of the invention, the marker strips can be assembled from elements of elastic prefabricated floor coverings (puzzle floor mats) (Fig. 5), which are engaged with each other in the working position, which ensures high geometric accuracy of the construction of the working area, the stability of its shape and size and, accordingly, simplifies the preparation of the VR / AR system for work. Markers can be installed at predetermined locations of such coating elements. The predetermined installation locations of the markers can be holes of the corresponding shape into which the markers are mounted, and unused holes can be closed with plugs of the same material as the coating itself. The power wires of the markers can be located under the coating and can be ribbon wires, which eliminates their manifestation in the form of irregularities on the working surface of the coating. The combined use of the marker strips shown in FIG. 3, with prefabricated floor coverings.

For amateur use, when ease of installation and ease of use are important factors, the floor-standing option may be preferable, in which the user can place marker strips on the floor, forming a figure close to a rectangle from them, and connect them to one or more power sources (Fig. . 6). The installation time for such a system is usually no more than two minutes.

For commercial use, in particular for VR parks, it may be important to place the tapes with markers out of the reach of users so that they could not change the characteristics of the working area defined during system initialization and / or specified in the system settings with their actions. In addition, commercial VR / AR systems are often multi-user, and when choosing the location of marker strips, it is advisable to take into account the possibility of shielding markers by users when they move around the work area. In this case, a variant with wall or ceiling arrangement of marker strips may be preferable (Fig. 7, the ceiling is not shown conditionally). It should be noted that with ceiling placement of marker strips, the size of the working area can significantly exceed the size of the figure limited by marker strips, however, in this case, several additional markers located on the floor may be required to determine the position of the floor plane relative to the marker strips, or use an additional calibration procedure systems, for example, with the temporary placement of one or more trackers on the floor during calibration.

The following describes the principles for detecting an optical pattern when determining a fixed (not requiring calibration) zone of optical tracking and during tracking of a moving object using the VR / AR system as an example and provides options for their practical implementation.

Tracking zone - an area of space in which tracking of a moving object is provided. According to the invention, in a VR / AR system, the tracked moving object may be a user or a part of a user's body, for example, a head, arm, leg, to which a tracker is attached. A tracked moving object may also be an object held by a user, for example, in a hand. However, the tracked moving object can be any other moving object, for example, a working body of a robotic system or a material handling vehicle in an internal logistics system.

Tracking of a moving object is carried out in the volume of the tracking zone. To automatically determine the tracking zone, the projection of this zone on the working (usually horizontal) surface, for example, the surface of the floor or ceiling in the room, is divided into fragments - cells, which can have a different shape. When using the orthogonal coordinate system, the square shape of the cells is preferred. The cell size is determined depending on the technical implementation of the VR / AR system - the minimum and maximum size of the tracking zone, the required tracking accuracy, the type of markers, the type and size of elementary patterns formed by markers, etc. In particular, when using a prefabricated floor covering (Fig. 5), the cell size may be equal to the element size of such a coating, for example, 60 × 60 cm. In this case, a marker strip is 1.5 m long with three markers, as shown in FIG. 3, is placed in three cells, and, depending on the orientation of the elementary pattern, markers can have different relative positions on the plane (Fig. 8). The orientation of the elementary pattern in FIG. 8 is conventionally shown by arrows on the marker strip. Of course, that the length of the marker strip can be greater or less than 1.5 m and it can be placed in more or less cells. For example, with a higher marker density, the marker strip corresponding to the elementary pattern can be located in two cells.

Thus, the surface is divided into cells, each of which has its own coordinates. Cell coordinates can be set in various ways, for example, in Cartesian or polar coordinates. In an illustrative embodiment of the invention, the cells are numbered in a predetermined manner and the coordinates of each cell are uniquely determined by its number. An example of cell numbering is shown in FIG. 9. The position of each elementary pattern in such a coordinate grid is determined by the number of the cell containing the center of the elementary pattern and the direction of this pattern. In FIG. 10, the conditional center of the elementary pattern is marked with a circle, and the arrow shows its conditional direction. The direction of the elementary pattern may have a numerical designation (coding), as shown in FIG. 10. In FIG. 11 shows an example of the arrangement of elementary patterns in a tracking area of a square shape. Of course, the tracking zone may have a shape other than square.

Elementary patterns form unique combinations, conditionally referred to as “constellations” below. In the VR / AR system, the position and orientation of the tracker in space is determined by the "constellations", like sea, air or land navigation in the starry sky. In other words, by identifying unique combinations of elementary patterns in the image received from the optical sensor of the tracker within the tracking zone, the VR / AR system is able to uniquely determine the position and orientation of the tracker. Therefore, the task of identifying and registering "constellations" at the stage of setting up the optical tracking zone before starting the VR / AR system is so important.

Combinations of patterns that are not unique are rejected by the algorithm for registering “constellations”. In FIG. 12A and FIG. 12B shows examples of a unique and non-unique combination of elementary patterns: shown in FIG. 12A, the combination of patterns is unique, and the one shown in FIG. 12B, the combination of patterns is not unique, since it has central symmetry and rotation of its image by 180 ° relative to the center of the image results in an identical image, technically indistinguishable from the original, therefore, the combination of elementary patterns in FIG. 12B cannot be used to determine the position and orientation of the tracker and the associated moving object.

It should be noted that in general, the size of the tracking zone may be larger than the size of the field of view of the tracker. This means that the tracker can receive an image not of the entire tracking area, but only of a certain part of it, while the tracker’s field of view at each moment of time depends on its position and orientation. In the future, the field of view of the tracker may be called the tracking window. To determine the position and orientation of the tracker according to the optical sensor, it is necessary that at least one “constellation” fall into the tracking window.

The following describes the sequence of actions of the "constellations" registration algorithm that is performed when setting up the tracking zone, in particular, when the VR / AR system is initialized after it is turned on or after reboot, due to a change in its configuration or due to a change in the working scenario. The result of this algorithm is a table containing records of all the "constellations" in the tracking zone. Each record contains the identifier (ID) of the "constellation" and the identifiers of the elementary patterns that make up this "constellation". In an illustrative embodiment of the invention, the constellation ID contains the coordinates of each elementary pattern in the constellation, which are represented by the cell number at which the center of this elementary pattern is located, and the direction of the elementary pattern, represented by the rotation code, and the identifier of the elementary pattern is its serial number specified in the configuration of the tracking zone when marking it. Of course, the constellation ID structure may differ from that described above.

The block diagram of the algorithm for identifying and registering "constellations" when setting up the tracking zone is shown in FIG. 13A, B.

At step S1, a tracking zone is determined which will be used in the operation of the tracking system. The determination of the tracking zone can be carried out by performing its marking or by reading the finished marking from the memory of the tracking system. The principles for marking the tracking zone are described in earlier patent applications PCT / IB2017 / 058068 and US15844967 of the same applicant, all of which are incorporated herein by reference.

In step S2, the size of the tracking window is selected based on the technical characteristics of the optical sensor (viewing angle, resolution, etc.), the image processing speed in the tracking system and the maximum complexity of the “constellation” (maximum number of elementary patterns in one “constellation” ), taking into account the division of the tracking zone into cells.

In an illustrative embodiment of the invention, the tracking window is taken to be a 4 × 4 square cell (in Fig. 9, the tracking window is shown in bold) and for each elementary pattern, 4 × 4 = 16 variants of the center of the pattern are possible (requires 4 bits) and four rotation options in 90 ° increments (requires 2 bits). If the maximum complexity of the "constellation" is limited to 8 elementary patterns, then the length of the "constellation" ID will be 8 × (4 + 2) = 48 bits.

Of course, the size of the 4x4 cell tracking window in an exemplary embodiment of the invention is selected to facilitate understanding. In practice, the size of the tracking window depends on the physical dimensions of the tracking zone, the purpose of the VR / AR system, the number of users, etc. and may be, for example, 6 × 6 cells or 10 × 10 cells. Moreover, the length of the ID "constellation" will be greater.

For example, if the tracking window is taken to be equal to a square of 6 × 6 cells, then for each elementary pattern, there are 6 × 6 = 36 options for positioning the center of the pattern (requires 6 bits) and four rotation options in 90 ° increments (requires 2 bits). If at the same time the maximum complexity of the "constellation" is limited to 8 elementary patterns, then the length of the ID of the "constellation" will be 8 × (6 + 2) = 64 bits, and if the maximum complexity of the "constellation" is limited to 16 elementary patterns, then the length of the "constellation" will be 16 × (6 + 2) = 128 bits.

Reducing the maximum complexity of the "constellation" and reducing the size of the tracking window allows you to speed up data processing and, therefore, reduce the overall delay in the response of the VR / AR system to user actions. At the same time, increasing the maximum complexity of the "constellation" allows you to increase the reliability of tracking, especially when shading markers by users in a multi-user VR / AR system. Thus, the choice of the size of the tracking window and the maximum complexity of the "constellation" is a compromise solution taking into account all the parameters of the VR / AR system.

Then the tracking window is set to its initial position. In an illustrative embodiment of the invention, the position in the lower left corner of the tracking zone is selected as the initial one. Of course, the starting point may be another convenient position.

At step S3, all elementary patterns that completely fall into the tracking window are determined.

At step S4, the ID of the “constellation” of the candidate is formed, which contains elementary patterns that fall into the tracking window.

In this case, during the operation of the tracking algorithm described below, a situation is possible when not all elementary patterns that fall into the tracking window are present on the image from the optical sensor of the tracker. This can be caused, for example, by shading markers with the hands or game equipment of the user with whom this tracker is connected, or by other users in the multi-user VR / AR system (see Fig. 14). In particular, if four elementary patterns get into the tracking window, only three or only two of them can be detected in the image from the optical sensor of the tracker. Therefore, for such an ID of a “constellation” candidate, in the general case, containing N elementary patterns, when setting up the tracking zone, the IDs of all its daughter “constellations” candidates containing N – 1, N – 2, N – 3, ... elementary patterns.

Only unique combinations of elementary patterns should fall into the final table of “constellations”. If the combination of elementary patterns is not unique, it cannot be used to determine the position and orientation of the tracker and the moving object associated with it.

Therefore, at step S5, the “constellation” of the candidate is checked for uniqueness, and if an identical ID of the “constellation” of the candidate already exists in the table, then the ID of such a “constellation” of the candidate and the ID of all its child “constellations” of the candidates are marked as non-unique.

After forming the ID of the “constellations” of candidates for the current tracking window, they are recorded in the table of “constellations” at step S6.

Then, in step S7, the tracking window is rotated by a predetermined angle, and in step S8, steps S2 – S6 are repeated with respect to the rotated tracking window. A rotation is required to verify the uniqueness of the "constellations" of candidates observed from different angles. In an exemplary embodiment of the invention, this rotation is performed 90 ° counterclockwise. Of course, this rotation can be performed at a different angle and / or clockwise. At step S9, the turn and steps S7, S8 are performed several times until the tracking area returns to the position before the start of the turn. In particular, when turning through 90 °, the turning and steps S7, S8 must be performed four times.

Subsequently, during the operation of the algorithm in step S10, the tracking window moves across the entire tracking zone and in step S11, for each window position, IDs of “constellations” are completely entered into this window, as described above for steps S2 – S9. In an illustrative embodiment of the invention, this movement is performed in two cells. Of course, this movement can be performed with another step, for example, one or three cells. In some embodiments, this movement may have different vertical and horizontal steps. For example, the step of moving the tracking window vertically can be one cell, and the step of moving the tracking window horizontally can be two cells, or vice versa.

Next, at step S12, the coverage of the entire tracking zone is checked and after the entire tracking zone was covered during the operation of the algorithm described above, all non-unique IDs of the “constellations” of candidates are removed from the “constellations” table in step S13. This completes the formation of the table of "constellations" and the work of the algorithm for registering "constellations" ends.

In the manner described above, the task of automatically adjusting the tracking zone is solved before the start of the tracking system. Further, during the operation of the tracking system, the tracking of a moving object with which the tracker is physically connected is carried out using the results of this automatic setup.

In the process of tracking a moving object, a tracking algorithm is executed, the block diagram of which is shown in FIG. 15.

At step S21, the image from the optical sensor of the tracker is read and its preliminary processing is performed, the algorithm of which is described in more detail below with reference to FIG. 20.

At step S22, all elementary patterns are detected that are present in the image from the optical sensor of the tracker and completely fall into the tracking window. The algorithm for detecting elementary patterns is described in more detail below with reference to FIG. 21.

In step S23, the “constellation” ID identified in the tracking window is generated. The “constellation” ID format and the method of its formation are generally similar to those described previously for the “constellation” registration algorithm.

At step S24, the correspondence of the generated constellation ID is determined to one of the unique combinations of elementary patterns fixed in the constellations table when setting up the tracking zone of the VR / AR system.

In step S25, based on the identified “constellations”, the position and orientation of the tracker in space is determined from the “constellations” table. Methods for determining the position and orientation of an object using external landmarks (in particular, navigation methods using starry sky maps) are well known to specialists in this field of technology, therefore, their description is omitted for brevity.

Then, when a new image arrives from the optical sensor of the tracker, steps S21 – S25 are repeated.

It should be noted that the tracking system must detect at least one “constellation” in order to determine the position and orientation of the tracker and begin tracking the moving object. Further, during the tracking process, the inability to detect one or more “constellations” in the next image from the optical sensor (for example, due to overlapping markers by the user with whom this tracker is connected, or by other users in the multi-user VR / AR system, or as a result of the user's unsuccessful zone of the tracker at some point in time) does not lead to a malfunction of the tracking system, since this system is able to determine the position and orientation of the tracker due to inertial tracking , Ie on the basis of data obtained from inertial sensors for a time sufficient to obtain a new image from the optical sensor and resume optical tracking.

In addition, in some embodiments of the invention, it is possible to continue tracking a moving object based not only on "constellations", but also on the basis of identified individual elementary patterns or even individual markers. For example, since the tracking system is able to determine the position and orientation of the tracker for some time due to only inertial tracking, based on the configuration information of the tracking zone, this system can determine the nominal position of individual markers (i.e., points or areas of space in which likely detection of markers). In this case, tracking can be continued by identifying only individual markers and reconciling their actual position with the nominal position. Obviously, the reliability of such tracking may be lower, but its use for some time, sufficient to obtain an image from an optical sensor with all the "constellations", can be fully justified.

These properties of the tracking system according to this invention can improve the accuracy and reliability of tracking, including in multi-user mode.

In an illustrative embodiment, the frequency of data received from the optical sensor is approximately 60 frames per second, and the frequency of data received from the inertial sensor is approximately 2000 samples per second. In other embodiments of the invention, the frequency of receipt of data from the optical sensor can be increased to approximately 400 frames per second. Technical solutions of optical tracking and inertial tracking and their interaction to solve this problem are described in earlier patent applications PCT / IB2017 / 058068 and US15844967 of the same applicant, the entire contents of which are incorporated into this application by reference.

The following describes in detail the algorithm for reading and preprocessing images, the block diagram of which is presented in FIG. 20.

In step S31, an image is read from the optical sensor. An example of an image obtained from an optical sensor is shown in FIG. 16. The dashed line conventionally shows the line connecting the emitters of one elementary pattern.

In step S32, the compensation of geometric distortions caused by the optics and deviations from the nominal dimensions in the design of the optical sensor of the tracker is performed. An example of an image after compensation for geometric distortions is shown in FIG. 17.

In step S33, the image is projected onto the work plane after compensation. Such projection of an image onto a plane is performed based on the direction of each marker (i.e., the geometric ray) in the coordinate system of the tracker in which this marker is detected, and the known upward direction, determined from the data of one or more inertial sensors. This direction can be determined in polar coordinates. A method for determining a beam directed to a specific marker is described in earlier patent applications PCT / IB2017 / 058068 and US15844967 of the same applicant, all of which are incorporated herein by reference. In some embodiments, projection of an image onto a work plane may not be required, in particular if the markers are already located on the work plane (for example, on a floor whose plane is accepted as the work plane).

In an illustrative embodiment, the upward direction is determined by accelerometer data. Of course, this direction can be determined by the data of another inertial sensor, for example, a gyroscope, or by the data of a group of homogeneous or heterogeneous inertial sensors, for example, an accelerometer and a gyroscope. Methods for determining the desired direction according to inertial sensors are well known to specialists in this field of technology, therefore, their description is omitted for brevity. In addition, the upward direction can be determined from other sensors, for example, an electric or magnetic field sensor.

The image projected onto the plane is the projection of the image received from the optical sensor onto the plane with respect to which the upward direction is normal, i.e. on the horizontal plane. In an illustrative embodiment, the horizontal plane is the floor plane of the room in which the VR / AR zone is located. This solution is optimal when marker strips are located on the floor. Of course, the horizontal plane can be another plane lying in the volume of the tracking zone, for example, the plane of the ceiling of a room with marker strips on the ceiling. Images projected onto parallel horizontal planes are related by a relation of mathematical similarity and their information value is essentially the same.

In step S34, the image is normalized, i.e. a grid is applied to this image, corresponding to dividing the tracking area into cells, as described above, then this image is rotated so that the direction of each elementary linear pattern is essentially parallel to one of the directions of the grid, and is shifted so that the center of each elementary linear pattern turned out to be essentially in the center of any cell. An example of the image of the tracking window after normalization is shown in FIG. 18, and in FIG. 19 for clarity, shows the position of elementary patterns relative to the grid. Methods of such image processing are well known to specialists in this field of technology, therefore, their description is omitted for brevity.

At step S35, a tracking window is highlighted in the image, the size and location of which is similar to the algorithm described in step S1, which is performed when setting up the tracking zone.

The algorithm for detecting elementary optical patterns, the block diagram of which is shown in FIG. 21.

In step S41, three points are randomly selected in the tracking window, presumably corresponding to markers.

In step S42, two reference points are determined from the selected three points. The reference points may be, for example, the points farthest from each other. In another example, the reference points may be the points closest to each other.

In step S43, the nominal position of the third point is determined for these two points, i.e. its position so that the entire triple of points corresponds to an elementary pattern.

In step S44, the deviation δ of the geometric position of the third point from its nominal position is determined. In FIG. 22 shows an example of the nominal and actual position of a third point located between two points farthest from each other.

At step S45, a check of the deviation δ is performed. If this deviation does not exceed a predetermined threshold value ε, then in step S46 the triple of points is fixed as an elementary candidate pattern, and if this deviation exceeds a predetermined threshold value ε, then in step S47 the triple of points is rejected, i.e. excluded from further consideration.

At step S48, a check is made to see if all combinations of the three points are processed in steps S41 – S47.

After processing of all combinations of three points in the tracking window is completed, the list of elementary candidate patterns in step S49 is sorted by their value δ, which is an indicator of the quality of elementary candidate patterns.

Then, in step S50, an elementary candidate pattern with a minimum value of δ is selected, and in step S51, it checks to see if there are points already used in the actual elementary patterns. If none of its points are marked as used, in step S52, such an elementary candidate pattern is moved to the list of valid elementary patterns, while the points included in this elementary pattern are marked as used. If at least one of its points is marked as used, then in step S53 such an elementary candidate pattern is rejected, i.e. excluded from further consideration.

Next, in step S54, steps S50 – S53 are repeated with respect to the elementary candidate patterns remaining in the list.

Of course, the algorithm for detecting elementary optical patterns described above can also be performed with respect to elementary optical patterns containing a different number of points, for example, four or five. Moreover, such an algorithm can divide the set of points of an elementary optical pattern into subsets of three points and, for each triple of points, perform the processing essentially as described above, and then aggregate the processing results, or such an algorithm can process the entire set of points at once to determine the values deviations δ ₁ , δ ₂ , etc.

The result of the algorithm for detecting elementary optical patterns is a list of valid elementary patterns, further used in the algorithm for registering "constellations" and in the tracking algorithm described above.

In an illustrative embodiment, the algorithm for detecting elementary optical patterns in steps S41 to S54 is described with respect to a linear three-point elementary pattern. Obviously, in other embodiments, the number of dots in an elementary pattern may differ from three and be, for example, four or five points. In addition, the appearance of an elementary pattern may differ from a linear one, i.e. points of an elementary pattern may not lie on one straight line, but form a predetermined geometric figure.

In the embodiments described above, the direction of each elementary linear pattern is chosen substantially parallel to one of the grid directions. However, in other embodiments, the linear patterns may be arranged differently, for example, in a sawtooth pattern as shown in FIG. 23. This arrangement of linear patterns avoids situations where the algorithm for detecting elementary patterns mistakenly takes the active marker of one linear pattern for the active marker of another (neighboring) linear pattern.

It should be noted that each process corresponding to steps S21 – S25 and S41 – S54 is performed independently on the data from each tracker, i.e. multi-threaded data processing is implemented, while the number of streams is a multiple of the number of active trackers in the VR / AR system.

It should also be noted that the above description reflects only those actions that are most essential to achieve the purpose of the invention. The specialist understands that for the functioning of the system it is necessary to perform other necessary actions, for example, connecting the equipment, initializing it, launching the corresponding software, transmitting and receiving commands and confirmations, exchanging service data, synchronization, etc., the description of which is omitted for brevity presentation.

In addition, it should be noted that the above description reflects only those parts of systems and devices that are most essential to achieve the purpose of the invention. One skilled in the art will appreciate that these systems and devices should or may contain other parts that support the functioning of the VR / AR system, the description of which is omitted for brevity.

The devices and their parts, methods and parts thereof, referred to in the description and drawings, relate to one or more specific embodiments of the invention, if they are mentioned with reference to a numerical reference designation, or to all variants of the invention, in which they can be used if they are mentioned without reference to a numerical reference designator.

The devices and their parts mentioned in the description, drawings and claims are hardware and software, while the hardware of some devices may differ, partially coincide or completely coincide with the hardware of other devices, unless otherwise indicated explicitly. The hardware of the devices may be located in different parts of other devices, unless otherwise specified explicitly. Software parts (modules) can be implemented in the form of program code contained in a storage device.

The sequence of actions in the description of the method is illustrative and in various embodiments of the invention, this sequence may differ from that described, provided that the function performed and the result achieved are maintained.

Parts and features of the present invention may be combined in various embodiments of the invention, if they do not contradict each other. The embodiments described above are provided for illustrative purposes only and are not intended to limit the scope of the present invention as defined by the claims. All reasonable modifications, upgrades and equivalent replacements in the composition, construction and principle of operation of the present invention, made within its essence, are included in the scope of this invention.

It should also be noted that the above description of the invention relates to the use of a method and / or system for detecting an optical pattern when tracking a moving object in virtual or augmented reality systems. However, this method and / or system is fully applicable in any other field for solving problems associated with determining the position and / or orientation and / or motion parameters of a moving object.

In particular, the technical solutions described above can be successfully applied to track the movements of goods and operators in warehouse, production or trade logistics systems, to track the movements of participants in the educational process in educational systems, to determine the position and orientation of the working bodies of robotic systems, to control manned or unmanned moving objects, including aircraft, and to solve a variety of tasks related to tracking moving objects In other systems, existing or those that will be developed in the future.

List of Non-Patent Documents

1. R. van Liere, J.D. Mulder. Optical tracking using projective invariant marker pattern properties. DOI: 10.1109 / VR.2003.1191138

2. Manuel Loaiza, Alberto Raposo, Marcelo Gattass. A novel optical tracking algorithm for point-based projective invariant marker patterns. ISVC 2007: Advances in Visual Computing, Part I, LNCS 4841, pp. 160–169, 2007.

3. Daniel Wagner, Dieter Schmalstieg. ARToolKitPlus for pose tracking on mobile devices. Computer Vision Winter Workshop, St. Lambrecht, Austria, February 6–8, 2007.

4. Thomas Pintaric, Hannes Kaufmann. Affordable infrared-optical pose-tracking for virtual and augmented reality. Proceedings of Trends and Issues in Tracking for Virtual Environments Workshop, 2007.

5. Arjen van Rhijn, Jurriaan D. Mulder. Optical tracking and calibration of tangible interaction devices. IPT & EGVE Workshop, 2005.

6. Klaus Dorfmüller-Ulhaas. Robust optical user motion tracking using a Kalman filter. Report 2003-06, Institut für Informatik, Universität Augsburg, Germany.

7. A. Kolahi, M. Hoviattalab, T. Rezaeian, M. Alizadeh, M. Bostan, H. Mokhtarzadeh. Design of a marker-based human motion tracking system. Biomedical Signal Processing and Control, 2, 2007, pp. 59–67.

8. David Claus, Andrew W. Fitzgibbon. Reliable automatic calibration of a marker-based position tracking system. Proceedings of IEEE Workshop on Motion and Video Computing Motion, 2005, Application of Computer Vision, 2005. WACV / MOTIONS '05 Volume.

9. Steven Maesen, Patrik Goorts, Philippe Bekaert. Scalable optical tracking for navigating large virtual environments using spatially encoded markers. Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology VRST '13, pp. 101-110.

10. Filippo Bergamasco, Andrea Albarelli, Emanuele Rodola, Andrea Torsello. RUNE-Tag: a high accuracy fiducial marker with strong occlusion resilience. DOI: 10.1109 / CVPR.2011.5995544.

11. Leonid Naimark, Eric Foxlin. Encoded LED system for optical trackers. DOI: 10.1109 / ISMAR.2005.28.

12. Yashar Deldjoo, Reza Ebrahimi Atani. A low-cost infrared-optical head tracking solution for virtual 3D audio environment using the Nintendo Wii-remote. Entertainment Computing, vol. 12, 2016, pp. 9–27.

Claims (56)

1. A method for tracking a moving object, including the following steps: - perform automatic adjustment of the tracking zone by identifying and registering unique combinations of elementary optical patterns - “constellations”: (S1) determine a tracking area; (S2) determine the size of the tracking window and set the tracking window to its initial position; (S3) determine in the tracking window elementary optical patterns formed by markers; (S4) form the ID of the "constellation" of the candidate; (S5) verify the uniqueness of each “constellation” of the candidate and mark the ID of the non-unique “constellation” of the candidate; (S6) write the ID of the "constellation" of the candidate in the table of "constellations"; (S7) rotate the tracking window by a predetermined angle; (S8) perform steps S2 – S6 for the rotated tracking window; (S9) check whether the required number of turns of the tracking window has been completed, and if necessary, repeat steps S7, S8; (S10) moving the tracking window with a predetermined step; (S11) perform steps S2 – S9 for the moved tracking window; (S12) check if the entire tracking area is covered, and if necessary, repeat steps S10, S11; (S13) remove non-unique IDs of the “constellations” of candidates from the table of “constellations”; - track the change in the position and / or orientation of the moving object by identifying unique combinations of elementary optical patterns and comparing them with elementary optical patterns registered when setting up the tracking zone. 2. The method of claim 1, wherein steps S4, S5, S6, and S13 are performed with respect to each candidate constellation ID and the ID of each of its child candidate constellations. 3. The method according to claim 1, wherein prior to performing step S2, the projection of the tracking zone is divided into cells and the size of the tracking window in step S2 is expressed in the number of such cells. 4. The method according to claim 1, in which the step of tracking changes in the position and / or orientation of the moving object includes the following steps: (S21) read the image from the optical sensor located on the moving object, and perform its preliminary processing; (S22) detecting elementary optical patterns formed by markers in the tracking window; (S23) form a constellation ID; (S24) identify the generated constellation ID from the constellations table; (S25) determine the position and orientation of the moving object based on the "constellation" identified in step S24. 5. The method of claim 4, wherein step S21 includes the following steps: (S31) read the image from the optical sensor; (S32) perform geometric distortion compensation of the read image; (S33) optionally projecting the image onto the work plane; (S34) normalize the image; (S35) highlight the tracking window on the image. 6. The method according to claim 4, in which step S22 includes the following steps: (S41) three points corresponding to markers are arbitrarily selected in the tracking window; (S42) from the selected three points, two reference points are determined; (S43) for the reference points, the nominal position of the third point is determined in accordance with the elementary optical pattern; (S44) determining a deviation δ of the geometric position of the third point from its nominal position; (S45) checking the value of δ; if the value of δ does not exceed a predetermined threshold value, then (S46) the three points are added to the list of elementary optical candidate patterns, and if the value of δ exceeds the predetermined threshold value, then (S47) the three points are rejected; (S48) check whether all combinations of the three points are processed and, if necessary, repeat steps S41 – S47; (S49) sorting the list of elementary optical candidate patterns by the value of δ; (S50) from the list of elementary candidate patterns, an elementary optical candidate pattern with a minimum value of δ is selected; (S51) checking for the presence of used points in the elementary optical candidate pattern; if at least one used point is detected in it, then (S52) such an elementary optical candidate pattern is rejected, and if no used points are found, then (S53) it is moved to the list of valid elementary optical patterns, while the points included in this elementary optical pattern, marked as used; (S54) perform steps S50 – S53 with respect to the remaining elementary optical candidate patterns. 7. A tracking system for a moving object, comprising: - at least one tracker located on a moving object, comprising an optical sensor and performing a method for tracking a moving object, as described in any one of paragraphs. 1-6; - at least one marker strip containing active markers forming elementary optical patterns in the image obtained from the optical sensor; - a central data processing device, in conjunction with the tracker, performing a method for tracking a moving object described in any one of paragraphs. 1-6. 8. The system of claim 7, wherein the optical sensor is a matrix optical sensor. 9. The system of claim 7, wherein at least a portion of the marker strips are located on a floor or other base. 10. The system of claim 9, wherein the marker strips are integrated with an elastic prefabricated floor covering. 11. The system of claim 7, wherein at least a portion of the marker strips are located on the supporting structures above the floor. 12. The system of claim 7, wherein the active markers are markers that emit light in the infrared range. 13. The system of claim 7, wherein the elementary optical pattern is a linear elementary optical pattern. 14. The system of claim 7, wherein the elementary optical pattern is a non-linear elementary optical pattern. 15. The system of claim 7, wherein the tracker and the central data processing device are connected by a wireless data transmission channel for transmitting setup data when the step of automatically adjusting the tracking area and data on the position and / or orientation of the tracker when performing the tracking step in the method, characterized in any one of paragraphs. 1-6. 16. The system of claim 7, configured to automatically adjust the tracking zone by identifying and registering unique combinations of elementary optical patterns in the manner described in any one of paragraphs. 1-6. 17. The system of claim 7, configured to track changes in the position and / or orientation of a moving object by identifying unique combinations of elementary optical patterns and comparing them with elementary optical patterns registered when setting up the tracking zone in a manner described in any one of paragraphs. 1-6. 18. The system of claim 7, wherein the tracker further comprises at least one inertial sensor and which is configured to track changes in the position and / or orientation of the moving object using the data of at least one inertial sensor during those periods time when it is impossible to identify the "constellations" in the manner described in any of paragraphs. 1-6.

Images