RU2773554C1 - Method for assembly of a demonstration arena (options), a system of assembly of a demonstration arena, a way to determine the position and orientation of the device within the environment, a system of demonstration arena for vehicles operating in the demonstration space (options), a system of demonstration arena (options) - Google Patents

Abstract

FIELD: assembly systems.

SUBSTANCE: inventions group relates to assembly method (options) and assembly system of a demonstration arena, a method for determining the position and orientation of a device within the surrounding structure, a demonstration arena system for vehicles operating in a demonstration space (options), a demonstration arena system (options). Common to all methods is that a code strip with code marks (a predetermined continuous pattern) is placed around the demonstration arena or on it (or on the surrounding structure) in a certain way, images are continuously captured by a positioning device, on the basis of which the position and three orientation parameters are determined. within the surrounding structure, which, in particular, are used in the motion control system of the device. Each of the variants of the system contains a code strip, a positioning device and a processor for implementing the operations of the respective methods.

EFFECT: improved positioning accuracy of devices during their movement within the demonstration (surrounding) space.

28 cl, 15 dwg

Description

Link to related applications

[01] The present invention claims priority over US16856230, filed 4/23/2020, and US16856256, filed 4/23/2020, which are given priority over provisional application US62837797, filed 4/24/2019. All of these applications are incorporated herein by reference in their entirety.

The field of technology to which the invention relates

[02] The present invention relates to robotic demonstration arenas and location systems based on patterns sensed by image sensors.

State of the art

[03] Various robots, including automated vehicles and other self-propelled devices, are designed to perform various functions or activities, often in a specific work area or demonstration arena. Such features may include, but are not limited to, production tasks, service tasks, and competitions. In demonstrations, it is often desirable or necessary for the robots to be in various positions in the demonstration arena, or move to these positions, or even be able to track themselves in order to determine and use their position in such demonstration arenas. Some such positioning systems are known, but require improvement in speed, simplicity, computational savings, and versatility.

[04] The above examples of the prior art and the disadvantages associated with them are illustrative and are not exclusive. Other disadvantages of prior art solutions and other examples of the prior art should be clear to experts in the art after reading the description and studying the drawings.

Disclosure of invention

[05] The following embodiments of the invention and their aspects are described and illustrated by systems, structures, methods and implementations, which should be considered as examples and illustrations, not limiting the scope of the invention. In various embodiments and implementations of the invention, one or more disadvantages are reduced or eliminated, and other embodiments are directed to other improvements and beneficial results.

[06] In one embodiment of the demo arena system for vehicles operating in the demo space, the demo arena system comprises a code strip having a start edge and a predetermined length to the end edge, comprising a predetermined number of intervals located from the start edge to the end edge , wherein the code strip may be set to a start position adjacent to the sample space and may occupy a predetermined number of slots adjacent to the sample space.

[07] In another embodiment, the code strip contains a center line of code and can be installed adjacent to the display space in such a way that it forms a reference plane above the display surface, while the sequence of code marks in the code strip contains a non-repeating sequence of blocks at regular intervals along the center line code.

[08] In another embodiment, the non-repeating sequence of blocks contains dark blocks interspersed with light blocks.

[09] In another embodiment, some of the dark and light blocks are located above the center line of the code, and some others of the dark and light blocks are located below the center line of the code.

[010] In another embodiment, any interval containing a dark block above the code center line contains a light block below the code center line, and any interval containing a light block above the code center line contains a dark block below the code center line, so that transitions light blocks to dark blocks above the center line of the code coincide with transitions of dark blocks to light blocks below the center line of the code, and transitions of dark blocks to light blocks above the center line of the code coincide with transitions of light blocks to dark blocks below the center line of the code, while matching the corners dark blocks above the center line of the code with the corners of the dark blocks below the center line of the code form characteristic points on the center line of the code.

[011] In another embodiment, the positioning device located in the demo space contains a digital camera for obtaining an image containing one or more parts of the mark sequence, while the digital camera contains a lens system for focusing the image in the image sensor, and the lens system has an optical center .

[012] In another embodiment, the processor is programmed to compare a digital camera image with a sequence of marks in memory in order to identify one or more parts of the sequence of marks in the image and generate signals indicative of the position and orientation of the positioner in the demo space.

[013] In another embodiment, the positioning device may be mounted on the vehicle such that the optical center is located in the reference plane.

[014] In another embodiment, the processor is programmed to identify the center line of the code from the feature points formed by matching the corners of the dark blocks above the center line of the code with the corners of the dark blocks below the center line of the code forming the feature points on the center line of the code.

[015] In another embodiment, the processor is programmed to identify the location of feature points in the image as the corresponding location of such feature points on the code strip, and thus the location of such feature points around the perimeter of the arena with respect to the initial position, by determining the location of the respective feature points in a sequence of labels and specific locations around the arena in memory.

[016] In another embodiment, the display surface is in the shape of a polygon, and the display space is surrounded by walls intersecting at the corners, with the start position being one of the corners.

[017] In yet another embodiment, the demo arena system comprises (1) a code strip located at least around a portion of the perimeter of the arena space above the demo surface and comprising a sequence of code marks beginning at a start position, with portions of the sequence of marks at any particular position next to the arena space relative to the starting position are unique to that specific position; (2) a positioning device located in the arena, containing a digital camera for obtaining an image, including one or more parts of a sequence of marks; and (3) a processor programmed to generate signals indicative of the position and orientation of the arena positioner based on part or more parts of a sequence of marks in the image.

[018] In another embodiment, a demo arena assembly method includes placing a code strip at least partially around or on the demo arena, wherein the code strip has a start edge and an end edge with a code length between the start edge and the end edge, placing the code strip strips at least partially around or on the demo arena includes placing a start edge at a start position relative to the demo arena having a particular location relative to the demo arena, the code strip contains a plurality of feature points along the length of the code between the start edge and the end edge, and each feature the point is at a predetermined distance from the start edge such that each of the feature points along the length of the code is at a corresponding predetermined distance from the start position.

[019] In another embodiment, the method includes assembling a plurality of walls around the demo space, intersecting them at a plurality of corners, designating one of the corners as a start position, and mounting the code strip on the walls with the start edge of the code strip located at the corner designated as the start position, wherein each of the characteristic points in the code strip is located at a corresponding predetermined distance from the corner designated by the initial position.

[020] In another embodiment, the method includes assembling a plurality of walls on a display surface and mounting a code strip on the walls such that a center line of code on the code strip is at a constant height above the display surface.

Brief description of the drawings

[021] the Attached drawings, included in this description and forming part of it, illustrate some, but not the only and not exclusive, embodiments and/or features. It is intended that the embodiments and drawings described herein be considered illustrative and not restrictive. The drawings contain the following.

[022] In FIG. 1 is an isometric view of an exemplary robot demonstration arena with a schematic of an exemplary positioning device mounted on the robot.

[023] In FIG. 2 is a schematic side view of the exemplary positioning device of FIG. one.

[024] In FIG. 3 is an enlarged view of a portion of the exemplary code strip of FIG. one.

[025] In FIG. 4 is a schematic plan view of the exemplary robot demonstration arena of FIG. 1 showing an exemplary coordinate system and local coordinates with exemplary positioning element diagrams.

[026] In FIG. 5 is a flow diagram of an exemplary process implemented by the positioning device to locate the optical center in the positioning device in the demo arena of FIG. one.

[027] In FIG. 6 is a schematic representation of an exemplary image of a portion of an exemplary code strip captured by the lens system in the positioner of FIG. one.

[028] In FIG. 7 is a schematic representation of an exemplary codeband feature dot map created by convolving the exemplary image of FIG. 6 with the core feature detection.

[029] In FIG. 8 is a schematic representation of an exemplary kernel for use in a convolution generating the exemplary feature point map of FIG. 7.

[030] In FIG. 9 is a schematic representation of the illustrative image of FIG. 6 with a line drawn thereon coinciding with the center line of the code strip, containing a schematic representation of an exemplary zero-mean filtering process used to detect code in an exemplary image.

[031] In FIG. 10 is an enlarged schematic view of the approximation line and the perpendicular line in pixel scale.

[032] In FIG. 11 is an image of an exemplary signal determined from an exemplary codeband using a zero-mean filtering process.

[033] In FIG. 12 is a side view to show the parameters of FIG. 4 when the positioner's camera is tilted down and the code bar is still visible at the top of the field of view.

[034] In FIG. 13 is a conceptual view of camera tilt and pan showing the parameters related to compensation and pinpointing.

[035] In FIG. 14 is a diagram illustrating lens distortion and the radial representation used to correct for such distortion.

[036] In FIG. 15 is a schematic plan view of the exemplary robot demonstration arena of FIG. 1 with several robots equipped with positioning devices and an exemplary stationary object in a demonstration arena.

Implementation of the invention

[037] An exemplary demonstration arena 10 for robots or other vehicles is shown in FIG. 1 with an exemplary positioning device 12 mounted on a vehicle, such as a robot 14 located in a demonstration arena 10. Code strip 16 is located at least partially around the perimeter of the demonstration space 17 above the demonstration surface 18. The vehicle 14 may be automated, self-propelled or other vehicle or device that can move or be moved along various paths on the display surface 18 to various positions within the display arena 10. For the purposes of this description, the terms "vehicle" and "robot" are used interchangeably for convenience in describing the entire system or method, since the specific type of vehicle 14 on which the positioning device 12 can be installed is not a feature limiting the invention. By way of example, and not limitation, the exemplary vehicle 14 shown schematically in FIG. 1 may be a competitive robot whose task may be to perform various prescribed actions, maneuvers or achieve various goals within the demonstration arena 10. The positioning device 12 mounted on the robot 14 provides the robot 14 with signals indicating the position and orientation of the positioning device in an arbitrary point in time or to change position and orientation in real time. The position signals may be used by the robot 14 in any manner desired by the designer or operator of the robot to perform actions or achieve goals. The robot 14 is shown to facilitate explanations of the usefulness of the demonstration arena 10 and the positioner 12, but the robot 14 itself and the specific programming of the robot 14 to use the position signals provided by the positioner 12 are not part of the present invention. Accordingly, the positioning device 12 is schematically shown as mounted on the robot 14 with an electrical data cable 20 for transmitting such position signals to the robot 14. However, such signals may be transmitted from the positioning device 12 to the robot 14 wirelessly, in any other convenient way or in another format known in the art, such as using frequency modulated electromagnetic radiation, infrared radiation, sound, and the like.

[038] Although an exemplary robot demonstration arena 10 is shown in FIG. 1 as a square arena, it may have other shapes such as rectangular, polygonal, round, oval, etc. In addition, illustrative elements such as the length and height of the sides or walls around the display arena 10, the area of the display surface 18, the dimensions of the robot 14 and the positioning device 12, and the like, are shown without being drawn to scale or in proportion to each other from - due to the limitations of the paper sheet, but the principles should be clear to those skilled in the art after reading the following explanations regarding these elements. As an illustrative example, the rectangular display arena 10 may have walls 22, 24, 26, 28, each twelve feet (3.66 m) long and twelve inches (0.3 m) high. Code strip 16 is shown in FIG. 1 as extending from its start edge 29 located at a predetermined start position 30 (sometimes referred to as start position 30 in this specification) around the perimeter of the display arena 10, for example, from a certain corner 31 of the display arena 10 along the perimeter of the display arena 10 to the end edge 33 code bar 16 back to the start position 30. However, code bar 16 may extend only along part or several parts of the perimeter, or it may pass through the sample space or around (or partially around) an object in the sample space, if the code bar 16 has a known initial position with respect to the demonstration arena 10 and at least part of the code strip 16 can be fixed on the image by the positioning device 12. In other words, the code bar 16 is located in some way adjacent to the demo space, or its initial position may be at a known location relative to the demo space in the demo arena, and the demo space positioner 12 can capture an image of at least a portion of the code bar. 16 containing a sufficient number of feature points to allow the positioning device 12 to locate the positioning device 12 in the display space. The exact location of each feature point 60 (FIGS. 3, 4, 6 and 7) of the code strip 16 is known from the code structure. In other words, the exact distance of each feature point 60 from the start edge 29 of the code bar 16 is known from the code structure. position 30, for example, at the designated corner 31 of the exemplary square display arena 10 shown in FIG. 1, the exact distance of each feature point 60 of the code bar 16 from the designated corner 31 of the demo arena 10 is known. Accordingly, the exact location of each feature point 60 of the code bar 16 relative to the assigned corner 31 is known.

[039] In this example, code strip 16 may be applied to walls 22, 24, 26, 28 in any convenient manner. For example, the code may be printed on any suitable substrate material 32, including being printed using any known printing processes and consumables on a material 32 such as paper, plastic, wood, or other material, and made suitable for installation. on walls 22, 24, 26, 28. In one example, backing material 32 may include a self-adhesive back surface that can be used to adhere code strip 16 backing material 32 to walls 22, 24, 26, 28. Exemplary backing material 32 may have a code strip printed or otherwise applied to the front (visible) surface of the substrate material 32 located up to the top edge 34 of the substrate material 32 so that the mounting of the substrate material 32 on the walls 22, 24, 26, 28 is flush with the top edge 34 of the substrate material 32 with top edges 42, 44, 46, 48 of walls 22, 24, 26, 28 parallel to the display surface 18, is a convenient way to mount coded The first stripe 16 is parallel to the display surface 18 at a constant height above the display surface 18. The code strip 16 may be silk-screened, laser engraved, printed, painted or formed in any other way. This makes it convenient and easy to assemble multiple robot demonstration arenas 10 of the same shape and size, such as demonstration arenas in different locations for robot competitions with standardized competition rules, characteristics, and capabilities, as described in more detail below. For example, multiple identical code bars 16 can be easily manufactured and installed in various demonstration arenas 10 that are sized and shaped to match the sizes and shapes for which such code bars 16 are designed. , equipped with a positioning device 12, can perform and compete without calibration in any of these competitions using demonstration arenas having the same dimensions and equipped with the same copies of such code strips 16. As another example, an "umbrella" sponsor or organizer of a robotics competition, in particular, the Robotics Education and Competition Foundation, may designate a specific shape and size for the demonstration arena 10 for robotics competitions held under its auspices, and such sponsor or organizer may print or otherwise produce multiple copies of the code strip 16 with exact dimensions corresponding to 10. A local organizer of a local robotics competition, such as an elementary school, high school, college, or club, wishing to host a local robotics competition according to the rules, objectives, and other requirements of the umbrella sponsor or organizer, can simply build a local robot demo arena 10 with the shape and dimensions specified by the umbrella sponsor or host, and the umbrella sponsor or host may provide code strip 16 to the local host to mount in a specified manner in the local host's demo arena 10. The code strip 16 may be a single piece or may be made up of several pieces, such as one piece for mounting on each of the four walls 22, 24, 26, 28 of the exemplary square display arena 10 of FIG. 1. Regardless of the number of parts, the code sequence extends from the start position 30 at least some distance from the demo space 17, which may at least partially (but need not) be located around the perimeter of the demo arena 10. In the exemplary demo arena 10 shown in fig. 1, the code sequence extends from the start position 30 around the entire perimeter of the display space 17 and returns to the start position 30. Accordingly, a large number, for example, hundreds of precisely located feature points 60 can be easily positioned and mounted by the user with minimal instructions and without the need to measure the position. each characteristic point around the display space 17. In the exemplary square display arena 10 described above, the local organizer only needs to mount the code strip 16 around the display arena 10 so that its starting edge 30 is located in the corner 31 of the display arena 10, after which the exact location of each characteristic point 60 of the code strip 16 relative to the angle 31 in the local demonstration arena 10 will be unambiguously known without the need for further measurements or alignment, which is true for any other local, regional or world demo onstration arena 10 built and equipped in the same way. Accordingly, robots equipped with a positioning device 12 as described above and programmed to operate in such a local demonstration arena 10 can perform in all other demonstration arenas 10 similarly constructed and equipped without reprogramming their positioning means. In another embodiment, the code strip 16 may also be formed from segments of unique codes individually positioned, for example, to align such segments with various predetermined alignment features on walls or other surfaces to which they are affixed. This arrangement makes it possible to simplify the manufacture or installation of code strips 16 on multiple replicas of the demo arena 16, while still ensuring that the code strip 16 is accurately located on each such demo arena 16 for use by the positioning device 12 as described herein.

[040] An exemplary code strip 16 is shown in FIG. 1, and an enlarged view of a portion of exemplary code strip 16 is shown in FIG. 3. The exemplary code strip 16 comprises a predetermined sequence of code marks 50 starting at a start position 30, continuing along the center code line 52 around the perimeter of the demonstration arena 10, and ending again at the start position 30. Fragments of the sequence of marks 50 at a particular position around the demonstration arena 16 unique to this position. The code mark sequence 50 comprises a non-repeating sequence of dark blocks 54, 56 and light blocks 55, 57 at intervals 58 along the code center line 52 such that intervals 58 and blocks 54, 55, 56, 57 are located from the starting position 30 around the perimeter of the display arena 10 and back to the start position 30. In the exemplary code strip 16, the intervals 58 are evenly spaced, that is, have the same width, but in some implementations they may have different widths. Some of the blocks, such as dark blocks 54 and light blocks 55, are above the center code line 52, and some of the blocks, such as dark blocks 56 and light blocks 57, are below the center code line 52. Dark blocks 54, 56 in the exemplary code strip 16 shown in FIG. 1 and 3 are shown as black blocks and the light blocks 55, 57 in exemplary code strip 16 are shown as white blocks. Such black blocks 54, 56 and white blocks 55, 57 are conveniently produced by simply printing the black blocks 54, 56 on the white substrate material 32, resulting in the white blocks 55, 57 being in the intervals 58 between the black blocks 54, 56. However, , the dark blocks 54, 56 and the light blocks 55, 57 may be of contrasting colors, patterns, or other forms. Accordingly, for the purposes of the present description, the terms "dark blocks" and "light blocks" include contrasting colors, patterns, and other implementations, regardless of intensity, provided that they are different from each other. For example, red and blue blocks may have the same intensity but be differentiated by suitable filtering or other processes. Also, for example, black blocks and white blocks are considered to be dark blocks and light blocks, respectively. Also, as shown in exemplary code strip 16, some of the dark blocks 54, 56 are located in adjacent intervals 58, effectively expanding such adjacent dark blocks 54, 56 to two adjacent intervals and giving the appearance of a double length dark block. Similarly, some of the light blocks 55, 57 are located in adjacent intervals 58, which also effectively expands such adjacent light blocks 55, 57 to two adjacent intervals and gives the appearance of a double length light block. Such dark blocks 54, 56 and light blocks 55, 57 are detectable.

[041] In the exemplary code strip 16 shown in FIG. 1 and 3, any interval 58 containing a dark block 54 above code center line 52 contains a light block 57 below code center line 52, and any interval 58 containing a light block 55 above code center line 52 contains a dark block 56 below the center line. 52 code so that the transitions of light blocks 55 to dark blocks 54 above code center line 52 coincide with transitions of dark blocks 56 to light blocks 57 below code center line 52, and transitions of dark blocks 54 to light blocks 55 above code center line 52 coincide with transitions of light blocks 57 to dark blocks 56 below the center line 52 of the code. Accordingly, the coincidence of the corners of the dark blocks 54 above the center line 52 of the code with the corners of the dark blocks 56 below the center line 52 of the code form characteristic points 60 on the center line 52 of the code, which are detectable.

[042] The exemplary code strip 16 shown in FIG. 1 and 3 and described above, can be constructed from a maximum length pseudo-random sequence of dark blocks 54, 56 and light blocks 55, 57, which extends in length from the start position 30 around the perimeter of the display arena 10 and back to the start position 30, as described above. Those skilled in the art will appreciate that the maximum length sequences are the largest code sequences that can be built without repetition in the smallest sliding window. Methods of generating such code sequences of maximum length are known to those skilled in the art, such as linear feedback shift registers (Galois configuration) or linear feedback shift registers (Fibonacci configuration). The MathWorks™ MATLAB program can be used to generate the required code sequences with many possible maximum lengths. For example, code strip 16 may be constructed from a maximum length pseudo-random sequence with a Manchester code of black (dark) blocks 54, 56 and white (light) blocks, which extends in length from the start position 30 around the perimeter of the demonstration arena 10 and back to the start position 30. Those skilled in the art will understand that in a Manchester code, each bit of an uncoded bit sequence is encoded as two bits of the Manchester code of that uncoded bit sequence. A convenient example is a Manchester sequence code with a maximum length of 255. If an uncoded sequence contains 255 bits, then the uncoded sequence's Manchester code contains 510 bits - exactly twice as many as the uncoded bit sequence. In particular, in the Manchester code, each MSB of the uncoded sequence is replaced by the MSB followed by the LSB in the Manchester code bit sequence. In Manchester code, each least significant bit of the uncoded bit sequence is replaced by the least significant bit followed by the most significant bit in the Manchester code bit sequence. In an exemplary code band, a 16-bit sequence generated by an 8-bit Linear Feedback Shift Register (LFSR) as a pseudo-random bit sequence of maximum length with a length of 255 is encoded with a Manchester code to result in a bit sequence of length 510 bits with Manchester code, while the window 59 (Fig. 3) with a width of fifteen (15) consecutive intervals 58 can slide over possible positions along the entire length of the code strip 16, including the transition through the initial position 30, while the 15-slot code sequence is not repeated in any way. in one place of the demonstration arena 10.

[043] However, it should be noted that such sequences need not be maximum length sequences. It is only necessary that the sequence does not repeat itself in the minimum sliding window. If the sequence is not a maximum length sequence, then a sliding window containing more intervals 58 must be used to ensure that the sliding window does not include any repeating pattern anywhere in the demo arena 16. In other words, using a non-maximum length sequence means that the image sensor 64 must capture more adjacent non-overlapping bits so that the imager 12 can unambiguously determine its position. Thus, unlike the case of using the maximum length sequence described above, using a non-maximum length sequence requires the image sensor 64 to capture more than a 15-slot code sequence.

[044] The exemplary codeband 16 with the maximum length sequence is further encoded such that each interval 58 contains a black block 54 above the center line 52 of the code and a white block 57 below the center line 52 of the code to indicate the MSB of the sequence with the Manchester encoding of the code strip 16, or a white block 55 above the center line 52 and a black block 56 below the center line 52 to indicate the least significant bit of the Manchester-encoded codeband 16 sequence. This arrangement allows codeband 16 or codeband 16 fragments to be decoded with a zero-mean filter, as in more detail described below. With a predetermined non-repeating code of the code strip 16 mounted around the perimeter of the demonstration arena 10, as shown in FIG. 1 and described above, the exact location in three dimensions of each feature point 60 of the code strip 16 is known from its structure, and there are hundreds of such feature points 60 at precisely known locations around the perimeter of the display arena 10. As noted above, these feature points 60 can be detected. .

[045] As shown in FIG. 1 and 2, exemplary positioner 12 includes a digital camera with a lens system 62 for focusing an image of at least a portion of code strip 16 in image sensor 64. The lens system 62 may be any of a variety of conventional digital camera lens systems for providing appropriate focus and zoom in the image sensor 64 that are well known to those skilled in the art of digital cameras, including, for example, a single lens or a combination of lenses. The camera lens system 62 has an effective optical center 66 for projecting and focusing the image in the image sensor 64 . The effective optical center 66 is the point near the lens system 62 at which the virtual point of best approximation of the image function of the lens system 62 in the image sensor 64 is located. In other words, the optical center 66 is the point through which all rays pass from the imaged objects to their respective images projected into the image sensor 64 by the lens system 62, as will be appreciated by those skilled in the art of digital cameras. In this case, the optical axis 68 of the lens system 62 passes through the optical center 66 perpendicular to the plane of the image sensor 64. The focal length F is the distance from the optical center 66 to the image sensor 64 . The viewing angle 70 is set by the focal length F of the lens system 62 and the dimensions of the image sensor 64 . The number of intervals 58 of the code strip 16 captured by the image sensor 64 for a particular field of view (angle) 70 varies depending on the distance from the optical center 66 to that part of the code strip 16 that is displayed in the image sensor 64. A wider field of view (angle) 70 is desirable to display a sufficient number of code marks 50 so that the positioner 12 can locate the exact position of the optical center 66 in the demonstration arena 16 even when the robot 14 and the positioner 12 are near the code strip 16. to accurately identify code blocks 54, 55, 56, 57 and feature points 60 during decoding, it is also desirable that the image of code strip 16 captured by image sensor 64 have as many pixels as possible in code block 54, 55, 56, 57, even when the robot 14 and the positioner 12 are located at a large distance from the code strip 16. Accordingly, the structure of the focal length F and the size of the image sensor 64 for the desired field of view 70 may represent a technical compromise between providing a sufficiently wide field of view 70 to ensure that when the robot 14 moves and positioning devices 12 in position, d sufficiently close to the code band 16, capturing the minimum number of code marks 50 and characteristic points 60, sufficient for the positioning device to determine the position of the optical center 66 on the demonstration arena 10 at such close distances, but at the same time a sufficiently narrow field of view 70 for the image sensor 64, providing capturing enough pixels in the code block 54, 55, 56, 57 by moving the robot 14 and the positioner 12 to a position in the demo arena 10 sufficiently far from the code bar 16 to allow the positioner 12 to determine the position of the optical center 66 in the demo arena 10 when such long distances.

[046] The exemplary positioning device 12 includes a digital processor 74, such as a Central Processing Unit (CPU), for determining the positions of the optical center 66 of the positioning device 12 based on the images of the code strip 16 fragments, as described in more detail below, and outputs signals indicating such positions. In general, the use of the terms "position" and "location" in relation to the optical center 66 refers to the spatial position and the angular orientation of the optical center 66, unless the context indicates otherwise. If the positioning device 12 is mounted on the robot 14, for example, as shown schematically in FIG. 1, any particular location of optical center 66 on display arena 10 can be correlated to specific locations of robot 14 or robot elements 14 on display arena 10 based on the known dimensions of robot 14 and the position of optical center 66 on robot 14. Outputs generated by processor 74 and indicating the location of the optical center 66 are transmitted by the processor 74 to the robot 14 via the electrical data cable 20 or by other means and can be used in any way provided by the design or determined by the robot 14 for use in performing actions, maneuvers or tasks.

[047] The code strip 16 is preferably mounted such that the center line 52 of the code is at the same height h above the display surface 18 around the entire perimeter of the display arena 10, as shown schematically in FIG. 1. As indicated above, the alignment of the top edge 34 of the substrate material 32 and, accordingly, the top of the code strip 16 with the top edges 42, 44, 46, 48 of the walls 22, 24, 26, 28 facilitates such mounting of the code strip 16 with the center line 52 code at the same height h above the display surface 18. Thus, the central line 52 of the code around the display arena 10 forms a plane (indicated schematically by dotted lines 76) at a height h above the entire display surface 18 of the display arena 10. The positioning device 12 is located on the display arena 10 , and the optical center 66 and the optical axis 68 are preferably, but not necessarily, located in the plane 76, that is, also at a height h above the display surface. Accordingly, a manufacturer of robot 14 desiring to use exemplary positioner 12 with robot 14 may mount positioner 12 on robot 14 in such a position on robot 14 that optical center 66 is at a height h above display surface 18, as shown in FIG. 1. In this position, the optical axis 68 of the camera in the positioning device 12 is located in the plane 76 and thus the feature points 60 in the acquired image 104 always lie in a straight line.

) определяется устройством 12 позиционирования посредством процесса, который начинается с фиксации изображения датчиком 62 изображения (фиг. 2) части кодовой полосы 16, содержащей больше некоторого минимального количества характерных точек 60, как более подробно описано ниже (и как упоминалось выше в описании скользящего окна 59 на фиг. 3), а затем выполняется первичное прогнозирование или предположение в отношении местоположения (x, y) оптического центра 66 и начальной угловой ориентации (ϕ, θ, ψ) оптической оси 68, из которого следует первичное прогнозирование или предположение относительно значений s′ на виртуальном экране 92, причем такое прогнозирование или предположение предусматривает, что соответствующие лучи 86, 88, 90 из соответствующих характерных точек 60 (например,

на фиг. 4) проходят через виртуальный экран 92. Затем измеряются фактические значения s в соответствующих точках 85, 87, 89 на виртуальном экране 92, при этом соответствующие лучи 86, 88, 90 из соответствующих характерных точек 60 (например,

на фиг. 4) проходят через виртуальный экран 92. Эти фактические значения s соответствуют пикселям на изображении, зафиксированном датчиком 62 изображения. После вычисления метрики E ошибки, указывающей на различия между прогнозными значениями s′ и измеренными значениями s, делается новый прогноз местоположения (x, y) оптического центра 66 и угловой ориентации (ϕ, θ, ψ) оптической оси 68 на основе градиента метрики E ошибки. Иными словами, выполняется новое предположение относительно координат (x, y, ϕ, θ, ψ) оптического центра 66 и оптической оси 68 и процесс повторяется итеративно до тех пор, пока метрика E ошибки не снизится до заранее заданного удовлетворительного или приемлемого значения. Последнее местоположение и угловые координаты (x, y, ϕ, θ, ψ), приводящие к удовлетворительному или приемлемому значению метрики E ошибки, выдаются устройством 12 позиционирования в качестве точного представления фактического местоположения и угловой ориентации устройства 12 позиционирования на демонстрационной арене 10 в этот момент. Естественно, этот процесс может быстро и многократно повторяться устройством 12 позиционирования, чтобы выдавать такие координаты положения и угловые координаты (x, y, ϕ, θ, ψ) по существу в режиме реального времени при перемещении устройства 12 позиционирования в пространстве демонстрационной арены 10, например, с помощью робота 14, на котором может быть установлено это устройство 12 позиционирования.[048] As shown in FIG. 4, a coordinate system, such as a Cartesian coordinate system 78, is associated with the demo arena 10. The origin 80 (0,0) of the coordinate system 78 is located at the center of the demo arena 10 with the positive x -axis 82 to the right and the positive y -axis 84 to the front (i.e., the positive 82 x axis is on the right in the paper plane, and the positive 84 y axis is on the top in the paper plane). The optical center 66 of the positioner 12 is located at some location ( x , y ) in the demonstration arena 10, delimited by the code strip 16, as described above. The location of the optical center 66 also has a value ( z ) vertically, but the optical center 66 is always in plane 76 at a height h above the display surface 18, as described above, so the value ( z ) vertically is unchanged and can be neglected. In addition, there are always three angles associated with the position of the optical center 66, ie compass angle or yaw (ϕ), positive or negative pitch (θ) and roll (ψ). Accordingly, a complete description of the location of the optical center 66 contains three spatial dimensions and three angular orientations ( x , y , z , ϕ, θ, ψ). However, for simplicity and to avoid unnecessary repetition, specific location values ( x , y , z , ϕ, θ, ψ) are used as needed for specific calculations or steps. In this example, if we neglect the vertical value ( z ) as described above, the actual position ( x , y , ϕ, θ, ψ) of the optical center 66 in the demonstration arena 10 (represented in FIG. 4 as

) is determined by positioner 12 through a process that begins with image sensor 62 (FIG. 2) capturing a portion of codeband 16 containing more than a certain minimum number of feature dots 60, as described in more detail below (and as mentioned above in the description of sliding window 59 in Fig. 3), and then an initial prediction or guess is made regarding the location ( x , y ) of the optical center 66 and the initial angular orientation (ϕ, θ, ψ) of the optical axis 68, from which follows the initial prediction or guess about the values of s ′ on the virtual screen 92, and such a prediction or assumption assumes that the corresponding rays 86, 88, 90 from the corresponding characteristic points 60 (for example,

in fig. 4) pass through the virtual screen 92. The actual values of s are then measured at the respective points 85, 87, 89 on the virtual screen 92, with the respective rays 86, 88, 90 from the respective characteristic points 60 (for example,

in fig. 4) pass through the virtual screen 92. These actual values of s correspond to the pixels in the image captured by the image sensor 62. After computing the error metric E indicative of the differences between the predicted values s ′ and the measured values s , a new prediction is made of the location ( x , y ) of the optical center 66 and the angular orientation (ϕ, θ, ψ) of the optical axis 68 based on the gradient of the error metric E . In other words, a new assumption is made about the coordinates ( x , y , ϕ, θ, ψ) of the optical center 66 and the optical axis 68 and the process is iteratively repeated until the error metric E decreases to a predetermined satisfactory or acceptable value. The last position and angular coordinates ( x , y , ϕ, θ, ψ) resulting in a satisfactory or acceptable value of the error metric E are output by the positioning device 12 as an accurate representation of the actual position and angular orientation of the positioning device 12 in the demonstration arena 10 at that moment . Naturally, this process can be quickly and repeatedly repeated by the positioning device 12 to provide such position coordinates and angular coordinates ( x , y , ϕ, θ, ψ) in substantially real time as the positioning device 12 moves in the space of the demonstration arena 10, for example , by means of a robot 14 on which this positioning device 12 can be mounted.

) образуют лучи 86, 88, 90 к оптическому центру 66, проходящие через виртуальный экран 92 в соответствующих точках 85, 87, 89, что соответствует фиксации света пикселей датчиком 64 изображения. Пересечения лучей 86, 88, 90 с виртуальным экраном 92 описывают ожидаемое положение пикселей характерных точек 60 (например,

) в зафиксированном изображении. Преобразования, описывающие такое отображение лучей в положение пикселей, известны специалистам в данной области техники, а один их пример более подробно описан ниже.[049] As shown in FIG. 5 and other figures, an exemplary process of determining the location and orientation ( x , y , ϕ, θ, ψ) of the optical center 66 of the positioning device 12 starts at point 100. axis 68 (FIGS. 1-3) and hence also image sensor 64. An example of such a portion of code strip 16 is shown in FIG. 3, and in the image capture example shown in FIG. 4, characteristic points 60 (for example,

) form rays 86, 88, 90 to the optical center 66 passing through the virtual screen 92 at the corresponding points 85, 87, 89, which corresponds to the fixation of the light of the pixels by the image sensor 64. The intersections of the beams 86, 88, 90 with the virtual screen 92 describe the expected position of the pixels of the feature dots 60 (for example,

) in the captured image. Transformations describing such ray-to-pixel position mappings are known to those skilled in the art, and one example is described in more detail below.

[050] As described above, with a predetermined non-repeating code, the code strip 16 mounted around the perimeter of the demonstration arena 10 as shown in FIG. 1 and described above, the exact location in three dimensions of each feature point 60 on the code strip 16 is known from its structure and is held in memory associated with the processor 74. There are hundreds of such points 60 at precisely known locations around the perimeter of the demo arena 10. here, one exemplary code strip 16 for the exemplary arena 10 described above, measuring 12 x 12 feet (3.68 x 3.68 m), contains five hundred and ten (510) intervals 58 of equal length, starting, for example, at an angle of 30 (Fig. 1 ). In this example, there are four walls 22, 24, 26, 28, each 12 feet long, so the total length of code strip 16 is forty-eight (48) feet or five hundred and seventy-six (576) inches (14.63 m). Accordingly, for a total length of five hundred and ten (510) intervals 58, each interval 58 is 576/510 = 1.129 inches (28.68 mm). As above, the predetermined sequence of blocks 54, 56 in exemplary codeband 16 is Manchester encoded to create a series of equidistant code bits corresponding to equidistant intervals 58. Exemplary codeband 16 shown in FIG. 1 and 3 encodes each bit as either a white-on-black interval or a black-on-white interval. As also described above, transitions of white blocks 55 to black blocks 54 above code center line 52 coincide with transitions of black blocks 56 to white blocks 57 below code center line 52, and transitions of black blocks 54 to white blocks 55 above code center line 52 coincide with transitions of white blocks 57 to black blocks 56 below the center line 52 of the code. Accordingly, the coincidence of the corners of the black blocks 54 above the center line 52 of the code with the corners of the black blocks 56 below the center line 52 of the code form characteristic detectable points 60 on the center line 52 of the code. Since some of the black blocks 54 are adjacent to each other and some of the white blocks 56 are also are adjacent to each other, there are fewer transitions of white blocks 55 to black blocks 54 and vice versa located above and below the center line 52 of the code than the number of available intervals 58. Accordingly, the number of identified characteristic points 60 is less than five hundred and ten (510) intervals 58 , but there are still hundreds of such identifiable feature points 60 of code strip 16 at precisely located and well-known locations ( x , y ) around the demo arena 10.

[051] In step 102 of capturing the image in FIG. 5, the image 104 captured by the positioner 12 contains a portion 106 of the code strip 16 as shown, for example, in FIG. 6. Next, at feature point detection step 108, feature points 60 in portion 106 of code band 16 are detected by processor 74, as shown schematically in FIG. 7. To reduce computation without sacrificing performance, the positioner 12 may use a monochrome image sensor 64. In the example of the feature point detection step 108, the convolution of the image 104 using the feature detection kernel 110 shown in FIG. 8 results in a feature map 112 with a series of responses for feature points 60 in a substantially linear pattern as shown in FIG. 7. In regression step 114 for the line of best fit, it is determined that line 116 best represents the response positions for feature points 60, as shown in FIG. 9. Best fit line 116 corresponds to center code line 52 of code strip 16 shown in FIG. 1 and FIG. 2. Best fit line 116 is used in signal extraction step 118 to control the zero-mean filtering process described below to decode the portion 106 of codeband 16 captured in image 104. To improve reliability and reduce computational overhead, codeband 16 is mounted so that so that the center code line 52 is coplanar throughout the demo arena 10 as described above and shown in FIG. 1. In addition, the positioning device 12 is mounted on the robot 14 so that the optical center 66 remains in the plane 76 at a height h above the display surface 18, as also described above and shown in FIG. 1. Thus, regardless of the position ( x , y ) and orientation (ϕ, θ, ψ), the center line 52 of the code is transformed into lines 116 of the best fit (Fig. 9) even at corners 30 in which the end of one part of the code strip 16 on one wall of the demo arena 10 meets the beginning of another part of code strip 16.

[052] The zero-mean filtering process in signal extraction step 118 of FIG. 5 is illustrated in FIG. 9 and FIG. 10. Pixels along line 120 perpendicular to best fit line 116 are processed to generate a detectable signal 123, an example of which is shown in FIG. 11. In the exemplary zero-mean filtering process shown in FIG. 10, pixel intensities at the top 124 of perpendicular line 120 above best fit line 116 are added to the sum, and pixel intensities at bottom 126 of perpendicular line 120 below best fit line 116 are subtracted from the sum. In other words, for the position of the perpendicular line 120 shown in FIG. 9, the amplitude of each pixel captured by the image sensor 64 in the upper portion 124 of the perpendicular line 120 above the best fit line 116 is multiplied by +1, and the amplitude of each pixel captured by the image sensor 64 in the lower portion 126 of the perpendicular line 120 below the best fit line 116, multiplied by -1 and added to the sum. This sum is calculated iteratively, once for each position of the perpendicular line 120, that is, once for each perpendicular column of pixels in the portion 106 of the image 104 that intersects the best fit line 116. Accordingly, the filter gives a zero response for any homogeneous region, i.e. it excludes the mean for that area. It is only sensitive to areas where the average brightness above the best fit line 116 differs from the average brightness below the best fit line 116. The number of perpendicular lines is 120, i.e. vertical columns of pixels at interval 58 (FIG. 3) (for each block 54, 55, 56, 57) changes dramatically depending on how far the lens system 62 of positioner 12 is from code strip 16 (FIG. 1). If the lens system 62 is located very close to code band 16, there may be many (for example, tens) of perpendicular lines 120 (FIG. 9) summed at interval 58. If the lens system 62 is located far from code band 16, for example, at one the wall of the demonstration arena 10 and is directed to the opposite wall, there may be only a few, perhaps even less than two vertical lines 120 summed in the interval 58.

[053] As a result of these zero-mean filtering operations, a detectable signal 123 is generated, as shown, for example, in FIG. 11. The Manchester code present in the detected signal 123 is decoded in the decode and pattern search step 128 shown in FIG. 5 using any of a variety of decoding techniques known to those skilled in the art to detect codestream 130 bits of a pseudo-random sequence of portion 106 of codeband 16 in captured image 104. In this example, black block 54 above white block 57 is encoded as 1, and the white block 55 above the black block 56 is encoded as 0. There is a minimum number of continuous non-overlapping intervals 58 that must be captured in the image 104 so that the processor 74 of the imaging device 12 can determine which part of the code it perceives, depending on the number of encoded bits in the smallest cluster or sequence of bits in a coded bitstream that does not repeat anywhere in codeband 16, i.e. unique in the encoded bitstream. In this exemplary codeband 16, as a maximum length pseudo-random sequence with a length of 255, created using an 8-bit linear feedback shift register and followed by Manchester encoding to obtain a 510-bit long bit sequence as described above, the captured image 104 should contain at least fifteen (15) contiguous non-overlapping intervals (e.g., 16.9 inches) to allow determination of which part of the code it represents, since sequences of fifteen (15) or more bits are not repeated anywhere in the bitstream. Accordingly, a piece of code makes it possible to identify the exact portion of code captured in image 104. In addition, the memory associated with processor 74 contains a sequence of bits for the entire code band 16. binds to known coordinates ( x , y ) corresponding to feature points 60 in this part of the code, as indicated in the known coordinates binding step 132 in FIG. 5. Finding the codeband image 106 in the captured image 104 takes relatively little computation time, and after generating the detectable signal 123, the position calculation is reduced to a one-dimensional problem of relatively low complexity instead of a calculation problem in a two-dimensional image space.

), лучше всего показанные на фиг. 4, и идентифицирует точку

по информации о кодовой полосе 16, хранящейся в памяти, связанной с процессором 74, как описано выше. Следовательно, из этой информации в памяти о характерной точке 60 известно фактическое местоположение (x, y) точки

. При этом процессор 74 вычисляет точку 85, в которой луч 86 из точки

проходит через виртуальный экран 92, на основе предположения или оценки местоположения (x, y, ϕ, θ, ψ) для оптического центра 66, обозначенного точкой

на фиг. 4. При таком известном местоположении (x, y) точки

и таком предполагаемом или оцениваемом местоположении (x, y, ϕ, θ, ψ) точки

расстояние 94 в направлении вперед можно определить по следующей формуле: глубина в направлении вперед

где оператор «•» означает скалярное произведение, а

- направленный вперед вектор, совпадающий с оптической осью 68. Кроме того, расстояние 96 по перпендикуляру можно определить по следующей формуле: расстояние по перпендикуляру

где оператор «•» означает скалярное произведение, а

- вектор 185, перпендикулярный направленному вперед вектору

Расстояние s′ для прогнозируемой точки 85, в которой луч 86 из точки

проходит через виртуальный экран 92, на основе фактического положения точки

и этого допущения или оценки положения точки

, можно определить по следующей формуле: s′ = d *(расстояние по перпендикуляру)/(глубина в направлении вперед), где d - расстояние между оптическим центром 66 и виртуальным экраном 92, причем точки 85, 87, 89 пересечения лучей 84, 86, 88 масштабируются в пиксели датчика изображения 64 изображения с соответствующим заранее заданным расстоянием d до виртуального экрана. Расстояние d до виртуального экрана является параметром, специфичным для данной конструкции, оно рассчитывается один раз на заводе и впоследствии считается одинаковым для каждого устройства, имеющего такую конструкцию. В описанном выше примере с учетом иллюстративного положения и ориентации устройства 12 позиционирования, показанного на фиг. 4, расстояние по перпендикуляру будет отрицательным (-) числом, а точка, расположенная в направлении направленного вперед вектора 68, будет иметь нулевое (0) расстояние по перпендикуляру. Аналогичные вычисления могут быть выполнены для других характерных точек 60, идентифицированных в зафиксированной части кодовой полосы 16, например, точек

и

на фиг. 4. Как упомянуто выше, измеряются фактические расстояния s для точек 85, 87, 89 пересечения, а метрика E ошибки, указывающая на различия между прогнозируемыми значениями s′ и измеренными значениями s, вычисляется на шаге 136 на фиг. 5. Этот процесс повторяется каждый раз с новым предполагаемым положением, пока метрика E ошибки не станет достаточно малой на шаге 138 на фиг. 5, чтобы предполагаемое местоположение и угловая ориентация (x, y, ϕ, θ, ψ) оптического центра 66, дающие такие прогнозируемые значения s′, достаточно точно определяли фактическое местоположение оптического центра 66 на демонстрационной арене 10, т.е. считались стабильными. Как указано выше, первичное предположение о местоположении оптического центра 66 на демонстрационной арене 10 может быть произвольным или оно может быть основано на некоторой оценке. Один пример оценки для начального местоположения может быть выполнен процессором 74 путем выбора характерной точки 60 на изображении 104, имеющей наибольшее отрицательное значение s, характерной точки 60 на изображении 104, имеющей наибольшее положительное значение s, и характерной точки 60 на изображении 104, ближайшей к середине, то есть имеющей наименьшее абсолютное значение s. Используя измеренные значения s, процессор способен аналитически вычислять начальную оценку местоположения (x, y) для оптического центра 66 и способен вычислять начальную оценку ориентации (ϕ, θ, ψ), используя ближайшее к середине расстояние s. Как упомянутая оценка положения, так и упомянутые оценочные вычисления углов основаны на аналитических решениях, которые могут быть получены специалистами в данной области техники.[054] After the known coordinates ( x , y ) corresponding to feature points 60 are determined in step 132, these coordinates ( x , y ) are converted to linear coordinates s ' in step 134 in FIG. 5 using a transformation process, sometimes referred to as perspective transformation, which is well known to those skilled in the art of computer graphics. In one example of such a conversion, positioner 12 in FIG. 1 displays the portion of codeband 16 containing feature dots 60 (for example,

), best shown in Fig. 4, and identifies a point

according to the information about the code strip 16 stored in the memory associated with the processor 74, as described above. Therefore, from this information in the characteristic point memory 60, the actual location ( x , y ) of the point is known

. In this case, the processor 74 calculates the point 85 at which the beam 86 from the point

passes through the virtual screen 92, based on a guess or estimate of the location ( x , y , ϕ, θ, ψ) for the optical center 66, indicated by the dot

in fig. 4. With such a known location ( x , y ) the points

and such assumed or estimated location ( x , y , ϕ, θ, ψ) of the point

distance 94 in the forward direction can be determined by the following formula: depth in the forward direction

where the operator "•" means the scalar product, and

is a forward vector coinciding with the optical axis 68. In addition, the perpendicular distance 96 can be determined by the following formula: perpendicular distance

where the operator "•" means the scalar product, and

- vector 185 perpendicular to forward vector

Distance s ′ for predicted point 85 at which ray 86 from point

passes through the virtual screen 92, based on the actual position of the point

and this assumption or estimate of the position of the point

, can be determined by the following formula: s ′ = d * (distance along the perpendicular) / (depth in the forward direction), where d is the distance between the optical center 66 and the virtual screen 92, and the points 85, 87, 89 of the intersection of the rays 84, 86 , 88 are scaled into the pixels of the image sensor 64 with an appropriate predetermined distance d from the virtual screen. The distance d to the virtual screen is a design-specific parameter, it is calculated once at the factory and is subsequently assumed to be the same for every device having this design. In the example described above, given the exemplary position and orientation of the positioning device 12 shown in FIG. 4, the perpendicular distance will be a negative (−) number, and the point located in the direction of the forward vector 68 will have a perpendicular distance of zero (0). Similar calculations can be performed for other feature points 60 identified in the fixed portion of codeband 16, such as points

and

in fig. 4. As mentioned above, the actual distances s for the intersection points 85, 87, 89 are measured, and an error metric E indicating differences between the predicted values s ' and the measured values s is computed at step 136 in FIG. 5. This process is repeated each time with a new estimated position until the error metric E is small enough at step 138 in FIG. 5 so that the predicted location and angular orientation ( x , y , ϕ, θ, ψ) of the optical center 66 giving such predicted values of s ' accurately determines the actual location of the optical center 66 in the demonstration arena 10, i.e. were considered stable. As indicated above, the initial guess about the location of the optical center 66 in the demonstration arena 10 may be arbitrary or it may be based on some estimate. One example of an estimate for a start location may be performed by processor 74 by selecting the feature point 60 in image 104 that has the largest negative value of s , the feature point 60 in image 104 that has the largest positive value of s , and the feature point 60 in image 104 that is closest to the middle. , that is, having the smallest absolute value s . Using the measured values of s , the processor is able to analytically calculate an initial position estimate ( x , y ) for the optical center 66 and is able to calculate an initial orientation estimate (ϕ, θ, ψ) using the distance s closest to the middle. Both said position estimate and said angle estimation calculations are based on analytical solutions that can be obtained by those skilled in the art.

и

обнаруживаемого сигнала 123, субпиксельная оценка выполняется с использованием линейной интерполяции:[055] At the same time, code signal 123 extracted at step 118 in FIG. 5 is corrected for any pitch, roll, or distortion of the lens in step 140. In addition, in step 142 of detecting the zero-crossing sign, the zero-crossing of the detected signal 123 is determined. the exact locations in three dimensions ( x , y , z ) are known, predetermined by the design and structure of the code strip 16, as well as its installation in the demonstration arena 10, as described above. In FIG. 12 shows an example of the parameters of FIG. 4 when the camera in the positioner 12 is tilted down, but the code bar 16 is still visible at the top of the field of view, and the roll angle (ϕ) is zero for ease of explanation. In FIG. 13 is a conceptual view of camera pitch and roll showing parameters related to compensation and fine position finding. In FIG. 14 illustrates the distortion introduced by the lens and the radial representation used to correct for such distortion. An exemplary process for establishing the exact location of the optical center 66 of the lens system 62 of the positioning device 12 in the demonstration arena 10 is based, as described above, on accurately measuring feature points 60 in the acquired image. In this exemplary process, sub-pixel measurement is used to improve accuracy when displaying feature dots 60. Thus, when a sign change is detected between adjacent elements

and

detected signal 123, sub-pixel estimation is performed using linear interpolation:

where s is a floating point interpolation of the integer index i of the detection matrix 122 describing the sub-pixel zero-crossing estimate.

[056] Instantaneous or continuous pitching of the camera in the positioning device 12 up or down causes the apparent distance d to the viewing screen (see FIG. 4) to change as a distance d' , resulting in a scaling of the distance parameter s , which is corrected by applying a factor k :

[057] In addition to a non-zero pitch orientation, the camera may have or experience non-zero roll. Such a roll has the effect of turning the best fit line 116 on the corresponding image 104 (FIG. 6). This roll also has the effect of shifting the apparent location of the point associated with parameter s =0 along the best fit line 116. The tilt offset p of FIG. 13 is located as

[058] With these two adjustments, the zero-crossings of the detected signal 123 are mapped to parameter s as

where x is the floating point zero-crossing pixel coordinate defined in the coordinate system of FIG. fourteen.

[059] Further improvement in accuracy can be achieved by applying lens correction approximated as follows:

where

as defined in FIG. 14. Accordingly, the zero crossings of feature points 60 are detected at step 142 (zero crossing flag detection) in FIG. 5 as

[060] The parameter mapping s ' is then computed in the linear coordinate conversion step 134 in FIG. 5 as

In these expressions

[061] Since, as noted above, the exact location of the code feature points 60 is known, and since these feature points 60 uniquely correspond to zero-crossings of the detected signal 123, and the ideal mapping of these feature points 60 in virtual screen coordinates 92 is also known, the error E at step error calculation 136 in FIG. 5 is defined as

[062] The calculated error E at step 136 in FIG. 5 is monitored in step 138 (stability check) to determine whether the error E has stopped decreasing, indicating that the best position has been found. If at step 138 it is determined that the error E has not yet decreased to the optimal value, the process proceeds to a step assuming a new position 144 in FIG. 5, in which a new trial position can be calculated, for example, based on Newton-Raphson or dichotomous division methods. Using the Newton-Raphson method known to those skilled in the art, the residual value r _i is first determined as

and C _i is defined as

The Jacobian can be calculated as

so the gradient of the folded error function is

[063] The step in Newton's method is then defined as

in this case, the Hessian is approximated as

предполагается на шаге предположения нового положения 144 на фиг. 5 и это предположение используется в качестве пробного варианта нового положения на шаге преобразования в линейные координаты 134 на фиг. 5. Повторение продолжается до тех пор, пока ошибка E не станет стабильной на шаге 138 на фиг. 5, после чего местоположение и ориентация оптического центра 66 на демонстрационной арене 10 считается последним предположенным положением

после шага предположения нового положения 144, как показано на фиг. 5. Как описано выше, значением (z) по вертикали можно пренебречь, поскольку оптический центр 66 всегда находится в плоскости 76 на высоте h (фиг. 1), а рыскание (ϕ) входит в уравнения, например, в приведенное выше уравнение (8), как

Тангаж (θ) и крен (ψ) определяются после завершения итерационного цикла какSo the new proposed position

assumed in the step of guessing a new position 144 in FIG. 5 and this assumption is used as a test of the new position in the linearization step 134 in FIG. 5. The repetition continues until the error E becomes stable at step 138 in FIG. 5, after which the location and orientation of the optical center 66 in the demonstration arena 10 is considered the last guessed position

after the guess new position step 144 as shown in FIG. 5. As described above, the value of ( z ) vertically can be neglected, since the optical center 66 is always in plane 76 at height h (Fig. 1), and yaw (ϕ) enters into equations, for example, in the above equation (8 ), how

Pitch (θ) and roll (ψ) are determined after completion of the iterative cycle as

и

.

and

.

[064] These calculations are not overly complex given the power of even conventional low-cost processors on the market, so that refresh rates in excess of ten (10) updates per second are easily achieved.

[065] Thus, image 104 captured by image sensor 64 corresponds to one and only one specific position and orientation of optical center 66. Therefore, each such position and orientation of optical center 66 corresponds to a unique and predictable location of feature points 60 in virtual image 92. Essentially, it is a guessing process for determining the position and orientation of the optical center 66 by guessing and then comparing the predicted image with the actually measured image. Accordingly, the processor 74 assumes many positions of the optical center 66 until it can independently recreate an image that matches the image received by the sensor 64.

[066] In practice, this process does not involve calculations for matching images, instead, the information from the images is converted into one-dimensional arrays. This is done by converting the image 104 captured by the image sensor 64 into a one-dimensional set of feature points 60 identified by their one-dimensional distance along the center line 52 of the code strip 16. The processor 74 also identifies each feature point 60 so that the ( x , y ) coordinates of each characteristic point 60 are known. This identification allows comparison of the error E of the metric used for the guess process with measured one-dimensional lists, instead of comparing the measured image 104 with a self-generated virtual image, which would be much more computationally expensive. Given the virtual trial position and orientation of the optical center 66, it is simple to determine which optical elements 60 can be seen and where exactly they can be located in the image and, therefore, one can form a one-dimensional list of virtual positions s of feature points 60 along the virtual center line. Since the measured image 104 from the image sensor 64 is converted into a one-dimensional list of distances s and the virtual one-dimensional list of distances s ' can be predicted for any trial position of the optical center 66, the process can be repeated by trying multiple positions and orientations until the self-generated one-dimensional list of positions s ' of feature point 60 will not match the 1D distances s obtained from image sensor 64. Finally, the loop converges to a position where the squared difference ( s ' - s ) between the 1D lists is at a minimum.

[067] As shown in FIG. 15, a plurality of positioning devices 12, for example mounted on several robots 14 moving independently of each other, can operate simultaneously on the same demonstration arena 10, as shown, for example, in FIG. 15. Each of the positioning devices 12 can determine the exact location of its optical center 66 in real time, as described above, and generate position signals for such locations for use by the robot 14 on which such positioning device 12 is mounted. Depending on the viewing angle 70 of the lens system 62, each camera of each positioner 12 can typically capture a portion of the code strip 16 in its field of view with enough feature dots 60 (FIG. 4) to locate its optical center 66, even if others are moving. robots 14 or random objects 96 cover parts of the code strip 16 from this camera. In one example, the addition of one or more additional positioners 12' mounted on the robot 14 and oriented in a different direction, for example, rotated 180 degrees relative to the first positioner 12, can further reduce the likelihood of a complete overlap of the code strip 16 for such a robot 14. In addition In addition, widening the field of view 70 reduces the likelihood of a complete overlap of the code band in the field of view 70 . However, as noted above, such an expansion of the field of view 70 can reduce the number of samples of signal 123 per interval 58 to an unacceptable value when positioner 12 is away from codeband 16.

[068] The positioning device 12 can also be used for additional image analysis, for example, to avoid the object 14, 96, determine the location of the object 14, 96, and the like. The equipment required for image processing 104 from the image sensor 64 and positioning, as described above, allows additional image analysis to be performed. For example, image 104 from image sensor 64 may first be processed to determine the location of optical center 66 as described above, and then image 104 may be further processed to determine the presence of other objects 14, 96 to determine the presence of such objects or obstacles.

[069] Accordingly, it is possible to use all suitable combinations, partial combinations, modifications and equivalents that fall within the scope of the invention defined by its features. As used herein, the expressions "comprise", "comprises", "comprising", "include", "including" and "includes" relating to features are intended to indicate the presence of those features, parts of the invention, elements or steps, but do not exclude the presence or adding one or more other elements, parts of the invention, elements, steps or groups thereof. In addition, directional terms such as "up", "down", "on", "out", "above", "under", "up", "down", etc. may and sometimes do refer to the orientation of elements and features as shown in the drawings and are not intended to require any particular physical orientation or any limitation on the orientation of a device or element in actual use, unless otherwise noted in the description.

Claims (44)

1. A method for assembling a demonstration arena, including: placing the code strip at least partially around or on the demo arena, wherein the code strip has a start edge and an end edge with a code length between the start edge and the end edge; when placing the code strip at least partially around or on the demo arena, the start edge is placed at a start position relative to the demo arena having a specific location relative to the demo arena; the code strip contains a plurality of feature points along the code length between the start edge and the end edge, and each feature point is at a predetermined distance from the start edge, so that each of the feature points along the code length is at a corresponding predetermined position relative to the start position; and different code bar segments are applied to a plurality of walls, with the center line of the code bar code in each segment on each wall being at a constant height. 2. A method for assembling a demonstration arena, including: placing the code strip at least partially around or on the demo arena, wherein the code strip has a start edge and an end edge with a code length between the start edge and the end edge; when placing the code strip at least partially around or on the demo arena, the start edge is placed at a start position relative to the demo arena having a specific location relative to the demo arena; the code strip contains a plurality of feature points along the code length between the start edge and the end edge, and each feature point is at a predetermined distance from the start edge, so that each of the feature points along the code length is at a corresponding predetermined position relative to the start position; and - assembling a plurality of walls around or near the demo space with the intersection of the walls at a plurality of corners, designating one of the corners as the initial position, and mounting the code strip on the walls, with the initial edge of the code strip located at the corner designated by the initial position, and each of the characteristic points in the code strip is located at a corresponding predetermined distance from the corner designated by the start position. 3. The method of claim 2, which comprises assembling a plurality of walls around or adjacent to the display surface such that the center line of the code stripe code forms a plane at a constant height above the display surface. 4. A method for determining the position and orientation of a device located within the surrounding structure and containing a motion control system that ensures the movement of the device within the surrounding structure, including: - placing a predetermined continuous pattern (code strip) on the surrounding structure, wherein the predetermined continuous pattern is printed in the form of a predetermined non-repeating sequence of black blocks and white blocks located at regular intervals, and some of the blocks are twice as long as others in length; - use in the device of a positioning system that continuously captures images of a continuous pattern; - determination of the three-dimensional position and three orientation parameters within the surrounding structure based on the captured images; and - the use of a motion control system when moving the device within the surrounding structure based on a certain three-dimensional position and three orientation parameters, at the same time, a positioning system capable of capturing images in the field of view makes it possible to determine a two-dimensional position with three orientation parameters relative to a continuous pattern. 5. The method according to claim 4, characterized in that the positioning system comprises a camera with a lens system having an effective optical center projected onto an image sensor, which is associated with computing means capable of determining the position and orientation of the device, regardless of the presence of communication outside it. 6. The method according to claim 5, characterized in that the surrounding structure is a polygonal structure, while the origin of the coordinate system is located in the center of this structure and determines the position of the x -axis to the right of the center and the positive y -axis in front of the center. 7. The method according to claim 5, characterized in that the code strip is aligned to the corners of the surrounding structure and does not require measuring the exact location of discrete markers, while the continuous code strip allows you to automatically determine its location at different characteristic points without any measurements, thus , excluding calibration and measurement data for the surrounding structure. 8. A demonstration arena system for vehicles that operate in a demonstration space, comprising a code strip having a start edge and a predetermined length to the end edge, and containing a predetermined number of predetermined intervals located from the start edge to the end edge, while the sequence of code marks is located in these intervals so that the code marks are located at predetermined distances from the start edge; the code strip is configured to be mounted so that the initial edge of the intervals is located at the initial position near the side of the display space, and covers a predetermined number of intervals from the initial position along at least part of the side at a certain height above the perimeter of the display surface; the image of at least part of the code strip is captured by a camera mounted on a vehicle operating on the demonstration surface of the demonstration space, the optical axis of the camera being directed to the side of the demonstration space, and the position of the code marks on this image is correlated with the initial position at predetermined intervals. 9. The system according to claim 8, characterized in that the intervals of the code strip are the same from the start edge to the end edge, and the sequence of code marks in the code strip is located at equal intervals along the center line of the code located from the start edge to the end edge. 10. The system of claim. 9, characterized in that the sequence of code marks contains a non-repeating sequence of blocks containing dark blocks interspersed with light blocks. 11. The system of claim. 10, characterized in that some of the dark and light blocks are located above the center line of the code, and some other of the dark and light blocks are located below the center line of the code. 12. A demo arena system for vehicles that operate in a demo space, comprising a code strip having a start edge and a predetermined length to the end edge, and containing a predetermined number of intervals from the start edge to the end edge, wherein the code strip contains a sequence of code marks , containing dark blocks interspersed with light blocks along the center line of the code, some dark blocks and light blocks are located at some intervals above the center line of the code, and some other dark blocks and light blocks are located at some intervals below the center line of the code so that each an interval containing a dark block above the center line of the code also contains a light block below the center line of the code, and each interval containing a light block above the center line of the code also contains a dark block below the center line of the code, so that the transitions of light blocks to dark blocks above the center line whether the transitions of dark blocks into light blocks below the center line of the code, the transitions of dark blocks into light blocks above the center line of the code coincide with the transitions of light blocks into dark blocks below the center line of the code, and the corners of the dark blocks above the center line of the code coincide with the corners of the dark blocks below the center line of the code form characteristic points on the center line of the code. 13. The system according to claim 12, characterized in that in the demonstration space there is a positioning device containing a digital camera for obtaining an image containing one or more parts of a sequence of code marks, while the digital camera contains a lens system for focusing the image in the image sensor, and the lens system has an optical center. 14. The system according to claim 13, characterized in that the code strip contains a central line of the code and is made with the possibility of mounting it next to the demonstration space in such a way that it forms a base plane above the demonstration surface, while the positioning device is made with the possibility of mounting it on vehicle in such a way that the optical center is located in the reference plane. 15. The system according to claim. 14, characterized in that it contains a processor programmed to compare the image obtained with a digital camera with a sequence of code marks in memory in order to identify one or more parts of the sequence of code marks in the image to generate signals indicating the position and orientation of the positioning device in the demo space. 16. Demonstration arena system, comprising: - a code strip located on at least a portion of one or more sides of the arena space above the display surface and containing a code mark sequence starting at a start position, wherein parts of the code mark sequence at any particular position adjacent to the arena space relative to the start position are unique to that particular position; - a positioning device located in the arena space and containing a digital camera with an optical axis directed to the side of the arena space, for obtaining an image containing one or more parts of the sequence of marks; and - a processor programmed to generate signals indicative of the position and orientation of the positioning device in arena space, based on part or several parts of the sequence of marks on the image. 17. The system of claim. 16, characterized in that the sequence of code marks contains a non-repeating sequence of blocks at intervals along the center line of the code. 18. The system of claim. 17, characterized in that the non-repeating sequence of blocks contains dark blocks interspersed with light blocks. 19. The system of claim. 18, characterized in that some of the dark and light blocks are located above the center line of the code, and some other of the dark and light blocks are located below the center line of the code. 20. The system of claim. 19, characterized in that some intervals contain a dark block above the center line of the code and a light block below the center line of the code, and other intervals contain a light block above the center line of the code and a dark block below the center line of the code, while the transitions light blocks to dark blocks above the code center line coincide with dark block transitions to light blocks below the code center line, dark block transitions to light blocks above the code center line coincide with light block transitions to dark blocks below the code center line, and dark block corners coincide above the center line of the code with the corners of the dark blocks below the center line of the code form characteristic points on the center line of the code. 21. The system of claim. 20, characterized in that the digital camera includes a lens system for focusing the image in the image sensor, while the lens system has an optical center and is centered with the image sensor along the optical axis. 22. The system according to claim 21, characterized in that the code strip is made with the possibility of its installation near the arena space in such a way that the central line of the code forms a reference plane above the demonstration surface, while the positioning device is made with the possibility of its installation on the vehicle in such a way so that the optical center is located in the base plane. 23. A demonstration arena system for vehicles operating in a demonstration space, comprising: - a code strip containing a sequence of code marks, including dark blocks interleaved with light blocks along the center line of the code, with some of the dark blocks and light blocks located above the center line of the code, and some others of the dark blocks and light blocks located below the center line code in such a way that under each dark block above the center line of the code there is a light block below the center line of the code, and that under each light block above the center line of the code there is a dark block below the center line of the code, so that the coincidence of the corners of the dark blocks above the center line of the code with the corners of the dark blocks below the center line of the code form characteristic points on the center line of the code; and - a processor programmed to identify the center line of the code in the image of a part of the code strip by characteristic points formed by the coincidence of the corners of the dark blocks above the center line of the code with the corners of the dark blocks below the center line of the code. 24. The system of claim. 23, characterized in that the processor is programmed to identify the location of the feature points in the image as the location of such feature points on the code strip and, thus, the location of such feature points around the perimeter of the demonstration arena on which the code strip is mounted, according to relative to the start position by determining the location of the respective feature points in the mark sequence and specific locations of the respective feature marks around the demo arena in the memory. 25. A demo arena system for vehicles that operate in a demo space, comprising a code strip having a start edge and a predetermined length to the end edge, and containing a predetermined number of intervals located from the start edge to the end edge, wherein the code strip is made with the possibility of its mounting in the initial position next to the display space and covers a predetermined number of intervals adjacent to the display space, located above the display surface, having the shape of a polygon, the display space is surrounded by walls intersecting at the corners, and the initial position is one of these corners. 26. Demonstration arena system, comprising: - a code strip located on at least a portion of the perimeter of the arena space above the display surface and containing a sequence of code marks starting at the start position, wherein (1) parts of the code mark sequence at any particular position adjacent to the arena space relative to the start position are unique for this particular position, (2) the code mark sequence contains a sequence of dark blocks interspersed with light blocks at intervals along the center line of the code, with some of the dark and light blocks located above the center line of the code, and some others of the dark and light blocks located below center line of the code, (3) some intervals contain a dark block above the center line of the code and a light block below the center line of the code, and other intervals contain a light block above the center line of the code and a dark block below the center line of the code, while the transitions of light blocks to dark th blocks above the center line of the code coincide with transitions of dark blocks to light blocks below the center line of the code, and transitions of dark blocks into light blocks above the center line of the code coincide with transitions of light blocks into dark blocks below the center line of the code, and (4) coincidence of the corners of the dark blocks above the center line of the code with the corners of the dark blocks below the center line of the code form characteristic points on the center line of the code; - a positioning device located in the arena space and containing a digital camera for obtaining an image containing one or more parts of the sequence of code marks; and - a processor programmed to identify the center line of the code by characteristic points formed by the coincidence of the corners of the dark blocks above the center line of the code with the corners of the dark blocks below the center line of the code. 27. The system of claim. 26, characterized in that the processor is programmed to (1) identify the location of feature points in the image as the location of such feature points on the code strip, and thus the location of such feature points with respect to the initial position by determining the location of the corresponding characteristic points in the sequence of labels and specific locations near the arena space stored in the memory, and (2) generating signals indicative of the position and orientation of the positioning device in arena space, based on part or more parts of the sequence of code marks in the image. 28. A demo arena system, comprising a code strip located on at least a portion of the perimeter of the arena space above the demo surface and containing a sequence of code marks starting at a start position, with parts of the sequence of code marks at any particular position adjacent to the arena space relative to the start position are unique to that particular location, the demo surface is polygonal, the arena space is surrounded by walls that intersect at the corners, and the start position is one of those corners.

Images