DE102019114531A1 - Device for position and position detection of markings in three-dimensional space - Google Patents

Abstract

There is a device for position and position detection of markings (2, 12) in three-dimensional space with at least one marking unit (1, 11, 23, 24, 25, 26, 27, 28), with an optical image acquisition unit (21), and with an evaluation unit (22) which is set up to determine the location and position of the marking unit (1, 11, 23, 24, 25, 26, 27, 28), wherein the marking unit (1, 11, 23, 24 , 25, 26, 27, 28) has at least six optically activatable communication elements (3, 13) for coding user data, which can be switched on or off individually depending on the user data, the image acquisition unit (21) has a series of images of the marking unit (1 , 11, 23, 24, 25, 26, 27, 28) with a predetermined image frequency, and the evaluation unit (22) is set up to determine the location and position of the marking unit (1, 11, 23, 24, 25 , 26, 27, 28) and from the entirety of the communication elements that are switched on or off n (34.42; 35, 43) to capture the coded user data.

Description

• - mit mindestens einer Markiereinheit, die an einem Objekt festlegbar ist, mit mehreren optisch aktiven (d.h. selbstleuchtenden) Markierungen oder passiven (d.h. aufgrund von Lichtreflexion sichtbaren) Markierungen,
• - mit einer oder mehreren optischen Bilderfassungseinheiten, die zur Aufnahme von Bildern der Markiereinheit eingerichtet sind, und
• - mit einer Auswerteeinheit, die zur eindeutigen Bestimmung der Lage und der Position der insbesondere an dem Objekt festgelegten Markiereinheit eingerichtet ist, so dass über die an dem Objekt festgelegte Markiereinheit erfindungsgemäß auch die Lage und Position des Objekts im dreidimensionalen Raum bestimmt ist.

A device for the location and position detection of markings in three-dimensional space is described

• - with at least one marking unit that can be fixed to an object, with several optically active (ie self-luminous) markings or passive (ie visible due to light reflection) markings,
• - With one or more optical image acquisition units which are set up to record images of the marking unit, and
• With an evaluation unit which is set up for the unambiguous determination of the location and the position of the marking unit, in particular fixed on the object, so that the location and position of the object in three-dimensional space is also determined according to the invention via the marking unit fixed on the object.

The proposed marking unit has at least four optically active markings for determining position and position, of which at least three of the markings are preferably in a common plane and at least one of the markings is outside this plane.

From the pamphlets EP 1 813 911 A1 , US 2005/0201613 A1 , EP 1 498 688 B1 , WO 2004/114 112 A1 , US 2008/0111985 A1 , WO 2006/069 748 A1 , U.S. 5,227,985 A , US 7 742 895 B2 and DE 10 2014 012 693 A1 systems and methods are known which can determine the location and position of an object in space by recording a marking unit fixed on the object. For this purpose, optically active, ie self-illuminating, or optically passive, ie light-reflecting, markings are provided in a fixed geometric arrangement on the marking unit. Many different arrangements for evaluation are possible. Most of the proposed arrangements provide at least four markings, of which at least three markings span a plane and at least one mark lies outside this plane. The at least three markings arranged in a plane can lie on at least two straight lines that are not parallel and span the plane. Such an arrangement of the markings enables a reliable location and position determination of the marking unit in space from a single two-dimensional image of the marking unit, which is recorded by means of an optical image acquisition unit (for example a photo apparatus or a camera).

The DE 10 2014 012 693 A1 describes a complex system and procedure. The position and orientation is determined on the basis of a single two-dimensional image of a marking arrangement provided with at least seven markings, which is attached to the object. The system further comprises an image acquisition unit for recording a two-dimensional image of the object or the marking arrangement arranged on the object and an evaluation unit for the unambiguous determination of the location and the position of the object on the basis of the recorded image. The seven markings of the marking arrangement are in a fixed, predetermined spatial relationship to one another, six of these markings forming a plane, while the seventh marking is arranged outside or at a distance from that plane. The six markings are divided into groups that lie on two different straight lines that intersect at an angle of 90 °. The first straight line comprises at least four markings, and the second straight line comprises at least two markings. In a plan view of the plane, the seventh marking outside the plane is also on this first straight line, specifically on one side of the second straight line which faces away from at least two markings of the first straight line. This is important for an unambiguous assignment of the image markings, ie the markings depicted in the image, and thus the reconstruction of the location and position of the markings in space. During the evaluation, several homographies are calculated for possible assignments. A position determination is reconstructed from these homographies. Based on this, the average reprojection error (compared to the known actual arrangement of the markings on the marking arrangement) is calculated for all image markings for each position determination. The homography that has the least error, that is correct and that is used for the unambiguous determination of the position.

In practice, this evaluation leads to very good results, but it is very complex with regard to the image evaluation and computing power of the system, in particular due to the need to calculate more than one homography. This leads to problems in the live tracking of fast moving objects, because the computing power of systems used in practice is insufficient, in particular if not only one marking arrangement is monitored in a system, but a plurality of marking arrangements.

In theory, each marking arrangement can be distinguished from other marking arrangements recorded in the image. However, this often requires an evaluation and detection of objects in the environment if a possible unique numbering on the marking arrangement, for example a sticker on the marking arrangement, cannot be recognized. The evaluation of the surroundings of the marking arrangement in the image, however, it regularly leads to ambiguities when tracking moving objects.

In principle, it would also be conceivable to arrange the markings on the marking arrangements in a geometrically different manner. However, this would further complicate the already complex evaluation.

In the industrial environment, it is desirable to be able to differentiate between up to several hundred marking units and / or to be able to transmit information from that transmitted with a significant bandwidth in a robust, less error-prone manner. It is also desirable to limit the energy consumption of such marking arrangements, some of which are battery-operated, in favor of longer running times.

It is therefore the object of the invention to provide a communication option with high bandwidth for a system for position and position detection of the type mentioned at the beginning, for example a transmission of at least about 2 bytes per second, preferably about 10 bytes per second and particularly preferably more than 100 Bytes per second for each marking unit. Depending on the application, the transmission capacity can be appropriately divided into the display or transmission of an identification and the communication of other information.

At the same time, the same invention can be used to reduce energy consumption by reducing the number of simultaneously active markings.

This object is achieved with the features of claim 1. For this purpose it is provided in particular that the marking unit has at least six optically activated, i.e. has communication elements for coding useful data that can be switched on and off by the marking unit, the marking unit being set up to switch the individual communication elements on or off depending on the useful data. In other words, the useful data are encoded in the communication elements. The marking unit thus generates a coding of the useful data in a form that can be transmitted by the communication elements. The communication elements connected at the same time form a symbol of the coding. A reduction in energy consumption can be achieved, if necessary, by reducing the communication bandwidth accordingly, i.e. by reducing the number of necessary, simultaneously active communication elements.

The image acquisition unit then records a series of images of the marking unit with a predetermined image frequency, with both the markings for position and position detection and the communication elements switched on or off according to the coding of the user data, i.e. each a symbol of the marking, recorded and displayed are or will be.

Correspondingly, the evaluation unit is set up to determine the location and position of the marking unit in an image from the recorded markings and to record the coded useful data from the entirety of the communication elements shown (switched on or off), i.e. to decode or to capture the symbols of the coding. If a series of images is recorded, all images from the recorded series of images are preferably evaluated. This makes it possible for the transmitted user data to be combined from the respectively decoded user data of the individual images of the image series. According to the invention, the useful data can thus be transmitted in a series of individual symbols and reassembled during decoding, preferably in that the evaluation unit is set up to carry out the entire communication sequence].

The evaluation unit will typically provide a position and orientation determination for each image. The user data may not be available for every picture. In unfavorable cases, the useful data of the first few images will be missing. Usually a continuous stream of user data arrives afterwards. The invention is not restricted to the fact that every image is actually evaluated and / or provides the desired information. The invention also covers cases of a selective image evaluation - whatever the result - with regard to the position and attitude determination and / or the transmission of useful data.

The useful data can contain information that is used to identify the marking unit, signaling a status of the marking unit, the object (on which the marking unit is fixed) and / or its surroundings. This represents a particularly preferred application in which the detection of the position of an object also enables object-related information to be transmitted at the same time. However, the device according to the invention is not limited to this particularly preferred application, in which the device is set up to carry out a method that extends a fundamentally known method for determining position and position by determining the state, without the generation and evaluation of the images by the image acquisition unit and the evaluation unit has to be technically fundamentally changed. It is sufficient to set up the marking unit and the evaluation unit in such a way that that optically active elements (as they can already be used as markings for the location and position device) are used as communication elements for the transmission of useful data.

The markings and the communication elements are fixedly arranged on the marking unit, i.e. they are invariably arranged at predetermined positions relative to one another on the marking unit. The positions of the markings and the communication elements on the marking unit, and thus their relative positions, are known in the evaluation unit.

The optically active communication elements and the optically active or passive markings emit light waves in the visible or invisible wavelength range, i.e. in the wavelength range of the spectral colors between about 380 nm to 750 nm, in the ultraviolet wavelength range between about 100 nm to 380 nm or according to a particularly preferred application in the infrared wavelength range between about 750 nm to 1400 nm. According to a particularly preferred embodiment, the (optical active) marking and / or the communication element can be an infrared LED (IR LED). The optical image acquisition unit is adapted to the optically active markings and / or communication elements of the marking unit, i.e. able to pick them up optically. The optical image acquisition unit can preferably be a digital camera, in the case of infrared light with a suitably designed IR blocking filter which does not block the IR wavelength of the markings or communication elements (so-called IR cameras). The markings and the communication elements are preferably formed by the same optically active components, in particular IR LEDs.

For the coding of the information, the marking unit has a computing unit (e.g. a processor or microprocessor) that is used to detect a status of the marking unit (or a status of an object fixed on the marking unit or a status transmitted to the marking unit e.g. in a local communication ) and is set up to switch the communication elements on and off, the communication elements being switched on and off according to an implemented coding rule. This is what is meant by coding the information.

The image frequency with which the optical image acquisition unit (also referred to as a camera for short) takes the images of the marking unit is preferably at least about 70 Hz, preferably more than 100 Hz or 150 Hz. Since the markings and communication elements are optically active elements or retroreflectors illuminated by an external source, ie luminous elements, no long exposure times of a single image are necessary to capture. In this way, image frequencies in a range of around 160 Hz to 300 Hz can also be achieved in an industrial environment with currently common optical elements, for example IR LEDs and IR cameras or IR emitters in combination with retroreflectors and IR cameras. Sensible image frequencies are limited upwards by the required exposure time. Given the sensitivity of currently common optical elements, a sensible upper limit for the image frequency is in the order of 1 kHz.

According to a preferred embodiment, the marking unit can be set up to use k-out-of-n coding for the coding of information, where n is the number of optically activated communication elements and k is the number of communication elements that are connected simultaneously during coding, i.e. of the communication elements forming the symbol.

According to the invention, at least 6 communication elements are preferably provided, i.e. n 6 applies. According to a particularly preferred embodiment, the number of communication elements n is between 10 and 20. This achieves a sensible balance between the achievable bandwidth and the number of communication elements to be accommodated on the marking unit. On the one hand, the size of the marking unit must not become so large that the marking unit can no longer be easily fixed on the objects. On the other hand, the distance between the various communication elements must be large enough that they can be easily resolved in the image of the camera. A preferred application is around n = 16 communication elements. For example, k = 1 to k = 3 communication elements are activated at the same time for the coding of information. With n = 16 communication elements, these represent preferred codings, i.e. a 1-out-of-16 coding, 2-out-of-16 coding or a 3-out-of-16 coding. Since the number of simultaneously active elements is determined via k, the energy requirement in the case of actively illuminated markings scales accordingly.

The number of symbols that can be distinguished in an image with a / (- from n coding corresponds to $$\left(\begin{array}{c}n\\ k\end{array}\right)=\frac{n!}{\left(n-k\right)!\cdot k!}$$

With a 2-out-of-16 coding there are 120 different combinations (symbols) of 2 illuminated out of a total of 16 communication elements. A 3-out-of-16 coding results in 560 Combinations (symbols) of 3 illuminated out of a total of 16 communication elements. With the energy-saving 1-out-of-16 coding, 16 different combinations remain. The “symbol” is understood to mean the number of k simultaneously connected communication elements in the arrangement of the totality of all n communication elements.

In other words, this means that with a 2-out-of-16 coding, up to 120 different marking units can be distinguished in one image if each marking unit is represented or identified by a different symbol on the total of 16 communication elements. With 3-out-of-16 coding, even up to 560 different marking units can be distinguished in one image.

In more general terms, 120 different information items can be represented in an image with 2-out-of-16 coding and 560 different information with 3-out-of-16 coding. The user data is transmitted in this information. The same applies to 1-out-of-16 coding in the case of a reduced bandwidth and, at the same time, a reduced energy requirement.

If this information presented in the communication elements of a marking unit does not change over time, the same information is decoded in every image. In this case, a symbol can be used as a tracker ID (i.e. as an identification code for a marking unit).

According to the invention, the marking unit can also be set up to display several symbols one after the other in chronological order.

In other words, this means that the marking unit is set up to carry out a data transmission in which various items of information in the useful data are represented in the communication elements in the communication elements in chronological succession. The user data are therefore encoded in a sequence of several symbols and displayed in a time sequence.

The image capture unit captures these various symbols in their chronological sequence by capturing the series of images. Each image or, more precisely, each different symbol is decoded in the evaluation unit and the useful data are composed of the sequence of decoded symbols. The amount of information is therefore not limited to the amount of information given by the coding.

Since the optical image acquisition unit or camera records the marking unit in a series of images anyway, in order to record changes in the position and position of the marking unit over time, any user data assigned to the marking unit can also be used in basically any form without changing the sequence during image acquisition and evaluation Transfer amount.

The symbols transmitted one after the other can also be the same (identical) symbols that are coded in immediate succession and thus transmitted.

According to a further development of this inventive concept, the marking unit can be set up to generate the time sequence of the plurality of symbols with a symbol frequency which corresponds at most to the image frequency of the image acquisition unit.

This ensures that every symbol is recorded in (at least) one image and thus transmission of all data is secured. If the symbol frequency corresponds to the frame frequency, the highest possible data rate is achieved in the transmission. For this it must be ensured that the user data in the image can also be reliably decoded. This can basically be achieved in different ways.

By synchronizing the representation of the symbols in the communication elements with the image recording, it can be achieved that exactly one symbol is represented in each image. The exposure time should then be selected so short that only one symbol is actually visible in the communication elements, i.e. the activation time of the communication elements for displaying the symbols must be so long or short that only this one symbol is reliably displayed during the exposure time of the image. A synchronization can be achieved in a manner known in principle to the person skilled in the art by transmitting suitably selected synchronization symbol sequences. However, this is complex. In addition, an existing synchronization is often lost again, especially in an industrial environment with mostly suboptimal transmission conditions, which would require a renewed synchronization for the continuation of communication.

Particularly preferred embodiments therefore provide asynchronous communication in which the switching between the symbols in the coding and the image recording are not synchronized. This means that two or more symbols can be shown superimposed in one picture. Such an image cannot be evaluated at first because the coding cannot be recognized and the image cannot therefore be decoded.

In order to nevertheless enable the decoding in a simple embodiment, it can be provided according to one embodiment that the marking unit is set up to generate the time sequence of the different symbols with a symbol frequency that corresponds to 1 / i of the image frequency of the image capture unit, where i any number is significantly greater than 1, ie for example i ≥ 1.5 or i ≥ 1.8. Preferably, i is equal to a number between approximately 2 and 4. In the case of i = 2, the symbol frequency corresponds to just half the image frequency, with the result that a symbol is always mapped in 2 successive images, in the case of i = 3 in three images, etc. By comparing two or more images, the symbol sequence can be reconstructed. However, the data rate is also reduced by a factor of 1 / i.

According to a particularly preferred embodiment, the invention proposes an extension of the k-out-of-n coding, in which the marking unit is set up to not the k connected communication elements of the preceding symbol in the cycle of the k-out-of-n coded symbols displayed one after the other to be used for the symbol below.

According to this embodiment it is provided that the evaluation unit is set up not to use the k connected communication elements of the preceding symbol for the decoding when decoding a symbol in an image. For this purpose, the evaluation unit can be set up to identify all connected communication elements - for example by means of image evaluation software - in the current image and to compare them with the connected communication elements of the previous symbol, with activated communication elements of the previous symbol when decoding the symbol in the current image not be considered.

In this embodiment, according to the invention, all connected communication elements differ between the preceding and the following symbol from the time sequence of the several symbols. This means that every image can be clearly identified with knowledge of the previous symbol. This also applies precisely to the preferred case in which the symbol frequency and the image frequency are chosen to be the same.

In this modified k-out-of-n coding for a sequence of coded symbols, in which the k communication elements from the respective preceding symbol are not used for the following symbol, the number of symbols that can be distinguished in a picture is reduced with a k-out -n coding accordingly $${\left(\begin{array}{c}n\\ k\end{array}\right)}^{*}=\frac{\left(n-k\right)!}{\left(n-2k\right)!\cdot k!}$$

For this modified 2-out-of-16 coding, there are only 91 (instead of 120) different combinations of 2 illuminated out of a total of 16 communication elements. With a 3-out-of-16 coding, there are accordingly only 286 (instead of 560) combinations of 3 illuminated out of a total of 16 communication elements. Accordingly, there are only 15 (instead of 16) combinations in the energy-saving 1-out-of-16 coding.

Since - particularly in this embodiment - the coding and decoding of a symbol depends on the preceding symbol in the sequence of several symbols, a symbol cannot be coded or decoded without knowledge of a preceding symbol. According to the invention, a defined starting point, which is implemented in the marking unit and the evaluation unit, is therefore established as the entry point for the time sequence. As a starting point for the sequence of several symbols, it can be implemented in the marking unit to generate a symbol that does not change for such a long time that this symbol can be seen in at least two consecutive images, preferably more than two, for example three to six images, the marking unit is still visible.

Such an entry into a chronological sequence of several symbols as the beginning of a communication can easily be identified in the evaluation unit. For example, a symbol can also be selected as an entry point that represents an impermissible symbol in the coding for communication. This can avoid confusion. This symbol is implemented accordingly in the marking unit and the evaluation unit so that it is available for coding and decoding. According to the invention, it is also possible to define a sequence of several different symbols as entry point, of which at least one, possibly all, are generated by the marking unit for as long as this symbol is visible unchanged in at least two successive images of the marking unit. The symbols can also include symbols in which all markings and / or communication elements are switched on or off. A marking unit with only switched off communication elements can also be understood as a symbol of the coding.

In an advantageous implementation, it can be provided that the symbol generated as the starting point and not changing for the prescribed long time is a unique identification of one of several marking units in the device proposed according to the invention. Such a configuration can arise, for example, in an industrial environment, if the position, the location and / or the state of several objects in space are to be recorded. In this case, many different marking units can be detected by one (or possibly also several) optical detection units of the device, which - in relation to the one device - have a unique identification. With the 2-out-of-16 coding described as an example, up to 120 different marking units of a device (in the sense of a system) can be uniquely identified, with a 3-out-of-16 coding this is even up to 560 different marking units.

In a further embodiment of the invention it can be provided that the marking unit is set up to generate the symbols displayed one after the other in the chronological sequence by means of different codings.

A specific embodiment can, for example, provide that a 2-out-of-16 coding is initially used in order to generate a unique identification of the marking unit (tracker ID) for up to 120 marking units. This unique identification can, for example, be sent as a starting point for communication as a symbol that does not change over several images recorded by the image acquisition unit. This can then be followed by communication with symbols according to 3-out-of-16 coding, in particular in accordance with the modified 3-out-of-16 coding already described, in which the symbol frequency corresponds to the image frequency. The assignment of the 2-of-16 coding and the 3-of-16 coding to the data content (unique identification and user data communication) can of course also be done the other way around. Here the person skilled in the art will select suitable assignments and / or codings, also taking into account the energy requirements. The same applies to the specific selection of the k-out-of-n coding. The number of different symbols, i.e. The person skilled in the art will select the number of simultaneously illuminated communication elements k in a suitable manner for the singular, whereby he will suitably weigh the boundary conditions “energy consumption” in the case of battery-operated marking units and “communication bandwidth”.

In any case, according to the invention, the transmitted useful data are assigned to exactly one identified marking unit. After communication has ended, according to the invention, in a communication variant, all communication elements can be switched off for a predetermined period until a new communication cycle is generated in the same way.

When decoding and evaluating the symbols, different information, such as the identification of the marking units and the communication of user data, can easily be identified and separated from one another by using different codings.

According to an arrangement of the device according to the invention, it is proposed that the at least four optical markings are arranged on the marking unit in such a way that two of the markings each define a straight line and the straight lines defined in this way intersect at a point of intersection. If one of the markings is at the intersection of the straight line, a total of three markings are sufficient to define the two straight lines. Then, even with four markings, one mark can lie outside the plane spanned by the two straight lines. Otherwise, according to the invention, at least five markings are necessary for this. This enables, in particular when preferably at least three of the markings are in a common plane and at least one marking is outside this plane, a precise location and position determination of the marking unit in only one two-dimensional image with only one two-dimensional camera. With the location and position determination of the marking unit, the position of the object on which the marking unit is fixed are also known.

According to another preferred arrangement of the device according to the invention, it is proposed that at least six optical markings are arranged on the marking unit in such a way that three of the markings each define a straight line and the straight lines defined in this way intersect at a point of intersection. If one of the markings is at the intersection of the straight line, a total of five markings are sufficient to define the two straight lines. Then even with six markings, one mark can lie outside the plane spanned by the two straight lines. Possibly. One of the communication elements that lies on one of the straight lines can also be used as a marker for determining the straight line.

This enables, especially if preferably at least five of the markings are in a common plane and at least one marking is outside this plane, in addition to the precise location and position determination of the marking unit in only one two-dimensional image with only one two-dimensional camera, it also enables the quick location and assignment of the Markings in the two-dimensional image. With the location and position determination of the marking unit, the position of the object on which the marking unit is fixed are also known.

The main features of the evaluation, which is based on geometric considerations and takes advantage of the fact that the relative geometric arrangement of the markings to one another on the marking unit is known, are known from the prior art mentioned at the beginning and are part of the general specialist knowledge of the person skilled in the art. With the minimum number of markings mentioned, it is possible to limit the number of homographs to be calculated to a single one. With regard to the location and position determination, the invention can be implemented with all suitable methods which accordingly need not be explained at this point.

In such an embodiment, it is particularly advantageous if the at least six, preferably all, communication elements are arranged along this at least one straight line between two adjacent markings.

In principle, and also differently from what is claimed in claim 1, the number of communication elements according to the invention can also be less than six. In order to use the proposed k-out-of-n coding, at least a number of n = 2 communication elements is necessary. An alternative embodiment of the invention also relates to a device with a marking device which has only two, three, four or five communication elements.

Instead of arranging all communication elements along a straight line between two markings, according to the invention the communication elements can also be arranged in a defined manner along several rows between the stationary markings, i.e. in a kind of matrix arrangement. The rows and columns can, for example, be parallel to one or both of the first and second straight lines or rotated relative thereto. Such an arrangement in several rows, for example in a matrix form, enables more compact marking units.

In any case, the communication elements can be easily identified in each two-dimensional image by the defined arrangement relative to the markings and the symbols can be decoded.

When the communication elements are arranged along one straight line (or each row in a matrix arrangement), the evaluation unit for decoding the symbols can be set up to determine the distance in the image between a selected one of the two adjacent markings and one of the switched on (in the sense of the To determine the symbol with forming) communication elements and to normalize them to the distance between the two adjacent markers, whereby the respective distance for all connected communication elements is entered in a result vector according to their size (ascending or descending). Checking the result vector with all possible coding vectors that can be expected on the basis of the codings used for the symbols for the best match, the expected coding vector with the best match defining the symbol used for decoding.

In other words, in the case of k-over-n coding, a vector with k length entries is defined along the one (or possibly also several) straight line, each length entry indicating a connected communication element. Since all communication elements preferably lie on a straight line that is precisely defined in length by the two delimiting marking elements, the coding vector can be robustly extracted from every two-dimensional image of the marking unit by normalizing the coding vector to the distance between the two adjacent markings that include the communication elements . In principle, a corresponding arrangement can also be achieved in a matrix arrangement with suitable (in particular asymmetrical) positioning of the matrix arrangement between the markings. A decoding achieved in this way is very reliable, in particular very stable against image noise.

wobei A1, A2, A3 die Bildpunkte der angeschalteten Kommunikationselemente und M0, M1 die benachbarten, die Kommunikationselemente umgebenden Markierungen sind.The coding vector V _image can be described for a 3-out-of-n coding by the following formula $${\text{V}}_{\text{image}}=\Vert \text{M1}-\text{A1}\Vert ,\Vert \text{M1}-\text{A2}\Vert ,\Vert \text{M1}-\text{A3}\Vert /\Vert \text{M1}-\text{M0}\Vert ,$$

where A1, A2, A3 are the pixels of the connected communication elements and M0, M1 are the adjacent markings surrounding the communication elements.

The coding vector V _image is compared to all expected coding vectors, each of which represents a symbol valid in the coding. This can be done by subtracting the coding vector V _image and the expected coding vector V _expected , the absolute value of the vector obtained (match) indicating the degree of correspondence $$\text{match}=\Vert {\text{V}}_{\text{image}}-{\text{V}}_{\text{expected}}\Vert$$

The expected coding vector expected with the smallest associated Mach is chosen as the vector indicating the symbol to be decoded.

A particularly reliable evaluation can be achieved if a marking is arranged in the plane at the point of intersection of the straight lines. This can be an additional marking or a component of the at least two markings that define one of the two straight lines. Since the marking located at the crossing point lies on both straight lines, three markings are sufficient according to the invention to define the two straight lines. However, it corresponds to a preferred embodiment that each straight line can be identified by at least two independent points.

The point of intersection is defined by the direction of the two straight lines and can be determined very reliably in the two-dimensional image. This point is therefore particularly suitable for determining the distance between the connected communication element and the marking arranged at the intersection.

It has been found to be a particularly preferred embodiment to define the plane by at least five of the markings, which are arranged in two straight lines with at least one common mark. Finding lines with three marks is more robust in an industrial environment than finding lines defined by only two marks. Furthermore, at least one of the named five markings can be implemented by a communication element that is also used for communication.

The free arrangement of the marking set off from the plane of the remaining markings has proven to be a further particularly preferred embodiment. Here, the markings in the plane are prevented from being covered by the offset marking when viewed from a large angle. The position of the marking offset from the plane is determined in an application-specific manner in accordance with the viewing angle to be expected in the specific application and is not in direct relation to the straight lines which are defined by the markings in the plane.

As a further particularly preferred embodiment, a variant with between 1-out-of-16, 2-out-of-16 and 3-out-of-16 switchable coding for the communication of the useful data has been found, with 16 here generally being read as n, with the Number of usable for communication, ie means the communication elements built into the marking unit as a whole.

The user of the marking unit can thus decide for himself whether the transmission speed or the energy consumption should be given priority to the marking unit. The number of markings used at the same time has a decisive influence on energy consumption when actively self-luminous markings are used.

Further advantages, features and possible applications of the invention can also be found in the following description of an exemplary embodiment and the drawing. All of the features described and / or illustrated are part of the present invention, regardless of how they are summarized in the claims or their references.

• 1 schematisch eine Aufsicht auf eine bevorzugte erste Ausführungsform der Markiereinheit der erfindungsgemäßen Vorrichtung;
• 2 schematisch eine Aufsicht auf eine bevorzugte zweite Ausführungsform der Markiereinheit der erfindungsgemäßen Vorrichtung;
• 3 eine schematische Darstellung einer Ausführungsform der Erfindung mit mehreren Markiereinheiten;
• 4 eine schematische Darstellung einer erfindungsgemäß bevorzugten Kodierung für die Kommunikation von Nutzdaten; und
• 5 eine schematische Darstellung einer nicht-synchronisierten Kommunikation zur Erläuterung;

Show it:

• 1 schematically a top view of a preferred first embodiment of the marking unit of the device according to the invention;
• 2 schematically a plan view of a preferred second embodiment of the marking unit of the device according to the invention;
• 3 a schematic representation of an embodiment of the invention with several marking units;
• 4th a schematic representation of a coding preferred according to the invention for the communication of useful data; and
• 5 a schematic representation of a non-synchronized communication for explanation;

In 1 is a preferred embodiment of a marking unit 1 a device for location and position detection according to the present invention. This shows the marking unit 1 with five marks 2.1 to 2.5 and sixteen optically activated communication elements 3.1 to 3.16 .

The markings 2.1 to 2.4 are arranged in a plane divided by two straight lines 4th and 5 is stretched. These straight lines 4th , 5 are in 1 represented by dash-dotted auxiliary lines.

The first straight 4th is through the markings 2.1 and 2.2 defined, with all communication elements 3.1 to 3.16 also on this straight 4th are arranged. The second straight 5 is through the markings 2.1 , 2.4 and 2.4 Are defined. The mark 2.1 lies at the intersection of the two straight lines 4th , 5 and is therefore part of the two straight lines 4th , 5 .

The mark 2.5 lies outside of the through the two straight lines 4th , 5 defined plane, preferably such that the vertical projection of the marking 2.5 on the plane not on either of the two straight lines 4th , 5 lies.

Using known geometric relationships between the markings 2.1 to 2.5 to one another which are sufficiently known to the person skilled in the art from the prior art cited at the beginning and therefore need not be explicitly described at this position, can be done using image recognition from two-dimensional images of the marking unit 1 the position and location of the marking unit 1 can be uniquely determined in a spatial coordinate system. The camera taking the picture (i.e. an in 1 optical image acquisition unit (not shown) is calibrated to this spatial coordinate system. In a simplest embodiment, a spatial coordinate system predetermined by the camera can be used. Alternatively, any coordinate system to which the camera is calibrated can also be used. This is also known to the person skilled in the art and does not need to be explained in more detail at this point.

With the marking unit shown 1 are the markers 2.1 to 2.5 and the communication elements 3.1 to 3.16 formed by different optically active or passive elements, for example IR LEDs of different sizes, wavelengths or identities. It is also conceivable that the markings 2.1 to 2.5 optically passive retroreflectors are formed.

The marking unit 1 has a, for example, cross-shaped base body on which the markings 2.1 to 2.5 and the communication elements 3.1 to 3.16 are set. For the sake of clarity, the base body is not shown in the drawings. A computing unit, also not shown, is integrated into the base body, which is set up according to the invention to carry out the communication in the manner already described, in particular to encode useful data to be transmitted and to activate and deactivate optically active elements (in particular the communication elements, possibly also the markings ). The computation units of the marking units are also not shown in the drawing for the sake of clarity.

In 2 is an alternative embodiment of a marking unit 11 shown in which the markings 12.1 to 12.5 and the communication elements 13.1 to 13.16 are formed by the same optically active elements, for example infrared LEDs according to a preferred embodiment. Otherwise, the embodiment corresponds to FIG 2 according to the embodiment 1 . The description that is understood therefore also applies accordingly to the embodiment according to FIG 2 .

3 shows schematically an overall view of an embodiment of the device proposed according to the invention 20th for position and position detection of markings 2 , 12 in space, on several marking units, in the example shown six marking units 23 to 28 , are arranged. The marking units 23 to 28 according to the 1 or 2 marking units described in more detail 1 , 11 and each have the five markings described 2 , 12 and the sixteen described communication elements 3 , 13 on, without the invention being limited to this precise number.

The device 20th comprises an optical image acquisition unit designed as a camera 21st who have several different marking units 23 to 28 can record in one picture. According to the invention, several cameras can also 21st (Optical image acquisition units) can be provided in order to be able to optically acquire a larger spatial area. The or each camera 21st is to an evaluation unit 22nd connected, in a known and basically already described manner for the unambiguous determination of the location and position of each of the marking units 23 to 28 is set up. The respective markings are used for this 2 , 12 the marking units 23 to 28 , possibly supplemented by the also known positions of one or more of the (illuminated) communication elements 3 , 13 furnished. The marking units 23 to 28 can differ from the optical imaging unit 21st be removed from what is in 3 due to the different sizes of the marking units 23 to 28 is symbolized. Their position in space can also be different, as shown.

It is also preferable if there are several cameras 21st a common evaluation device 22nd provided to which all cameras 21st are connected.

According to the invention, the arithmetic units (not shown) are the marking units 23 to 28 set up for the transmission of user data, for example an identification of the marking unit 23 to 28 and / or other information, for example a tool that is fixed to the marking unit. For this purpose, the processing unit encodes the user data and places them on the communication elements (sixteen in the example shown) 3 , 13 the marking units 23 to 28 represent.

The optical imaging unit (camera) 21st takes the marking units in the image area 23 to 28 with the connected communication elements. The images are evaluated in the connected evaluation unit so that the information encoded in the image on the communication elements can be decoded for each marking unit.

In the in 3 example of the sixteen communication elements shown (cf. 1 or 2 ) two different communication elements are activated or switched on and the remaining 14 communication elements are deactivated or switched off. Each of the Communication elements provide different information, for example for identifying the marking units 23 to 28 (Tracker ID or identification code) can be used. If any marking unit 23 to 28 displays their identification code permanently or at predetermined intervals, the marking units 23 to 28 can also be tracked in several consecutive images, even if the marking units 23 to 28 change their position and / or location in space over time.

In terms of communication, the invention is not limited, according to the invention, to an identification code for each marking unit 23 to 28 to be transmitted, but also enables the transmission of further information, such as, for example, the actuation of a tool on which the marking unit is fixed, and which can be detected in any way via the arithmetic unit of the marking unit.

For this purpose, the identification code and other useful data can be transmitted alternately by the arithmetic unit encoding the data to be transmitted one after the other and displaying it on the communication elements. The optical image capture unit 21st takes several pictures of the marking units 23 to 28 one after the other, so that depending on the information just transmitted, different communication elements each have one of the marking units 23 to 28 are activated. These are determined in the evaluation unit and the entire user data is thus composed of the data transmitted one after the other, ie the communication elements activated in each case in sequence.

Such a transmission can basically be realized by switching the activation of the communication elements on the marking units 23 to 28 and the image acquisition are synchronized. In this way, exactly one coding (symbol) can be displayed in an image. Such a coding (also according to the invention) is complex and requires the regular exchange of synchronization character strings in order to maintain the synchronization. This requires on the part of the (often battery-operated) marking units 23 to 28 a considerable additional energy expenditure.

However, if synchronization is lost during transmission, it can happen that various communication elements are switched on and off during the exposure time of an image. This is in 5 shown by way of example for a 2-out-of-16 coding, as already described at the beginning. The connected communication elements 34 ("On") are dark and the communication elements are switched off 35 are ("off") are shown brightly.

A total of sixteen communication elements are provided, which are numbered in steps of 5 on the left-hand side for easy referencing. While taking a picture 30th the arithmetic unit of the marking unit sequentially displays two symbols on the communication elements of the marking unit, which are shown in 5 as the first state 31 of the communication elements during image acquisition and the second state 32 of the communication elements are shown during the image acquisition. In the first state 31 only the communication elements No. 5 and No. 9 are switched on (and form the coded symbol). In the second state, the symbol is formed by the communication elements No. 4 and No. 7 that are switched on.

The overall image resulting from switching the symbols during the image exposure time, ie during the image acquisition 33 of the communication elements shows a symbol with a total of four connected communication elements, namely with communication elements no. 4, no. 5, no. 7 and no. 9. This is an impermissible symbol in the coding and can therefore not be decoded.

Because a synchronization of image recording and coding of the symbols on the communication elements is complex, the invention proposes as a particularly preferred communication method that the marking unit, or the arithmetic unit of the marking unit, is set up to generate the time sequence of the different symbols with a symbol frequency, which corresponds at most to the frame rate of the image capture unit. This ensures that in the image of the camera, which continuously records images of the marking unit, at most two states of the communication elements, as in FIG 5 through the first and second states 31 , 32 symbolizes, are included.

Furthermore, the marking unit, or the computing unit of the marking unit, is set up not to use the k connected communication elements of the preceding symbol for the following symbol in the cycle of the k-out-of-n coded symbols displayed one after the other.

• Der Zyklus der Kommunikation umfasst zwei verschiedene Datenarten, die durch unterschiedliche Kodierungen über den Zeitstrahl t dargestellt werden.

This is in 4th for a preferred embodiment of the communication proposed according to the invention, for the implementation of which the marking units are set up:

• The communication cycle comprises two different types of data, which are represented by different codings over the time line t.

The first type of data 40 is an identification code (tracker ID) for the respective marking unit and is coded in a 2-out-of-16 coding. The second type of data 31 is a user data coding in which any other data is coded. The user data is encoded in a 3-out-of-16 coding. The choice of the specific coding in this example is only used for exemplary explanation. The invention is not limited to this example.

The start of communication, ie the starting point for the sequence of several symbols, is determined in this example by the first data type 40 , ie the transmission of the identification code. This is in 4th represented by the left column of communication elements which contain the first coding of the identification code (first symbol 45 ) describes. In the example chosen, the identification code of the marking unit is due to the activated communication elements 42 with the numbers no. 5 and no. 9 of the total of sixteen communication elements shown in the column. The remaining communication elements 43 are turned off.

That first symbol 45 the sequence of several symbols of symbols displayed one after the other according to the time line t 45 , 46 , 47 , 48 , 49 is a non-changing symbol that has been generated for such a long time that this symbol 45 is visible unchanged in at least two successive images of the marking unit. On the one hand, this allows the starting point of a communication to be displayed. In addition, it is ensured that this information can also be decoded at least in the last of the at least two images. A larger number of images can preferably also be selected in which the coding of the first symbol is located 45 does not change.

The second symbol in the sequence 46 is made by the second type of data 41 , ie formed by useful data different from the identification code. According to the 3-out-of-16 coding, the symbol is activated by communication elements 42 formed with the numbers No. 4, No. 7 and No. 12. As the communication elements with numbers No. 5 and No. 9 in the previous first symbol 45 these are elements of communication 44 with the numbers No. 5 and No. 9 blocked when the second symbol is transmitted. Knowing this previous first symbol 45 can also use the second symbol 46 can be reliably decoded in the recorded image even if the symbols with the numbers No. 5 and No. 9 could still be recognized as switched on in the second image. These communication elements with the numbers No. 5 and No. 9 would, when decoded, as communication elements blocked for the second symbol 44 ignored.

The communication elements that are not switched on and the communication elements that are not blocked are communication elements that are switched off 43 .

For the third symbol 47 and the fourth symbol 48 that is also the second type of data 41 (User data), the explanation for the second symbol applies 46 corresponding.

The fifth symbol 49 is according to the example 4th again a symbol of the first type of data 40 (Identification code), which is generated as described for such a long time that this symbol 49 is visible unchanged in at least two consecutive images of the marking unit. It is therefore not absolutely necessary to set certain communication elements as blocked communication elements 44 excluded from decoding. The second image at the latest enables the information identifier to be decoded. Due to the different coding (in the example only two instead of three illuminated communication elements 42 ) the identification code can also be recognized as such.

As a result of the solution according to the invention, communication of additional useful data with a moderate bandwidth is achieved with no or without significant additional energy requirement in addition to the position and position detection of the marking units. A higher bandwidth can be achieved at any time - with the acceptance of a higher energy requirement - by increasing the information density by using more communication elements connected simultaneously in a symbol.

List of reference symbols

1

Marking unit

2.1 to 2.5

mark

3.1 to 3.16

Communication element

4th

first straight

5

second straight

11

Marking unit

12.1 to1 2.5

mark

13.1 to 13.16

Communication element

14th

first straight

15th

second straight

20th

Device for position and position detection

21st

optical image capture unit

22nd

Evaluation unit

23

Marking unit

24

Marking unit

25th

Marking unit

26th

Marking unit

27

Marking unit

28

Marking unit

30th

Duration of the image acquisition

31

first state of the communication elements during image acquisition

32

first state of the communication elements during image acquisition

33

Image of communication elements

34

activated communication element ("on")

35

deactivated communication element ("off")

40

first type of data (identification code, tracker ID)

41

second data type (user data)

42

activated communication element ("on")

43

deactivated communication element ("off")

44

blocked communication element ("block")

45

first symbol of the sequence of several symbols

46

second symbol of the sequence of several symbols

47

third symbol of the sequence of several symbols

48

fourth symbol of the sequence of several symbols

49

fifth symbol of the sequence of several symbols

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 1813911 A1 [0003]
• US 2005/0201613 A1 [0003]
• EP 1498688 B1 [0003]
• WO 2004/114112 A1 [0003]
• US 2008/0111985 A1 [0003]
• WO 2006/069748 A1 [0003]
• US 5227985 A [0003]
• US 7742895 B2 [0003]
• DE 102014012693 A1 [0003, 0004]

Claims (14)

Device for position and position detection of markings (2, 12) in three-dimensional space with - at least one marking unit (1, 11, 23, 24, 25, 26, 27, 28), which can be fixed on an object, with several markings ( 2, 12), - an optical image acquisition unit (21) which is set up to record images of the marking unit (1, 11, 23, 24, 25, 26, 27, 28), and - with an evaluation unit (22) which is set up for the unambiguous determination of the location and the position of the marking unit (1, 11, 23, 24, 25, 26, 27, 28), the marking unit (1, 11, 23, 24, 25, 26, 27, 28) has at least four markings (2, 12) for determining location and position, of which at least three of the markings (2.1 to 2.4, 12.1 to 12.4) are preferably in a common plane and at least one marking (2.5, 12.5) is outside this plane, characterized in that - the marking unit (1, 11, 23, 24, 25, 26, 27, 28) has at least six optically activatable com communication elements (3, 13) for coding user data, the marking unit (1, 11, 23, 24, 25, 26, 27, 28) being set up to switch on the individual communication elements (3, 13) depending on the user data or the communication elements (34, 42) connected at the same time forming a symbol (45, 46, 47, 48, 49) of the coding, - the image acquisition unit (21) a series of images of the marking unit (1, 11, 23, 24, 25, 26, 27, 28) with a predetermined image frequency, and - the evaluation unit (22) is set up to determine the location and position of the marking unit (1, 11, 23, 24) in an image from the markings (2, 12) , 25, 26, 27, 28) to be determined and from the totality of the switched on or off communication elements (34, 42; 35, 43) to capture the coded user data. Device according to Claim 1 , characterized in that the marking unit (1, 11, 23, 24, 25, 26, 27, 28) is set up to use a k-out-of-n coding for the coding of information, where - n is the number of are optically activated communication elements (3, 13) and - k is the number of communication elements (34, 42) that are switched on simultaneously during the coding. Device according to Claim 1 or 2 , characterized in that the marking unit (1, 11, 23, 24, 25, 26, 27, 28) is set up so that several symbols (45, 46, 47, 48, 49) are displayed one after the other in a time sequence. Device according to Claim 3 , characterized in that the marking unit is set up to generate the time sequence of the different symbols (45, 46, 47, 48, 49) with a symbol frequency which corresponds at most to the image frequency of the image acquisition unit (21). Device according to one of the Claims 2 and 3 or 4th , characterized in that the marking unit (1, 11, 23, 24, 25, 26, 27, 28) is set up in the cycle of the successively displayed, k-out-n coded symbols (45, 46, 47, 48, 49) not to use the k connected communication elements (34, 42) of the preceding symbol (44) for the following symbol (45, 46, 47, 48, 49). Device according to one of the Claims 3 to 5 , characterized in that the marking unit (1, 11, 23, 24, 25, 26, 27, 28) is set up as the starting point for the sequence of several symbols (45, 46, 47, 48, 49) a non-changing Generate symbol (45, 49) for such a long time that this symbol (45, 49) is visible unchanged in at least two successive images of the marking unit (1, 11, 23, 24, 25, 26, 27, 28) . Device according to one of the Claims 3 to 6th , characterized in that the marking unit (1, 11, 23, 24, 25, 26, 27, 28) is set up to encode the symbols (45, 46, 47, 48, 49) shown one after the other in the chronological order to create. Device according to one of the preceding claims, characterized in that the at least four markings (2, 12) are arranged on the marking unit (1, 11, 23, 24, 25, 26, 27, 28) in such a way that two of the markings ( 2, 12) define a straight line (4, 5; 14, 15), and the straight lines (4, 5; 14, 15) thus defined intersect at a point of intersection. Device according to Claim 8 , characterized in that the at least six communication elements are arranged along one of these straight lines (4, 14) between two adjacent markings (2.1, 2.2; 12.1; 12.2). Device according to Claim 9 , characterized in that the evaluation unit (22) is set up to - in the image in each case determine the distance between a selected one of the two markings (2.1, 2.2; 12.1; 12.2) and one of the connected communication elements (34, 42) and on normalize the distance between the two adjacent markings (2.1, 2.2; 12.1; 12.2), with the respective distance for all switched on Communication elements (34, 42) sorted according to their size is entered in a result vector, and - to check the result vector with all possible coding vectors that can be expected based on the codings used for the symbols (45, 46, 47, 48, 49) for the best match , the expected best match coding vector defining the symbol (45, 46, 47, 48, 49) used for decoding. Device according to one of the Claims 8 to 10 , characterized in that a marking (2.1, 12.1) is arranged at the intersection of the straight lines (4, 5; 14, 15). Device according to Claim 10 and 11 , characterized in that the distance between the connected communication element (34, 42) and the marking (2.1, 12.1) arranged at the intersection is determined. Device according to one of the preceding claims, characterized in that the at least six communication elements (3, 13) are arranged along several rows between two adjacent markings (2.1, 2.2; 12.1; 12.2). Device according to one of the Claims 8 to 13 , characterized in that the at least one marking (2.5, 12.5) is arranged outside the plane defined by the at least three of the markings (2.1 to 2.4, 12.1 to 12.4) in such a way that a perpendicular projection of the one lying outside this plane Marking (2.5, 12.5) in the plane does not lie on one of the straight lines (4, 5; 14, 15) which are defined by the at least three markings (2.1 to 2.4, 12.1 to 12.4) lying in the plane.

Images