DE102018209366B4 - Method for determining a position and / or orientation of a device - Google Patents

Abstract

Method for determining a position (7) and / or orientation (6) of a device (1), in particular a motor vehicle, environmental data which are recorded in the environment of the device (1 ) arranged marker element (3), the environmental data being processed by a processing algorithm in order to determine the relative position (7) and / or orientation (6) of the device (1) with respect to the marker element (3), the environmental data being a two-dimensional one Surface (24) of the marking element (3) which is mapped for at least two different orientations (18, 19) of the surface (24) with respect to the detection means (4) with different textures (25, 26), each of which is an image of the same three-dimensional pattern from a perspective assigned to the respective orientation (18, 19), and / or a three-dimensional structure of the marking element map ts (3) and the processing algorithm determines the position (7) and / or orientation (6) of the device (6) depending on the texture (25, 26) and / or the mapping of the three-dimensional structure, with at least two in the environmental data Substructures (15, 16, 17) of the three-dimensional structure or the three-dimensional pattern and / or the at least one section (13) of a first substructure (12) of the three-dimensional structure or the three-dimensional pattern is covered by at least a second substructure (11) of the three-dimensional structure or the three-dimensional pattern can be detected, the position (7) and / or orientation (6) of the device (1) being determined as a function of the relative position of the substructures (15, 16, 17) or of the masking.

Description

The invention relates to a method for determining a position and / or orientation of a device, in particular a motor vehicle, wherein environmental data are captured by a detection means of the device, which represent a marking element arranged in the environment of the device, the environmental data being processed by a processing algorithm, to determine the relative position and / or orientation of the device with respect to the marking element. In addition, the invention relates to a marking element and a motor vehicle.

Automated management of moving devices, especially motor vehicles, is becoming increasingly relevant. Here, the robustness of the guidance can be increased significantly if markers are used which identify certain positions and / or orientations. Such markers can be detected by a detection device in order to determine the position or orientation of the device with high accuracy and thus, for example, to implement highly precise guidance to a specific desired position, for example to a parking position, a loading plate or the like. The problem here is that in order to determine a position or orientation with sufficient accuracy, at least two markers typically have to be recorded at the same time. As a result, the effort for guiding facilities or motor vehicles is significantly increased by appropriate markers.

Alternatively, natural markings in the area can be used. Such a procedure is, for example, from the publications DE 10 2015 220 831 A1 and WO 2010/141209 A1 known. To do this, however, it is first necessary to map the positions of corresponding landmarks with sufficient accuracy, which also results in a relatively large outlay.

The publication DE 10 2013 110 785 A1 discloses a positioning method for positioning a mobile device relative to a security feature of a document. The actual position of the mobile device with respect to the document is determined on the basis of a perspective distortion of a document feature and the position of a security feature is determined. The security feature can be a hologram, for example. To validate the hologram, the mobile device can be positioned with the aid of a guiding approach, which is based on the rear sight and grain and a virtual horizon, in such a way that a view of the hologram is shown that corresponds to a pre-recorded view.

A method for determining an own position of the motor vehicle is from the document US 2017/0016740 A1 known. In this case, landmarks are recognized in image data from a camera and are stored in digital map data. Since the heights, sizes, shapes and the like of the landmarks are known, the landmarks recognized in image data can be used to determine the distance.

From the dissertation by T. Kühnl, Road Terrain Detection for Advanced Driver Assistance Systems, Bielefeld University, 2013, it is known to use feature recognition trained by machine learning in order to assign pixels to a camera to a specific lane. This can be used to determine a lane used by your own vehicle.

The invention is therefore based on the object of specifying an improved way of determining position or orientation, which in particular achieves good accuracy in determining the position or orientation of the device even with a few or a single marking element.

The object is achieved according to the invention by a method of the type mentioned at the outset, on the one hand the environmental data being a two-dimensional surface of the marking element, which is mapped with at least two different orientations of the surface with respect to the detection means with textures different from one another, each representing an image of the same three-dimensional pattern a perspective assigned to the respective orientation, and / or depict a three-dimensional structure of the marking element and the processing algorithm determines the position and / or orientation of the device as a function of the texture and / or the mapping of the three-dimensional structure, with at least two partial structures of the three-dimensional structure or the three-dimensional pattern and / or the masking of at least a portion of a first partial structure of the three-dimensional structure or the three-dimensional pattern du rch can be recognized by at least a second substructure of the three-dimensional structure or the three-dimensional pattern, the position and / or orientation of the device being determined as a function of the relative position of the substructures or the concealment. The processing algorithm can be parameterized by means of several processing parameters that are or are determined as part of a machine learning process.

Within the scope of the invention it was recognized that by using three-dimensional structures on marking elements, ie For example, of projections or recesses, or of surfaces that show different textures depending on the viewing angle, for example of holograms or “tilting images” formed by lenticular screens or parallax barriers, a relative position and orientation of the detection means and thus the device with respect to the marking element can be determined particularly easily can. When using purely two-dimensional markings, the problem arises that a relative orientation of the detection means and the marking can only be recognized due to a compression of the depicted marking. Thus, in some applications it can be difficult to distinguish between a pure scaling of the marking due to a change in distance and a compression due to a change in orientation. Therefore, several markings are used in the prior art to determine a position and orientation of a device with respect to these markings. By using viewpoint-dependent textures or by using three-dimensional structures, a distinction can be made between these cases, whereby the use of a single marking element can be sufficient to determine the position or orientation of the device with sufficient accuracy.

In addition, the position or orientation determination can also be improved by determining these variables using a machine learning method. This can be used in conjunction with the above-described marking element with a three-dimensional structure or with an orientation-dependent texture, but methods of machine learning can also achieve good accuracy in determining the position or orientation with sufficient training if only a two-dimensional marker with a fixed texture is used . In this case, the marking element or the texture can preferably have a predetermined dimension, in particular a predetermined length and width, in order to enable a robust determination of the position and / or orientation. For example, the marking element can have the same dimensions as the marking element used to train the processing algorithm. Here, marking elements with different dimensions can be used if each dimension, e.g. a unique texture is assigned to each pair of length and width, so that the given dimensions are known by recognizing the texture. The dimensions do not have to be determined explicitly by the processing algorithm, but the relationship between texture and dimension of the texture or the marking element in the recorded environmental data on the one hand and the position and / or orientation on the other hand can be learned directly by the processing algorithm.

The environmental data can be acquired two-dimensionally by the acquisition means. For example, the environmental data can be captured by a camera. The position or orientation is determined in particular from a single image of the camera. Alternatively, it would be possible to record the environmental data in such a way that they include depth information on individual pixels. For example, a time-of-flight camera, a laser scanner or the like can be used. If such depth information is available and a marking element with a three-dimensional structure is used, the relative position and orientation can be determined with particularly high accuracy and / or particularly easily.

The different textures can be generated by a hologram or a lenticular screen or a parallax barrier of the marking element. The marking element can, for example, consist exclusively of a hologram, or it can comprise a holder which holds a hologram. By arranging a lenticular grid or a parallax barrier from a corresponding image, a viewing angle-dependent image display can be achieved. In particular, an image that is dependent on the viewing angle can be represented on the surface as the texture. In principle, the different textures can be completely different images, so that, for example, different patterns or different geometric shapes are shown for different angles. This enables a particularly simple distinction between the individual images and thus between the individual viewing directions, but typically a relatively coarse angle gradation is achieved.

It is also possible that the textures mapped for the different orientations of the surface with respect to the detection means are each an image of the same three-dimensional pattern from a perspective assigned to the respective orientation. This can be achieved in particular by generating the textures using a hologram. In this case, the surface essentially behaves as if the three-dimensional pattern was a three-dimensional structure that is actually arranged in the region of the surface. The approaches for evaluating detected three-dimensional structures for determining the position and orientation, which are explained in more detail below, can thus also be used for such viewing angle-dependent textures.

At least two partial structures of the three-dimensional structure or of the three-dimensional pattern and / or the masking of at least a portion of a first partial structure of the three-dimensional structure or of the three-dimensional pattern by at least a second partial structure of the three-dimensional structure or of the three-dimensional pattern are recognized in the environmental data, the position and / or the orientation of the device is determined as a function of the relative position of the substructures or the concealment. This takes advantage of the fact that, in the case of three-dimensional surfaces, in particular surfaces with protrusions or depressions, which in particular have steep walls, projecting parts of the structure cover different parts of the structure further behind, depending on the viewing angle, or the substructures lying at different distances in a two-dimensional image are dependent on the viewing angle and thus, depending on the position and orientation of the detection means with respect to the marking element, are displaced relative to one another. In comparison to a two-dimensional pattern, changes in orientation for a three-dimensional structure can thus be detected much more easily even with a two-dimensional image. As explained above, the same effect can also be used if a corresponding influence of the perspective on concealments or relative positions of substructures of a three-dimensional pattern is imaged depending on the viewing angle by means of a hologram or the like.

The position and / or orientation of the device can be determined in addition or relatively depending on an expansion of the image of the surface and / or the texture and / or a section of the texture and / or the three-dimensional structure and / or a section of the three-dimensional structure. It can be used here that the mapping of the surface or the texture or the three-dimensional structure scales with the distance between the detection means and the marking element. In particular, as explained above, a viewing direction of the detection means onto the marking element and then the distance depending on the extent can be determined.

As explained at the beginning, it can be advantageous to use a machine learning method to determine the position or orientation from the environmental data. In this case, for example, a neural network, in particular a convolutional neural network, can be used as the processing algorithm. A training can be carried out with the aid of training data sets, in particular a method of error feedback or gradient descent can be used to train the neural network. Other methods of machine learning can also be trained using training data sets.

A training data record for monitored learning can each include input data for the processing algorithm and a target result. The processing algorithm can then be applied to the input data for all training data sets or a part of the training data sets, and a deviation from the target results can be minimized by varying the processing parameters.

The processing algorithm can be trained in particular by means of a plurality of training data sets which comprise images of the marking element from different angles and / or distances. The images are preferably recorded by the detection means or a further detection means which is constructed in a similar manner or at least results in similar environmental data as a result. The target results can indicate the viewing angle or the distance for the respective image. In this case, the trained processing algorithm also initially outputs the viewing angle and the distance with which the marking element is detected. The position and / or orientation of the device can be calculated by corresponding offsets, which are known due to the known arrangement of the detection means in the device. Alternatively, corresponding offsets can already be taken into account during training, that is to say in the target result.

In addition to the method according to the invention, the invention relates to a marking element which is set up to participate in the method according to the invention. In particular, the marking element can have a three-dimensional structure, in particular projections and / or depressions on a side facing the detection means. As an alternative or in addition, at least one surface of the marking element can have a viewing angle-dependent texture, which can be realized, for example, by having a hologram, a lenticular screen, a parallax barrier or the like.

In addition, the invention relates to a device, in particular a motor vehicle, with a detection means for recording environmental data and a processing device for processing the environmental data, the processing device being set up to process the environmental data in accordance with the inventive method in order to determine the position and / or orientation of the Determine device with respect to the marking element. In particular, the device can be a motor vehicle that is set up to be partially or fully automated as a function to drive the determined position and / or orientation. For example, the motor vehicle can be guided into a specific parking position or onto a loading plate using the method explained.

• 1 ein Ausführungsbeispiel einer erfindungsgemäßen Einrichtung,
• 2 - 4 verschiedene Ausführungsbeispiele eines erfindungsgemäßen Markierungselements, und
• 5 ein Ablaufdiagramm eines Ausführungsbeispiels eines erfindungsgemäßen Verfahrens.

Further advantages and details of the invention emerge from the following exemplary embodiments and the associated drawings. Here show schematically.

• 1 an embodiment of a device according to the invention,
• 2nd - 4th different embodiments of a marking element according to the invention, and
• 5 a flow diagram of an embodiment of a method according to the invention.

1 shows a facility 1 , namely a motor vehicle that is in a target pose 2nd should be performed. This can take place, for example, as part of an automated parking process or as part of the positioning of the motor vehicle with respect to a charging device, for example a charging plate. In order to enable, in particular, fully automated positioning, the position and orientation of the motor vehicle should be recorded as precisely as possible. For this purpose, an artificial marking element 3rd used, for example on a wall 10th is arranged. Alternatively, the marking element 3rd also be attached to a floor, ceiling or other surfaces.

Through a means of registration 4th Ambient data of the motor vehicle are recorded, which is the marking element 3rd map at least partially. These are processed by the processing facility 5 processed to the position 7 and orientation 6 of the motor vehicle with respect to the marking element 3rd to investigate. Are a target position and target orientation for the target pose 2nd regarding the marking element 3rd Known, a trajectory can be planned based on this data to the motor vehicle in the target pose 2nd respectively.

As part of the processing of the environmental data can be a distance 9 between the detection means 4th and the marker element 3rd as well as a perspective 8th in which the marking element 3rd is detected, determined. This information correlates directly with the orientation 6 and the position 7 .

So far, two-dimensional marking elements have been used in the application shown 3rd or certain patterns on walls or floors. This is usually when using a single marking element 3rd not sufficiently precise position and orientation determination possible. The accuracy and robustness of the position and orientation determination can be improved on the one hand by using the following, with reference to 2nd - 4th is explained, special marking elements are used, and / or that, as later with reference to 5 will be explained, a method of machine learning is used to determine the position and orientation.

In the in 2nd shown embodiment has the marking element 3rd a three-dimensional structure. The subtree 11 forms a projection that of the substructure 12th , the base of the marker 3rd , protrudes. The marker element 3rd can, for example, be rotationally symmetrical with respect to this projection.

Will the marker element 3rd in the recording geometry shown by the detection means 4th the section is shown 13 of the substructure 12th through the substructure 11 covered. This can be recognized by processing the environmental data, especially if the surface 14 of the substructure 12th has a pattern that clearly shows the degree of concealment. The position and extent of the section 13 , which can be determined by processing the environmental data, depend on the position and orientation of the detection means 4th and thus the facility 1 regarding the marking element 3rd from. Thus, compared to a two-dimensional marking, there is additional information regarding the position and orientation of the device 1 regarding the marking element 3rd ready. Thus, due to the three-dimensional structure of the marking element 3rd improves the accuracy and robustness of the position and orientation determination.

Becomes a three-dimensional structure of the marking element 3rd pictured, can be related to the one related to 2nd described concealment or alternatively several substructures 15 , 16 , 17th of the three-dimensional marking element 3rd can be recognized, which are exemplary in 3rd are shown. A change in orientation 18th , 19th the facility 1 or the detection means 4th regarding the marking element 3rd leads to images of substructures 15 , 16 that are on different heights, change the distance 20 , 21 that is far larger than the change in distance 22 , 23 the images of substructures 16 , 17th that are on the same high projections or on one level. The three-dimensional structure of the marker 3rd can thus be used to identify different substructures and to determine their spacing, with corresponding spacing in a two-dimensional image of the marking element 3rd can vary significantly more with the point of view than at a purely two-dimensional marking element would be the case.

A particularly strong dependence on the illustration of the marking element 3rd of orientation 18th , 19th can, as schematically in 4th is achieved by having a two-dimensional surface 24th of the marking element 3rd is used for two different orientations 18th , 19th the surface 24th regarding the detection means 4th with different textures 25th , 26 is mapped. The pictures 27 , 28 show schematically the content of the environmental data when the surface 24th from the perspective 18th , 19th is mapped. The surface 24th itself is mapped similarly in both cases, with a texture shown on the surface 25th , 26 is different. In the example shown, simple geometric shapes are used as textures 25th , 26 used for the viewing angles that allow a distinction between two or more viewing angles. However, more complex patterns or holographic or pseudo-holographic images can also be used.

The shown illustration of different textures 25th , 26 depending on the point of view 18th , 19th can be technically realized by using the surface 29 a certain pattern is used up, by applying a parallax barrier 30th or a lenticular grid 31 on the surface 29 depending on the point of view 18th , 19th different parts of this pattern are shown. Such “tilt pictures” are well known in the prior art and should not be explained in detail.

In an alternative embodiment, the surface could 24th instead be formed by a hologram. It is possible here that the entire marking element is a hologram or that a hologram is applied to one or more surfaces of the marking element 3rd is applied. If a hologram is used, a three-dimensional pattern imaged by the hologram becomes perspective correct depending on the viewing angle 18th , 19th reproduced. Thus, those related to 2nd and 3rd Explained approaches to exploit the concealment or change in the relative position of partial structures of a three-dimensional structure can also be used for partial structures of this three-dimensional pattern, the marking element as a whole being simple to set up, since instead of a three-dimensional structure, a flat surface with a hologram attached to it is used can.

The dimension 32 the surface 24th or the texture 25th , 26 or part of this texture 25th , 26 or one of the in the 2nd and 3rd pictured substructures 11 , 12th , 15 , 16 , 17th or the entire three-dimensional structure can be taken into account when determining the position and / or orientation. In particular, as explained above, there can be between different angles 18th , 19th a distinction is made, based on a relative orientation of the detection means 4th and thus the facility 1 to the marker element 3rd can be closed. A distance 9 between the detection means 4th and thus the facility 1 and the marker element 3rd can then depend on the dimension 32 be determined.

As explained in detail above, three-dimensional structures or viewpoint-dependent textures of the marking element can be evaluated by a manually parameterized algorithm in order to determine a relative position and orientation of the device 1 to the marker element 3rd to determine. However, it may also be advantageous to use a machine learning method instead. In particular, this can also enable a relative position and orientation of the device when a single two-dimensional marking element is used 1 to determine this marking element with sufficient accuracy. An example of such a procedure is given below with reference to the in 5 shown flowchart explained.

The steps S1 - S3 are used to train the processing algorithm, for example a neural network, that is to say to determine its processing parameters. These steps can be separate from the following steps S4 - S6 be carried out, especially outside the facility whose position and / or orientation is to be determined.

In step S1 First, images of the marking element are recorded from different angles and distances. For this purpose, the registration means 4th or a further, preferably identical or at least similarly designed further detection means can be used. In step S2 will be an assigned position and orientation for the facility for each of these recordings 1 given. For this purpose, the viewing angle and the distance are determined, for example by a highly precise measurement of the relative position and orientation of the detection means and the marking element with respect to one another, and the position and orientation of the device are determined from this by means of appropriate offsets 1 given. Alternatively, it would also be possible to train the processing algorithm in such a way that it first determines the viewing angle and distance and only then adds the corresponding offsets.

In step S3 the training data records are used to train the processing algorithm. For this purpose, the processing algorithm can be applied to the individual images and a deviation of the sizes determined from those in step S2 predetermined sizes can be minimized. Appropriate approaches, in particular error feedback or a gradient method for training neural networks and other methods of machine learning, are known in the prior art and are not to be explained in detail.

In step S4 the environmental data through the detection means 4th the facility 1 detected. These are in step S5 through the in step S3 parameterized processing algorithm processed to a position and orientation of the device 1 regarding the marking element 3rd to investigate. In step S6 For example, these variables can be used to set up a trajectory for automated operation of a device 1 determine used motor vehicle.

Claims (7)

Method for determining a position (7) and / or orientation (6) of a device (1), in particular a motor vehicle, environmental data which are recorded in the environment of the device (1 ) arranged marker element (3), the environmental data being processed by a processing algorithm in order to determine the relative position (7) and / or orientation (6) of the device (1) with respect to the marker element (3), the environmental data being a two-dimensional one Surface (24) of the marking element (3) which is mapped for at least two different orientations (18, 19) of the surface (24) with respect to the detection means (4) with different textures (25, 26), each of which is an image of the same three-dimensional pattern from a perspective assigned to the respective orientation (18, 19), and / or a three-dimensional structure of the marking element map ts (3) and the processing algorithm determines the position (7) and / or orientation (6) of the device (6) depending on the texture (25, 26) and / or the mapping of the three-dimensional structure, with at least two in the environmental data Substructures (15, 16, 17) of the three-dimensional structure or the three-dimensional pattern and / or the at least one section (13) of a first substructure (12) of the three-dimensional structure or the three-dimensional pattern is covered by at least a second substructure (11) of the three-dimensional structure or the three-dimensional pattern can be detected, the position (7) and / or orientation (6) of the device (1) being determined as a function of the relative position of the substructures (15, 16, 17) or of the masking. Procedure according to Claim 1 , characterized in that the environmental data are recorded in two dimensions by the detection means (4). Procedure according to Claim 1 or 2nd , characterized in that the different textures (25, 26) are generated by a hologram or a lenticular screen (30) or a parallax barrier (31) of the marking element (3). Method according to one of the preceding claims, characterized in that the position (7) and / or orientation (6) of the device (1) as a function of an extent (32) of the image of the surface (24) and / or the texture (25, 26) and / or a section of the texture (25, 26) and / or the three-dimensional structure and / or a section of the three-dimensional structure. Method according to one of the preceding claims, characterized in that the processing algorithm is parameterized by a plurality of processing parameters which are or are determined in the context of a method of machine learning. Procedure according to Claim 5 , characterized in that the processing algorithm is trained by a plurality of training data sets, which comprise images of the marking element (3) from different angles and / or distances. Device, in particular motor vehicle, with a detection means (4) for recording ambient data and a processing device (5) for processing the ambient data, characterized in that the processing device (5) is set up to process the environmental data according to the method according to one of the Claims 1 to 6 to process in order to determine the position (7) and / or orientation (6) of the device (1) with respect to the marking element (3).

Images