DE102021207180B3 - Method and device for locating a vehicle in an environment - Google Patents

Abstract

The invention relates to a method for locating a vehicle (50) in an environment, in which clear optical markers (40) with known properties are or will be arranged in the environment, with an environment image (10) of the environment using at least one camera (51) is detected, with at least one optical marker (40) being detected in the captured environment image (10), with an identifier (41) of the at least one optical marker (40) being read out, with a marker position (20) of the detected at least one optical marker (40) is determined on the basis of the identifier (41) read in each case in a card (15), with a relative position (21) of the at least one camera (51) being determined for the detected at least one optical marker (40), and with a Camera position (22) of the at least one camera (51) and / or a vehicle position (23) of the vehicle (50) in the map (15) based on the specific relative ven position (21) and the determined marker position (20) is determined and provided, the determination of the camera position (22) and / or the vehicle position (23) taking into account camera properties (51-1). The invention also relates to a device (1) for locating a vehicle (50) in an area.

Description

The invention relates to a method and a device for locating a vehicle in an environment.

A core problem in partially automated or automated driving of a vehicle is localization of the vehicle in an environment. In addition to localization via satellite-based position determination systems (e.g. Global Positioning System, GPS), landmark-based systems are also known in which objects in the area are recognized and used for localization on a map.

From the CN 112304305A a method and system for initial positioning of a vehicle is known. The method comprises the steps of: scanning and creating a map of an underground car park, fixing a marker image and the position of the marker image in the map, calculating a rotation R and a translation T from a camera to the marker image by using a perspective-n-point ( PnP) algorithm, calculating the coordinates of the camera relative to the map and completing the initial positioning. According to the method using the initial positioning task based on the marker images, only a plurality of marker images need to be arranged in advance at the entrance to the underground car park to complete the calculation of the initial position.

An omnidirectional camera model is known from D. Scaramuzza, Omnidirectional Vision: from Calibration to Robot Motion Estimation, ETH Zurich, PhD Thesis no. 17635, Zurich, February 22, 2008.

From the DE 10 2014 002 150 B3 a method is known for determining the absolute position of a mobile unit, which has at least one optical environment sensor recording a detection range of the environment surrounding the mobile unit, in a predetermined navigation environment, wherein optical, distinguishable markers that can be detected by the environment sensor are used in the navigation environment, each of which is assigned an absolute position that can be called up from a map data set, with at least one marker being detected and identified in sensor data recorded by at least one of the at least one environmental sensors using an image processing algorithm to determine the position, and taking into account the position of the marker in the sensor data, a relative position between the mobile unit and the marker is determined, and an absolute position of the mobile unit is determined as a function of the relative position and the absolute position of the marker, with Ber Taking into account the last determined position of the mobile unit, markers that can be detected by the environmental sensors and the setting data relating to their position in relation to the mobile unit are determined using the map data set and are used to limit the sensor data from the environmental sensors to be evaluated for the detection and identification of the at least one marker.

From the DE 10 2018 210 765 A1 a method for operating a localization system is known, in which map data is provided that includes position data for recorded landmarks, each recorded landmark being assigned a landmark type. Surrounding data are recorded and real landmarks are detected using the surrounding data. A self-position is determined on the basis of the detected real landmarks and the map data. In the method, the real landmarks are detected by means of at least one detector module, each detector module being assigned to a landmark type. A new landmark type is determined on the basis of the map data and descriptor information for the new landmark type is provided. A new detector module for the new landmark type is generated on the basis of the descriptor information.

From the WO 2009/098319 A2 a navigation system is known comprising a navigation device suitable for being mounted on or carried by a vehicle and at least one position marker which is in a fixed position relative to a highway along which the vehicle can travel. The navigation device comprises processing means arranged to receive a feed from a camera containing an image of a scene in front of the vehicle, to process the image to identify any position markers in the scene, to determine the absolute position of the position marker using information associated with the marker, determine the distance between the vehicle and the position marker by processing the image supplied by the camera; and determine an estimate of the position of the vehicle from the absolute position and distance.

From the WO 2018/044191 A1 a vehicle positioning system is known that uses multiple optical cameras with adjacent fields of view to read coded markers with predetermined positions to determine the position of a vehicle within a structure with a high degree of accuracy. The vehicle position The positioning system allows for the direct installation and use of a positioning device on a vehicle with a limited number of coded markers to determine the position of the vehicle with millimeter accuracy.

From the U.S. 2021/0 017 859 A1 a system for controlling a mining machine inside an underground mine is known. A rotatable laser source emits laser light and a taillight sensor receives reflected laser light and provides an indication of distance and taillight intensity at several different rotation angles. A coordinate reference point comprises a pattern of varying reflectivity and provides at least one 2D coordinate location. A processor determines an absolute coordinate position of the mining machine in space as the mining machine moves through the underground mine. The processor collects reflected laser light intensity values for a plurality of respective rotation angles, and recognizes the pattern of the reference point in the plurality of reflected laser light intensity values, and determines the absolute coordinate position in space of the mining machine based on spatial information of the recognized pattern.

From the DE 10 2017 123 515 B4 a method for lateral vehicle position determination is known.

The invention is based on the object of creating a method and device for locating a vehicle in an environment in which localization is improved.

The object is achieved according to the invention by a method having the features of patent claim 1 and a device having the features of patent claim 8 . Advantageous configurations of the invention result from the dependent claims.

In particular, a method for locating a vehicle in an environment is provided, in which clear optical markers with known properties are or will be arranged in the environment, with an environment image of the environment being captured by at least one camera, with at least one in the captured environment image optical marker is detected, with an identifier of the at least one optical marker being read, with a marker position of the detected at least one optical marker being determined on a map based on the identifier read out in each case, with a relative position of the at least one camera to the detected at least one optical marker is determined, and wherein a camera position of the at least one camera and/or a vehicle position of the vehicle on the map is determined and provided on the basis of the determined relative position and the determined marker position, w wherein the camera position and/or the vehicle position is/are determined taking into account camera properties.

Furthermore, in particular a device for locating a vehicle in an environment is created, comprising at least one camera and a position determination device, wherein the at least one camera is set up to capture an image of the environment, and wherein the position determination device is set up in the captured image of the environment to recognize at least one clear optical marker arranged in the environment, to read out an identifier of the at least one optical marker, to determine a marker position of the detected at least one optical marker on a map based on the respectively read out identifier, a relative position of the at least one camera to the detected at least to determine an optical marker, and a camera position of the at least one camera and/or a vehicle position of the vehicle in the map based on the determined relative position and the determined markerp to determine and provide position, the camera position and/or the vehicle position being determined taking into account camera properties.

The method and the device make it possible to locate a vehicle on a map. This is done by capturing and recognizing unique optical markers around the vehicle. An identifier of a detected and recognized optical marker is read out from the captured environment image and a marker position belonging to the optical marker is determined on the basis of the identifier in a map. The unique optical markers are stored in the map at associated positions, so that the respective position of an optical marker can be retrieved from the map based on the identification code read out. A position of the at least one camera relative to the detected at least one optical marker is determined. This is done in particular taking into account a beam path of the image. In particular, a position of the at least one optical marker is determined in a coordinate system of the at least one camera. In other words, a position of the at least one optical marker in the coordinate system of the camera is determined as the relative position. Based on the determined relative position and the Marker position on the map, a camera position of the at least one camera and/or a vehicle position of the vehicle on the map is determined. For this purpose, if the position or pose of the at least one camera in a vehicle coordinate system is known, a corresponding transformation can be carried out. In other words, a position of the optical marker is determined in particular in the camera coordinate system. Together with the marker position known from the map, a camera position and/or a vehicle position, which in particular has a fixed relationship to the camera position, can then be determined. When determining the camera position of the at least one camera and/or the vehicle position of the vehicle, camera properties, in particular intrinsic camera properties, are taken into account. Since the camera properties are taken into account when determining the camera position of the at least one camera and/or the vehicle position of the vehicle, the camera position and/or the vehicle position can be determined in an improved manner.

The at least one camera is, for example, a camera with a fisheye or wide-angle lens. The (intrinsic) camera properties then result in particular from the imaging properties of the fisheye or wide-angle optics. When determining the position of the at least one camera relative to the at least one optical marker, a camera model is taken into account in particular, with imaging properties of a camera lens, for example, being taken into account in the camera model.

Parts of the device, in particular the position determination device, can be designed individually or combined as a combination of hardware and software, for example as program code that runs on a microcontroller or microprocessor. However, it can also be provided that parts are designed individually or combined as an application-specific integrated circuit (ASIC) and/or field-programmable gate array (FPGA).

It is provided that the camera properties include a subdivision of a viewing area of the camera into areas of different quality, the quality being taken into account when determining the relative position of the at least one camera and/or when determining the camera position and/or the vehicle position. As a result, the camera position and/or the vehicle position can be determined in such a way that a known or determinable uncertainty when determining the relative position has an influence on the specific value that is determined for the camera position and/or the vehicle position.

One embodiment provides for the relative position of the at least one camera to be determined based on a picture element position of the at least one optical marker in the captured image of the surroundings and known geometric properties of the at least one optical marker. In this case, it is provided in particular that corners of a square optical marker are determined. On the basis of lines of sight from the at least one camera to the respective corners, which are determined from the respective image element position in the captured image of the surroundings, and the known geometric properties, in particular an extent of the optical marker or a distance between adjacent corners of the optical marker, a Position of the optical marker can be determined in the camera coordinate system.

In a further developing embodiment it is provided that the quality of a respective area of the field of view of the at least one camera is empirically determined or has been determined. This can be done, for example, as part of a calibration of the at least one camera. For example, it can be determined that a quality due to camera optics-caused distortion in an edge region in a captured environment image decreases with a distance from a center point of the captured environment image. A specific relative position, which is based on an optical marker detected in an edge area, then has a lower quality than a specific relative position, which was based on an optical marker detected in the center of the image.

One embodiment provides that specific relative positions of the at least one camera to a plurality of optical markers detected in the captured image of the surroundings are taken into account in a weighted manner, starting from a corresponding quality when determining the camera position and/or the vehicle position. In this way, given a plurality of relative positions, the individual relative positions can be weighted according to their respective uncertainty, which improves the determination of the camera position and/or the vehicle position. In particular, relative positions that correspond to a higher quality can be weighted more heavily than relative positions that correspond to a lower quality.

In one embodiment, it is provided that specific relative positions of the at least one camera to a plurality of optical markers detected in the captured image of the surroundings, based on a respective corresponding quality when determining the camera position and/or the Vehicle position are taken into account or discarded depending on at least one predetermined threshold value. As a result, certain relative positions that correspond to a quality below the at least one predefined threshold value can be ignored when determining the camera position and/or the vehicle position.

It can also be provided that a combination of weighting and threshold-dependent consideration is used. For example, it can be provided that relative positions that correspond to a quality below the specified threshold value are not taken into account when determining the camera position and/or the vehicle position, whereas all relative positions that correspond to a quality equal to or above the specified threshold value are weighted are taken into account according to their respective quality.

In one embodiment it is provided that the clear optical markers in the area are designed as Aruco markers. In particular, the Aruco markers have predetermined properties, in particular a predetermined and known geometric size. In particular, the Aruco markers are placed on the vertical walls of buildings and other objects in the area. It is provided in particular that an upper and a lower edge of the Aruco marker are or will be arranged horizontally.

In one embodiment, it is provided that at least one other object with known properties is detected in the captured environment image, with a further relative position of the at least one camera to the detected at least one other object being determined, and with the determined further relative position when determining the camera position the at least one camera and/or the vehicle position of the vehicle in the map is taken into account. This can further improve the determination of the camera position and/or the vehicle position. Based on the known properties, in particular geometric properties such as dimensions, etc., a position relative to the other object that has been detected can be determined. This is also done in the camera's coordinate system, taking camera properties into account. In particular, it is further provided that the detected object is searched for in the map and a position of the detected object is retrieved from the map. A camera position of the at least one camera and/or a vehicle position of the vehicle on the map can then also be determined via the position of the detected object on the map and the determined relative position, which when determining the camera position and/or the vehicle position via the optical marker can be taken into account. Another object can be a traffic sign, a bridge, a house, a door, etc., for example. In this context, it is assumed in particular that a size of the further object (from the map) is known, as is a position or pose of the further object in the map.

Further features for the configuration of the device result from the description of configurations of the method. The advantages of the device are in each case the same as in the embodiments of the method.

Furthermore, in particular, a vehicle is also created, comprising at least one device according to one of the described embodiments.

• 1 eine schematische Darstellung einer Ausführungsform der Vorrichtung zum Lokalisieren eines Fahrzeugs in einem Umfeld;
• 2 eine schematische Darstellung zur Verdeutlichung einer Ausführungsform der Vorrichtung;
• 3 eine schematische Darstellung einer Ausführungsform eines eindeutigen optischen Markers (Aruco-Marker);
• 4 eine schematische Darstellung zur Verdeutlichung des Bestimmens einer relativen Position in einem Koordinatensystem einer Kamera aus einer erfassten Umfeldabbildung;
• 5 eine schematische Darstellung zur Verdeutlichung von Sichtstrahlen einer Kamera;
• 6 ein schematisches Ablaufdiagramm einer Ausführungsform des Verfahrens zum Lokalisieren eines Fahrzeugs in einem Umfeld.

The invention is explained in more detail below on the basis of preferred exemplary embodiments with reference to the figures. Here show:

• 1 a schematic representation of an embodiment of the device for locating a vehicle in an environment;
• 2 a schematic representation to illustrate an embodiment of the device;
• 3 a schematic representation of an embodiment of a unique optical marker (Aruco marker);
• 4 a schematic representation to clarify the determination of a relative position in a coordinate system of a camera from a captured image of the surroundings;
• 5 a schematic representation to clarify visual beams of a camera;
• 6 a schematic flowchart of an embodiment of the method for locating a vehicle in an environment.

In the 1 a schematic representation of an embodiment of the device 1 for locating a vehicle 50 in an environment is shown. The device 1 is arranged in the vehicle 50 and comprises a camera 51 and a position determination device 2. The position determination device 2 comprises, for example, a computing device 3, eg a microprocessor, and a memory 4. In particular, the device 1 executes the method described in this disclosure. The arithmetic unit 3 can carry out arithmetic operations to carry out measures of the method.

The camera 51 is, for example, a wide-angle camera or a fisheye camera. The camera 51 captures an image 10 of the surroundings. Clear optical markers with known properties are arranged in the environment, for example on buildings or facilities of a traffic infrastructure.

The position determination device 2 is set up to recognize at least one clear optical marker arranged in the surroundings in the recorded surroundings image 10 . This is done, for example, by means of pattern recognition methods known per se. After recognition, image elements (pixels) in the recorded environment image can be assigned to the optical marker. In particular, values and a respective position of these picture elements are subsequently processed.

The position determination device 2 reads an identifier of the at least one optical marker. For this purpose, for example, a bit sequence encoded in the optical marker is determined from the values of the picture elements. Based on the identifier read out in each case, the position determination device 2 determines a marker position 20 of the recognized at least one optical marker in a map 15 by comparison. The map 15 is stored in the memory 4, for example. The marker position 20 is given in particular in absolute coordinates of the map 15, in particular in coordinates of a global coordinate system.

The position determination device 2 determines a relative position 21 of the at least one camera 51 to the detected at least one optical marker. Furthermore, the position determination device 2 determines a camera position 22 of the at least one camera 51 and/or a vehicle position 23 of the vehicle 50 in the map 15 based on the determined relative position 21 and the determined marker position 20 and makes them available, for example in the form of an analog or digital Signal, for example as a digital data packet.

The specific camera position 22 and/or the specific vehicle position 23 are supplied as input data, for example, to a navigation device 52 of the vehicle 50 .

The determination of the camera position 22 and/or the vehicle position 23 is carried out taking into account camera properties 51-1. These camera properties 51-1 can include, for example, a distortion of imaging optics (e.g. a lens system) etc.

Provision can be made for the relative position 20 of the at least one camera 51 to be determined based on a picture element position of the at least one optical marker in the captured image of the surroundings 10 and known geometric properties of the at least one optical marker.

In particular, it is provided that the clear optical markers in the area are designed as Aruco markers.

It can be provided that the camera properties 51-1 include a subdivision of a field of view of the camera 51 into areas 12-x of different quality 13, the quality 13 when determining the relative position 21 of the at least one camera 51 and/or when determining the camera position 22 and/or the vehicle position 23 is taken into account. This embodiment is in 2 illustrated schematically using a captured environment image 10. It is assumed that the captured environment image 10 coincides with the field of view of the camera. If, for example, there is a distortion in the direction of the image areas on the outside with respect to an image center 11 of the environment image 10, the areas 12-x on the outside can correspond to a lower quality 13 or these areas 12-x have a lower quality 13 assigned (see graph in 2 to the right). It can also be provided that a function is used and/or determined (e.g. the function is determined empirically as part of a calibration of the camera 51) starting from an image center 11 (or another reference position of the image elements of the environment image 10), wherein the Function describes a quality 13 that is dependent on a distance r from the image center 11 .

It can be provided that the quality 13 of a respective area 12-x of the field of view of the camera 51 is empirically determined or is determined. In particular, this can be done as part of a calibration of the camera 51 .

It can be provided that specific positions 21 of the camera 51 relative to several in the recorded environment image 10 ( 1 ) detected optical markers based on a corresponding quality 13 ( 2 ) are weighted when determining the camera position 22 and/or the vehicle position 23 . The weighting is then selected in particular in such a way that coefficients correspond to the respective quality 13, with a relative position 21 being weighted all the more heavily the greater a quality 13 corresponding thereto is.

It can be provided that certain relative positions 21 of the camera 51 to a plurality of optical markers detected in the captured image of the surroundings 10, based on a respective quality 13 corresponding thereto, when determining the camera position 22 and/or the vehicle position 23 as a function of at least one predetermined threshold value 16 be taken into account or discarded. In particular, the quality 13, which corresponds to a certain relative position 21, is compared with the threshold value 16. If the quality 13 reaches or exceeds the threshold value 16, the relative position 21 (weighted in some embodiments) is taken into account, otherwise the relative position 21 is not taken into account and is discarded.

Provision can be made for at least one further object with known properties to be recognized in the captured environment image 10, with a further relative position 30 ( 1 ) of the camera 51 is determined for the detected at least one further object, and the determined further relative position 30 is taken into account when determining the camera position 22 of the camera 51 and/or the vehicle position 23 of the vehicle 50 in the map 15. In this embodiment too, a quality factor 13 corresponding to the further relative position 30 can be taken into account, as described.

In the 3 1 is a schematic representation of a unique optical marker 40, the unique optical marker 40 being embodied as an Aruco marker. The optical markers 40 each have a unique identifier 41 which is graphically encoded as a bit pattern. Furthermore, the optical markers 40 are dimensioned and arranged in such a way that dimensions 42 (here by way of example only vertical dimensions) are known, so that the dimensions 42 can serve as a size reference when determining the relative position. A relative position between the optical marker and the camera can be determined via the dimensions 42; in particular, a position of the optical marker in a coordinate system of the camera can be determined. For this purpose, in particular, the four corners of the Aruco marker are recognized in a captured image of the surroundings, for example with the aid of known methods of computer vision and/or pattern or object recognition. The unique identifier 41 encoded in the bit pattern can be read out from the captured image of the surroundings.

In the 4 a schematic representation is shown to clarify the determination of a relative position in a coordinate system of a camera 51 from a captured image 10 of the surroundings. This is based on an omnidirectional camera model, as is used in "fish-eye optics" (cf. also D. Scaramuzza, Omnidirectional Vision: from Calibration to Robot Motion Estimation, ETH Zurich, PhD Thesis no. 17635, Zurich, February 22, 2008) . The goal is to find a relationship between a given two-dimensional pixel (point p with coordinates u and v) and a three-dimensional vector P originating from an effective viewpoint O. In general, this means that intrinsic camera parameters and intrinsic mirror properties have to be found (which in particular correspond to the camera properties in this disclosure). In this case, the omnidirectional camera model treats the imaging system as a uniform, compact system, that is to say it does not differentiate whether a mirror 61 or a fisheye optic is used in combination with the camera 51 .

In the model, the following assumptions are made: A mirror-camera system 60 is a central system, that is, there exists a point in the mirror 61 through which each reflected ray passes (viewpoint O). This point forms the origin of the camera coordinate system with the coordinates x, y and z. A camera axis and a mirror axis are aligned with each other, which means that only small deviations in rotation are taken into account in the model. The mirror 61 is rotationally symmetrical with respect to its axis. A lens distortion of the camera 51 is not taken into account in the model. For fisheye lenses, camera lens distortion can be integrated in a projection function f introduced below.

mit u, v den Koordinaten eines Bildelementes in einer erfassten Umfeldabbildung 10. Die Funktion, die eine Kalibrierung schätzt ist die Funktion, die einen Bildpunkt p in der erfassten Umfeldabbildung auf einen korrespondierenden dreidimensionalen Vektor P abbildet. Daher lässt sich schreiben: $$P=\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=\left[\begin{array}{c}a\cdot u\\ a\cdot v\\ ƒ\left(u,v\right)\end{array}\right]$$

Assuming that a camera axis and a mirror axis are aligned, then: $$\left[\begin{array}{c}x\\ y\end{array}\right]=a\cdot \left[\begin{array}{c}\mathrm{and}\\ v\end{array}\right]$$

with u, v the coordinates of a pixel in a captured environment image 10. The function that a calibration estimates is the function that maps a pixel p to a corresponding three-dimensional vector P in the captured environment image. Therefore one can write: $$P=\left[\begin{array}{c}x\\ y\\ \mathrm{e.g}\end{array}\right]=\left[\begin{array}{c}a\cdot \mathrm{and}\\ a\cdot v\\ ƒ\left(\mathrm{and},v\right)\end{array}\right]$$

The prefactor α can be included in the function ƒ, resulting in the following expression: $$P=\left[\begin{array}{c}x\\ y\\ \mathrm{e.g}\end{array}\right]=\left[\begin{array}{c}\mathrm{and}\\ v\\ ƒ\left(\mathrm{and},v\right)\end{array}\right]$$

It is important to remember that P is not a three-dimensional point but a vector, so such a simplification is allowed. Since the mirror 61 is rotationally symmetrical, the function ƒ(u, v) is only dependent on a distance ρ of a point from the image center 11: $$\rho \left(\mathrm{and},v\right)=\sqrt{{\mathrm{and}}^2+{\mathrm{<figure-callout\; id="2"\; label="v"\; filenames="DE102021207180B3\_0008.png"\; state="\{\{state\}\}">v</figure-callout>}}^2}$$

This results in: $$P=\left[\begin{array}{c}x\\ y\\ \mathrm{e.g}\end{array}\right]=\left[\begin{array}{c}\mathrm{and}\\ v\\ ƒ\left(\rho \right)\end{array}\right]$$

The function ƒ(ρ) is required for calibration. A polynomial can be used for this, for example, whose coefficients are determined as part of an (empirical) calibration: $$ƒ\left(\rho \right)=a_0+a_1\rho +a_2{\mathrm{<figure-callout\; id="2"\; label="\rho "\; filenames="DE102021207180B3\_0008.png"\; state="\{\{state\}\}">\rho </figure-callout>}}^2+a_3{\mathrm{<figure-callout\; id="3"\; label="\rho "\; filenames="DE102021207180B3\_0008.png"\; state="\{\{state\}\}">\rho </figure-callout>}}^3+\cdots$$

A relative position of an optical marker can be determined with the aid of the calibrated model, starting from a captured image 10 of the surroundings.

It may be that the camera axis and the mirror axis are not perfectly aligned. Also, the pixels may not be square when captured. This causes a round edge of the mirror 61 to appear as an ellipse. In order to take this into account, the incorrect alignment and digitization artifacts can be taken into account by an affine transformation with the parameters c, d, e, l and c', which are also carried out as part of a calibration of the camera 51 (or the mirror camera system 60) can be determined: $$\left[\begin{array}{c}\mathrm{and}\text{'}\\ v\text{'}\end{array}\right]=\left[\begin{array}{cc}c& \mathrm{i.e}\\ e& l\end{array}\right]\cdot \left[\begin{array}{c}\mathrm{and}\\ v\end{array}\right]+\left[\begin{array}{c}xc\text{'}\\ yc\text{'}\end{array}\right]$$

This equation relates the distorted coordinates (u',v') to the ideal undistorted coordinates (u,v).

In 5 A schematic representation is shown to clarify visual beams 62 of a camera 51 . In the simplified representation, the camera 51 has a standard lens (radius=1 or vector length=1). In this case, the visual beams 62 are composed of the vectors 63 (which correspond in particular to the vector P described above) which, starting from the positions of the image elements of corner points of the optical marker 40 in the captured environment image, in particular taking into account the calibration of the camera 51 described above , to be determined. These vectors 63 are lengthened until a distance between them corresponds to a known dimension 42 of the optical marker 40. Based on this, the relative position can then be determined, i.e. a position of the optical marker 40 in the coordinate system of the camera 51.

In 6 a schematic flowchart of an embodiment of the method for locating a vehicle in an environment is shown. Distinct optical markers with known properties are arranged in the environment. In particular, the optical markers can be designed as Aruco markers.

In a measure 100, an image of the surroundings is recorded using at least one camera.

In a measure 101, at least one optical marker is detected in the recorded environment image. If there are several optical markers, all optical markers are recognized.

In a measure 102, an identifier of the at least one optical marker is read out. If several optical markers were detected, this is done for all detected optical markers.

In a measure 103, a marker position of the detected at least one optical marker is determined on the basis of the respectively read identification on a map. If several optical markers are detected, this is done for all detected optical markers.

In a measure 104, a position of the at least one camera relative to the detected at least one optical marker is determined. In other words, a position of the at least one optical marker is determined in the coordinate system of the at least one camera. If several optical markers were detected, this is done for all detected optical markers. In the case of multiple cameras, this is done for each of the multiple cameras.

In a measure 105, a camera position of the at least one camera and/or a vehicle position of the vehicle on the map is determined and provided based on the determined relative position and the determined marker position provides, the determination of the camera position and / or the vehicle position taking into account camera properties. If a number of optical markers were detected, the number of optical markers can be taken into account as a function of a quality corresponding to image areas in the captured environment image in which the markers were detected, for example by a quality-dependent weighting and/or a quality-dependent selection of the number of optical markers.

In a measure 106, the specific camera position and/or the specific vehicle position can be supplied to a navigation device of the vehicle, for example.

Provision can be made for the method to be repeated continuously, so that localization can always take place depending on the current environment.

reference list

1

contraption

2

positioning device

3

computing device

4

Storage

10

captured environment mapping

11

center of the picture

12-x

Range (camera field of view)

13

quality

15

Map

16

threshold

20

marker position

21

relative position

22

camera position

23

vehicle position

30

further relative position (further object)

40

optical marker

41

identifier

42

dimension

50

vehicle

51

camera

51-1

camera properties

52

navigation device

60

mirror camera system

61

mirror

62

visual beam

63

vector

100-106

Measures of the procedure

O

vantage point

p

Point

P

vector

u, v

Coordinates of an image element

x,y,z

Coordinates (camera coordinate system)

Claims (9)

Method for locating a vehicle (50) in an environment, in which clear optical markers (40) with known properties are or will be arranged in the environment, wherein an image (10) of the surroundings is captured by means of at least one camera (51), wherein at least one optical marker (40) is recognized in the recorded environment image (10), wherein an identifier (41) of the at least one optical marker (40) is read out, wherein a marker position (20) of the detected at least one optical marker (40) is determined on the basis of the respectively read identification code (41) in a card (15), wherein a relative position (21) of the at least one camera (51) to the detected at least one optical marker (40) is determined, and wherein a camera position (22) of the at least one camera (51) and/or a vehicle position (23) of the vehicle (50) on the map (15) is determined based on the determined relative position (21) and the determined marker position (20) and provided, the camera position (22) and/or the vehicle position (23) being determined taking into account camera properties (51-1), wherein the camera properties (51-1) include a subdivision of a field of view of the camera (51) into areas (12-x) of different quality (13), the quality (13) when determining the relative position (21) of the at least one camera ( 51) and/or when determining the camera position (22) and/or the vehicle position (23). procedure after claim 1 , characterized in that the relative position (21) of the at least one camera (51) is determined based on a picture element position of the at least one optical marker (40) in the captured image of the surroundings (40) and known geometric properties of the at least one optical marker (40). becomes. procedure after claim 1 or 2 , characterized in that the quality (13) of a respective area (12-x) of the field of view of the at least one camera (51) is or is determined empirically. Method according to one of the preceding claims, characterized in that certain relative positions (21) of the at least one camera (51) to a plurality of optical markers (40) detected in the captured environment image (10) based on a respective quality (13) corresponding thereto Determining the camera position (22) and/or the vehicle position (23) are weighted. Method according to one of the preceding claims, characterized in that certain relative positions (21) of the at least one camera (51) to a plurality of optical markers (40) detected in the captured environment image (10) based on a respective quality (13) corresponding thereto Determining the camera position (22) and/or the vehicle position (23) as a function of at least one predetermined threshold value (16) are taken into account or discarded. Method according to one of the preceding claims, characterized in that the clear optical markers (40) in the area are designed as Aruco markers. Method according to one of the preceding claims, characterized in that at least one further object with known properties is recognized in the recorded environment image (10), with a further relative position (30) of the at least one camera (51) being determined for the recognized at least one further object and wherein the determined further relative position (30) is taken into account when determining the camera position (22) of the at least one camera (51) and/or the vehicle position (23) of the vehicle (50) on the map (15). Device (1) for locating a vehicle (50) in an environment, comprising: at least one camera (51), and a position determination device (2), wherein the at least one camera (51) is set up to capture an image of the surroundings, and wherein the position determination device is set up to detect at least one unique optical marker arranged in the environment in the captured image of the surroundings, to read out an identifier of the at least one optical marker, to determine a marker position of the detected at least one optical marker on a map based on the identifier read out in each case to determine a relative position of the at least one camera to the detected at least one optical marker, and to determine and provide a camera position of the at least one camera and/or a vehicle position of the vehicle on the map based on the determined relative position and the determined marker position, wherein the camera position and/or the vehicle position is determined taking into account camera properties, wherein the camera properties (51-1) include a subdivision of a field of view of the camera (51) into areas (12-x) of different quality (13), the quality (13) when determining the relative position (21) of the at least one camera ( 51) and/or when determining the camera position (22) and/or the vehicle position (23). Vehicle comprising at least one device claim 8 .

Images