DE102016223072B4 - Determination of a vehicle's own movement on the basis of stationary objects - Google Patents

Abstract

A method for determining the proper movement of a vehicle (10), comprising capturing image data using sensor means (1, 2) of the vehicle (10), identifying stationary objects using a classifier based on the image data, which is based on prior knowledge of the appearance of stationary objects works, and determining the proper movement as a function of the stationary objects, wherein a relative movement of at least one of the stationary objects is determined using the image data, and wherein the relative movement is used to determine the proper movement.

Description

The present invention relates to an estimation of the vehicle's own motion, which is based on the fact that objects detected by the vehicle are classified as stationary.

the DE 10 2014 223 363 A1 discloses a localization of a motor vehicle in a fixed reference map. Object positions in the surroundings of the motor vehicle are determined from a radar impulse response, the currently determined object positions forming a map of the surroundings. The position of the motor vehicle in the reference map is determined by comparing the environment map with the reference map. A classification for the object positions is carried out, and a classification of the objects within areas of the reference map is carried out with the aid of machine learning methods.

the DE 10 2015 005 987 A1 describes a storage of reference data describing static features for a route covered by the motor vehicle in order to be able to carry out the evaluation of currently recorded radar data with reduced effort when driving this route again, taking into account the reference data.

the DE 10 2011 118 147 A1 discloses determining a speed of a vehicle. For this purpose, it is determined with respect to an object detected by a sensor of the vehicle whether the detected object is stationary. This determination takes place either on the basis of an image evaluation as a function of a change in the size of an object in an image or on the basis of vehicle-to-infrastructure communication.

the DE 10 2012 203 037 A1 also describes the determination of a speed of a vehicle. For this purpose, data on the surroundings of the vehicle relating to a non-moving object are determined, which sends out information about its non-movement. A relative movement of the vehicle and, from this, the speed of the vehicle are determined from the environment data of the vehicle in relation to the non-moving object.

the DE 10 2011 119 762 A1 discloses the positioning of a vehicle. For this purpose, the vehicle includes a digital map in which data on landmarks are assigned to a geographical position. For this purpose, movement data are recorded based on translational movements and rotational movements of the vehicle. The vehicle position is determined by means of the movement data based on the landmarks.

According to the prior art, the stationary objects are determined using a digital map or using a communication link. Therefore, the determination of stationary objects according to the prior art is not possible if either the communication link cannot be established or if no digital map is available for the current position of the vehicle.

The present invention therefore has the task of determining the proper movement of a vehicle on the basis of stationary objects independently of a communication link or of a digital map.

According to the invention, this object is achieved by a method for determining a vehicle's own movement according to claim 1, by a computer program according to claim 7 and by a control device according to claim 8. The dependent claims are advantageous and preferred embodiments of the present invention.

• • Erfassen von Bilddaten mit Hilfe von Sensormitteln des Fahrzeugs. Die Sensormittel können dabei beispielsweise einen LIDAR-Scanner, einen ESP-Sensor, ein Radar und/oder eine Kamera umfassen. Mit anderen Worten können die Bilddaten mit einem oder mit mehreren Sensoren detektiert werden. Beim Einsatz von mehreren Sensoren können die Ergebnisse dieser Sensoren zu den Bilddaten fusioniert werden. Dadurch ist es erfindungsgemäß möglich, die Odometrie redundant zu erfassen und gegebenenfalls die Genauigkeit zu erhöhen.
• • Identifizieren von ortsfesten Objekten mit Hilfe eines Klassifikators abhängig von den erfassten Bilddaten. Dabei arbeitet der Klassifikator mit einem Vorwissen über das Aussehen und/oder die Form von ortsfesten Objekten. Mit anderen Worten kann der Klassifikators durch sein Vorwissen in den Bilddaten ortsfeste Objekte erkennen und bestimmen. Bei solchen ortsfesten Objekten oder Landmarken kann es sich z.B. um pfostenförmige Objekte (Pfosten), Fahrbahnmarkierungen, Bordsteine, Gulli-Deckel, Ampeln oder Verkehrsschilder handeln.
• • Bestimmen der Eigenbewegung in Abhängigkeit von den ortsfesten Objekten. Diese ortsfesten Objekte oder Landmarken werden innerhalb der ständig von den Sensormitteln des Fahrzeugs erfassten Bilddaten von einem Zeitpunkt zum nächsten verfolgt und abhängig von der Bewegung eines ortsfesten Objekts in den Bilddaten über der Zeit wird die Eigenbewegung des Fahrzeugs geschätzt oder bestimmt. Dabei wird vorausgesetzt, dass ein ortsfestes Objekt statisch ist und seine Position nicht verändert.

In the context of the present invention, a method for determining a proper movement of a vehicle is provided, which comprises the following steps:

• • Acquisition of image data with the aid of sensor means in the vehicle. The sensor means can include, for example, a LIDAR scanner, an ESP sensor, a radar and / or a camera. In other words, the image data can be detected with one or more sensors. When using several sensors, the results of these sensors can be merged into the image data. This makes it possible according to the invention to record the odometry redundantly and, if necessary, to increase the accuracy.
• • Identification of stationary objects with the aid of a classifier depending on the captured image data. The classifier works with prior knowledge of the appearance and / or the shape of stationary objects. In other words, the classifier can recognize and determine stationary objects through its prior knowledge in the image data. Such stationary objects or landmarks can be, for example, post-shaped objects (posts), lane markings, curbs, manhole covers, traffic lights or traffic signs.
• • Determination of the proper movement as a function of the stationary objects. These stationary objects or landmarks are tracked from one point in time to the next within the image data that are constantly recorded by the sensor means of the vehicle and are dependent on the movement of a stationary object in the Image data over time is used to estimate or determine the vehicle's own movement. It is assumed that a stationary object is static and does not change its position.

In order to identify stationary objects, the classifier neither requires communication from the vehicle to an object located outside the vehicle nor any card. The method according to the invention therefore fulfills the object set above.

According to the state of the art, the proper movement of a vehicle is often determined on the basis of scan or image points that belong to dynamic (i.e. non-stationary) objects. This disadvantageously falsifies the determination of the proper movement by the movement of the dynamic objects. In that, according to the invention, the intrinsic movement or odometry is determined as a function of stationary objects, this problem which frequently occurs according to the prior art is advantageously avoided.

According to a preferred embodiment according to the invention, the classifier is trained before it is used to identify stationary objects on the basis of objects that are stationary and on the basis of objects that are not stationary. According to this embodiment, the classifier learns by evaluating image data of objects for which the classifier knows whether they are stationary or not. According to the invention, an SVM classifier (“Support Vector Machine” classifier) is frequently used as the classifier. The classifier (regardless of the classifier type used) is trained accordingly in order to be able to classify an object as stationary or non-stationary.

This embodiment has the advantage that it can be used for any type of stationary object, since the method according to the invention (more precisely the classifier according to the invention) can be learned accordingly for every type of stationary objects.

According to a further preferred embodiment according to the invention, the classifier determines an object in the image data as being stationary when the classifier detects a predetermined pattern in the image data. In this embodiment, the classifier is programmed, for example, with the aid of which pattern in the image data a stationary object is recognized.

This embodiment has the advantage over the previous embodiment that no training of the classifier is required.

A relative movement of at least one stationary object is determined on the basis of the image data. Based on this relative movement of the stationary object to the vehicle (own vehicle), the own movement of the vehicle is then determined.

The relative movement of the stationary object can be determined with an iterative closest point method, with which a movement of the stationary object is determined in the image data over time. With the iterative closest point method, the distance between a point cloud representing a stationary object in the image data at a first point in time and a point cloud representing the same stationary object in the image data at a second point in time can be determined. For this purpose, the point cloud of one point in time is shifted in such a way that the distances between corresponding points in the two point clouds are minimal. The relative movement of the stationary object from the first point in time to the second point in time can then be determined on the basis of the displacement.

• • Bestimmen von Odometrie-Positionsdaten des Fahrzeugs in Abhängigkeit von den ortsfesten Objekten. Die Odometrie-Positionsdaten indizieren eine Relativbewegung des Fahrzeugs, zum Beispiel in einem willkürlich festgelegten Referenzkoordinatensystem oder im sogenannten Fahrzeugkoordinatensystem („body frame“). Dabei wird die Relativbewegung des Fahrzeugs insbesondere jeweils relativ in Bezug auf eine vorangegangene Position des Fahrzeugs abhängig von zumindest einem ortsfesten Objekt indiziert bzw. bestimmt.
• • Erfassen von Absolut-Positionsdaten des Fahrzeugs. Das Erfassen der Absolut-Positionsdaten kann vorteilhafterweise mit einfachen Absolut-Positioniersystemen (z.B. einem auf GPS basierenden Navigationssystem) durchgeführt werden. Die Absolut-Positionsdaten geben die gemessene Position des Fahrzeugs zu einem bestimmten Zeitpunkt in absoluten Werten, beispielsweise in einem UTM- oder WGS 84-Referenzkoordinatensystem, an. Optional können die Absolut-Positionsdaten auch eine Orientierung bzw. Richtung aufweisen, welche eine aktuelle Bewegungsrichtung des Fahrzeugs anzeigt. Eine Kombination aus Position und Orientierung wird häufig als Pose bezeichnet. Die zu einem bestimmten Zeitpunkt ermittelten Absolut-Positionsdaten können demnach ausreichen, um eine Pose zu diesem bestimmten Zeitpunkt zu bestimmen. Diese Pose gibt dann zu dem bestimmten Zeitpunkt die Position und die Fahrrichtung des Fahrzeugs an.
• • Erzeugen eines Posen-Graphen, bei welchem die Kanten und die Knoten des Posen-Graphen abhängig von den Odometrie-Positionsdaten und den Absolut-Positionsdaten erzeugt werden. Dabei wird unter einem Posen-Graphen ein Graph verstanden, dessen Knoten Posen und dessen Kanten Bedingungen entsprechen, welche zwischen denjenigen Knoten, die von der jeweiligen Kante verbunden werden, existiert. Ein Knoten des Posen-Graphen kann entweder ein Fixknoten oder ein Optimierungsknoten sein. Ein Fixknoten repräsentiert dabei eine absolute Positionsschätzung oder Positionsmessung. D.h. die Pose des Fixknotens entspricht der Position und der Richtung, die bei der entsprechenden Positionsschätzung bzw. Positionsmessung geschätzt bzw. gemessen wurden. Ein Optimierungsknoten entspricht einer Pose zu einem beliebigen Zeitpunkt, zu dem es keinen Fixknoten geben muss. Häufig ist jedoch ein Optimierungsknoten direkt mit einem oder mit mehreren Fixknoten verbunden.

According to a further embodiment according to the invention, the method according to the invention comprises the following further steps:

• • Determination of odometry position data of the vehicle as a function of the stationary objects. The odometry position data indicate a relative movement of the vehicle, for example in an arbitrarily defined reference coordinate system or in the so-called vehicle coordinate system (“body frame”). The relative movement of the vehicle is in particular indicated or determined in each case relative to a previous position of the vehicle as a function of at least one stationary object.
• • Acquisition of absolute position data of the vehicle. The acquisition of the absolute position data can advantageously be carried out with simple absolute positioning systems (for example a GPS-based navigation system). The absolute position data indicate the measured position of the vehicle at a specific point in time in absolute values, for example in a UTM or WGS 84 reference coordinate system. Optionally, the absolute position data can also have an orientation or direction which indicates a current direction of movement of the vehicle. A combination of position and orientation is often referred to as a pose. The absolute position data determined at a specific point in time can accordingly be sufficient to determine a pose at this specific point in time. This pose then indicates the position and direction of travel of the vehicle at the specific point in time.
• • Generation of a pose graph in which the edges and the nodes of the pose graph are generated depending on the odometry position data and the absolute position data. A pose graph is understood here to be a graph whose nodes pose and whose edges correspond to conditions that exist between those nodes that are connected by the respective edge. A node of the Posen graph can either be a fixed node or an optimization node. A fixed node represents an absolute position estimate or position measurement. That is to say, the pose of the fixed node corresponds to the position and the direction that were estimated or measured in the corresponding position estimation or position measurement. An optimization knot corresponds to a pose at any point in time at which there does not have to be a fixed knot. However, often an optimization node is directly connected to one or more fixed nodes.

The pose graph merges the recorded odometry position data and absolute position data. Several odometry estimators or odometry positioning systems and also several absolute positioning systems can be used to create the pose graph. If several odometry estimators are used, there can also be many edges between two optimization nodes corresponding to the number of odometry estimators.

• • Optimieren des Posen-Graphen abhängig von den Odometrie-Positionsdaten und den Absolut-Positionsdaten. Bei der Optimierung des Posen-Graphen werden die Optimierungsknoten (genauer die Posen der Optimierungsknoten) bestimmt. Dagegen werden die Fixknoten (genauer die Posen der Fixknoten) durch die Optimierung nicht beeinflusst. Zur Optimierung werden insbesondere Fehlerterme definiert, welche durch die Optimierung möglichst klein gehalten werden. Dabei beschreibt eine Art von Fehlerterm beispielsweise den Unterschied der Pose des Optimierungsknoten von einer Pose eines mit diesem Optimierungsknoten verbundenen Fixknotens. Eine andere Art eines Fehlerterms beschreibt den Unterschied zwischen den Posen der Optimierungsknoten und Odometriedaten (z.B. den Odometrie-Positionsdaten), welche anhand eines Odometrieschätzers zum Zeitpunkt eines Optimierungsknotens bestimmt werden. Durch die Optimierung des Posen-Graphen wird insbesondere versucht, einen möglichst guten Kompromiss zwischen den Absolut-Positionsdaten und den Odometrie-Positionsdaten zu finden.
• • Bestimmen der Eigenbewegung in Abhängigkeit von dem Posen-Graphen.

The fixed nodes are determined on the basis of the absolute position data. Using the odometry position data, the edges are determined which connect two optimization nodes or one optimization node with a fixed node.

• • Optimizing the pose graph depending on the odometry position data and the absolute position data. When optimizing the pose graph, the optimization nodes (more precisely the poses of the optimization nodes) are determined. In contrast, the fixed nodes (more precisely the poses of the fixed nodes) are not influenced by the optimization. In particular, error terms are defined for optimization, which are kept as small as possible by the optimization. A type of error term describes, for example, the difference between the pose of the optimization node and a pose of a fixed node connected to this optimization node. Another type of error term describes the difference between the poses of the optimization nodes and odometry data (for example the odometry position data), which are determined using an odometry estimator at the time of an optimization node. By optimizing the pose graph, an attempt is made in particular to find the best possible compromise between the absolute position data and the odometry position data.
• • Determining the proper motion as a function of the pose graph.

Since the pose graph includes absolute position data in addition to the odometry position data, which (in particular in the case of the fixed nodes) are based on an absolute position measurement, the intrinsic movement can advantageously be estimated even more precisely.

As already described above, error terms can be defined in connection with the pose graph. These are usually larger (smaller), the further (less far) the positions estimated by the optimization (the optimization nodes) are removed from the outputs of the positioning systems used and the smaller (larger) the variance of the corresponding outputs of the positioning systems. As a result of the optimization, the pose graph can be changed in such a way that the error terms are as small as possible. In other words, the aim of the optimization is to globally minimize a corresponding error function under boundary conditions. The optimization can be done numerically. In addition, the optimization can take place iteratively and thus comprise several optimization iterations. The optimization can include, for example, a technique of nonlinear least squares. In addition, the optimization can take place by means of a Gauss-Newton technique and / or by means of a Levenberg-Marquardt technique.

Determining the proper movement according to the invention on the basis of the pose graph offers two advantages. On the one hand, the acquisition or determination of the odometry position data, which is necessary for determining the pose graph, is improved through the knowledge of stationary objects. On the other hand, the determination of the proper movement is further improved, since when determining the proper movement using the pose graph, the absolute position data (of the pose graph) are also taken into account.

According to the invention, the specific intrinsic movement of the vehicle can be used to carry out an automatic driving function or driver assistance function of the vehicle. Such a driver assistance function could include autonomous, partially automatic or highly automatic driving of the vehicle. The driver assistance function can include, for example, a lane departure warning assistant, a distance assistant and a parking assistant.

Many driver assistance systems and automatic driving functions of a vehicle require an estimation of its own movement in order to be able to estimate the trajectory of the vehicle over short periods of time. The present invention enables precisely this estimation of the trajectory driven with an accuracy that is better than that of the prior art.

Furthermore, the present invention describes a computer program or software which can be loaded into a memory of a programmable controller or a computing unit of a vehicle. With this computer program, all or various previously described embodiments of the method according to the invention can be executed when the computer program is running in the control or control device of the vehicle. The computer program may need program resources, e.g. libraries and auxiliary functions, in order to implement the corresponding embodiments of the method. In other words, the claim directed to the computer program is intended to protect software in particular with which one of the above-described embodiments of the method according to the invention can be carried out or which carries out this embodiment. The software can be a source code (e.g. C ++) that is still compiled (translated) and linked or that only needs to be interpreted, or it can be an executable software code that only needs to be entered into the corresponding processing unit or control device for execution loading is.

In the context of the present invention, a control device is also provided which comprises an interface and a processor. The interface is designed to capture image data from sensor means in the vehicle. The processor is designed to identify stationary objects with the aid of a classifier on the basis of the image data and to determine their own movement as a function of these stationary objects.

The advantages of the control device according to the invention essentially correspond to the advantages of the method according to the invention, which are explained in detail beforehand, so that a repetition is dispensed with here.

According to an embodiment according to the invention, the control device is designed to carry out any embodiment of the method according to the invention.

In the context of the present invention, a vehicle is also provided which comprises the control device according to the invention.

In the following, the present invention is described in detail on the basis of preferred embodiments according to the invention with reference to the figures.

The present invention offers the advantage that the proper movement or odometry is determined with fewer errors, since it does not have to be estimated whether or not certain image features are objects suitable for determining odometry (i.e. stationary objects). If the results of several sensors or detectors are merged to acquire the image data, the procedure according to the invention is advantageously robust against a sensor failure.

1 shows schematically a vehicle according to the invention with a control unit according to the invention.

In 2 the flow chart of a method according to the invention is shown.

In 1 is a vehicle according to the invention 10 shown, which is next to a LIDAR scanner 1 a camera 2 and a control device according to the invention 20th includes. The control unit 20th itself includes an interface 3 to get image data from the lidar scanner 1 and the camera 2 to capture and a processor 4th . With the processor 4th Fixed objects are identified in the image data with the help of a classifier and the vehicle's own movement 10 determined depending on the stationary objects.

In 2 the flow chart of a procedure according to the invention is shown.

In the first step S1 image data are captured with sensor means in the vehicle. These image data are then used in step S2 Objects determined. In the next step S3 the stationary objects are determined as such among these objects with the aid of a classifier. With the help of these stationary objects in step S4 Odometry position data determined. In step S5 a pose graph is generated with this odometry position data and with additionally recorded absolute position data. Finally, in step S6 determines the proper movement of the vehicle as a function of this pose graph.

List of reference symbols

1

LIDAR scanner

2

camera

3rd

interface

4th

processor

10

vehicle

20th

Control unit

S1-S6

Process step

Claims (9)

A method for determining the proper movement of a vehicle (10), comprising capturing image data using sensor means (1, 2) of the vehicle (10), identifying stationary objects using a classifier based on the image data, which is based on prior knowledge of the appearance of stationary objects works, and determining the proper movement as a function of the stationary objects, wherein a relative movement of at least one of the stationary objects is determined based on the image data, and wherein the relative movement is determined based on the relative movement. Procedure according to Claim 1 , characterized in that the classifier is trained on the basis of objects which are known to be stationary and on the basis of objects which are known to be non-stationary, before stationary objects are identified by means of the classifier. Procedure according to Claim 1 or 2 , characterized in that the classifier identifies an object as stationary when the classifier recognizes a predetermined pattern in the image data. Method according to one of the preceding claims, characterized in that the relative movement of at least one of the stationary objects is determined using the image data using an iterative closest-point method with which a movement of the stationary object is determined in the image data over time. Method according to one of the preceding claims, characterized by determining odometry position data of the vehicle (10) as a function of the stationary objects, acquiring absolute position data of the vehicle (10), generating a pose graph in which edges and nodes of the pose Graphs are generated depending on the odometry position data and the absolute position data, optimizing the pose graph depending on the odometry position data and the absolute position data, and determining the proper movement depending on the pose graph. Method according to one of the preceding claims, characterized in that the intrinsic movement is used to carry out an automatic driving function of the vehicle (10). Computer program with a program code in order to carry out all the steps of the method according to one of the preceding claims, when the program code is carried out by a processor in a vehicle (10). Control unit comprehensive an interface (3) which is designed to capture image data from sensor means (1, 2) of a vehicle (10), a processor (4) which is designed to identify stationary objects by means of a classifier on the basis of the image data and to determine the proper movement as a function of the stationary objects, wherein the control device (20) is designed to determine a relative movement of at least one of the stationary objects on the basis of the image data, and to determine the proper movement based on the relative movement. Control unit after Claim 8 , characterized in that the control device (20) for performing the method according to one of the Claims 1 - 6th is designed.

Images