DE102020213697A1 - Method for detecting road boundaries and a system for controlling a vehicle - Google Patents

Abstract

Method for detecting road boundaries for a vehicle (1), in which sensor data comprising detection points are recorded using a first sensor and the road boundaries are estimated using the sensor data from the first sensor, the road boundaries being determined using the sensor data using the detection points of a plurality of polygons are determined, in particular continuously, so that a polygon network is created, which is used to estimate the road boundaries, objects being assigned to the detection points and a distinction being made as to whether it is a static object or a moving object, and a second sensor is provided , by means of which static objects are detected, and the static objects of the first sensor and the static objects of the second sensor are merged in order to generate a static environment, the static environment being used to estimate the road boundaries.

Description

The present invention relates to a method for recognizing road boundaries of a vehicle and a system for controlling a vehicle, in which road boundaries are recognized using the method according to the invention.

Technological background

Modern means of transportation such as B. motor vehicles or motorcycles are increasingly equipped with driver assistance systems, which use sensor systems to detect the environment, recognize the traffic situation and support the driver, z. B. by a braking or steering intervention or by the output of a visual or acoustic warning. Radar sensors, lidar sensors, camera sensors or the like are regularly used as sensor systems for detecting the surroundings. From the sensor data determined by the sensors, conclusions can then be drawn about the environment, e.g. B. an object and / or environment classification can be made. Furthermore, the detection of the surroundings is almost indispensable in the field of (partially) autonomous driving. By processing the sensor and environment data, assistance functions can then be carried out, e.g. B. Accidents with other road users can be avoided or complicated driving maneuvers can be made easier by supporting or even completely taking over the driving task or vehicle guidance (partly / fully automated).

The detection and assessment of road boundaries or roadsides (or road edges or roadsides) is an elementary function in modern driver assistance systems and z. B. used in an automatic distance control (ADR) or adaptive cruise control (ACC) or an emergency brake assistant (EBA) to estimate the movement path or the roadway of the respective means of transport or vehicle. The road boundary is usually expressed in the form of a clothoid. As a clothoid is used in modern road construction z. B. denotes the transition curve between a straight line and a curve. The respective course of the clothoids can, for. B. can be estimated based on static targets or objects that are located in the area of the road boundary and were detected by radar measurements. For example, such an estimation of the road boundary is based on a Kalman filter, which z. B. serves to reduce errors in real measured values and to provide estimates for non-measurable system variables.

Environment detection using radar sensors is often used in modern means of transportation. This is based on the emission of bundled electromagnetic waves (radar waves or radar signals) and their reflection, e.g. B. by other road users, obstacles on the road or the edge of the road. The differentiation of obstacles or objects such. B. other road users, traffic signs, road markings, peripheral buildings, crash barriers and the like is of great importance to determine when z. B. a braking or steering intervention is to be initiated.

Printed state of the art

From the U.S. 6,751,547 B2 a method for accurately estimating the geometry of a vehicle's forward path is known, which is based on a two-clothoid road model. Here road data provided by a camera or a radar system is collected. A measurement transfer function of the two-clothoid road model is then calculated using the road data. The near and far range clothoid coefficients are estimated simultaneously and the forward path of the vehicle is estimated from the data provided by the two clothoid model.

Furthermore, the DE 10 2016 107 705 A1 a method in which a path for an autonomously driving vehicle is planned using Delaunay triangulation, collecting all detections including static and moving objects. Then the triangulation method and looking for triangle edges (triangle segments) that lie on the road surface and meet a set of criteria, one of the criteria being that each edge is not a road boundary edge. After finding the possible edges or the possible paths, the vertices of these edges are then extracted, at which the path is planned. A function of the various path costs is then estimated, choosing the path with the lowest cost.

In addition, methods are known in which sensor fusion is used in order to better estimate the movement path or the road boundaries surrounding the movement path. For example, describes the CN102398598B a Lane Fusion system of front and rear cameras or the CN101793528B a method of lane estimation using sensor fusion of multiple sensors (camera, radar, lidar).

Furthermore, the DE 10 2018 215 448 B3 a generic method for a vehicle according to the preamble of claim 1.

Of particular importance in the known methods is that the road estimate must be available on time and without assumptions, where assumptions mean the maps that were created at an earlier point in time, i. H. factory before the vehicle is used. The geometry of these maps can change, e.g. B. due to construction sites or parked vehicles in urban scenarios. Recorded streets and paths must therefore be available in good time and up-to-date. Accordingly, the road boundary or lane boundary should be available in every operating cycle and with a high level of accuracy, since driving safety is generally required to meet high requirements. As a result, there is increased interest in developing methods that improve the accuracy of road boundary determination.

Object of the present invention

Proceeding from the prior art, the object of the present invention is now to provide a method in which the detection of road boundaries is improved in a simple and cost-effective manner, particularly with regard to accuracy and safety, and which eliminates the disadvantages of the prior art be overcome.

solution of the task

The above object is achieved by the entire teaching of claim 1 and the independent claim. Expedient developments of the invention are claimed in the dependent claims.

In the method according to the invention for detecting road boundaries or for estimating (the geometry of) the movement path of a means of transportation, in particular a vehicle, sensor data are collected by means of a sensor, with the road boundaries being recognized using the sensor data. This is done in that, in particular, static targets or objects along the movement path are determined using the sensor data and location points are defined for the respective position of the targets and the sensor. Furthermore, a polygon is determined based on the location points, z. B. by serving a definable number of location points as corner points of the polygon. In particular, the polygon can include location points on each side of the street. The polygon is then used to estimate the movement path or to identify the road boundaries. This determination can e.g. B. be implemented in the form of an algorithm, i. H. With the aid of the invention, the inner road boundary of the road can be determined or calculated based on the sensor measurements. Because a number of polygons are also determined, in particular continuously, so that an entire polygon network is determined, the polygon network can be used to estimate the movement path. According to the invention, objects are assigned to the detection points of the first sensor and a distinction is made as to whether the object is static or moving. Furthermore, a second sensor is provided, by means of which static objects can be detected (and of course, if necessary, also moving objects). The static objects from the first sensor and the static objects from the second sensor are then merged to create a static environment, which static environment is used to estimate the path of travel. This results in the advantage that there is a natural superimposition of sensor information or sensor data. Furthermore, the static environment is merged with the surrounding traffic (moving objects and their lanes) for the estimation of the movement path and other tactile sensors. Although the algorithm is complex, it can be implemented in a simple and cost-effective manner and retrofitted in existing systems.

The method according to the invention can expediently be a purely computer-implemented method, with the term “computer-implemented method” in the sense of the invention describing a flow plan or procedure that is implemented or carried out using a computer. The calculator, such as B. a computer, a computer network, a control unit (z. B. ECU or Electronic Control Unit or ADCU or Assisted & Automated Driving Control Unit) or a computing device therein or any other programmable device known from the prior art, can process the corresponding data using programmable calculation rules. With regard to the method according to the invention and the system according to the invention, the essential properties z. B. be caused by a new program, new programs, an algorithm or the like, which is executed on the control unit.

According to a preferred embodiment, a radar sensor is provided as the first sensor, which generates radar detections due to the emission of bundled electromagnetic waves (radar waves or radar signals) and their reflection, z. B. other road users, obstacles on the road or the surrounding buildings the road surface. However, other sensors known from the prior art can also be provided as the first sensor, e.g. B. camera, ultrasonic or lidar sensors, whose detection points are fed to the algorithm. Furthermore, a camera sensor is preferably provided as the second sensor, by means of which road edges or road or lane markings can be detected as static objects. However, other known sensors can also be used here.

It has proven to be particularly advantageous if the polygon is a triangle, since this can be calculated extremely easily with little computing effort. In the same advantageous manner, an entire polygon network can be created using triangular formations. This can be done in a simple way using a Delaunay triangulation or Delaunay triangle formation, as already described in DE 10 2018 215 448 B3 described. In particular, the term Delaunay triangulation also includes other known derivatives of a Delaunay triangulation, such as. B. Constrained Delaunay triangulation.

Crash barriers, infrastructure, parked road users, road markings, roadside buildings and the like can expediently be provided as static objects.

The polygon network of the first sensor is preferably merged with the static objects of the second sensor, or the polygon network serves as input for the first sensor for the fusion.

The middle of the road can expediently be determined between the road boundaries recognized on the left and right by the ego vehicle. Based on the distance between the road boundaries and the middle of the road, it can then be determined whether the road boundaries widen or narrow and/or whether the detections or objects are incorrect detections (so-called “outliers”) or not.

Furthermore, the present invention includes a system for controlling a vehicle. The system for controlling the ego vehicle includes a control device for controlling the vehicle and for processing data from the sensors, a first sensor for detecting and distinguishing between moving and static objects and a second sensor for detecting static objects. The ego vehicle is controlled by the control device in that the control device can determine the road boundaries by means of a method according to one of the preceding claims. For this purpose, the control device can expediently be set up in such a way that it can access actuators of the ego vehicle in order to carry out corresponding driving and assistance functions in a supportive and/or (partly) autonomous manner.

A camera radar sensor (or alternatively also a lidar sensor) is preferably provided as the first sensor and a camera (or alternatively also a lidar sensor) is provided as the second sensor, the radar sensor being configured to detect and distinguish between moving and static objects, and the camera being configured is to recognize static objects, in particular roadsides and/or road markings (lanemarking or lane detection). As a result, the measurements of the first sensor can be used at the point at which the camera or the second sensor can no longer detect (e.g. due to insufficient range), e.g. B. lidar or radar. This creates additional redundancy and also verifies the measurement results.

According to a preferred embodiment of the invention, the control device first merges the static and moving objects detected by the first sensor (first sensor fusion of the sensor data from the first sensor). Thereafter, the static objects detected by the second sensor can then be merged with the static objects of the first sensor in order to generate a static environment (second sensor fusion), with the static environment and the moving objects then being used to determine the road boundaries.

character list

• 1 eine vereinfachte schematische Darstellung eines Fahrzeuges, bei dem eine Schätzung der Geometrie eines Bewegungsweges anhand des erfindungsgemäßen Verfahrens erfolgt;
• 2 eine vereinfachte schematische Darstellung von Radardetektionen in einem Koordinatensystem, wobei der Bewegungsweg anhand einer Triangulierung durchgeführt wurde;
• 3 eine weitere vereinfachte schematische Darstellung des Koordinatensystems aus 2, mit zusätzlich erfassten Straßenmarkierungen, sowie
• 4 eine vereinfachte Darstellung der Abhängigkeit zwischen Länge der Straßengrenze und der Position des Fahrzeuges.

In the following, the invention is explained in more detail by means of expedient exemplary embodiments. Show it:

• 1 a simplified schematic representation of a vehicle in which the geometry of a movement path is estimated using the method according to the invention;
• 2 a simplified schematic representation of radar detections in a coordinate system, wherein the movement path was carried out using a triangulation;
• 3 another simplified schematic representation of the coordinate system 2 , with additionally recorded road markings, as well as
• 4 a simplified representation of the relationship between the length of the road boundary and the position of the vehicle.

Reference number 1 in 1 denotes an ego vehicle with a control device 2 (ECU, Electronic Control Unit or ADCU, Assisted and Automa ted Driving Control Unit), through which sensor control, sensor data fusion, environment and/or object recognition, trajectory planning and/or vehicle control can take place in particular (partially) autonomously. To control the vehicle, the control device 2 can access various actuators (steering 3, motor 4, brake 5). Furthermore, the ego vehicle 1 has various sensors for detecting the surroundings (lidar sensor 6, camera 7, radar sensor 8 and ultrasonic sensors 9a-9d). Advantageously, the sensor data can be used for environment and object detection, so that various assistance functions such. B. emergency brake assistant (EBA, Emergency Brake Assist), distance following control (ACC, Automatic Cruise Control), lane departure control or a lane departure warning system (LKA, Lane Keep Assist) or the like can be realized. Furthermore, the assistance functions can also be executed via the control device 2 or another control unit provided for this purpose. The trajectory is planned based on an estimate of the geometry of the movement path according to the method according to the invention.

In the method according to the invention, the geometry of the movement path is estimated by determining or calculating the inner road boundary on the basis of sensor measurements. Exemplary embodiments are described below in which a radar sensor is used as the sensor. However, other sensors known from the prior art, such as e.g. B. lidar or camera sensors. In each scan cycle, the radar sensor provides a list of objects or targets with the corresponding relative position of the respective targets and their speed in relation to the ego vehicle, which includes the radar sensor and corresponds to the location 0 (x=0; y=0). . Based on the relative speed to the ego vehicle, the objects are then classified as static or mobile targets. In the present case, however, only the static targets to which a location point can be assigned are used for estimating the road boundary. The estimation then takes place by calculating or inserting a polygon network or a Delaunay triangulation (Delaunay triangle formation or Delaunay triangulation) on the basis of the position of the static targets or their location points and that of the ego vehicle. The creation of the polygon mesh or the Delaunay triangulation is carried out in particular by a method or a method configuration according to FIG DE 10 2018 215 448 B3 , to which direct reference is made.

2 shows a traffic scenario in which the ego vehicle 1 is equipped with a radar sensor 8 and is on a roadway or a road with surrounding traffic, shown in an overview in which the Delaunay triangles (or their triangle edges), the system estimated path, driven paths of other road users are shown. The radar sensor 8 is designed in such a way that the system can distinguish between static and moving objects. This embodiment is well known and is not intended to form part of this invention. Information about the road geometry in front of the ego vehicle 1 can be derived from the static objects. However, such information can also be ambiguous: for example, there are scenarios where only corridors based on static information are good candidates for roadside estimation. In order to eliminate such ambiguities, the lane information from moving objects can also be used to estimate the roadside. The traces of the detected moving objects indicate the position history or movement history of the objects or road users in the traffic surrounding the ego vehicle 1 . In 2 the traces are shown as an example using a dot-dash line. In traffic scenarios, e.g. B. at point A, where two opposite values are found, the dotted line indicates a road corridor, and the lanes (from the same direction of travel) indicate a road corridor to the right. Conveniently, the road corridor can then be selected, which is displayed from the lanes in the same direction of travel (ie the right road corridor in this case). This is because the path indicated by the lanes is already a confirmed road that other road users in the area have already traveled through, as evidenced by their traveled paths. The path given by the dotted line is hypothetical and based only on assumptions. In order to verify these assumptions, a two-stage fusion process is carried out.

The first step in the fusion process is to fuse the radar information to itself, i. H. the surrounding traffic information (other road users or surrounding moving objects) is merged or brought together or fused with the static information (static objects such as roadside buildings, crash barriers and the like). The result of this merging is a set of road boundaries where both the search lines and the traces from the surrounding traffic intersect the same road boundaries. In the event that conflicting situations arise, only edges cut off from surrounding traffic in the same traffic direction are considered road boundaries.

After the road boundaries have been identified, this data can be analyzed again by taking a closer look at the result of the fusion of the radar data between the static and dynamic environment (represented by the lines to the left and right of the ego vehicle 1 in 3 ). Outliers can be detected along the road borders between these two lines, which e.g. B. can lead to the fact that exits or narrowing of the streets are recognized.

For such outlier detection, the longitudinal position of the center of the detected road boundaries (road center) with respect to their length can be specified as in 4 shown. Analyzing the peaks of such a plot, it can be seen that the width of the street or street boundary at point A is much greater than the width of the rest. Due to this deviation, this point A can be classified as an outlier, which can thus be filtered out. Another piece of information that could be used in the example is a lane narrowing approximately before and after point A. If the lengths of road boundaries before and after point A are analyzed, a decrease in the average width of the road surface between road boundaries can be observed. This decrease can then be interpreted as a road narrowing. In the same way, the opposite would suggest a road expansion. Correspondingly, this method can also be used to recognize lane entrances (e.g. road entrances and road exits on freeways).

Furthermore, the ego vehicle 1 has additional sensors that can be used to detect lane markings (e.g. lidar sensor 6 or camera 7). The lane markings or road markings detected by these sensors can now be overlaid or added to the previously merged information, as in 3 shown. In this case, a second fusion step takes place, in which the road boundaries identified by means of radar data are fused with the identified lane markings. The own lane of the ego vehicle 1 is to be selected here as a pair of lines from the left and right of the ego vehicle 1 that are closest to the ego vehicle 1 . As in 3 shown, the ego lane can be estimated using the lane detection information far ahead up to point A, while the continuation of the lane from point A can only be determined using the radar data.

In addition to estimating the free space, detecting road crossings and forks, detecting road widening, road narrowing or road access, the method according to the invention can also be used in the field of robotics, e.g. B. in the trajectory planning of a robot.

In summary, the method according to the invention can be carried out using the following steps: First, the fusion of the static environment or static objects and moving objects (eg also the paths of the surrounding road users) takes place using a triangulation method. After that, the static environment is represented by fusing sensor information from other sensors (lidar sensor 6 or camera 7) and radar information, using triangulation as an input, i. H. the fusion of the static environment with objects or road markings from other sensors. Finally, discontinuous structures can be filtered out by analyzing the road boundaries, e.g. B. parking bays, exits, construction sites and the like. Furthermore, the fusion algorithms of two sensors known from the prior art usually take place in a model-based environment, i. H. different software modules (e.g. from camera and/or radar) each estimate clothoid functions and pass them on to a central processing unit, which then uses the models, e.g. B. compares as polynomials and then merges them. Although the advantage here is that only the coefficients of the polynomials are compared, thereby saving computing capacity, each sensor unit carries out its own averaging of the lane, which can lead to errors. In order to avoid such errors, a type of deep fusion is carried out according to the invention, i. H. the data is processed before own models/lane averaging are generated.

Reference List

1

ego vehicle

2

control device

3

steering

4

engine

5

brake

6

lidar sensor

7

camera

8th

radar sensor

9a-9d

ultrasonic sensor

QUOTES INCLUDED IN DESCRIPTION

This list of the documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• US 6751547 B2 [0005]
• DE 102016107705 A1 [0006]
• CN 102398598B [0007]
• CN 101793528B [0007]
• DE 102018215448 B3 [0008, 0015, 0024]

Claims (10)

Method for detecting road boundaries for a vehicle (1), in which sensor data comprising detection points are recorded using a first sensor and the road boundaries are estimated using the sensor data from the first sensor, the road boundaries being determined using the sensor data using the detection points of a plurality of polygons are determined, in particular continuously, so that a polygon network is created, which is used to estimate the road boundaries, characterized in that the detection points are assigned objects and a distinction is made as to whether it is a static object or a moving object, and a second Sensor is provided by means of which static objects are detected, and the static objects of the first sensor and the static objects of the second sensor are merged to generate a static environment, the static environment is used to estimate the road boundaries. procedure after claim 1 , characterized in that at least one sensor from the following sensors is provided as the first and/or second sensor: - lidar sensor (6), - camera (7), - radar sensor (8), - ultrasonic sensor (9a-9d). procedure after claim 1 or 2 , characterized in that the polygon is a triangle or the polygons are triangles. procedure after claim 3 , characterized in that the polygon mesh is determined using a Delaunay triangulation. Method according to at least one of the preceding claims, characterized in that crash barriers, infrastructure, parked road users, road markings, roadside buildings and the like are provided as static objects. Method according to at least one of the preceding claims, characterized in that the polygon network of the first sensor is merged with the static objects of the second sensor. Method according to at least one of the preceding claims, characterized in that the middle of the road between the road boundaries recognized on the left and right by the ego vehicle (1) is determined and, based on the distance between the road boundaries and the middle of the road, it is determined whether the road boundaries widen or narrow and/or or whether the detections or objects are false detections. System for controlling an ego vehicle (1), with a control device (2), a first sensor for detecting and distinguishing between moving and static objects, a second sensor for detecting static objects, wherein an ego vehicle (1) is controlled by the control device (2) in that the control device (2) determines the road boundaries using a method according to one of the preceding claims. system after claim 8 , characterized in that a radar sensor (8) is provided as the first sensor and a camera (7) is provided as the second sensor, the radar sensor (8) being configured to detect and distinguish moving and static objects, and the camera (7) is configured to recognize static objects, in particular roadsides and/or road markings. system after claim 8 or 9 , characterized in that the control device (2) first merges the static and moving objects detected by the first sensor, then merges the static objects detected by the second sensor with the static objects of the first sensor to generate a static environment, and then the static Environment and the moving objects are used to determine the road boundaries.

Images