DE102019008093A1 - Method for merging sensor data from a large number of detection devices using a sparse occupancy grid, as well as a driver assistance system - Google Patents

Abstract

The invention relates to a method for merging (16) a large number of sensor data (18) from a large number of detection devices (20) by means of an electronic computing device (22) of a driver assistance system (12) of a motor vehicle (10), in which by means of the large number of detection devices (20) an environment (24) of the motor vehicle (10) is recorded and the sensor data (18) recorded by the respective recording device (20) are transmitted to the electronic computing device (22) and the transmitted sensor data (18) by means of the electronic computing device (22) are fused on the basis of at least one occupancy grid of a respective detection device (20), the plurality of sensor data (18) for merging (16) being classified in a respective sparse occupancy grid (14) and one within the respective sparse occupancy grid (14) Object image (26) of at least one in the environment (24) of the motor vehicle gs (10) detected object is carried out, associated sensor data (18) of the at least one object of the plurality of detection devices (20) being connected to one another. The invention also relates to a driver assistance system (12).

Description

The invention relates to a method for merging a large number of sensor data from a large number of detection devices by means of an electronic computing device of a driver assistance system of a motor vehicle according to the preamble of claim 1. The invention also relates to a driver assistance system.

A fundamental task of sensor fusion in connection with the detection of the surroundings of motor vehicles is to bring together corresponding measured variables or sensor data from the various sensors used, which can in particular also be referred to as detection devices, and to indicate in a joint, coordinated representation of the surroundings where the detection devices are in the detection area corresponding objects are located. When using detection devices, in particular for driver assistance systems in motor vehicle construction, usually several detection devices based on different physical principles, for example a camera, radar or lidar, are used in order to achieve sufficient accuracy with regard to the position and size of detected objects in the vicinity of the motor vehicle . In particular, supplementary or complementary sensor combinations are preferred here.

Driver assistance systems place very high demands on the detection rates of objects, since accident-free driving can only be achieved with very high detection rates. A low detection rate means that objects in the vicinity of the motor vehicle are not recognized and the necessary reactions to these objects, for example warning or braking or evasive action, cannot be carried out. In order to avoid these situations, on the one hand, a very high detection rate is required of the detection device and, on the other hand, the measurements of several detection devices of a sensor fusion are combined. Two approaches in particular are known for this from the prior art, one approach being grid-based and the other approach being object-based.

From the DE 10 2015 201 747 A1 A sensor system for a vehicle is known, with a multiplicity of sensor devices which are designed to capture raw data on the surroundings of the vehicle and to calculate and output at least one occupancy grid from the captured raw data for each capture time, and with a control device designed to do so is to calculate a merged occupancy grid based on the individual occupancy grids of the sensor device and to accumulate the merged occupancy grids calculated for different detection times over time and output an accumulated occupancy grid.

The object of the present invention is to create a method and a driver assistance system by means of which a merger of sensor data from a large number of detection devices can be implemented with reduced effort.

This object is achieved by a method and by a driver assistance system according to the independent claims. Advantageous embodiments are specified in the subclaims.

One aspect of the invention relates to a method for merging a large number of sensor data from a large number of acquisition devices by means of an electronic computing device of a driver assistance system of a motor vehicle, in which an environment of the motor vehicle is acquired by means of the plurality of acquisition devices and the sensor data acquired with the respective acquisition device are displayed the electronic computing device are transmitted and the transmitted sensor data are merged by means of the electronic computing device on the basis of at least one occupancy grid of a respective detection device.

It is provided that the plurality of sensor data for merging is classified in a respective sparse occupancy grid and an object mapping of at least one object detected in the surroundings of the motor vehicle is carried out within the respective sparse occupancy grid, with related sensor data of the at least one object of the plurality of detection devices with one another get connected.

This makes it possible, in particular, that the driver assistance systems or the driver assistance system can be used to continuously record a static environment while driving and make it available to the processing chain. If there are static objects in the lane, a corresponding system reaction can take place. This can include displaying a warning, switching off a driving function, assisted or automatic braking or assisted or automatic evasive action. In particular, the method can be used to recognize objects in the vicinity of autonomous driving functions. This can be done through the efficient representation of the sensor measurement data as polylines and the computational time-saving fusion in the sparse occupancy field, which corresponds to the sparse occupancy grid Implement cost-saving fusion system. The method according to the invention is not restricted to static objects; it can also be applied to dynamic objects.

In other words, the invention proposes a method for detecting a vehicle environment by means of sensor fusion and using occupancy grids. According to the invention, sparse occupancy fields are used, the resolution of occupancy grids being automatically adapted to the resolution of a respective sensor. Measurement data are first mapped onto the occupancy grid and merged therein. An object is then created within the respective occupancy grid, with related entries being linked to one another.

According to an advantageous embodiment, a resolution of a respective sparse occupancy grid is automatically adapted as a function of the resolution capacity of the respective detection device.

It is also advantageous if the sensor data of a respective detection device are represented as at least one polyline within the sparse occupancy grid.

It is also advantageous if a shape of the sparse occupancy grid is adapted as a function of a detection type of the detection device.

In a further advantageous embodiment, when the sensor data are merged, the vehicle's own speed is taken into account.

A further aspect of the invention relates to a driver assistance system for a motor vehicle with a plurality of detection devices and with an electronic computing device, the driver assistance system being designed to carry out a method according to the preceding aspect. In particular, the method is carried out by means of the driver assistance system.

Yet another aspect of the invention relates to a motor vehicle with a driver assistance system. The motor vehicle is designed in particular as a passenger vehicle. In particular, the motor vehicle can also be designed as an at least partially autonomous motor vehicle, in particular as a fully autonomous motor vehicle.

Advantageous embodiments of the method are to be regarded as advantageous embodiments of the driver assistance system and of the motor vehicle. For this purpose, the motor vehicle and the driver assistance system have objective features which enable the method or an advantageous embodiment thereof to be carried out.

Further advantages, features and details of the invention emerge from the following description of preferred exemplary embodiments and with reference to the drawings. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures can be used not only in the respectively specified combination, but also in other combinations or on their own, without the scope of the Invention to leave.

• 1 eine schematische Ansicht von spärlichen Belegungsgittern; und
• 2 eine schematische Ansicht einer Ausführungsform einer elektronischen Recheneinrichtung eines Fahrerassistenzsystems.

Show:

• 1 a schematic view of sparse occupancy grids; and
• 2 a schematic view of an embodiment of an electronic computing device of a driver assistance system.

Identical or functionally identical elements are provided with the same reference symbols in the figures.

1 shows an embodiment of a motor vehicle in a schematic plan view 10 with a driver assistance system 12 . In particular, the 1 different sparse occupancy grids 14th .

In particular, the 1 a method of merging 16 ( 2 ) of sensor data 18th ( 2 ) from a variety of detection devices 20th ( 2 ) by means of an electronic computing device 22nd ( 2 ) of the driver assistance system 12 of the motor vehicle 10 , in which by means of the plurality of detection devices 20th an environment 24 of the motor vehicle 10 is detected and each with the respective detection device 20th captured sensor data 18th to the electronic computing device 22nd and the transmitted sensor data 18th by means of the electronic computing device 22nd based on at least one occupancy grid of the respective detection device 20th to be merged.

It is provided that the multitude of sensor data 18th for merging 16 in the respective sparse allocation grid 14th is classified and within the respective sparse occupancy grid 14th an object image 26th ( 2 ) at least one in the area 24 of the motor vehicle 10 detected object is carried out, with related sensor data 18th of at least one object of Variety of detection devices 20th be connected to each other.

In particular, it can be provided that a respective sparse occupancy grid is resolved 14th depending on the resolution of the respective detection device 20th automatically adjusted.

It can also be provided that the sensor data 18th a respective detection device 20th as at least one polyline within the sparse occupancy grid 14th be represented.

Furthermore, it can be provided that depending on a type of detection of the detection device 20th a form of the sparse occupancy grid 14th is adjusted. This is particularly due to the different occupancy grids 14th in the 1 shown.

It can also be provided that during the merger 16 the sensor data 18th an airspeed of the motor vehicle 10 is taken into account.

In particular, the 1 thus that to merging 16 the sensor data 18th according to the invention, the individual detections initially in the sparse occupancy grid 14th be classified. As a sparse occupancy grid 14th can be any division of the environment 24 serve in segments in which a limited number of hypotheses are allowed in each segment. These sparse occupancy grids 14th are particularly in the 1 shown.

An occupancy grid has proven to be particularly advantageous 14th if it reflects the properties of the sensor image. For beam sensors, i.e. for sensors that are installed at a fixed point and the environment 24 These are sparse occupancy grids 14th also formed like a jet, for example in the 1 at the right allocation grids 14th is shown. This is the resolving power of the sparse occupancy grid 14th automatically to that of the recording devices 20th customized. In this way you avoid unnecessary memory and computing requirements. In the two sparse occupancy grids on the left 14th of the 1 on the other hand, the resolution for the entire detection area is determined by the maximum required resolution at one location in the detection area.

A sparse allocation grid is particularly advantageous with regard to the computing time required 14th even if the affiliation of a sensor measurement to a segment of the occupancy field can be calculated efficiently. This is, for example, in the three sparse occupancy grids on the left 14th of the 1 the case. It is disadvantageous in terms of computing efficiency if the occupancy field has areas whose resolution is not required for further functional use. For example, in the third example, this is the 1 the case. Right in front of the vehicle 10 the lateral resolution is very high. It therefore proves to be particularly advantageous to have an origin 28 the sparse occupancy field 14th behind the motor vehicle 10 to lay. As a result, the association with a segment can be calculated quickly and easily, and the segment resolution is adapted to both the sensory resolution and the functional resolution required in driver assistance functions.

2 shows in a schematic view an embodiment of the electronic computing device 22nd of the driver assistance system 12 . In particular, the method presented thus consists of three steps. In particular, in a first step a on the sparse occupancy grid 14th instead of. For this purpose, a detection of the time of the fusion, a preselection of objects, a scanning, an entry in the occupancy grid 14th and enrichment with sensor properties, for example quality, accuracy, classes or connections, can be carried out. The second step is the merger 16 . Both the merger 16 as well as tracking and classification can be implemented. The object is mapped in the next step 26th this in particular by connecting related entries in the sparse occupancy grid 14th is realized.

Requirement for the merger 16 the sensor data is that all sensor measurement data are represented in the same way before the actual fusion step can be carried out. The representation of objects at the entrance of the method described here as polylines, that is to say sequences of point coordinates that are connected, proves to be particularly advantageous. They describe those for the acquisition device 20th visible outer contour of an object. Model-based representations such as boxes, rod objects or three-point objects would cause problems in meeting the accuracy requirements of the driver assistance system 12 to lead.

For the mapping of the objects in the sparse occupancy grid 14th become the objects of the detection device 20th initially sampled so that there is a sampling point for each segment that intersects the polyline representation. The objects that are already present and tracked in the occupancy field are treated in the same way. It must be ensured that both sensor measurement data and the objects already represented in the occupancy field were predicted to the same point in time or projected into the past by retrodiction.

Further processing steps in the mapping onto the occupancy field can be, for example, a preselection of the sensor objects that are relevant for the driver assistance function either on the basis of their class or location. In addition, classification results, accuracies, quality measures and connection information, which enables a reconstruction of the previously connected points, can be "inherited" at the sampling points from the object description to the point description.

The actual merger 16 begins, as is usual in fusion processes, with the association step. Here it is decided whether sampling points are merged. The association range is given here on the one hand by the division of the field segments and on the other hand by the measurement uncertainty within a field segment. It has proven to be advantageous to allow an association with points in adjacent field segments, in particular at the object ends. In contrast to methods with complete occupancy fields, in which the position uncertainty is implemented by "smearing" onto neighboring field elements, the proposed method is able to track the objects with the help of filters, for example a Kalman filter, as in object tracking without occupancy fields and to output an improved estimated value for the object position or for the object scanning point.

Finally, points that belong together can be connected again in the object formation and as individual objects in the further processing chain of the driver assistance system 12 to provide. In object generation, it has proven to be particularly advantageous if the inherited object properties and the former sensor and fusion object connections are taken into account. The connections from the last processing cycle can thus be used in the current processing cycle to decide whether points should be connected again.

• Insbesondere wird geprüft, ob ein Sensorpunkt mit einem bestehenden Fusionspunkt fusioniert werden kann. Falls ja, wird das Update durchgeführt, falls nein wird geprüft ob noch eine Hypothese in dem Feldsegment frei ist. Falls ja wird dieses belegt, falls nein wird eine Hypothese gelöscht und mit einer neuen belegt. In die Entscheidung welche gelöscht wird, kann die Existenzwahrscheinlichkeit und das Risiko,
• beispielsweisenäherliegende Hindernisse sind gefährlicher als entfernte, berücksichtigt werden.

The savings potential compared to a merger 16 with a complete occupancy field is the restriction to a limited number of hypotheses per field segment. The number of hypotheses that is required depends on the frequency of incorrectly measured obstacles (false-positive measurement) of the detection devices 20th from. Usually it is used for the driver assistance system 12 only the first obstacle in the corridor is required. However, since this can also be a false-positive measurement, so many hypotheses must be possible that the first real obstacle is detected with sufficient certainty, which, in contrast to the false-positive measurements, is then confirmed in the following measurements . If more than the limited number of sampling or fusion points is generated during the sensor update, a strategy is integrated in the presented method that decides which hypotheses "survive":

• In particular, it is checked whether a sensor point can be fused with an existing fusion point. If so, the update is carried out; if not, it is checked whether a hypothesis is still free in the field segment. If yes, this is proven, if not, a hypothesis is deleted and a new one is assigned. In the decision which is deleted, the probability of existence and the risk,
• For example, obstacles lying closer are more dangerous than distant obstacles are taken into account.

Overall, the invention shows a fusion of sensor data for vehicle environment detection with sparse occupancy fields, in particular sparse occupancy grids 14th .

List of reference symbols

10

Motor vehicle

12

Driver assistance system

14th

sparse occupancy grid

16

fusion

18th

Sensor data

20th

Detection device

22nd

electronic computing device

24

Surroundings

26th

Object illustration

28

origin

30th

Illustration

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102015201747 A1 [0004]

Claims (6)

A method for merging (16) a large number of sensor data (18) from a large number of detection devices (20) by means of an electronic computing device (22) of a driver assistance system (12) of a motor vehicle (10), in which, by means of the large number of detection devices (20), a Environment (24) of the motor vehicle (10) is recorded and the sensor data (18) recorded by the respective recording device (20) are transmitted to the electronic computing device (22) and the transmitted sensor data (18) are transmitted by means of the electronic computing device (22) The basis of at least one occupancy grid of a respective detection device (20) are fused, characterized in that the large number of sensor data (18) for merging (16) is classified in a respective sparse occupancy grid (14) and an object image is displayed within the respective sparse occupancy grid (14) (26) at least one in the surroundings (24) of the motor vehicle (10) detected object is carried out, associated sensor data (18) of the at least one object of the plurality of detection devices (20) being connected to one another. Procedure according to Claim 1 , characterized in that a resolution of a respective sparse occupancy grid (14) is automatically adapted as a function of a resolution capability of the respective detection device (20). Procedure according to Claim 1 or 2 , characterized in that the sensor data (18) of a respective detection device (20) are represented as at least one polyline within the sparse occupancy grid (14). Method according to one of the preceding claims, characterized in that a shape of the sparse occupancy grid (14) is adapted as a function of a detection type of the detection device (20). Method according to one of the preceding claims, characterized in that the merging (16) of the sensor data (18) takes into account the speed of the motor vehicle (10). Driver assistance system (12) for a motor vehicle (10) with a plurality of detection devices (20) and with an electronic computing device (22), the driver assistance system (12) for performing a method according to one of the Claims 1 to 5 is trained.

Images