DE102023002030B3 - Procedure for creating a digital map - Google Patents

Abstract

The invention relates to a method for creating a digital map (K), whereby aerial photographs (L) of a given geographical area are captured. According to the invention, trajectories (T1 to Tm) with courses of geographical positions of the respective vehicle (F1 to Fn) are recorded by means of a plurality of vehicles (F1 to Fn) when driving in the geographical area. Segmented trajectory groups (TG1 to TGy) are formed from the recorded trajectories (T1 to Tm), wherein the segmented trajectory groups (TG1 to TGy) and the aerial photographs (L) are aligned with one another and lane segments of roadways are determined from the aerial photographs (L) and the segmented trajectory groups (TG1 to TGy) aligned with them by means of a trained convolutional neural network (CNN) and are included in the digital map (K).The invention further relates to the use of a digital map (K) created in such a method in a vehicle (F1 to Fn) for executing a navigation function and/or for executing an automated driving operation.

Description

The invention relates to a method for creating a digital map according to the preamble of claim 1.

The invention further relates to a use of such a digital map.

High-resolution digital maps, also known as high-definition maps, and the information they contain on lane geometry and topology are of great importance for automated, for example highly automated or autonomous, vehicle operation, as they provide the necessary information about relevant roads and the vehicle's surroundings that go beyond the detection limits of on-board sensors. Such digital maps therefore make a decisive contribution to the safety and reliability of the automated vehicle.

Lane geometries stored in the digital maps provide the vehicle with a physical route of a road as well as a number and actual global positions of specific lanes on the road. Based on this information, a system designed for automated driving of the vehicle can make decisions about future maneuvers, a direction of travel and necessary lane changes and plan a specific future trajectory.

Lane topologies stored in the digital maps describe how the lanes relate to each other and how they change in relation to intersections, junctions and other road features. This information is also important because it helps algorithms of the system intended for the automated driving of the vehicle to navigate safely and efficiently through intersections and other complex traffic situations.

• - Eingeben eines zu erkennenden Satellitenbilds in ein trainiertes LaneNet-Netzwerkmodell, um ein Satellitenbild markiert mit Koordinatenpunkten von Fahrspurlinien auszugeben,
• - Gruppieren der Koordinatenpunkte ausgegebener Fahrspurlinien, um einen Koordinatenpunkt zu erhalten, der zu jeder Fahrspurlinie gehört,
• - Anpassen gemäß dem Koordinatenpunkt, der zu jeder Fahrspurlinie gehört, um jede Fahrspurlinie zu erhalten, wobei das LaneNet-Netzwerkmodell in einem Training mit einem Trainingssatz gebildet wird, der eine Vielzahl von Satellitenbildern und in jedem Satellitenbild markierte Koordinatenpunkten für Fahrspurlinie umfasst, und
• - Verwendung des LaneNet-Netzwerkmodells, um die Fahrspurlinien in einem erfassten Satellitenbild zu identifizieren und eine Navigationskarte zu erstellen.

The CN112418037A describes a method and a system for detecting lane markings in a satellite image. The method comprises the following steps:

• - Inputting a satellite image to be recognized into a trained LaneNet network model to output a satellite image marked with coordinate points of lane lines,
• - Grouping the coordinate points of output lane lines to obtain a coordinate point belonging to each lane line,
• - adjusting according to the coordinate point corresponding to each lane line to obtain each lane line, wherein the LaneNet network model is formed in training with a training set comprising a plurality of satellite images and lane line coordinate points marked in each satellite image, and
• - Using the LaneNet network model to identify lane lines in an acquired satellite image and create a navigation map.

From the EN 10 2022 001 030 B3 A method for creating a digital map is known in which sensor data is recorded by a plurality of vehicles while driving in a geographical area and in which a road map is created from the recorded sensor data using machine learning methods, which road map contains a description of driving path segments that have been recorded in the past.

From the EN 10 2019 215 522 A1 A method for updating attributes of a digital map using aerial photographs is known in which attributes of roadways in the respective segments are determined from segments of the aerial photographs using a trained neural network and are used to update or complete attributes stored in the map.

The invention is based on the object of specifying a novel method for creating a digital map and a use of such a digital map.

The object is achieved according to the invention by a method which has the features specified in claim 1 and by a use which has the features specified in claim 10.

Advantageous embodiments of the invention are the subject of the subclaims.

In a process for creating a digital map, aerial photographs of a given geographical area are captured.

According to the invention, trajectories with courses of geographical positions of the respective vehicle are recorded by means of a plurality of vehicles when driving in the geographical area, with segmented trajectory groups being formed from the recorded trajectories. The segmented trajectory groups and the aerial photographs are aligned with one another, with lane segments being formed from the aerial photographs and the segmented trajectory groups aligned with them by means of a trained convolutional neural network. elements of roadways are determined and included in the digital map.

Lane markings can be extracted from aerial photographs, such as satellite images or aerial photographs. However, there is a risk that if road markings are missing, unclear or ambiguous, aerial photographs will provide unclear representations of the road layout. This can lead to misjudgments when determining lane geometries and lane topologies. For example, if the lanes are obscured by bridges, tunnels, buildings, peripheral buildings or vegetation, incompleteness can occur in the derivation of lanes. When deriving a lane topology based solely on actual vehicle trajectories, only the driving behavior of the vehicles is taken into account. Physical lane markings are not taken into account. This in turn leads to limitations in the recognition of the actual lane geometry and the complete lane topology.

Due to the combination of the aerial photographs with the correlating segmented trajectory groups, a particularly high level of accuracy and correctness of the digital map can be achieved using this method. This is particularly possible even when road markings are missing or ambiguous. This combination can be fully automated by using algorithms from the field of machine learning and further processing steps, i.e. in particular by using the trained convolutional neural network.

In particular, when road markings are missing or ambiguous, such as in road crossing areas or in poor road conditions, the combination of the aerial images with the correlating segmented trajectory groups in lane detection enables information about the lane geometry and topology to be extracted from the trajectory courses and combined with the aerial images, so that a complete representation of roads with lanes is generated despite the missing information in the aerial images.

By combining the aerial photographs with the correlating segmented trajectory groups, it is also possible to bridge shadows and occlusions in the aerial photographs. The impairments caused by shadows, structural occlusions or other obscuring objects, such as vehicles, in the aerial photographs, which can lead to incorrect and incomplete detection of lane markings and road edges, can be remedied by adding the recorded trajectory courses, since the geographical positions of the vehicles are recorded independently of lighting conditions or other influences and can thus be projected into the corresponding aerial photograph.

The additional use of the trajectories also makes it possible to derive lane topologies, i.e. lane connections that comply with regulations, turning processes at road junctions and lane changes, without errors. This is not possible when using aerial photographs alone, as restrictions, for example due to traffic signs, cannot be recognized in images from a bird's eye view. The combination with the trajectories and thus the combinations actually driven by the vehicles enables a robust recognition of the lane topologies.

Furthermore, the use of the learning-based convolutional neural network and its generalization ability enables special cases to be robustly recognized and classified or derived accordingly. In comparison to heuristic or classical image recognition methods, not all special cases have to be known a priori, but can be solved by the method itself based on previously learned rules.

Furthermore, the present method leads to a high level of scalability. In addition to the fully automated execution after successful training, processing data in areas segmented by the segmented trajectory groups enables the subproblem to be broken down, which means that the map creation process can be parallelized. This reduces the time required enormously compared to conventional mapping processes.

According to a possible embodiment of the method, lane segments determined for neighboring geographical areas are combined to form lane paths of roadways on the digital map. This allows a global digital map to be created.

According to another possible embodiment of the method, the digital map is created on a backend server. This enables centralized creation of the digital map, which minimizes the computing power required for data processing in the individual vehicles. Furthermore, the created digital map can be made available centrally for a large number of vehicles.

According to another possible embodiment of the method, geographical positions of the trajectories are optimized using a clustering method. This makes it possible to correct inaccuracies in a temporal sequence of geographical positions that form the trajectories, for example due to so-called multi-path effects. This results in increased accuracy of the digital map generated.

According to another possible embodiment of the method, geographical positions of the trajectories are aligned with each other in the lateral and longitudinal directions, whereby the alignment is carried out depending on vehicle observations of localization objects by carrying out an error correction based on the same geographical localization features observed by several vehicles. By optimizing trajectory courses in this way, actual lane courses can be represented much better.

According to another possible embodiment of the method, the segmentation of the trajectory groups is carried out based on the direction of travel and/or speed of the vehicles. The segmentation increases the performance of the learning-based method carried out using the convolutional neural network. The segmentation can be carried out by highlighting patterns in the input data and reducing the complexity of this input data for the learning-based method. The segmentation can be carried out very precisely and efficiently using a clustering method, for example a so-called DBSCAN method (DBSCAN = Density-Based Spatial Clustering of Applications with Noise).

According to another possible embodiment of the method, the segmentation of the trajectory groups is carried out based on a mathematical model in a regression method. Using such a regression method, the complexity of the input data can be further reduced and a large data reduction can be achieved.

According to another possible embodiment of the method, the aerial photographs are pre-processed in a radiometric calibration, whereby certain features in the aerial photographs are highlighted during the pre-processing. This enables improved processing of the aerial photographs in the learning-based method.

According to another possible embodiment of the method, the alignment of the trajectory groups and the aerial photographs to each other is carried out based on a respective geographical location and/or respective height information in the trajectories of the trajectory groups and/or localization features of the trajectories of the trajectory groups detected in the aerial photographs. This alignment can improve a combination of both data sources in a data representation. By using height information in the trajectories, for example, optical distortions in the aerial photographs can be compensated. Furthermore, recognized localization features of the trajectories, which can also be found within the aerial photographs, such as lane markings, can be used to align the trajectory groups based on the aerial photographs. This can also avoid orientation deviations, among other things.

When using a digital map created in an aforementioned method, it is intended to be used in a vehicle to carry out a navigation function and/or to carry out automated driving. By using a digital map that is thus very accurate and correctly executed, the availability and reliability of navigation of the vehicle and of automated, in particular highly automated or autonomous, driving of the vehicle can be significantly improved.

Embodiments of the invention are explained in more detail below with reference to drawings.

• 1 schematisch ein Luftbild einer Straßenkreuzung mit einer Mehrzahl von Fahrzeugen und Trajektorien,
• 2 schematisch ein Luftbild der Straßenkreuzung gemäß 1 mit Trajektoriengruppen vor einer Optimierung und das Luftbild der Straßenkreuzung gemäß 1 mit Trajektoriengruppen nach einer Optimierung,
• 3 schematisch ein Luftbild der Straßenkreuzung gemäß 1 mit Trajektorien nach einer Segmentierung von Trajektoriengruppen und
• 4 schematisch einen Ablauf eines Verfahrens zur Erstellung einer digitalen Karte.

Showing:

• 1 schematically an aerial photograph of a road junction with a plurality of vehicles and trajectories,
• 2 schematically an aerial photograph of the road junction according to 1 with trajectory groups before optimization and the aerial image of the road junction according to 1 with trajectory groups after optimization,
• 3 schematically an aerial photograph of the road junction according to 1 with trajectories after a segmentation of trajectory groups and
• 4 schematically shows a process for creating a digital map.

Corresponding parts are provided with the same reference numerals in all figures.

In 1 is a plan view of a road intersection with a plurality of vehicles F1 to Fn and trajectories T1, T2 driven by several of these vehicles F1 to Fn.

In a possible embodiment of a method according to the invention for creating a 4 The digital map K shown in more detail is intended to provide aerial photographs L of a given geographical area with correlating segmented trajectory groups TG1 to TGy, which are 2 and 3 This is done in a learning-based process, in particular by means of a 4 Convolutional Neural Networks (CNN), which are described in more detail, were used.

For this purpose, aerial photographs L of the given geographical area are recorded. This recording is carried out, for example, using satellites as satellite images and/or using at least one camera as an aerial photograph. The geographical area is represented by means of an aerial photograph L from a bird's eye view.

The aerial photograph L can, for example, be in an RGB format or another suitable format.

Furthermore, trajectories T1, T2 with courses of geographical positions of the respective vehicle F1 to Fn are recorded by means of a plurality of vehicles F1 to Fn when traveling in the geographical area.

Subsequently, data processing is carried out for both data sources, i.e. for the captured aerial photographs L and the recorded trajectories T1, T2. In particular, data processing is carried out separately for both data sources.

The data preparation is based on the 2 and 3 explained in more detail.

2 shows the aerial photograph L of the road junction according to 1 with a variety of trajectories T1 to Tm before optimization (left illustration) and the aerial image L of the road junction according to 1 with trajectories T1 to Tm after optimization (right illustration).

The trajectories T1 to Tm of the vehicles F1 to Fn consist at least essentially of a temporal sequence of geographical positions, for example GPS positions, of the vehicles F1 to Fn, which together form a trajectory group TG1 to TGy, also referred to as a trajectory track or trace.

Such trajectory groups TG1 to TGy can be inaccurate for various reasons, such as so-called multi-path effects. In order to improve this inaccuracy, an optimization of the geographical positions of the trajectories T1 to Tm of the trajectory groups TG1 to TGy is carried out in an optimization of the trajectory groups TG1 to TGy. This can be done, for example, using a method from the class of so-called clustering methods. Furthermore, an alignment of the geographical positions in the longitudinal and lateral direction to each other is carried out. Existing vehicle observations of localization objects are used and an error correction is carried out based on the same observed geographical localization features. Through this optimization of the trajectory groups TG1 to TGy, actual lane courses can be represented much better - as shown in the illustration on the right.

In order to increase the performance of the learning-based method, patterns in the input data are highlighted in a pre-processing step so that the complexity of the input data can be reduced. This is done by segmenting the trajectory groups TG1 to TGy, which is achieved by displaying them in 3 is made clear.

This segmentation is carried out based on defined features, such as a direction of travel and/or speed of the vehicles F1 to Fn, which each record the trajectories T1 to Tm to create the trajectory groups TG1 to TGy. The segmentation can be carried out, for example, by using additional clustering methods, such as the so-called DBSCAN method. A data reduction realized by applying such clustering methods can be supplemented by applying regression techniques in which a mathematical model is used to segment the trajectory groups TG1 to TGy.

By means of segmentation, further features can be provided to a predictive model used in the learning-based method, so that its performance can be increased. In possible embodiments, such clusters are found in the data itself using the learning-based method.

Furthermore, the aerial photographs L are also prepared for use with the learning-based method. In order to highlight certain features in the aerial photographs L, these can be pre-processed in a separate pre-processing step using methods such as radiometric calibration in order to optimize subsequent processing using the learning-based method.

Following the preprocessing of the input data from both data sources, i.e. following the segmentation of the trajectories groups TG1 to TGy and the possible processing of the aerial photographs L, both data sources are combined into one data representation.

To do this, the trajectory groups TG1 to TGy and the aerial photographs L are first aligned with each other. In addition to a simple alignment based on a respective geographical location, further alignment steps can be used. For example, height information from the trajectory groups TG1 to TGy can be used to compensate for optical distortions in the aerial photograph L. Furthermore, similar to the described preprocessing of the trajectory groups TG1 to TGy, recognized localization features of the trajectory groups TG1 to TGy, which can also be found within the aerial photographs L, such as lane markings, can be used to align the trajectory groups TG1 to TGy based on the aerial photographs L. This can also avoid orientation deviations, among other things.

The joint representation thus generated later serves as the data basis for the learning-based procedure, which is 4 The lane geometries FG and lane topologies FT shown in more detail should interfere.

The combination of both data sources creates a comprehensive dataset of images that contain both an aerial photograph L of the streets and the projected trajectory groups TG1 to TGy. Data processing and data preparation are completed by dividing the dataset into smaller, homogeneous data points, each of which contains individual street areas.

To actually derive lane geometries FG and lane topologies FT within the processed data, a learning-based model from the field of AI-based image understanding is used, such as a Convolutional Neural Network CNN. This shows 4 .

Here, an inference of the desired data is formulated as a so-called instance segmentation problem, in which a neural network of the convolutional neural network CNN is to recognize each instance of a trajectory group TG1 to TGy in the aerial images L and output it accordingly. The choice of the neural network motivates the previously described preparation of the data in the form of processed aerial images L and projected trajectory groups TG1 to TGy as input data. These are used to predict the lane geometries FG and lane topologies FT in the digital map K as output information.

The use of learning-based methods always requires training the model used with suitable, preselected data. Image understanding falls into the area of so-called supervised learning, which requires so-called ground truth data to successfully train the model. This data has previously been attributed by a human and is used to adapt parameters of the neural network.

The convolutional neural network CNN first solves the problem of detecting the lane segments in the input image, i.e. in the aerial image L, and their actual geographical extent. To fully generate a lane layer, the lane topology FT, i.e. a relationship between the lane segments found, must also be determined. Two approaches are suitable for this. Firstly, the learning-based model can be supplemented so that the relationship is already solved within the model. Alternatively, a topology of the detected lane segments in the aerial image L can be derived by taking into account the directions of travel of the corresponding trajectory group TG1 to TGy in a subsequent heuristic post-processing step. Both variants can be integrated into the process in a fully automated manner.

A final processing step for generating the digital map K includes the re-transformation of the image data into a global geographical representation and the merging of the respective image segments into a global digital map K. The results of the divided aerial photographs L are merged again at image boundaries. This means that lane segments determined for neighboring geographical areas are combined to form lane courses of roadways in the digital map K.

Claims (10)

Method for creating a digital map (K), whereby aerial photographs (L) of a given geographical area are recorded, characterized in that - by means of a plurality of vehicles (F1 to Fn) during travel in the geographical area, trajectories (T1 to Tm) with courses of geographical positions of the respective vehicle (F1 to Fn) are recorded, - segmented trajectory groups (TG1 to TGy) are formed from the recorded trajectories (T1 to Tm), - the segmented trajectory groups (TG1 to TGy) and the aerial photographs (L) are aligned with one another, and - from the aerial photographs (L) and the segmented trajectory groups (TG1 to TGy) lane segments of roads are determined using a trained convolutional neural network (CNN) and included in the digital map (K). Procedure according to Claim 1 , characterized in that lane segments determined for adjacent geographical areas are combined to form lane courses of roadways of the digital map (K). Procedure according to Claim 1 or 2 , characterized in that the creation of the digital map (K) is carried out on a backend server. Method according to one of the preceding claims, characterized in that the geographical positions of the trajectories (T1 to Tm) are optimized by means of a clustering method. Method according to one of the preceding claims, characterized in that - the geographical positions of the trajectories (T1 to Tm) are aligned with one another in the lateral and longitudinal direction, wherein - the alignment is carried out as a function of vehicle observations of localization objects by carrying out an error correction on the basis of identical geographical localization features observed by several vehicles (F1 to Fn). Method according to one of the preceding claims, characterized in that the segmentation of the trajectory groups (TG1 to TGy) is carried out based on a direction of travel and/or speed of the vehicles (F1 to Fn). Procedure according to Claim 6 , characterized in that the segmentation of the trajectory groups (TG1 to TGy) is carried out based on a mathematical model in a regression procedure. Method according to one of the preceding claims, characterized in that - the aerial photographs (L) are pre-processed in a radiometric calibration, wherein - in the pre-processing, certain features are highlighted in the aerial photographs (L). Method according to one of the preceding claims, characterized in that the alignment of the trajectory groups (TG1 to TGy) and the aerial images (L) to one another is carried out on the basis of a respective geographical position and/or respective altitude information in the trajectories (T1 to Tm) of the trajectory groups (TG1 to TGy) and/or localization features of the trajectories (T1 to Tm) of the trajectory groups (TG1 to TGy) detected in the aerial images (L). Use of a digital map (K) created in a method according to one of the preceding claims in a vehicle (F1 to Fn) for executing a navigation function and/or for executing an automated driving operation.