DE102020111540B3 - System and method for the automatic recording of extreme driving situations, especially for highly automated driving functions - Google Patents

Abstract

The invention relates to a method for the automatic detection of extreme driving situations for highly automated driving functions, comprising the extraction of various parameters from several different data sources in a data module; transmitting the parameters from the data module to a parameterization module; the transmission of situation data from a shadow module to the parameterization module; the parameterization of traffic situations with the parameters received from the parameterization module; transferring the parameterized traffic situations from the parameterization module to a classifier and to an output module; extracting relevant features from the parameterized traffic situations from the classifier; transferring the extracted features from the classifier to the output module; the calculation of extreme driving situations for a predetermined geographical and / or temporal area from the output module; and the outputting of a route planning for the predetermined geographical and / or temporal area, which includes an optimized number of extreme situations, from the output module.

Description

The invention relates to a system and method for the automatic detection of extreme driving situations, in particular for highly automated driving functions.

The trend towards highly automated driving functions in motor vehicles requires extensive safeguarding strategies, since the responsibility for driving the vehicle no longer lies entirely with the driver, but active functions are taken over by computer units in the vehicle. It must be ensured that autonomous systems only have a very low error rate. This requires the targeted testing of extreme situations (corner cases). Such extreme situations result from a special combination of different factors. Examples of this are infrastructural features such as the type of street, the development on the edge of a street, the quality of the markings, but also environmental conditions such as weather conditions, the time of day and the season. In addition, the behavior of other road users is important.

However, a simulation of all possible scenarios on a test site provided for this purpose cannot be achieved with a reasonable amount of effort. Therefore, empirical values from test drivers and developers are currently used to predict and identify critical situations using test software and to use them to optimize highly automated driving functions.

After the software version for the highly automated driving functions has successfully passed the simulation phase, the software is tested on test tracks. In this phase, test drivers or remote-controlled vehicles carry out scenarios from a predefined test catalog. However, in contrast to simulation, weather conditions, seasons and geographical conditions limit the variety of scenarios. Such tests can therefore only cover a small part of the possible parameter range. As a result, there will be situations that are not covered by the highly automated driving function.

The document US 2020/0 101 974 A1 describes a device and a method for the selection of an optimal travel route taking into account a real-time driving environment in an autonomously driving vehicle. The method comprises receiving environmental situation information or road situation information from a computer cloud, calculating an evaluation for each lane of a multi-lane road based on the environmental situation information or the road situation information, and configuring a travel route based on the evaluation.

The document DE 10 2017 114 049A1 describes a system for determining and executing a route for an autonomously driving vehicle with a storage medium. The storage medium contains factor data for determining a relevant vehicle route and a route generation module that determines the route based on route complexity indices.

The document WO 2016/200 762 A1 describes a method for analyzing historical accident information for an autonomously driving vehicle. Using historical accident information, a route can be selected that has a lower accident risk than other routes.

The document WO 2018/026 603 A1 describes a system and a method for determining an optimal lane and route for an autonomously driving vehicle under changing road conditions. The road conditions are analyzed in order to determine a safe lane position.

The document US 2019/0 049 259 A1 describes a method for selecting a route for an autonomously driving vehicle. Driving factors such as map availability, weather conditions or road conditions are assigned to the route segments of the route. An evaluation of the respective route segment is determined from the driving factors, which in turn correlates with an autonomy level.

The object on which the invention is based now consists in creating a method and a system for the automatic detection of extreme driving situations, in particular for highly automated driving functions, which is characterized by high reliability and accuracy and is easy to implement.

According to the present invention, a method is proposed by means of which a machine detection of extreme driving situations is made possible in order to create, for example, the basis for the provision of highly automated driving functions, which are also characterized by a high degree of safety and reliability in extreme driving situations.

According to the invention, this object is achieved with regard to a method by the features of patent claim 1, and with regard to a system by the features of patent claim 5. the further claims relate to preferred embodiments of the invention.

• - Extrahieren von verschiedenen Parametern aus mehreren unterschiedlichen Datenquellen in einem Datenmodul, wobei das Datenmodul eine relationale Datenbank umfasst und als Datenquellen kartographische Informationen, Daten zu Verkehrszahlen, Luftbildaufnahmen und/oder Daten zu Wetterinformationen und/oder Daten von Kraftfahrzeugen verwendet werden;
• - Übermitteln der Parameter von dem Datenmodul an ein Parametrisierungsmodul;
• - Übermitteln von Situationsdaten, wie insbesondere Unfalldaten aus einem Unfallmodul und/oder Daten aus einem Schattenmodul an das Parametrisierungsmodul, wobei das Parametrisierungsmodul ein tiefes neuronales Netz zur Extraktion von Parametern und zur Mustererkennung umfasst;
• - Parametrisieren von Straßenstrecken und/oder Verkehrssituationen mit den empfangenen Parametern und den übermittelten Unfalldaten und/oder Daten aus einem Schattenmodul von dem Parametrisierungsmodul;
• - Transferieren der parametrisierten Straßenstrecken und/oder Verkehrssituationen von dem Parametrisierungsmodul an einen Klassifizierer und an ein Ausgabemodul;
• - Extrahieren von relevanten Merkmalen aus den parametrisierten Stra-ßenstrecken und/oder Verkehrssituationen von dem Klassifizierer;
• - Transferieren der extrahierten Merkmale von dem Klassifizierer an das Ausgabemodul;
• - Berechnen von fahrtechnischen Extremsituationen für einen vorbestimmten geographischen und/oder zeitlichen Bereich von dem Ausgabemodul;
• - Ausgeben einer Routenplanung für den vorbestimmten geographischen und/oder zeitlichen Bereich, die eine optimierte Anzahl an Extremsituationen umfasst, von dem Ausgabemodul.

According to a first aspect, the invention relates to a method for the automated detection of extreme driving situations, in particular for highly automated driving functions, comprising the following method steps:

• Extraction of various parameters from several different data sources in a data module, the data module comprising a relational database and cartographic information, data on traffic figures, aerial photographs and / or data on weather information and / or data from motor vehicles being used as data sources;
• - Transmission of the parameters from the data module to a parameterization module;
• Transmission of situation data, such as accident data in particular from an accident module and / or data from a shadow module, to the parameterization module, the parameterization module comprising a deep neural network for extracting parameters and for pattern recognition;
• Parameterization of road sections and / or traffic situations with the received parameters and the transmitted accident data and / or data from a shadow module from the parameterization module;
• - Transferring the parameterized road routes and / or traffic situations from the parameterization module to a classifier and to an output module;
• - Extraction of relevant features from the parameterized road routes and / or traffic situations from the classifier;
• Transferring the extracted features from the classifier to the output module;
• - Calculation of extreme driving situations for a predetermined geographical and / or time range from the output module;
• Output of a route plan for the predetermined geographical and / or temporal area, which includes an optimized number of extreme situations, from the output module.

The data module advantageously has an autoencoder and / or generative opposing networks (GAN) and / or image processing algorithms.

In a further embodiment, the neural network is trained with historical parameters and / or historical accident data.

In particular, filtering of the transmitted data by means of predetermined hyperparameters and / or a gradient optimizer and / or activation functions can be provided for training the neural network in order to ensure normalized data input.

According to a second aspect, the invention relates to a system for the automated detection of extreme driving situations, in particular for highly automated driving functions. The system comprises a data module which is configured to extract various parameters from several different data sources and to transmit the parameters to a parameterization module. Furthermore, situation data modules, such as in particular an accident module that is configured to transmit accident data to the parameterization module, and a shadow module that is configured to transmit data to the parameterization module, are provided. The parameterization module is configured to parameterize road sections and / or traffic situations with the received parameters and the transmitted accident data and / or data from a shadow module and to transfer the parameterized road sections and / or traffic situations to a classifier and to an output module. The classifier is configured to extract relevant features from the parameterized road routes and / or traffic situations and to transfer the extracted features to the output module. The output module is configured to calculate extreme driving situations for a predetermined geographical and / or temporal area and to output route planning for the predetermined geographical and / or temporal area, which includes an optimized number of extreme situations. The data module comprises a relational database and cartographic information, data on traffic figures, aerial photographs and / or data on weather information and / or data from motor vehicles are used as data sources. The parameterization module comprises a deep neural network for the extraction of parameters and for pattern recognition.

In a further embodiment it is provided that the data module has an autoencoder and / or generative opposing networks (GAN) and / or image processing algorithms.

The neural network is advantageously trained with historical parameters and / or historical accident data.

In particular, filtering of the transmitted data by means of predetermined hyperparameters and / or a gradient optimizer and / or activation functions can be provided for training the neural network in order to ensure normalized data input.

The invention is explained in more detail below with reference to the exemplary embodiments shown in the drawing.

• 1 ein Blockdiagramm zur Erläuterung eines Ausführungsbeispiels eines erfindungsgemäßen Systems;
• 2 ein Blockdiagramm zur Erläuterung der Datenbankstruktur des Datenmoduls des erfindungsgemäßen Systems;
• 3 ein Blockdiagramm zur Erläuterung der Struktur des Parametrisierungsmoduls des erfindungsgemäßen Systems;
• 4 ein Flussdiagramm zur Erläuterung der einzelnen Verfahrensschritte des erfindungsgemäßen Verfahrens.

It shows:

• 1 a block diagram to explain an embodiment of a system according to the invention;
• 2 a block diagram to explain the database structure of the data module of the system according to the invention;
• 3 a block diagram to explain the structure of the parameterization module of the system according to the invention;
• 4th a flow chart to explain the individual method steps of the method according to the invention.

Additional features, aspects and advantages of the invention or its exemplary embodiments will become apparent from the detailed description in conjunction with the claims.

According to the present invention, two types of extreme driving-specific situations are covered. The first type is extreme situations caused by human error, including accidents in particular. The other case includes extreme situations that are only relevant for the Advanced Driver Assistance System (ADAS; Highly Automated Driving, HAD) but have no negative impact on a human driver. What these extreme situations have in common is that they can be characterized by certain parameters and can thus be represented as an image in a parameter space. A parameter is a variable that has a direct or indirect effect on the occurrence of extreme driving situations. Examples of parameters are, for example, the weather conditions, the signage and the road infrastructure. Accident research, for example, deals with statistical methods and derived probability distributions of individual parameters such as the hour of the day or the amount of precipitation related to the occurrence of accidents. In addition to a linear influence on an extreme situation, the various parameters can also have complex interdependencies and / or exert a non-linear and / or multidimensional influence on an extreme situation.

In order to be able to predict extreme driving situations, road scenarios such as road sections and / or traffic situations are parameterized according to the invention by means of mathematical algorithms, in particular algorithms of artificial intelligence. In addition to the manual entry of numerous parameters, databases provided by the authorities can be used. For example, the Federal Statistical Office provides a nationwide database for recording all accidents with serious injuries. There are similar data sets in many other countries. To expand the parameter space, additional data sources are assigned to each accident, the data sources ranging from weather data to entries in social media. In addition, commercial map data is available that can be used.

1 shows the system according to the invention 100 for the automated recording of extreme driving situations for highly automated driving functions. The data are from a data module 10 in a parameterization module 20th entered, further processed and then in an output module 30th output as results, in particular in the form of route planning with suggested navigation and time information. This allows a test route to be selected that covers a maximum of extreme situations. In order to use the route planning and navigation algorithms, the map data are preferably converted into a graph network. A cloud computing platform can be provided for the calculation and storage functionalities.

The data module 10 includes various data sources or databases for the further parameterization of road scenarios. This includes, in particular, road network data 12th with road specifications such as lanes and bridges. Furthermore, there are data 12th in relation to the road infrastructure such as the road surface, the perimeter development and / or the road layout. The calculation and storage of all geographical objects is preferably carried out in the EPSG 25832 coordinate system (Universal Transverse Mercator (UTM) Zone 32N). This system is also used by authorities in Germany. The lateral and longitudinal positions are shown in meters. In addition, global reference systems such as the “World Geodetic System 1984 (WGS 84)”, which is also used in GPS receivers (Global Positioning System), can be used. So can the whole Map content from countries such as Germany can be imported.

Furthermore, the traffic figures are 14th of interest. The traffic data 14th have two components, namely the annual average daily traffic, which is independent of time in relation to a certain year, and the real hourly traffic volume. Both components are determined by sensors that are either embedded in the road surface or in reflector pillars. The total number of traffic counting stations on German autobahns is around 950 machines. The measurement data includes the total number of vehicles and the traffic volume for eight different types of motorized vehicles in both directions. The data are made freely accessible by the Federal Highway Research Institute. In addition, data from commercial map service providers can be used.

Another source of data is aerial photographs 16 such as those made available by Google Maps, for example.

Furthermore, weather data 17th to the data module 10 transmitted. The weather data 17th include historical weather measurements and future weather forecasts.

Further data that is in the parameterization module 20th are entered are historical accident data 40 as well as trigger data of a shadow module 50 . Through these different data sources 10 , 40 , 50 detection and prediction of extreme situations for highly automated driving functions is possible.

With the shadow module 50 it is a database that is triggered by background functions when a vehicle is moving and is referred to as shadow mode. The vehicles can be test vehicles or customer vehicles. In the shadow mode, triggers are provided that respond to a specific behavior of individual components. For example, exceeding certain acceleration values can trigger a trigger, so that the relevant vehicle data is stored in a data memory or directly to the shadow module 50 be transmitted. The data preferably consist of sensor data and camera images.

The parameterization module 20th preferably comprises a deep neural network 22nd (eng. deep neural network) and a classifier 70 . The neural network 22nd preferably has three layers. The neural network 22nd can also be designed as a convolutional neural network. In addition, reinforcement algorithms with weightings can be used to reinforce the detection of extreme situations. In particular, it is planned to use Python as the programming language for the deep neural network 22nd because it is an easy-to-use neural network programming language and is currently a widely used language for machine learning applications. Since Python is open source software, a large number of libraries are available. This is of particular interest for very specific weather data formats. In addition, Python has different interfaces for several database systems. However, it can also be provided that programming languages such as C ++ or MATLAB are used.

2 shows the storage and processing of the individual data sources 12th , 14th , 16 and 17th in the data module 10 . For improved efficiency and increased computing speed, predictions are preferably carried out only once for all time-independent parameters. The results are then permanently stored, preferably in an internal relational database 18th such as a database defined by means of SQL (Structured Query Language). In contrast to this, time-dependent values remain in their respective format and only certain parts are read in with each request. As a relational database, SQL has the advantage of interactively processing structured data in tables. For example, the SQL client PostgreSQL offers the integration of spatial extensions via PostGIS. PostGIS is a very advanced and scalable concept for geographic information databases and is used in numerous projects, including Open Street Map (OSM) and Crash Forecasting. PostGIS saves geographic objects such as points, lines and polygons as hash keys. Different geographic operations are directly integrated by given functions.

In addition, the data module 10 preferably an auto-encoder 11 or generative opposing networks (GAN) for processing aerial photographs and other special image processing algorithms 19th to analyze scanned street images. The data module extracts from the data 10 relevant parameters, in particular using mathematical methods of principal component analysis. Preferably 439 parameters are extracted. To improve efficiency, a distinction is made between time-independent and static parameters.

3 shows the training of the neural network 22nd in the parameterization module 20th . For the Training the neural network 22nd can filter 23 the transmitted data must be provided. A vectorization 26th of the entered data is done in order to avoid inequalities in the data set. A normalization is carried out by using a normalizer 27 and is required to normalize the initial training set to minimum and maximum values. You can also use a gradient optimizer 24 and activation functions 25th be provided for a normalized data entry for training the neural network 22nd to enable.

The output format 28 of the neural network 22nd is preferably designed as a Softmax function, which leads to at least two-dimensional output between 0 and 1. The output is used as input to the classifier 70 passed on, which then classifies each input with the respective feature, either extreme situation or no extreme situation, depending on which probability is higher.

The implementation of the neural network 22nd is preferably done with TensorFlow. TensorFlow is an open source ML library available for various programming languages including Python and C ++. It has compilers for both central processing units (CPUs) and graphics processing units (GPUs).

After an initial training of the neural network 22nd with historical accident data 40 was carried out is an additional training with historical parameters, for example from field operation data, for example with data from the shadow module 50 , possible. This allows the neural network 22nd are further developed in such a way that extreme situations specific to driving are recognized for highly automated driving functions.

In the 4th the method steps of the method according to the invention are shown up to the output of the route to a user. The selection of the route is based on the parameterization of the road conditions obtained from the database 10 and the pre-trained deep neural network 22nd provided.

In a step S10, various parameters are stored in a data module 10 from several different data sources 12th , 14th , 16 , 17th extracted.

In a step S20, the parameters are from the data module 10 to the parameterization module 20th transmitted.

In a step S30, situation data, such as accident data in particular, are obtained from the accident module 40 and / or data from a shadow module 50 to the parameterization module 20th transmitted.

The parameterization module parameterizes in a step S40 20th Road routes and / or traffic situations with the received parameters and the transmitted accident data from the accident module 40 and / or data from a shadow module 50 .

In a step S50, the parameterization module transfers 20th the parameterized road routes and / or traffic situations to a classifier 70 and to an output module 30th .

In a step S60 the classifier extracts 70 relevant features from the parameterized road sections.

In a step S70 the classifier transfers 70 the extracted features to the output module 30th .

In a step S80, the output module calculates 30th Extreme driving situations for a predetermined geographical and / or temporal area.

In a step S90 the output module outputs 30th a route planning for the predetermined geographical and / or temporal area, which includes an optimized number of extreme situations.

With the method according to the present invention, a combination of different data sources for predicting extreme driving situations for highly automated driving functions is possible. The basis for this is accident data from government authorities, which can be called up with location details, time stamps and additional information on the course of the accident. In addition, individual program parts of the highly automated driving functions are executed in the background in a shadow mode in an independent test fleet of vehicles. In the event of malfunctions, the shadow mode determines the current driving situation and saves the data. Weather conditions and traffic density are also taken into account. These data are used to parameterize traffic situations in road traffic and / or road routes in relation to weather situations, traffic scenarios and infrastructure. A classification is then carried out in order to select features that characterize extreme scenarios. In an output module, extreme technical driving situations for highly automated driving functions are recognized by means of the parameterization and the extracted features, which enables the targeted planning of test drives with a high probability of malfunctions occurring. In addition, the results can be compared with manually planned test drives in order to be able to make a statement about the relevance of the determined extreme situations.

According to the present invention, improved test route planning is thus possible in which extreme situations in terms of driving technology occur in an accumulated manner.

List of reference symbols

10

Data module

11

Auto encoder

12th

cartographic data

14th

Traffic data

16

Aerial photos

17th

Weather information data

18th

relational database

19th

Image processing algorithms

20th

Parameterization module

22nd

neural network

23

Filtering

24

Gradient optimizer

25th

Activation functions

26th

Vectorization

27

Normalizer

28

Output format of the neural network

30th

Output module

40

historical accident data

50

Shadow module

70

Classifier

100

system

Claims (8)

Method for the automatic recording of extreme driving situations, in particular for highly automated driving functions, comprising the following process steps: - Extraction (S10) of various parameters from several different data sources (12, 14, 16, 17) in a data module (10), the data module (10) comprising a relational database (18), and cartographic information (12) as data sources , Data on traffic figures (14), aerial photographs (16) and / or data on weather information (17) and / or data from motor vehicles are used; - Transmission (S20) of the parameters from the data module (10) to a parameterization module (20), the parameterization module (20) comprising a deep neural network (22) for the extraction of parameters and pattern recognition; - Transmission (S30) of situation data, such as, in particular, accident data from an accident module (40) and / or data from a shadow module (50) to the parameterization module (20); - Parameterization (S40) of road sections and / or traffic situations with the received parameters and the transmitted accident data from the accident module (40) and / or data from a shadow module (50) from the parameterization module (20); - Transferring (S50) the parameterized road routes and / or traffic situations from the parameterization module (20) to a classifier (70) and to an output module (30); - Extracting (S60) relevant features from the parameterized road routes and / or traffic situations from the classifier (70); - Transferring (S70) the extracted features from the classifier (70) to the output module (30); - Calculation (S80) of extreme driving situations for a predetermined geographical and / or temporal area from the output module (30); - Output (S90) of a route plan for the predetermined geographical and / or temporal area, which includes an optimized number of extreme situations, from the output module (30). Procedure according to Claim 1 wherein the data module (10) has an auto-encoder (11) and / or generative opposing networks (GAN) and / or image processing algorithms (19). Procedure according to Claim 1 or 2 , training of the neural network (22) with historical parameters and / or historical accident data is provided. Method according to one of the preceding claims, wherein filtering (23) of the transmitted data by means of predetermined hyperparameters and / or a gradient optimizer (24) and / or activation functions (25) are provided for training the neural network (22) in order to normalize data input to ensure. System (100) for the automatic detection of extreme driving situations, in particular for highly automated driving functions, comprising a data module (100) which is used for extracting various parameters from several different data sources (12, 14, 16, 17) and transmitting the parameters to a The parameterization module (20) is configured; Situation data modules such as an accident module (40) that is configured to contain accident data to transmit to the parameterization module (20), and / or a shadow module (50) which is configured to transmit data to the parameterization module (20); wherein the parameterization module (20) is configured to parameterize road sections and / or traffic situations with the received parameters and the transmitted accident data from the accident module (40) and / or data from a shadow module (50) and to transfer the parameterized road sections and / or traffic situations to one Transferring the classifier (70) and to an output module (30); wherein the classifier (70) is configured to extract relevant features from the parameterized road routes and / or traffic situations and to transfer the extracted features to the output module (30); wherein the output module (30) is configured to calculate extreme driving situations for a predetermined geographical and / or temporal area and to output a route plan for the predetermined geographical and / or temporal area, which includes an optimized number of extreme situations; wherein the data module (10) comprises a relational database (18); cartographic information (12), data on traffic figures (14), aerial photographs (16) and / or data on weather information (17) and / or data from motor vehicles are used as data sources; and wherein the parameterization module (20) comprises a deep neural network (22) for extracting parameters and for pattern recognition. System (100) according to Claim 5 wherein the data module (10) has an auto-encoder (11) and / or generative opposing networks (GAN) and / or image processing algorithms (19). System (100) according to Claim 5 or 6th , training of the neural network (22) with historical parameters and / or historical accident data (40) being provided. System (100) according to one of the preceding Claims 5 - 7th , filtering (23) of the transmitted data by means of predetermined hyperparameters and / or a gradient optimizer (24) and / or activation functions (25) being provided for training the neural network (22) in order to ensure normalized data input.

Images