DE102020117423A1 - System and method for scene-based route planning for a motor vehicle - Google Patents

Abstract

The invention relates to a method for scene-based route planning for a motor vehicle based on at least one selected scenario, a class of possible scenarios being predefined and stored in a database and a specific scenario being determined by a combination of several properties, comprising the following method steps:- receiving data and images captured by detectors and sensors and associated with geographic coordinates and other metadata from an extraction module;- extracting properties of possible scenarios from the transmitted data in the extraction module;- transmitting the properties from the extraction module to a classification module;- assigning the properties to one or more of the scenarios and determining the appropriate scenario from the classification module;- storing the identified scenario in a storage module;- selecting at least one scenario ums from an output module;- calculating a route from an output module in a given geographic area such that an optimized number of identified scenarios for the selected scenario is provided on the route;- presenting the route with the identified scenarios for the selected scenario on a digital map.

Description

The invention relates to a system and method for scene-based route planning for at least one moving object, in particular a motor vehicle based on at least one selected scenario type, a class of possible scenario types being predefined and stored in a database and a specific scenario type is determined by a combination of several properties.

The trend towards driver assistance systems and highly automated driving functions in motor vehicles, but also unmanned aircraft (drones) or watercraft, requires extensive security strategies, since the responsibility for driving the vehicle no longer lies entirely with the driver, but active functions are taken over by computer units in the vehicle. In the case of an unmanned object, a driver is also not provided at all. It must therefore be ensured that objects moving autonomously only have a very low error rate. This requires targeted testing of both extreme situations (corner cases) and everyday situations. Such extreme situations result from a special combination of different factors. Examples of this are infrastructural features such as the type of street, the development on the edge of a street, the quality of the markings, but also environmental conditions such as weather conditions, the time of day and the season. In addition, the behavior of other road users is important. In the case of aircraft and watercraft, the geographic topography and weather conditions play a particularly important role.

However, all possible types of scenarios that occur are sometimes difficult to find in the real world, especially since some events and situations rarely occur in the real world. A simulation of all possible scenario types on a test site provided for this purpose cannot be achieved with a reasonable amount of effort.

It is therefore desirable to be able to consider and specifically select certain scenario types such as motorway bridges, turning lanes, tunnels, roundabouts or also specific visibility conditions at dusk or in sunlight or certain geographic topography conditions for test drives of moving objects.

The object on which the invention is based is thus to create a method and a system for planning a route, in particular for test purposes, for at least one moving object, such as in particular a motor vehicle, which comprehensively takes into account possible scenario types that may occur along a route .

According to the present invention, a method is proposed by means of which a scene-based route planning is made possible in order, for example, to create the basis for the provision of driver assistance systems and highly automated driving functions and / or flight situations that occur both in everyday situations and in extreme driving situations are characterized by a high level of security and reliability.

According to the invention, this object is achieved with regard to a method by the features of claim 1, and with regard to a system by the features of claim 8. The further claims relate to preferred embodiments of the invention.

• - Empfangen von Daten und Bildern, die von Detektoren und Sensoren erfasst wurden und denen geographische Koordinaten und andere Metadaten zugeordnet sind, und/oder von Daten und Bilder aus anderen Datenquellen von einem Extraktionsmodul;
• - Extrahieren von Eigenschaften von möglichen Szenarien-Typen aus den übermittelten Daten und Bildern in dem Extraktionsmodul;
• - Übermitteln der Eigenschaften von dem Extraktionsmodul an ein Klassifizierungsmodul;
• - Zuordnen der Eigenschaften zu ein oder mehreren der vordefinierten Szenarien-Typen und Identifizieren des jeweils passenden Szenarium-Typs von dem Klassifizierungsmodul;
• - Speichern des identifizierten realen Szenariums mit den zugehörigen geographischen Koordinaten und/oder weiteren Metadaten in einem Speichermodul;
• - Auswählen zumindest eines Szenarium-Typs von einem Ausgabemodul;
• - Berechnen einer Route von einem Ausgabemodul in einem bestimmten geographischen Gebiet derart, dass zumindest eine optimierte Anzahl von identifizierten realen Szenarien für den zumindest einen ausgewählten Szenarium-Typ auf der Route vorgesehen ist;
• - Darstellen der Route mit den identifizierten Szenarien für den zumindest einen ausgewählten Szenarium-Typ auf einer digitalen Karte.

According to a first aspect, the invention relates to a method for scene-based route planning for at least one motor vehicle based on at least one selected scenario type, a class of possible scenario types being predefined and stored in a database and a scenario type being a combination of several Properties is determined, comprising the following process steps:

• - Receipt of data and images that have been recorded by detectors and sensors and to which geographic coordinates and other metadata are assigned, and / or of data and images from other data sources from an extraction module;
• - Extraction of properties of possible scenario types from the transmitted data and images in the extraction module;
• - Transmission of the properties from the extraction module to a classification module;
• - Assignment of the properties to one or more of the predefined scenario types and identification of the respectively appropriate scenario type from the classification module;
• - Storing the identified real scenario with the associated geographic coordinates and / or further metadata in a storage module;
• - Selecting at least one scenario type from an output module;
• - Calculating a route from an output module in a specific geographical area such that at least one optimized Number of identified real scenarios for the at least one selected scenario type is provided on the route;
• - Displaying the route with the identified scenarios for the at least one selected scenario type on a digital map.

In a further development it is provided that the digital map is displayed on an output medium or a screen of the output module.

Cartographic information, data on traffic figures, aerial photographs and / or data on weather information and / or data from a shadow module are advantageously used as further data sources.

In an advantageous further development, a user interface for communication with the output module is provided.

In particular, a scenario type represents a street scene and / or a traffic situation and / or a geographical topography situation.

In a further embodiment, the classification module has algorithms for classification.

The output module advantageously includes route planning and navigation algorithms.

According to a second aspect, the invention relates to a system for automated route planning for a motor vehicle based on one or more selected scenario types, a class of possible scenario types being predefined and stored in a database and a scenario type being a combination of several Properties is determined. The system comprises an extraction module which is designed to receive data and images that have been recorded by detectors and sensors and to which geographic coordinates and other metadata are assigned, and / or of data and images from other data sources and to receive properties of possible scenario types extract from the transmitted data and images; a classification module to which the extraction module transmits the properties and which is designed to assign the properties to one or more of the predefined scenario types and to identify the respectively appropriate scenario type, a storage module being provided, the identified real scenario with the associated geographical Save coordinates and / or other metadata; and an output module which is designed to select at least one scenario type and to calculate a route in a specific geographical area such that at least an optimized number of identified real scenarios for the at least one selected scenario type is provided on the route, and to display the route with the identified real scenarios on a digital map.

In a further development it is provided that the digital map is displayed on an output medium or a screen of the output module.

In a further embodiment, cartographic information, data on traffic figures, aerial photographs and / or data on weather information and / or data from a shadow module are used as further data sources.

A user interface for communication with the output module is advantageously provided.

In particular, a scenario type represents a street scene and / or a traffic situation and / or a geographical topography situation.

In a development it is provided that the classification module has algorithms for classification.

In a further embodiment, the output module includes route planning and navigation algorithms.

According to a third aspect, the invention provides a computer program product comprising an executable program code which is configured such that it executes the method according to the first aspect when it is executed.

The invention is explained in more detail below with reference to the exemplary embodiments shown in the drawing.

• 1 ein Blockdiagramm zur Erläuterung eines Ausführungsbeispiels eines erfindungsgemäßen Systems;
• 2 eine Darstellung einer digitalen Karte mit ausgewählten realen Szenarien;
• 3 eine weitere Darstellung einer digitalen Karte mit ausgewählten realen Szenarien;
• 4 ein Flussdiagramm zur Erläuterung der einzelnen Verfahrensschritte des erfindungsgemäßen Verfahrens;
• 5 schematisch ein Computerprogrammprodukt gemäß einer Ausführungsform des dritten Aspekts der Erfindung.

It shows:

• 1 a block diagram to explain an embodiment of a system according to the invention;
• 2 a representation of a digital map with selected real scenarios;
• 3 another representation of a digital map with selected real scenarios;
• 4th a flow chart to explain the individual method steps of the method according to the invention;
• 5 schematically a computer program product according to an embodiment of the third aspect of the invention.

Additional features, aspects and advantages of the invention or its exemplary embodiments will become apparent from the detailed description in conjunction with the claims.

In order to enable automated route planning with selected scenario types on a traffic route, a class of scenario types is defined according to the invention and recorded in a database or file. A scenario type is understood to mean, for example, a certain scene such as a bridge, a tunnel, a multi-lane roundabout or traffic lights on the motorway. This class of scenario types can be dynamically supplemented and adapted at any time, in particular by a system administrator.

What these scenario types have in common is that they can be characterized by certain features that are each assigned to a scenario type. Conversely, a scenario type can be determined by combining certain properties. Examples of properties are the signage or the road infrastructure such as the type of road surface, the number of lanes, and possible construction sites. The scenario type of a bridge is determined by properties such as a railing to the left and right of the road and a visible transition on the road. The scenario type of a motorway tunnel can be characterized by properties such as ceiling lights, a wall with doors and an overall dark field of view.

1 shows a system 100 according to the invention for scene-based route planning for at least one moving object, which is based on the selection of predefined scenario types. The moving object can be a motor vehicle such as a passenger vehicle or a truck, an aircraft or a watercraft, the aircraft being in particular unmanned aircraft (threats) and the watercraft being in particular ships, boats and ferries . In order to be able to plan a traffic route that should have certain scenario types, such as a high number of bridges or tunnels, street scenes and / or traffic situations or geographical topographies are recorded with various detectors 10 and sensors 12 according to the invention. The detectors 10 and sensors 12 are preferably one or more cameras that depict a specific scene and / or a traffic situation. But it can also be acoustic sensors, LIDAR systems or radar systems. The data therefore preferably consist of sensor data and camera images. The images and data can be recorded by test objects such as test vehicles or test drones, which travel or fly a certain route and record this route with the detectors 10 and sensors 12. The data and images are stored either in corresponding storage units of the detectors 10 and sensors 12 or in external storage modules. In particular, the captured data and images are stored with geographic coordinates, the time of data capture and other metadata.

In connection with the invention, a “memory unit” or “memory module” and the like can mean, for example, a volatile memory in the form of a random access memory (RAM) or a permanent memory such as a hard disk or a data carrier or, for example, an exchangeable memory Storage module or a cloud-based storage solution.

In connection with the invention, a “module” can be understood to mean, for example, a processor and / or a memory unit for storing program commands. For example, the processor and / or the control unit is specially set up to execute the program commands in such a way that the processor and / or the control unit executes functions in order to implement or realize the method according to the invention or a step of the method according to the invention.

The data can, however, also be recorded by a shadow module 20 in a vehicle. The shadow module 20 has its own data acquisition systems 22 or is connected to the detectors 10 and sensors 12.

The shadow module 20 triggers the data acquisition systems 22 and / or detectors 10 and sensors 12 by means of background functions in a moving vehicle and therefore works in a so-called shadow mode. The vehicles can be test vehicles or customer vehicles. This also applies in the case of aircraft or watercraft. In the shadow module 20, triggers are provided that respond to a specific behavior of individual components of the vehicle. For example, exceeding certain acceleration values can trigger a trigger so that the corresponding data acquisition systems 22 then acquire a street scene or a traffic situation. The from detectors 10, sensors 12 and data The data and images recorded by the detection systems 22 are either stored in a data memory of the detectors 10 and sensors 12 or transmitted directly to the shadow module 20. This recorded data and images are also saved with spatial coordinates, the time of data recording and other metadata.

In addition, further data sources 30 or databases can be used. These include, in particular, databases that contain data on the road network with road specifications such as lanes and bridges, the road infrastructure such as the road surface, the peripheral development, the road layout and are made available by authorities. But the nationwide database for recording all accidents with serious injuries, which is made available by the Federal Statistical Office, is an important source of data. There are similar data sets in many other countries.

In addition, traffic figures such as the real hourly traffic volume at a certain traffic scene are of interest for certain types of scenarios such as a traffic jam. The data are made freely accessible by the Federal Highway Research Institute.

Another data source 30 are aerial photographs, for example from Google Maps. Mapillary can also be used for street images. Mapillary collects geo-tagged user-generated street images recorded by dashcams and smartphones. These images are available under an open source license.

Since the weather conditions can also define a scenario type, weather data are a further data source 30. Weather data include historical weather measurements and future weather forecasts.

The calculation and storage of all geographic objects is preferably carried out in the EPSG 25832 coordinate system (Universal Transverse Mercator (UTM) Zone 32N). This system is also used by authorities in Germany. The lateral and longitudinal positions are shown in meters. In addition, global reference systems such as the “World Geodetic System 1984 (WGS 84)”, which is also used in GPS receivers (Global Positioning System), can be used. For example, the entire content of the map can be imported from Germany.

The data are passed on from the detectors 10, sensors 12, the shadow module 20 and the other data sources 30 to an extraction module 50. The extraction module 50 extracts properties from the data which are passed on to a classification module 60. The classification module 60 has algorithms for classification and assigns one or more suitable scenario types S1, S2,..., Sn to the properties. These scenario types S1, S2,..., Sn were predefined and are stored in a database 65.

An identified real scenario Sj (x, y, z) is stored in a memory module 70 together with the geographical position and other metadata such as the time of the data acquisition.

An output module 80 has route planning and navigation algorithms and can access the memory module 70. The output module 80 is connected to a user interface 90, via which a user can enter a scenario type that he wishes. The output module 80 then calculates a route that comprises an optimized number of identified real scenarios Sj (x, y, z) for the selected scenario type Sj in the form of a route planning with suggested navigation and time information. Several scenario types Sj, Sk can also be entered, so that the output module 80 then calculates a route that contains an optimized number of identified real scenarios Sj (x, y, z), Sk (x, y, z) for the selected scenario types Sj, Sk. It is thus possible to select and filter certain types of scenarios, such as right-turn lanes in a city or a motorway with many bridges. It is also possible for the output module 80 to propose several alternative routes, each of which has a slightly different number of identified real scenarios Sj (x, y, z).

The route with the identified real scenarios Sj (x, y, z) for the selected scenario type Sj (or the selected scenario types) in a specific geographical area is then displayed on an output medium or a screen 85 of an output module 80 geographical position shown on a digital map 82. In 3 a digital map 82 is shown with the scenario types of a tunnel, the occurrence of backlighting and a bridge.

The screen 85 of the output module 80 is preferably the screen of the navigation system of the vehicle. However, it is also conceivable that a mobile terminal of the user such as a smartphone is used alone or in addition as output module 80 and user interface 90. It is also conceivable that a computer is used for is provided as an output module 80 in a test and development center, for example.

In addition, additional information can be displayed for individual real scenarios, for example the length of a tunnel or a bridge or the possibility of backlighting in the appropriate weather conditions. Images of the real scenarios, for example a bridge, can also be displayed on the screen 85. In the 4th a digital map 82 is shown with images of the real scenarios of a tunnel and the occurrence of backlighting.

According to the invention, real scenarios can thus be found for certain selected scenario types Sj and displayed on a digital map 82. In this way, route planning can be simplified in which certain types of scenarios are to be run or flown.

A further development of the invention provides for the identified real scenarios to be checked regularly in order to determine changes. For example, a level crossing previously marked as unrestricted may have been provided with a level crossing so that the scenario needs to be updated.

It can also be provided that certain scenario types are only permitted for certain objects, since they are only of interest for these.

In the 4th the method steps of the method according to the invention are shown up to the output of a traffic route to a user.

In a step S10, data and images captured by detectors 10 and sensors 12, with the data and images each being assigned geographic coordinates, and / or data and images from other data sources 30 are received by an extraction module 50.

In a step S20, properties of possible scenario types are extracted from the transmitted data in the extraction module 50.

In a step S30, the properties are transmitted from the extraction module 50 to a classification module 70.

In a step S40, the classification module 70 assigns one or more predefined scenario types S1, S2,..., Sn to the extracted properties and identifies at least one suitable scenario type Sj.

In a step S50, the identified real scenario Sj (x, y, z) with the associated geographic coordinates and further metadata is stored in the memory module 70.

In a step S60, the output module 80 selects at least one scenario type Sj.

In a step S70, the output module 80 calculates a route in a specific geographical area in such a way that an optimized number of identified real scenarios Sj (x, y, z) for the selected scenario type Sj is provided on the route.

In a step S80, the route with the identified real scenarios (Sj, x, y, z) for the selected scenario type is displayed on a digital map 82.

5 FIG. 3 schematically illustrates a computer program product 200 comprising executable program code 205 which is configured to carry out the method according to the first aspect of the present invention when it is carried out.

The method according to the present invention enables route planning based on the selection of predefined scenario types. In order to find possible scenario types in a geographic area such as, for example, along a certain road, data and images are determined in the geographic area, the exact geographic coordinates being assigned to the data and images. Specific properties are extracted from the images and data. A combination of one or more properties each defines a certain predefined scenario type. The real scenarios determined in this way at the corresponding geographic locations are then displayed on a digital map 82. This enables scene-based route planning for at least one moving object such as a motor vehicle, an aircraft or a watercraft. In particular, it can be provided to carry out route planning for a fleet of moving objects, it being possible for the individual objects of the fleet to travel or fly off different routes from a starting point to a destination point.

List of reference symbols

10

detector

12th

sensor

20th

Shadow module

30th

Data Source

50

Extraction module

60

Classification module

65

Database

70

Memory module

80

Output module

82

digital map

85

screen

90

User interface

100

system

200

Computer program product

205

Program code

Claims (15)

Method for scene-based route planning for at least one moving object, such as a motor vehicle in particular, based on at least one selected scenario type (Sj), a class of possible scenario types (S1, S2, ..., Sn) being predefined and in a database (65) is stored and a specific scenario type (Si) is determined by a combination of several properties, comprising the following method steps: - Receiving (S10) of data and images that have been recorded by detectors (10) and sensors (12) and to which geographic coordinates and other metadata are assigned, and / or of data and images from other data sources (30) from an extraction module ( 50); - Extracting (S20) properties of possible scenario types (S1, S2, ..., Sn) from the transmitted data and images in the extraction module (50); - Transmission (S30) of the properties from the extraction module (50) to a classification module (60); - Assigning (S40) the properties to one or more of the predefined scenario types (S1, S2, ..., Sn) and identifying the respectively appropriate scenario type (Sj) from the classification module (60); - Saving (S50) the identified real scenario (Sj (x, y, z)) with geographic coordinates and / or further metadata in a memory module (70); - Selecting (S60) at least one scenario type (Sj) from an output module (80); - Calculating (S70) a route from an output module (80) in a specific geographical area such that an optimized number of identified real scenarios (Sj (x, y, z)) for the at least one selected scenario type (Sj) the route is planned; - Representation (S80) of the route with the identified real scenarios (Sj (x, y, z)) for the at least one selected scenario type (Sj) on a digital map (82). Procedure according to Claim 1 wherein the digital map (82) is displayed on an output medium or screen (85) of the output module (80) Procedure according to Claim 2 wherein cartographic information, data on traffic figures, aerial photographs and / or data on weather information and / or data from a shadow module (20) are used as further data sources (30). Method according to one of the preceding claims, wherein a user interface (90) is provided for communication with the output module (80). Method according to one of the preceding claims, wherein a scenario type (Sj) represents a street scene and / or a traffic situation. Method according to one of the preceding claims, wherein the classification module (60) has algorithms for classification. Method according to one of the preceding claims, wherein the output module (80) comprises route planning and navigation algorithms. System (100) for scene-based route planning for at least one motor vehicle based on at least one scenario type (Sj), a class of possible scenario types (S1, S2, ..., Sn) being predefined and stored in a database (65) and a specific scenario type (Si) is determined by a combination of several properties; comprising an extraction module (50) which is designed to extract data and images that have been acquired by detectors (10) and sensors (12) and to which geographic coordinates and other metadata are assigned, and / or of data and images from other data sources (30 ) to receive and to extract properties of possible scenario types (S1, S2, ..., Sn) from the transmitted data and images; a classification module (60) to which the extraction module transmits the properties and which is designed to assign the properties to one or more of the predefined scenario types (S1, S2, ..., Sn) and the respectively appropriate scenario type (Sj) to identify, wherein a memory module (70) is provided to store the identified real scenario (Sj (x, y, z)) with the associated geographic coordinates and / or further metadata; and an output module (80) which is designed to select at least one scenario type (Sj) and to calculate a route in a specific geographical area in such a way that at least an optimized number of identified real scenario types (Sj (x, y, z)) are provided for the at least one selected scenario type (Sj) on the route, and to display the route with the identified real scenarios (Sj (x, y, z)) on a digital map (82). System (100) according to Claim 8 wherein the digital map (82) is displayed on an output medium or screen (85) of the output module (80) System (100) according to Claim 8 or 9 wherein cartographic information, data on traffic figures, aerial photographs and / or data on weather information and / or data from a shadow module (20) are used as further data sources (30). System (100) according to one of the preceding Claims 8 - 10 , wherein a user interface (90) is provided for communication with the output module (80). System (100) according to one of the preceding Claims 8 - 11th , whereby a scenario type (Sj) represents a street scene and / or a traffic situation and / or a geographical topography situation. System (100) according to one of the preceding Claims 8 - 11th wherein the classification module (60) has algorithms for classification. System (100) according to one of the preceding claims, wherein the output module (80) comprises route planning and navigation algorithms. Computer program product (200) having executable program code (205) which is configured, when executed, to perform the method according to Claim 1 to execute.

Images