The invention relates to a system, methods and algorithms for controlling a vehicle and in particular an autonomously driving vehicle with the aid of a virtual environment, which is capable of a real-time 3D reconstitution of real traffic in a particular geographical area or a particular region. The virtual environment is transmitted to a viewed vehicle through an associated server and made available. The invention also relates to the creation and operation of this real-time 3D reconstitution of traffic in the server, which is connected on both sides with the individual road users and constantly receives data on the current traffic situation from at least some road users.
Vehicles are becoming increasingly intelligent and equipped with ever better wireless communication capabilities. They are increasingly controlled entirely or partially by computer. Satellite navigation systems have not only allowed automobiles to accurately determine their own location, but also enabled the vehicle user to determine the destination of a trip and drive to the destination.
The invention relates to the control of a considered vehicle based on the virtual environment. The considered vehicle is preferably autonomously controlled. At least one central server is used, to which the vehicle under consideration and a large number of other vehicles are connected. They are collectively referred to as connected vehicles. They and other road users transmit the information recorded by their sensors via e.g. Speed, location, driver assistance mode, drive scenario, other road users, traffic situation, etc. to the central server. This collects the data and merges it with other data, in particular map data, weather data, congestion information. He creates a virtual three-dimensional image of the real traffic and makes it available to the considered vehicle. The server can process additional information, in particular missing data, extended data and recommendations for activated automated functions in the vehicle under consideration and send this processed data to the vehicle under consideration. There is a two-way traffic at least between the vehicle considered and the central server. In a further enhancement, the individual vehicles may be equipped with only one module for broadband and high-fidelity connectivity, which transmits the status and position of the considered motor vehicle to the server. The server acts as a virtual sensor array and transmits the virtual sensor information back to the viewed vehicle.
The method can be used in particular for partially or fully automatic vehicles. The application is also possible for other vehicles and at all for any road user.
Developing and offering additional automated driving functions also requires customizing the control algorithms. In general, automated driving functions need to be able to interact with other road users in accordance with the rules of road traffic. Control algorithms are the building blocks of automated driving functions. Control algorithms are usually responsible for the longitudinal and lateral steering of the vehicle and work together with sensors that i.a. are arranged on the considered vehicle. Since there is a wide variety of different driving situations, e.g. City traffic, motorway traffic, country road, etc., further comprises a plurality of different vehicle sensors, e.g. Ultrasound, laser, radar, lidar, camera, etc., and taking into account the different ranges and areas in which and for the information needed, the control algorithms must also be able to work with incomplete information. Sensors are limited in detection, e.g. Radar and camera typically limited to less than 200 meters. In addition, for example, the vehicle under consideration, no matter how it is controlled, can not obstruct, e.g. recognize another vehicle when that obstacle is hidden behind a hill, another motor vehicle or building, and so on.
There are solutions based on V2X communication to reduce and potentially eliminate these limitations of the sensors of the subject motor vehicle and associated control algorithms. The use of such V2X communication is very demanding, it requires secure high-speed connections and also additional computing power in each vehicle, e.g. an extra CPU, additional memory. All this also means extra power consumption. Furthermore, the embedded control algorithms must be extended to handle additional information provided by the dynamic number of V2X channels (C2C, I2C ...).
From the DE 11 2012 004 771 T5
For example, a method and a vehicle data collection system are known in the traffic field. Described is a central server that is connected to a network, this is with different Sensors, connected to the vehicles of the vehicle fleet and information about traffic, road signs and the like. The publication describes systems and methods for detecting precise traffic situations and the traffic environment in which data is collected from the infrastructure, the vehicles, etc., and this data is aggregated and processed in the central server. The data can then be forwarded to the individual road users. This system is comparable to already existing information about the current traffic with the difference that the data are much more precise and also contain information for the route of the vehicle under consideration, eg arrival time, etc.
Out US 2014/0063064 A1
is a system known that provides navigation and driving information to so-called head-up displays or similar displays, this in real time. Methods for augmented reality are also described in order to match the images displayed on the displays with the real objects.
Out US 2016/0189544 A1
is a system similar to the one mentioned DE 11 2012 004 771 T5
The data for maps and navigation are improved in real time and these data are forwarded to the vehicle under consideration.
The object of the invention is to improve the previously known solutions and to provide a central platform that provides a virtual image of real traffic in road users, in real time. It should be worked more economical and easier than with, for example, V2X communication.
This object is achieved by a method for controlling a vehicle under consideration in a real traffic environment, which is in particular in autonomous driving, by means of a virtual traffic environment, for a geographical area on a server in real time as a 3-dimensional virtual representation of the current traffic situation the vehicle under consideration is generated and transmitted to this vehicle in real time, wherein the server is connected to a plurality of road users, including the vehicle under consideration, bidirectionally via a wireless communication system and at least some of the road users, including the vehicle under consideration equipped with sensors that detect the area around the respective sensor and transmit the corresponding information to the server, the existing map data and the information obtained, the 3-dimensional virtual representation of the current traffic situation generated.
The virtual environment can be used for the following purposes: It can be used as a virtual sensor for the vehicle under consideration, which is connected to the server that makes the virtual environment available. It also provides information about obstacles, infrastructure and the like, which the sensors of the considered vehicle alone do not provide. It transmits missing information to the sensors of the vehicle in question, e.g. over the line of sight and the field of view of a sensor, etc. It works as a virtual control algorithm for the vehicle under consideration. The virtual control algorithm processes the virtual environment control data for the vehicle under consideration and sends the result to that vehicle in the real world, e.g. via engine torque request, brake torque request, etc. The virtual controls may be used for redundancy, for example to validate a steering command to the vehicle under consideration, or may serve as a replacement for the real controls, particularly in a further development. Finally, the control algorithms of each vehicle may be provided with additional information to improve the accuracy of the control.
The virtual traffic environment is implemented for a specific geographic area. It can preferably be implemented centrally for a large geographic area such as a country, a region, a continent. However, it can also be implemented in a distributed architecture, with many small virtual transport platforms being implemented on virtual servers. In doing so, each traffic platform can be dynamically reconfigured for a smaller geographic area, depending on e.g. the current traffic density in order to improve the accessibility and accuracy of the service. This approach is preferred. It can e.g. more or less virtual traffic platforms are assigned to monitoring a city at traffic rush times than normal, for example at night time. Or the system can be configured to allow more overlaps between each virtual transport platform. A cellular structure as in mobile networks is possible and is sought.
An embodiment of the invention will be described in more detail below, this is done with reference to the drawing. The embodiment is not intended to be limiting, it serves to explain the invention. In this drawing show:
- 1 FIG. 4 is an exemplary block diagram for a system based on a virtual traffic environment. FIG.
- 2 : a schematic, three-dimensional representation of a traffic situation in the real world, represented is, inter alia, a viewed vehicle that wants to park,
- 3 : the representation according to 2 in the representation as a virtual traffic situation, a three-dimensional map and the position of the considered vehicle are used,
- 4 : the representation according to 3 However, now as a traffic situation in the real world, the observed vehicle recorded with its sensors the traffic situation, as it exists in the real world, it is carried out a so-called detection step,
- 5 : a representation like 4 , now in the representation as a virtual traffic situation as a virtual reconstitution following the detection step, using the information of the 4 .
- 6 : a representation like 5 However, as a representation of the traffic situation in the real world and now with currently added road users, namely a bicycle, which is located in the blind area of the considered vehicle, and a third vehicle, and
- 7 : a representation according to 6 in the virtual traffic environment after the bicycle and also the third vehicle were detected.
Out 1 the main components of the system can be seen. A server 20 houses a managed directory for storing and describing digital objects for a digital archive. The system has at least one server 20 It can work on multiple servers 20 run, which communicate with each other. A server connection unit 22 COMM allows the individual servers 20 communicate with each other and exchange data with the outside world. A virtual traffic environment 24 VIRTUAL is a three-dimensional graphic representation in real time of real world traffic, including roads, road users, obstacles, buildings, etc. in a given supervised geographical area. This real-time reconstitution results at least from the following information: The location of the vehicle under consideration via, for example, GPS, the sensor information of the vehicle under consideration, digital maps, etc.
In a further development of the virtual traffic environment 24 could be the virtual traffic environment 24 to be upgraded to an X-in-the-loop environment to increase the accuracy of the map to better predict the evolution of traffic in the virtual world, etc.
A digital mapping and 3D map 26 MAP covers the supervised geographical area and offers enough details and resolution. This can be merged with existing maps such as Google maps. The maps can be classified by region or can be queried from a dedicated database for a particular region. The maps can be continuously improved and updated by information obtained from satellites, drones, and other monitoring means.
A supervised geographic region is with 28 REGION presented. In a preferred embodiment of the invention, each individual virtual traffic platform is responsible for a particular geographic region 28 , This region 28 can either be dedicated to a virtual platform, the server 20 the virtual platform preferably physically in or near the region 28 is arranged. The region 28 can also be adapted dynamically, depending on the traffic situation, the time (traffic rush hours), the possible failure of another server, etc., to allow sufficient coverage over the entire traffic area. In this case, it is preferable to work with a magnifying glass function in order to be able to view and treat a traffic area that requires a high degree of attention, for example a critical intersection in a city, in a particularly intensive way. In one embodiment of the invention, the monitored geographical area that is assigned to a specific server is defined such that there is an overlap with monitored regions that are assigned to other servers.
A wireless communication system 30 RX TX is designed as a communication network with a very large bandwidth, it allows the connection of the server 20 with the individual road users 32 PARTIC in real time. For example, a 5G technique or higher-quality connection technology is used.
As a road user 32 Each participant is called who moves and interacts with the traffic. It can be a vehicle, a bicycle, a pedestrian, an animal or any other movable thing.
A communication unit 34 PCOMM of the individual road user 32 allows the road users equipped with it 32 , with the server 20 communicate and exchange data. It can be a smartphone of a pedestrian, a smart collar for a dog, a router for a vehicle or the like.
A road user sensor 36 SENSOR allows information about the traffic in the environment of the road user 32 in a narrower (eg up to 100 m) and a medium (eg up to 200 m) distance range to capture and retrieve .. It can also sensors 36 be used with other detection areas. Preferably, several sensors 36 used. Such a sensor 36 for a pedestrian can be a smart phone, smart glasses, smart watch or the like. A sensor of a vehicle may be a camera, lidar, radar, ultrasonic sensor, laser or the like. The information collected by the sensors becomes the server 20 transfer. This information may include geographic coordinates or position (eg, GPS), an image captured by the camera, for example, including a collection of three-dimensional echoes, such as lidar, radar, ultrasound, laser, relative to the location of other road users.
A road user control 38 CONTR is intended for all users who move motorized. It can also be provided for others. Users of non-motorized mobility solutions can use their communication device 34 be informed about possible dangers, events, critical situations, recommendations etc. In a preferred implementation of the invention, the elements describe the road user control 38 Control software of a vehicle that interacts with the virtual traffic plan form to perform at least some of the following activities: Virtual sensing (using the virtual traffic platform information to replace a missing sensor). Sensor extension (using information from the virtual transport platform, to improve the accuracy or range of a sensor). Virtual controls (using the virtual transport platform algorithm to replace a missing control algorithm aboard the considered vehicle). Expansion of the control using the virtual transport platform algorithm to improve the accuracy of embedded control software.
A module for road user monitoring 40 MONIT determines which virtual platform environment for a connected road user 32 relevant based on the location of that road user 32 , The virtual platform environment can be a virtual platform with which the road user 32 Can exchange data. When the road user 32 Located in an overlap area covered by at least two virtual traffic environments, a handover is prepared to provide a seamless connection of the road user 32 with the next virtual traffic environment, as is known in mobile networks. An algorithm for path prediction may help to determine which virtual environment best suits the expected, future location of the road user 32 suitable is.
A module for virtual control and support 42 MODULE represents the services provided by the virtual traffic environment to a road user 32 can be offered. Typically, the virtual environment can be used by road users 32 Provide remote or other far-reaching information that is generally endowed with the ability to perform near range or mid range detections. These services may be simple traffic information such as notification, advice, notification. Or they can be control information, such as virtual sensors (using the virtual transport platform information to replace a missing sensor). A sensor extension (using virtual platform information to improve the accuracy and / or range of a sensor). Virtual control with algorithm of the virtual transport platform to replace a missing control algorithm aboard the considered vehicle or traffic participant). Control an extension (using the virtual transport platform algorithm to improve the accuracy of embedded control software).
The following embodiment relates to a virtual assistance control for a parking operation. For this purpose, the system according to 1 used. 2 shows a scene in the real world. A considered vehicle 50 wants to get in behind a curb 52 space between two parked vehicles 54 . 56 Read Maps. There are buildings 60 on the side of this parking.
The sequence of 2 to 7 shows step by step how the 3-dimensional virtual representation of the current traffic situation is generated.
The considered vehicle 50 is with the virtual transport platform (server 20 ) and transmits its coordinates and other information. The virtual platform has access to a high-resolution three-dimensional map 26 of the geographical area 28 , In this map is also the presence of a 10 cm high curb 52 The map also contains information about a two-lane road and some buildings, including the buildings 60 etc .. The virtual Traffic platform, however, knows the two parked vehicles 54 . 56 not because they are not equipped with connectivity. They are passive, they are not constantly present on the site.
3 shows the three-dimensional reconstitution of the virtual transport platform.
In the real world the considered vehicle activates 50 First, a detection mode, it performs a so-called detection phase. It tries to identify a suitable parking space. Among other things, a number of echoes (represented by stars) of the signals emitted by the vehicle under consideration are detected when the vehicle under consideration 50 the position 58 has reached, this is in 4 shown. The considered vehicle 50 that continues to be in position 58 has the parking bay 62 recorded and identified. Information during the acquisition phase is transmitted to the virtual transport platform. This adds the parked vehicles 54 . 56 as virtual objects in their representation. During the acquisition phase, the vehicle in question transmits 50 his current position 58 ,
The advanced three-dimensional virtual reconstitution of the environment is in 5 shown. At this stage, the virtual traffic environment knows the properties and position of the buildings 60 as well as the curb 52 , In addition, the position 58 of the vehicle under consideration. This information could already be given to the system for parking the vehicle under consideration in order to refine the planning of the trajectory (eg to specify a certain engine speed around the curb 52 to roll over in a gentle and secure manner).
In the real world, the parking system (parking assistant) now starts a maneuver with improved trajectory and control. Exactly during this process, however, is approaching a third vehicle 64 , This captures a bike with the help of its own camera sensor 66 , which also approaches, but in the blind area of the considered vehicle 50 located. Without a reaction of the considered vehicle 50 There could be a collision with the bike 66 come. The situation in the real world is in 6 shown.
The third vehicle 64 sends its camera information and sends more information to the virtual transport platform. This realizes the bike 66 and position it accordingly in the three-dimensional reconstitution. The virtual traffic platform could now even determine the risk of a collision and an immediate brake command to the vehicle under consideration 58 Send this with a notification to stop the maneuver. Thereafter, she could issue a command to continue parking to the vehicle under consideration 50 send as soon as the bike 66 is outside a collision area. Alternatively, the virtual traffic platform but could also control the parking of the considered motor vehicle 50 the position and the trajectory of the bicycle 66 to transfer. Then their control algorithm can include the new information in their own strategy and decide for themselves whether the parking maneuver is stopped until the bike 66 has passed or not.
7 illustrates the virtual traffic reconstitution including the third vehicle 64 and the bike 66 as well as the current position 58 of the considered vehicle 50 which were all transferred to the virtual transport platform.
The method and the algorithms have at least the following parts: a hosting server, a high-speed communication system, at least one road user and a collection of virtual services and virtual support from the virtual environment to the road user. The methods and algorithms for implementing a virtual traffic environment on a hosting server have in particular: A server communication unit for data exchange with the outside world, a real-time SD graphic reconstruction of the real world traffic (road, building, obstacle, road users ...) the hosting server has access to a database of 3D high resolution cartography or map and the hosting server is assigned a geographic area for which it is to reconstitute real world traffic.
The methods and algorithms for interacting with road users preferably have at least one, preferably more of the following features:
- The road users are connected to the server by means of intelligent devices, for example smartphones, routers, and are moving in traffic, for example a vehicle, a bicycle, a pedestrian, an animal.
- - The road users are equipped with at least one sensor, which is designed at least for the monitoring of a short-haul and / or a medium range region around the road users.
- - The road users continuously transmit the following information to the virtual traffic environment:
- a) Your position or geographic coordinate (eg GPS).
- b) The information collected by the sensors (camera, lidar, radar, laser, ultrasound ...).
- c) information related to their condition when available (eg speed, direction of movement, greed, inclination, role ...).
The methods and algorithms for monitoring communication between road users and hosting servers preferably have the following features:
- - The road users are assigned to at least one hosting server based on their location.
- A handover procedure makes it possible to change the hosting server currently assigned to a road user to another server of another geographic region when the road user leaves a certain region.
The methods and algorithms for real-time reconstitution of real traffic in a virtual environment preferably have the following features:
- The reconstitution is based on at least one, preferably some high-resolution 3D maps.
- - The reconstitution depends on the location of the connected road users.
- - Reconstitution takes place on the basis of the sensor information transmitted by the connected road users.
The methods and algorithms for defining a monitored region by a hosting server preferably have at least one, but in particular more of the following features:
- - The hosting server monitors a fixed, predefined geographic area
- The region monitored by a hosting server may overlap, for example a maximum of 5%, in particular a maximum of 2% of the monitored area, with the monitored area of another hosting server.
- - The geographic scope can be dynamically assigned to a hosting server according to the following criteria:
- i) The hosting server is located within or near the monitored area
- ii) The date and time (eg peak traffic times) are used as a criterion
- iii) traffic density is used as a criterion (eg assigned to a server with a high traffic density and congestion)
- iv) A prediction based on historical data is used as a criterion.
The methods and algorithms in order to provide the traffic users with virtual control buttons from the virtual traffic environment preferably have at least one, but in particular more of the following features:
- The functionality can be virtual sensing (with the information of the virtual traffic platform to replace a missing sensor of the road user)
- The functionality may be sensor expansion (with the information of the virtual traffic platform to improve the accuracy / range of a sensor of the road user).
- The functionality may be virtual control (with the virtual transport platform algorithm to replace a missing on-board control algorithm of the road user).
- The functionality may be extension of the control (with algorithm of the virtual transport platform to improve the accuracy of an embedded control software of the road user).
The control of a considered vehicle 50 in a real traffic environment, which is in particular in autonomous driving, takes place by means of a virtual traffic environment 24 representing a geographical area on a server 20 created in real time. A thereby obtained 3-dimensional virtual representation of the current traffic situation around the regarded vehicle 50 around it is transmitted in real time. The server 20 is with a variety of road users 32 to which also the considered vehicle 50 belongs, bidirectionally over a wireless communication system 30 connected. At least some of the road users 32 , including the vehicle considered 50 , are with sensors 36 equipped, which covers the area around the respective sensor 36 capture and the appropriate information to the server 20 transmit, which generates from existing map data and the information obtained a 3-dimensional virtual representation of the current traffic situation.
LIST OF REFERENCE NUMBERS
- Server connection unit
- virtual traffic environment
- Digital mapping and 3D map
- relevant geographic region covered
- Wireless communication system
- road users
- Communication unit (of the individual road user)
- Road users Sensor
- Road users control
- Road users monitoring
- Module for virtual control and support
- considered vehicle
- parked vehicle
- parked vehicle
- actual position
- Park bay
- third vehicle
QUOTES INCLUDE IN THE DESCRIPTION
This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.
Cited patent literature
- DE 112012004771 T5 [0007, 0009]
- US 2014/0063064 A1 
- US 2016/0189544 A1