EP4006870A1 - Method, vehicle and system for automated warning of other road users about hazard objects - Google Patents

Abstract

The invention relates to a method and a system for automatically warning other road users of a dangerous object, in particular a dangerous object related to road traffic, by a first vehicle, the first vehicle having suitable electro-optical sensors for detecting object data of the vehicle environment, a suitable On-board computer for evaluating the object data and a transceiver, the method having the following steps carried out by the first vehicle: • Receiving sensor data from the vehicle environment of the first vehicle, the sensor data being measured by means of at least one first sensor unit of the first vehicle; • Evaluating the received sensor data by an algorithm implemented on the on-board computer of the first vehicle and classification of objects based on the sensor data, the algorithm generating a danger object marking when one of the objects is classified as a dangerous object, wherein the dangerous object marking is represented by a data tuple G, which comprises at least the local position P_GO of the dangerous object, ie G(P_GO);• sending of the data tuple G(P_GO) as a broadcast signal.

Description

The invention relates to a method, vehicles and a system for automatically warning other road users of dangerous objects.

Modern cars these days are mostly equipped with sensors that should at least assist the driver while driving. The transport industry is also preparing for semi-autonomous or fully autonomous driving. For a vehicle to be able to drive completely autonomously, for example, it must be able to assess traffic just as well as humans. For this purpose, some vehicles already have several different sensors that can scan the roadway in different ways. It is known that different sensor types have specific advantages and disadvantages, so that in particular a clever combination of the data collected from different sensor types is able to compensate for the respective weaknesses of a single sensor type. Current autonomous test vehicles often orient themselves using data from the following sensor units: radar, optical cameras, infrared cameras, ultrasound and laser scanners (lidar).

The detection properties of the sensor units are briefly outlined below.

For example, lidar can be adversely affected by weather conditions, but has good low resolution, high accuracy and a range of up to 200m.

For example, radar is not heavily influenced by lighting or weather conditions, has good depth resolution, medium accuracy and a long range of around 250 m.

Ultrasound, for example, is not heavily influenced by lighting or weather conditions, has good depth resolution, but only a very short range.

The optical camera has characteristics that most closely resemble those of a human eye. For this reason, the camera is affected by strong lighting and weather, but can detect colors. In principle, the optical camera also has a very long range, which, however, depends on how much light the object emits, since the optical camera is a passive sensor.

An infrared camera is particularly well suited to detecting different heat profiles.

Despite all these different sensor units, it still happens, especially during test drives with test vehicles, that they do not correctly assess the dangers in road traffic, resulting in accidents. It is quite possible that a vehicle from one manufacturer correctly assesses a risk situation, whereas a vehicle from another manufacturer incorrectly assesses the same risk situation. Sometimes this can even occur in vehicles from the same manufacturer.

It is therefore the object of the invention to provide a method and vehicles that make it possible to better assess dangerous situations, so that accidents in particular can be avoided.

This problem is solved with the features of the independent claims.

The features of the various aspects of the invention described below or of the various exemplary embodiments can be combined with one another unless this is explicitly ruled out or is technically imperative.

• Empfangen von Sensordaten von der Fahrzeugumgebung des ersten Fahrzeugs mittels zumindest einer ersten Sensoreinheit des ersten Fahrzeugs;
  ∘ die Sensordaten können hierbei von dem Fahrzeug entweder in einer 360° Rundumsicht oder in Fahrtrichtung erfasst werden. Die 360° Rundumsicht bietet den Vorteil, dass auch Sensordaten erfasst werden können, die nicht in Fahrtrichtung des Fahrzeugs liegen, wobei eine Erfassung der Sensordaten nur in Fahrtrichtung eine Reduzierung des Rechenaufwands darstellt. Die Fahrtrichtung ist als besonders wichtige Richtung ausgezeichnet ist. Die Sensordaten können hierbei von einer einzelnen Sensoreinheit stammen, es können aber auch Daten von mehreren Sensoreinheiten, insbesondere den vorstehend beschriebenen Sensoreinheiten, stammen und zu Gesamtdaten kombiniert werden. Die Gesamtdaten weisen eine höhere Informationsdichte auf und stellen demnach eine umfassendere Beschreibung der Fahrzeugumgebung dar, erhöhen aber auf der anderen Seite den Rechenaufwand und den Detektionsaufwand.
• Auswerten der empfangenen Sensordaten durch einen auf dem Bordcomputer des ersten Fahrzeugs implementierten Algorithmus und Klassifizierung von Objekten anhand der Sensordaten, wobei der Algorithmus bei der Klassifizierung eines der Objekte als Gefahren-Objekt eine Gefahren-Objektmarkierung generiert, wobei die Gefahren-Objektmarkierung durch ein Daten-Tupel G repräsentiert wird, das zumindest die örtliche Position P_GO des Gefahren-Objekt umfasst, also G(P_GO);
  ∘ der Algorithmus zum Auswerten der Sensordaten kann insbesondere ein neuronales Netzwerk umfassen, dass durch Gefahren-Objekte trainiert worden ist. Der Index "GO" steht für "Gefahren-Objekt". Die örtliche Position des Gefahren-Objekts kann auf standardisierten Koordinaten (x, y, z), wie etwa dem GPS-System, beruhen. Dies hat den Vorteil, dass alle Fahrzeuge die Positionsangabe in standardisierten Koordinaten gleich interpretieren. Eine Alternative oder zusätzliche Möglichkeit ist es, die örtliche Position des Gefahren-Objekts als Position des Fahrzeugs und einem Richtungsvektor anzugeben, wobei der Richtungsvektor von den Koordinaten des Fahrzeugs auf das Gefahren-Objekt hinzeigt. Ein solcher Richtungsvektor kann von dem Algorithmus des Fahrzeugs errechnet werden. Dies hat den Vorteil, dass durch den Richtungsvektor eine Information extrahiert werden kann, von welcher Richtung das Fahrzeug auf das Gefahren-Objekt blickt.
• Versenden des Daten-Tupels G(P_GO) als ein Broadcast-Signal mittels des Transceivers.
  ∘ Hierbei kann das Daten-Tupel G einmalig oder auch für einen bestimmten Zeitraum mehrmals ausgesendet werden. Das einmalige Aussenden hat den Vorteil, dass der Datenverkehr bei einer "Vehicle-to-Vehicle" (V2V) Kommunikation reduziert wird, wobei das mehrmalige Absenden über den bestimmten Zeitraum, beispielsweise alle 10 Sekunden für den Zeitraum von 1 Minute, den Vorteil hat, dass das Daten-Tupel G zuverlässiger von weiteren Fahrzeugen empfangen werden kann. Da das erste Fahrzeug keine so starke Sendeleistung wie eine Basisstation aufweist und typischerweise nicht in einer erhöhten Position angeordnet ist, ist eine Abschattung des entsprechenden Broadcast-Signals relativ wahrscheinlich, sodass durch ein mehrmaliges Senden die Chance erhöht wird, dass ein weiteres Fahrzeug das Signal empfängt. Hierbei wird das Broadcast-Signal ohne eine bestimmte Adresse versendet und kann somit prinzipiell von jeder Empfängereinheit empfangen werden.

According to the invention, a method for automatically warning other road users of a dangerous object, in particular a dangerous object related to road traffic, is specified by a first vehicle, the first vehicle having suitable electro-optical sensors for detecting object data of the vehicle environment, a suitable on-board computer for Evaluating the object data and a transceiver includes. With the appropriate electro-optical sensors, it can be in particular the sensors listed above. The method has the following steps carried out by the first vehicle:

• receiving sensor data from the vehicle surroundings of the first vehicle by means of at least one first sensor unit of the first vehicle;
  ∘ The sensor data can be recorded by the vehicle either in a 360° all-round view or in the direction of travel. The 360° all-round view offers the advantage that sensor data can also be recorded that is not in the direction of travel of the vehicle, with the recording of the sensor data only in the direction of travel representing a reduction in the computing effort. The direction of travel is marked as a particularly important direction. In this case, the sensor data can come from a single sensor unit, but data can also come from a plurality of sensor units, in particular the sensor units described above, and be combined to form overall data. The overall data have a higher information density and therefore represent a more comprehensive description of the vehicle environment, but on the other hand increase the computational effort and the detection effort.
• Evaluation of the received sensor data by an algorithm implemented on the on-board computer of the first vehicle and classification of objects based on the sensor data, the algorithm generating a dangerous object marking when one of the objects is classified as a dangerous object, the dangerous object marking being generated by a data Tuple G is represented, which includes at least the local position P _GO of the danger object, ie G(P _GO );
  ∘ The algorithm for evaluating the sensor data can in particular include a neural network that has been trained by dangerous objects. The index "GO" stands for "Danger Object". The local position of the hazard object can be specified on standardized coordinates (x,y,z), such as the GPS system based. This has the advantage that all vehicles interpret the position information in standardized coordinates in the same way. An alternative or additional possibility is to specify the local position of the dangerous object as the position of the vehicle and a directional vector, with the directional vector pointing from the coordinates of the vehicle to the dangerous object. Such a direction vector can be calculated by the vehicle's algorithm. This has the advantage that the direction vector can be used to extract information about the direction from which the vehicle is looking at the dangerous object.
• Sending the data tuple G(P _GO ) as a broadcast signal by means of the transceiver.
  ∘ Here, the data tuple G can be sent once or several times for a certain period of time. Sending once has the advantage that the data traffic is reduced in "vehicle-to-vehicle" (V2V) communication, with repeated sending over a specific period of time, for example every 10 seconds for a period of 1 minute, having the advantage that the data tuple G can be received more reliably by other vehicles. Since the first vehicle does not have as strong a transmission power as a base station and is typically not located in an elevated position, the corresponding broadcast signal is relatively likely to be blocked, so that repeated transmission increases the chance that another vehicle will receive the signal . In this case, the broadcast signal is sent without a specific address and can therefore in principle be received by any receiver unit.

This method advantageously makes it possible for the first vehicle to carry out a decentralized assessment of dangerous situations, which can be flexibly transmitted to other vehicles by means of the broadcast signal, without any additional infrastructure. So it can in particular, the infrastructure of a mobile network can be dispensed with. As a result, the other vehicles can be informed about the dangerous object at an early stage, which significantly increases road safety by making the position P _GO of the dangerous object known to the other vehicle. In particular, the data tuple G can be received by a second vehicle and also sent on again. This leads to an efficient propagation of the signal over a long period of time and over a large range, with the transmission range of the first vehicle being considerably extended by the "chain-like" transmission of the data tuple G by the other vehicles. The process works without involving a "central authority" to manage the hazard reports.

In one embodiment, the data tuple G additionally has a time stamp t _F1 of the detection of the dangerous object by the first vehicle, ie G(P _GO , t _F1 ).

This advantageously enables the other vehicles in particular to calculate how long it has been since the first vehicle detected the dangerous object. With a classification of the dangerous object, this information can also help to determine whether the dangerous situation still exists. Depending on the class in which the dangerous object is classified, each class can be assigned a specific period of validity for the data tuple G, since it can generally be assumed that different dangerous situations last for different lengths of time. For example, it can be assumed that a dangerous situation in the event of an accident between two cars has a longer period of validity than, for example, an animal crossing the roadway. In the first case, for example, a validity period of 1 hour and in the second case a validity period of 5 minutes could be specified. The further classification of a dangerous object to a specific ID _GO can be carried out, for example, by the first vehicle and then added to the data tuple G(P _GO , t _F1 , ID _GO ) as additional information.

The data tuple G preferably also has a quality value GW _F1 of the detection of the dangerous object by the first vehicle, ie G(P _GO , GW _F1 ).

This has the advantage that the quality value GW _F1 indicates the statistical certainty with which a dangerous object, in particular a specific dangerous object, was detected. The other vehicles receiving the data tuple G can use the quality value GW _F1 to assess the degree of certainty with which they can trust the information received. The more the other vehicles trust the data tuple G, the more the hazard object is taken into account for further driving decisions by the on-board computer of the other vehicles. However, the quality value GW _F1 can also play a role if one of the other vehicles verifies the data tuple G or updates the data. If, for example, the quality value GW _Fn of one of the other vehicles is significantly worse than the quality value GW _F1 of the first vehicle, the data tuple G is preferably not updated since no improvement can be achieved with the new data.

The data tuple G preferably also has a speed vector v _F1 and a local position P _F1 of the first vehicle at time t _F1 , ie G(P _GO , t _F1 , V _F1 , P _F1 ).

This data can be used in an advantageous manner to determine what view the first vehicle has of the dangerous object, in particular whether it is approaching the dangerous object from the front or from behind. This information can be relevant for the other vehicles both when checking or updating the data tuple G and when considering their own driving decisions. The more similar the approach direction of one of the other vehicles is compared to the first vehicle, the more relevant the data tuple G is, as a rule, if we assume that the current time is only a sufficiently short time difference relative to the point in time t _F1 .

The data tuple G expediently also has a speed vector v _GO and of the dangerous object, ie G(P _GO , t _F1 , v _GO ).

Advantageously, the speed vector v _GO and the time stamp t _F1 of the detection can be used to determine where the hazardous object is located after a specific time. This can be used by the on-board computer of one of the other vehicles to assess whether the dangerous situation is still relevant.

It is also preferably possible for the first vehicle to use its sensors to determine further parameters and to add these to the data tuple G individually or in any combination. These parameters are explained below:
ID _GO : a unique identification of the dangerous object can be created. On the one hand, this enables the on-board computers to assess the dangerous situation and, on the other hand, to assess whether their respective sensors detect the same dangerous object.

ID _F1 , ..., ID _Fn : each vehicle can add an identity assigned to the data tuple G to it. This identity can be a vehicle number, for example, or it can also be represented by an (e)SIM number. This advantageously enables a clear assignment of which vehicle generated a specific data tuple G. Under certain circumstances, this can also be used for fault diagnosis if it turns out that the data tuple G of a specific vehicle consistently has a poor quality value GW _Fn .

V _GO : each vehicle can add an estimate of the volume of the hazardous object to the data tuple G, in particular with expansion components in all three spatial directions. This advantageously enables the other vehicles to estimate the extent of the dangerous object.

Update counter: each time the data tuple G is updated by a vehicle, the update counter can be increased by one. This shows how often the danger object was checked and updated.

In one embodiment, the broadcast signal is sent in a mobile radio standard, in particular the PC5 standard.

This has the advantage that modern vehicles do not have to be specially equipped with their own hardware, since modern vehicles generally have the ability to send using a mobile radio standard. In particular, the vehicles are set up to transmit via LTE and/or 5G. In particular, the PC5 broadcast communication specified by 3GPP in mobile networks can be used for this purpose, which is suitable for both LTE and 5G.

In a preferred embodiment, the broadcast signal is sent as a PC5 sidelink.

This offers the advantage that no mobile network, in particular no mobile phone masts/base stations, is required for communication between the vehicles. The PC5 Sidelink can therefore be used as a "pure""vehicle to vehicle" means of communication. As a result, the method can also be used outside the range of mobile phone networks. The V2V PC5 specification was specified for LTE and is specified for 5G, in particular to be able to communicate between end devices using a transmission mast. However, according to this preferred embodiment variant, the invention uses only the PC5 side link functionality for spatially limited terminal-to-terminal communication without the support of the cell phone masts. In the locally limited variant, terminal-to-terminal communication is generally the communication of a transmitter S1 to all accessible receivers E with E2, E3, ..., En. So in particular the communication between the vehicles. The vehicles then function as it were as a movable radio mast, with the transmission strength of the respective vehicle determining the transmission radius R. However, R is also dependent, in a known manner, on the environmental parameters and the reception properties of the receiver. In white papers, R=400 meters is currently assumed for the PC5 sidelink. In this case, the transmitter and receiver cannot be more than 400 meters apart. This rather arbitrary value of 400m follows from the fact that a terminal device in the general specification with a transmission mast must always be able to reach a transmission mast given the current cell sizes.

The broadcast signal is preferably sent as a broadcast signal set up for a base station, in particular if the first vehicle does not receive an acknowledgment of receipt of its broadcast signal from another vehicle.

If the first vehicle does not receive an acknowledgment of receipt of its broadcast signal from one of the other vehicles, there is a high probability that the hazard report for a following vehicle has been lost. By sending the broadcast signal to a base station, the base station can forward the transmitted data tuple G to the other vehicles in the same and/or neighboring mobile radio cells via the mobile radio network for a certain period of time and thus ensure that the hazard warning is not lost . As soon as another vehicle confirms receipt of the data tuple G via the base station's cellular network, the base station can interrupt transmission via the cellular network as long as the hazard warning is once again taking place via V2V communication. In particular, depending on the classification of the dangerous object ID _GO , the vehicle can always send the data tuple G to the base station. This is particularly the case when there is a dangerous situation in which, for example, the police and/or fire brigade must be informed. The base station can forward the data tuple G(P _GO , t _F1 , ID _GO ) to a central server, which automatically informs the authorities.

The data tuple G is preferably received by a second vehicle and evaluated by an on-board computer of the second vehicle.

This achieves several advantages. On the one hand, the data tuple G is made available to the second vehicle, so that it is possible for a potential dangerous situation to be made known to it in the first place. On the other hand, it is made more difficult for a potential threat to post incorrect warnings. The potential endangerer cannot simply grossly falsify the hazard message verified by several participants or even communicate it as no longer existing without this change not being recognized by other examiners, since there is always one defined minimum number of participants required for final verification for significant changes. The data tuple G can be marked as verified and/or one of the vehicles decides independently, based on the history of the data tuple G, about its validity. A hazard report is more credible if it also contains history information from previous messages and can therefore be better evaluated by recipients.

In one embodiment, a driver of the second vehicle is warned and/or the second vehicle automatically takes into account the information in the data tuple G.

The driver of the second vehicle can be warned about the potentially dangerous situation by his vehicle, for example by means of a signal lamp and/or a corresponding audio signal. However, the vehicle can also independently take into account the information in the data tuple G and, for example, automatically reduce the speed.

In one embodiment, the second vehicle checks and/or updates the data tuple G with regard to the dangerous object.

As a result, a hazard report can be updated decentrally, so that the quality of the information is consistently high. In addition, the information density of the data tuple G is steadily increased. The risk situation is thus fully verified at a later point in time, if necessary, slightly changed, or it is communicated as no longer existing.

In one exemplary embodiment, a weighted consideration taking into account a history of the data tuple G is applied during the check. For example, each of the vehicles can add to the data tuple G its own measurements. When checking the dangerous object, the last measurement is then weighted the most. This is realized by a corresponding change in the data tuple G by the respective vehicle. For example, if the volume of history data is limited to the last ten measurements, in which case only this maximum volume can be added to the data tuple G in order to reduce data traffic, then the following method can be used the most recent measurement gets a weighting factor of 10/10, the previous measurement a weighting factor of 9/10, ..., and the first measurement a weighting factor of 1/10. On the one hand, this ensures that the most recent measurement is taken into account to the greatest extent, but on the other hand it also ensures that the most recent measurement cannot completely "override" the previous measurements. This serves as a security aspect, as it can happen that the most recent measurement is based on "wrong data". In abstract terms, the weighting is to be selected in such a way that the sum of the weights of the previous measurements is greater than the weighting of the most recent measurement.

The first and/or the second vehicle are preferably self-driving cars.

This offers the advantage that they can fully consider the data tuple G in their driving decisions.

According to a second aspect of the invention, a vehicle is specified which is set up to generate, send, receive and process a broadcast signal according to one of the preceding claims.

Such a vehicle has the advantage that it can serve as a decentralized danger signal detector and decentralized mobile radio mast of danger signals and can thus effectively warn other vehicles of a dangerous object. In addition, such a vehicle can itself benefit from a hazard warning from other vehicles by taking the data tuple G into account in its driving decision. In particular, the vehicle is fully self-driving. This concept enables a comprehensive monitoring of dangerous situations, which would otherwise only be possible with a very cost-intensive comprehensive equipping of the streets with detectors. In terms of hardware, such a vehicle is equipped with the corresponding sensors mentioned above, with algorithms implemented on the on-board computer being set up to generate the data tuple G, to send it, to receive it and to process it accordingly.

According to a third aspect of the invention, a vehicle is specified which is set up to receive and process a broadcast signal according to one of the preceding claims.

This has the advantage that the vehicle can be constructed more cost-effectively according to the third aspect of the invention since it can do without the sensor units, but the vehicle can still benefit from the data tuple G and take into account the risk situation when making its driving decision and/or the driver indicate a possible risk.

According to a fourth aspect of the invention, a system is specified, the system having a vehicle as described above and a mobile radio station with access to a mobile radio network. The system is set up to perform one of the methods described above.

This system offers the advantage that the data tuple G sent by a first vehicle can also reach a second vehicle when it is outside the transmission range of the first vehicle.

Fig. 1:

zeigt schematisch ein System zum Ausführen des erfindungsgemäßen Verfahrens.

Fig. 2:

zeigt eine Aktualisierung eines Daten-Tupel G durch einen auf einem Bordcomputer eines Fahrzeugs implementierten Algorithmus.

Fig. 3:

zeigt einen beispielhaften Aufbau des Daten-Tupels G.

Fig. 4:

zeigt ein Flussdiagramm des Verfahrens, insbesondere auf dem Bordcomputer des Fahrzeugs.

Fig. 5:

zeigt exemplarisch ein Fahrzeug, das zur Durchführung des erfindungsgemäßen Verfahrens geeignet ist.

Preferred exemplary embodiments of the present invention are explained below with reference to the accompanying figure:

Figure 1:

shows schematically a system for carrying out the method according to the invention.

Figure 2:

shows an update of a data tuple G by an algorithm implemented on an on-board computer of a vehicle.

Figure 3:

shows an example structure of the data tuple G.

Figure 4:

shows a flowchart of the method, in particular on the vehicle's on-board computer.

Figure 5:

shows an example of a vehicle that is suitable for carrying out the method according to the invention.

Numerous features of the present invention are explained in detail below on the basis of preferred embodiments. The present disclosure is not limited to those specifically mentioned Combinations of features limited. Rather, the features mentioned here can be combined as desired to form embodiments according to the invention, unless this is expressly excluded below.

1 FIG. 1 schematically shows a system 10, in particular a communication system 10, for carrying out the method according to the invention. The system 10 includes at least a first vehicle 15 that can perceive a dangerous object 30 while driving on a roadway 35 . When this dangerous object 30 is detected, an on-board computer of the first vehicle 15 can generate a dangerous object marking and send this as a broadcast signal. This broadcast signal no further vehicles, in 1 received and processed by a second vehicle 20 and a third vehicle 25 . In addition, the broadcast signal can be received by a base station 40 of a mobile radio network.

In order to be able to effectively detect the dangerous object 30, the first vehicle 15 is provided with electro-optical sensors, in particular various types of sensors. The vehicle 15 therefore has an intelligent sensor system for recognizing the surroundings and for determining its own position and a transceiver for wireless communication. This is the case, for example, with autonomous or semi-autonomous vehicles. The vehicle 15 has a unique vehicle ID in the ITS "Intelligent Transport System". This can be the license plate, for example, but also an ID in the mobile network (MSISDN). A list of hazard IDs is defined in the ITS, which defines the type of hazard notification.

These sensors can be, for example, lidar, radar, infrared and/or digital stereo cameras. Infrared cameras are for living objects and living objects at night, since "inanimate" materials at night are more likely to be the same temperature as their surroundings.

These sensors are particularly suitable for detecting visible surfaces of objects. The visible surface is the surface of the object that can be detected by the respective sensor. With the sensors lidar, radar and digital stereo camera, this is identical to what a human eye would also see if it were close enough, i.e. the geometric surface of the "object-material".

In the case of the infrared cameras, the "thermal" surface may be determined slightly differently from the geometric one, since in this case the temperature transfer of the deeper layers of material also influences the heat of the geometric surface.

• Bestimmung der Geometrie: Überschreitungen von Grenzwerten zu Größe/Ausdehnung.
• Bestimmung der Position: Wenn sich das Objekt 30 auf der Fahrbahn/Fahrfläche 35 befindet und sich von der normalen/erwarteten Fahrbahn/-fläche signifikant unterscheidet.
• Bestimmung der Kinematik und Position: Wenn sich das Objekt 30 in Richtung Fahrbahn/Fahrfläche 35 bewegt und diese bald erreichen könnte.
• Datenbankabgriff/Vergleich: bzgl. bekannter Klassifizierungen mit den erfassten Eigenschaften
  • ∘ initial gescannter Objektoberflächenabschnitte/Konturlinien und
  • ∘ später ggfs. vervollständigter 3D Rundum Vervollständigung und
  • ∘ der Objekt Temperatur.

The determination of an object as a dangerous object can be implemented technically as follows:

• Geometry Determination: Exceeding size/extent limits.
• Determination of the position: When the object 30 is on the roadway/roadway 35 and differs significantly from the normal/expected roadway/roadway.
• Determining the kinematics and position: When the object 30 is moving in the direction of the roadway/roadway 35 and could soon reach it.
• Database access/comparison: regarding known classifications with the recorded properties
  • ∘ initially scanned object surface sections/contour lines and
  • ∘ later, if necessary, completed 3D all-round completion and
  • ∘ the object temperature.

All of the sensors mentioned can determine the direction to any object point. The direction can be determined in a global/standardized coordinate system that all vehicles 15, 20, 25 know.

Each vehicle 15, 20, 25 also has a local vehicle-fixed coordinate system that moves in translation and rotation relative to the global coordinate system as a function of the vehicle movement. This transformation from one system to another is known as a function of time/vehicle motion. Each sensor coordinate system is usually rigid to the vehicle coordinate system and the transformation from one system to another is known. Even movements of the sensor coordinate system in the vehicle can be converted into the vehicle coordinate system, since these movements are determined by the control systems. Thus, all direction measurements from the sensors are known as rays in the global coordinate system. Since the 3D position of one point of the beam in the vehicle is known, the beam is also clearly defined in space.

With radar and lidar, the distance between one point in the vehicle and the object surface point is determined with great precision.

As a result, the initial coordinates, direction and distance to the object point are known for each ray, from which the 3D position of the object surface point can be determined in a global coordinate system via so-called polar appending. The visible surface is determined in 3D by fast scanning.

Mono cameras scan without any time lag since the entire recording was made at the same time. They only provide directions, not distances. Camera observations are image coordinates on the CCD/CMOS chip of imaged object points.

Stereo cameras work differently when measuring distance. There, the distance to the object point can be determined via the beam intersection of two different recordings (other camera positions and possibly also different camera orientations) and their respectively assigned camera beams. At the same time, the 3D position on the object is clearly determined directly in 3D by the beam section, whereby the distance is then determined as the distance between the camera projection center and the object position using a 3D Pythagoras.

All sensors check each other and have complementary properties in terms of speed of detection, accuracy in terms of object distance, etc.

It can be assumed that an autonomously driving vehicle knows its 3D position in the global coordinate system with an accuracy of 1 cm and can also measure a roadway geometry with an accuracy of 1-3 cm. The sensors of the vehicle 15, 20, 25 enable the object points (roadside edges or dangerous objects) to be determined with an accuracy of at least 1 cm to 5 cm, depending on the distance. This object accuracy of a dangerous object that has been detected multiple times can be even higher, especially when the object 30 is static.

A concrete exemplary embodiment of the method is described below:
A dangerous object 30 is located on a busy road 35, such as a defective vehicle, an animal or dangerous road damage. The first vehicle 15 recognizes the danger using its electro-optical measuring instruments (camera/sensors). An autonomously controllable vehicle will have sensors that can detect a dangerous object 30 point by point on the visible surface of the object and thus a 3D model of the visible surface with respect to a 3D coordinate system (with z-axis vertical in the direction of gravity). , which is known to all autonomously driving vehicles 15, 20, 25. This is ensured by the fact that the first vehicle 15 can determine the direction and distance to each visible surface point sufficiently at the same time.

From these surface points and their coordinates, the software of the first vehicle 15 implemented on an on-board computer can determine a center of gravity S of the dangerous object 30 and from this, for example, calculate the extensions in the direction of the coordinate axes, whereby such an object can be determined in its position by three coordinates (Sx , Sy, Sz) and is determined in extension by six distances (-dx, +dx, -dy, +dy, -dz, +dz) relative to the center of gravity S. For each vehicle 15, 20, 25 that carries out this determination, these determination variables only depend on the quality G of the sensors of the vehicle 15, 20, 25, on the position P of the vehicle 15, 20, 25 and on the alignment A of the vehicle 15, 20, 25 in the 3D coordinate system relative to the danger object 30 depending if the object is a static one and undeformed object with respect to all different detection times.

This detection and object description is only a preferred embodiment. Other technical implementations are possible, e.g. the six distances do not have to be defined in the axis directions of the superordinate coordinate system, but can be defined as usual with regard to the directions x=in the direction of travel of the road and y=transverse to the direction of travel of the road and z-axis. As a result, “better” information would be available if the vehicle 15 were to drive past. In addition, the outer contours to the left and right of the direction of travel could also be determined as additional information. The determination of all these geometric quantities are known to those skilled in the art. If the variables are determined by over-determination, e.g. from lidar and camera measurements at the same time or from a vehicle 15 at slightly different points in time, improved accuracy dimensions can be determined for all geometric variables, e.g. Gaussian standard deviations, which account for the approx. 68% confidence intervals specify the parameter. "Further improved" refers to a determination accuracy with only a single method, e.g. B. Lidar only. If only one method is used, however, one obtains accuracy measures, since one can then also refer to the known internal measurement accuracy of this method in the current technical system design.

If the measurements are not available in sufficient density and quality, this information can be identified in the broadcast signal, in particular in the data tuple G, in an information field about the size of the standard deviations of the measurements and recognized and taken into account by recipients of the message.

The following data can be measured and/or evaluated in the data tuple G.

A unique ID of the hazard report generated by the first vehicle 15 itself can be composed of the following information: i) hazard ID specifying the type of hazard; ii) own vehicle ID, which clearly defines the vehicle from which the data-critical tuple G is sent; iii) a sequence number of the hazard message so that it can be identified whether a hazard has already been sent; iv) one or more time stamps, a first time stamp indicating, for example, the first time the object 30 was detected and a second time stamp indicating the last time the object 30 was detected; v) a position of the dangerous object 30, in particular in the standardized coordinates; vi) an extent and/or orientation of the danger object 30; vii) Information about the quality of the measurements of the sensors, in particular in standard deviations of the individual measurements; viii) an information flag identifying the danger object 30 as a static or moving object.

The first vehicle 15 can send the hazard warning, in particular as the data tuple G, with regard to the hazard object 30 by means of the broadcast signal to the other vehicles 20, 25, which are following vehicles and/or oncoming vehicles . This hazard warning can be taken into account by the other vehicles 20, 25 when making their driving decision and can also be checked by the other vehicles 20, 25 in order to improve the quality of the information in the hazard warning/hazard warning.

In order to transmit the broadcast signal, the vehicle 15 uses a communication method such as the PC5 sidelink, which in current implementations in vehicles has a transmission radius of around 400 m and is particularly suitable for direct “vehicle-to-vehicle” communication. The PC5 communication also works advantageously outside of mobile radio networks. This means that vehicles that are dialed into other mobile radio networks can also be reached. The more vehicles that take part in the process, the faster and better the process works. In addition, the method also works in areas where no mobile network is available, the mobile network has failed, or mobile networks are incompatible with each other. However, in the event that no other vehicle 20, 25 can be reached by means of the PC5 broadcast signal, the broadcast signal is sent as a cell broadcast to a reachable base station 40 sent. This base station 40 can then emit the broadcast signal for a predefined period of time at regular intervals, so that if new vehicles drive into its radio range, warnings of the danger are effectively given. The base station 40 can also forward the data tuple G to a server 45 . The server 45 can in turn inform other base stations about the hazard warning and/or forward the hazard warning to rescue services, for example.

After the data tuple G has been sent by the first vehicle 15 , it is received by the second vehicle 20 . Using the position information of the hazard object 30 in the data tuple G hazard and its own position and movement information (and possibly its current navigation route), the second vehicle 20 can extrapolate when it will drive past the hazard and can prepare accordingly ( e.g. drive slower). The second vehicle 20 takes into account both the information about its current location and its speed vector as well as its planned route in relation to the time required to reach the dangerous object 30.

As soon as the second vehicle 20 can detect the dangerous object 30 itself, it evaluates the danger warning, ie the data tuple G, which it received from the first vehicle 15 .

In this case, the second vehicle 20 has in particular the option of confirming the dangerous object, confirming it and, if necessary, updating the information in the data tuple G or rejecting the danger warning. In particular, the second vehicle 20 uses its own electro-optical sensors to compare the data received with those of the data tuple G received.

Here, the second vehicle 20 can insert its own FZG-ID with a time stamp and possibly a different location, enriched by its own risk assessment, into the risk report, i.e. the data tuple G: (risk ID; first vehicle 15-ID; first Vehicle 15 serial number; first vehicle 15 location; first vehicle 15 time stamp; second vehicle 20 ID, [second vehicle 20 place]; second vehicle 20 timestamp, second vehicle 20 threat assessment (confirm, change, reject)).

The second vehicle 20 can in turn sign data tuple G sent by it to other vehicles, in particular to the third vehicle 25, with its unique vehicle ID. In this case, the initial unique ID of the hazard generated by the first vehicle 15 is generally retained.

Should the deviation of the values measured by the second vehicle 20, which characterize the dangerous object 30, deviate too much from the correspondingly measured values of the first vehicle 15, and the dangerous object will still be in the vicinity of the position to be communicated by the first vehicle 15 detected, the second vehicle 20 will use a new unique ID for this hazard and then generate a corresponding data tuple G. In this case, the second vehicle 20 will consequently also reject the hazard warning from the first vehicle 10 . Whether the different danger positions obtained are "too large" must also be assessed in relation to the time difference between the respective measurements.

In a next step, further vehicles, in particular third vehicle 25, receive the broadcast signal now transmitted by second vehicle 20. If technically possible, each vehicle carries out the same steps as the vehicle from which it received the broadcast signal. As a result, an ever increasing information density of the dangerous object 30 is generated. Preferably, the other vehicles evaluate their own measurement results of the dangerous object 30 not only relative to the first vehicle 15, but relative to all other vehicles, which in turn have updated and/or confirmed the data tuple G, i.e. in particular always relative to the most recent one Confirmation.

In the simplest case, a vehicle only uses the latest from the previous hazard observation/evaluation. However, a current vehicle can also summarize all previous hazard observations/evaluations of the previous vehicles using a special weighting function rate to compare that particular overall rating to its own more recent hazard observation/assessment. The rule here is that the more the measurement of a specific vehicle is weighted, the more the corresponding measured values are taken into account when generating a modified hazard warning in the form of the data tuple G.

This procedure is particularly expedient if a moving dangerous object 30 is to be evaluated, since speed vectors and/or also acceleration vectors can be calculated from the previous positions of dangerous object 1 to (n−1). Both speed and acceleration vectors can be characteristic/plausible for different recognized dangerous objects.

The quality of the individual measurements of each previous vehicle and that of the current vehicle can also be taken into account in the weighting function in order to take the measurements of particularly good sensors of one of the vehicles 15, 20, 25 more into account. For example, a standard deviation can be determined and sent for the hazard position, extent and/or orientation. After each additional detection by other vehicles or later repeated detection of the same vehicle, the hazard information is confirmed/condensed/improved or rated as no longer existent. The quality of the information improves as more vehicles are included in the evaluation.

As a rule, all vehicles 15, 20, 25 coming from the same direction, provided there is no obscuration of objects, will geometrically detect the dangerous object 30 in the sequence of their movement from almost the same positions with almost the same alignments and are therefore easily comparable deliver digital object descriptions.

Vehicles traveling to the object from other routes will measure slightly different descriptions of the hazard object 30 depending on the extent of the object. The measured values are therefore dependent on the following parameters, especially in the case of larger extents and occlusions Approach - depending on the following factors, for example: the direction of travel of the roadway, the choice of lane, the current street from which the object can be detected, etc.

Each vehicle can determine these parameters of the approach with its sensors and/or its navigation system and add it to its object description as additional parameters, so that a vehicle, which carries out the object recognition later than other vehicles, can find out with which parameters it uses its own measured values of the dangerous object 30 can compare and in particular which measured values can be improved by his own current measurements.

The improvement of the transmitted measured values of the data tuple G then takes place via the best possible digital "superimposition" of potentially almost identical object views. The general procedure is known, for example, from photogrammetry as the "multiple images of a 3D object with multiple cameras from different directions and from different positions and at different times". These multi-image recordings are then processed using so-called bundle block adjustment and provide significantly more precise information than individual recordings. These bundle block adjustments can also successively add individual images (or individual object descriptions) to already highly accurate, previously determined multi-image object descriptions/object models (keyword: recursive parameter estimation / sequential adjustment). These processes run fully automatically and sufficiently quickly for our invention.

2 shows an update of a data tuple G by an algorithm implemented on an on-board computer of vehicles 15, 20, 25.

Since the hazard warnings are sent in the form of the data tuple G via the broadcast signal from different vehicles, vehicles can receive several competing hazard warnings for a hazard, which have different “confirmation chains”.

In the case of competing messages, a vehicle can use the unique ID to identify whether the same hazard is meant. If this is the case, the vehicle can understand the exact history based on the individual signatures/timestamps. It can therefore combine the two hazard statements into a single hazard statement and continue the procedure.

A fifth vehicle may receive a first broadcast signal 50 from the third vehicle 25 in which the data tuple G reported by the first vehicle 15 is from the second vehicle 20 and from the third vehicle 25 and a second broadcast signal 55 from a fourth vehicle in which the threat reported by first vehicle 15 was confirmed only to the fourth vehicle. The fifth vehicle can then combine the two broadcast signals 50, 55 into a new data tuple G 60 and thus has the confirmations from the second, the third and the fourth vehicle.

If a certain number of vehicles reject a hazard message in a row, the hazard message is discarded. The danger no longer exists or has been eliminated and the procedure can be completed.

If a certain number of vehicles agree as a result of a danger message, then this is considered "generally" accepted.

A "generally" recognized hazard warning can be sent from one of the vehicles, in particular including the entire data tuple G history, to the base station 40 of the mobile radio cell, which is in principle possible through a broadcast signal, since a base station 40 can also receive this . In this case, in particular, a flag can be set in the data tuple G in such a way that the base station 40 can determine that the corresponding broadcast signal is specifically intended for it. The mobile radio cell can use the handover history to determine the direction of travel of the arriving vehicles. In this way, the mobile radio cell can notify a vehicle entering the cell of the danger message at an early stage. In addition, the mobile radio cell can send the danger report to a central Authority (CA) - implemented in particular on the server 45 on the Internet - forward.

The weighting function is explained in detail below:
First, using a simple variant of the weighting function:
In the case of static dangerous objects, the positions and their standard deviations of all determined vehicles 1 to n can be combined to form a weighted average position, which also has a resulting standard deviation. This statistical adjustment procedure can also be used to eliminate outliers in the individual items, for example from individual vehicles. This individual vehicle can be the vehicle currently detecting the new hazard, which decides that the "old" hazard has been eliminated and that there is a new hazard in the vicinity of the old hazard position, or that there is no hazard at all.

Another sample variant of the weighting function:
In the following, "n" denotes the most recent measurement by vehicle "n", "n-1" denotes the previous measurement by vehicle "n-1". For objects in motion, the last velocity vector v(n-1) of the hazard object 30 at time t(n-1) and the time difference dt=t(n)-t(n-1) between vehicle determinations (n-1) and the vehicle (n) are taken into account in order to determine the hazard position Pos(n) of the vehicle (n) at t(n) for plausibility with regard to the hazard object 30 determined by the vehicle (n-1). and the corresponding position Pos(n-1).

• Posk(n)=Pos(n-1)+v(n-1)*(t(n)-t(n-1)) für antizipierte konstante Objektgeschwindigkeit vk=v(n-1);
• Posh(n)=Pos(n-1)+vh*(t(n)-t(n-1)) für antizipierte im Mittel erhöhte Objektgeschwindigkeit vh;
• Post(n)=Pos(n-1)+vt*(t(n)-t(n-1)) für antizipierte im Mittel reduzierte Objektgeschwindigkeit vt;

wobei vh und vt nach bekannten Formeln aus den möglichen Erhöhungs- und Abbremsbeschleunigungen b=bh und bt=-b des Gefahrenobjektes berechenbar sind. Die von dem n-ten Fahrzeug zum Zeitpunkt t(n) ermittelte Gefahrenposition kann somit geprüft werden, ob sie in dem folgenden Intervall liegt [Posh; Post]. Liegt sie nicht in diesem Intervall, könnte eine fehlerhafte Messung vorliegen.The possible acceleration vector b of the recognized object 30 (for example a deer) can also be taken into account here. It is important to consider the ideal border relationships:

• Posk(n)=Pos(n-1)+v(n-1)*(t(n)-t(n-1)) for anticipated constant object velocity vk=v(n-1);
• Posh(n)=Pos(n-1)+vh*(t(n)-t(n-1)) for anticipated object speed vh increased on average;
• Post(n)=Pos(n-1)+vt*(t(n)-t(n-1)) for anticipated mean reduced object velocity vt;

where vh and vt can be calculated according to known formulas from the possible increase and deceleration accelerations b=bh and bt=-b of the dangerous object. The danger position determined by the nth vehicle at time t(n) can thus be checked as to whether it lies in the following interval [Posh; Post]. If it is not within this interval, there could be an incorrect measurement.

A further weighting is also possible in this case, whereby b can be systematically varied in its three components bx, by and bz and thus a three-dimensional interval can be obtained for the above-mentioned one-dimensional interval. In this case, it is taken into account that the position of the dangerous object 30 does not just wander along a straight line, but may move within a cuboid—this tends to correspond to a 3-dimensional confidence ellipse, which is thereby abstracted. Taking into account known error propagation rules and assumptions about the error ranges of all parameters, a separate uncertainty measure can also be calculated for the cuboid for each coordinate direction. If the position Pos(n) of the dangerous object 30 determined by the nth vehicle lies outside the cuboid increased by its uncertainty mass, then this is an indication that a new dangerous object is involved. This thesis can be tested using well-known statistical hypothesis tests.

3 shows an example structure of the hazard warning as a data tuple G 65.

4 shows a flowchart of the method, in particular on the vehicle's on-board computer.

In step 70 the on-board computer receives sensor data from the vehicle surroundings of the first vehicle from at least one first sensor unit of the first vehicle;

In step 75, the on-board computer evaluates the received sensor data using an implemented algorithm and classifies objects based on the Sensor data, the algorithm generating a danger object marking when classifying one of the objects as a danger object, the danger object marking being represented by the data tuple G 65, which includes at least the local position P _GO of the danger object, ie G(P _GO );

In step 80, onboard computer 110 triggers figure 5 a sending of the data tuple G(P _GO ) 65 as a broadcast signal by a transmitting device of the vehicle.

figure 5 FIG. 1 shows an example of a vehicle 100 that is suitable for carrying out the method according to the invention. For this purpose, vehicle 100 has at least one of the electro-optical sensors 105 mentioned above for detecting and measuring dangerous object 30 . The measurement data are first transmitted via a data line 120 to the on-board computer 110, which evaluates the measurement data using an algorithm implemented on it. In particular, the data tuple G 65 can be generated here, which is transmitted via the data line 120 to a transmitting and/or receiving unit 115, in particular a transceiver 115, of the vehicle 100. The vehicle sends or receives the data tuple G 65 by means of the transmitting and/or receiving device 115. The vehicle 100 can be the first vehicle 15 in particular.

Claims (15)

Method for automatically warning other road users of a dangerous object (30), in particular a dangerous object (30) related to road traffic, by a first vehicle (15), the first vehicle (15) having suitable electro-optical sensors for detecting Object data of the vehicle environment, a suitable on-board computer (110) for evaluating the object data and a transceiver, the method having the following steps carried out by the first vehicle (15): • receiving sensor data from the vehicle environment of the first vehicle (15), the sensor data being measured by at least one first sensor unit (105) of the first vehicle; • Evaluation of the received sensor data by an on-board computer (110) of the first vehicle (15) implemented algorithm and classification of objects based on the sensor data, the algorithm generating a danger object marking when classifying one of the objects as a dangerous object (30). , wherein the dangerous object marking is represented by a data tuple G (65) which comprises at least the local position P _GO of the dangerous object, ie G(P _GO ) (65); • Sending the data tuple G(P _GO ) (65) as a broadcast signal. Method according to one of the preceding claims, characterized in that the data tuple G additionally has a time stamp t _F1 of the detection of the hazardous object by the first vehicle, ie G(P _GO , t _F1 ). Method according to one of the preceding claims, characterized in that the data tuple G additionally has a quality value GW _F1 of the detection of the dangerous object by the first vehicle, ie G(P _GO , GW _F1 ). Method according to one of Claims 2 to 3, characterized in that the data tuple G additionally has a speed vector v _F1 and a local position P _F1 of the first vehicle at time t _F1 , i.e. G(P _GO , t _F1 , V _F1 , P _F1 ). Method according to one of Claims 2 to 4, characterized in that the data tuple G additionally has a speed vector v _GO and of the dangerous object, ie G(P _GO , t _F1 , v _GO ). Method according to claim one of the preceding claims, characterized in that the broadcast signal is sent in a PC5 standard. Method according to Claim one of the preceding claims, characterized in that the broadcast signal is sent as a PC5 sidelink. Method according to Claim one of the preceding claims, characterized in that the broadcast signal is sent as a broadcast signal set up for a base station, in particular if the first vehicle does not receive confirmation of receipt of its broadcast signal from another vehicle. Method according to one of the preceding claims, characterized in that the data tuple G is received by a second vehicle and evaluated by an on-board computer of the second vehicle. Method according to Claim 9, characterized in that a driver of the second vehicle is warned and/or that the second vehicle automatically takes the information in the data tuple G into account. Method according to one of Claims 9 to 10, characterized in that the second vehicle checks the data tuple G with regard to the dangerous object. Method according to Claim 11, characterized in that a weighted consideration, taking into account a history of the data tuple G, is used during the check. Method according to one of the preceding claims, characterized in that the first and/or the second vehicle are self-driving cars. Vehicle set up for generating, sending, receiving and/or processing a broadcast signal according to one of Claims 1-13. System comprising a vehicle according to claim 14 and a mobile radio station with access to a mobile radio network, set up to carry out any of claims 1-13.

Images