DE102020202163A1 - Method and device for detecting objects and / or structures in the vicinity of a vehicle - Google Patents

Abstract

In a method according to the invention for detecting objects and / or structures in the vicinity of a vehicle (F), the position of the vehicle is determined (1). Localization data (LD) for the determined position are transmitted to a central server (S) (2). Swarm data (SD) for this position and / or a subsequent future position when driving on a route are received (3) by the central server (S), the swarm data being based on a statistical evaluation of sensor data that was recorded at an earlier point in time at this and / or the future position was recorded by a large number of other vehicles. The swarm data (SD) received are used to check (4) whether sensory detection of objects and / or structures in the vicinity of the vehicle is impaired.

Description

The present invention relates to a method for detecting objects and / or structures in the vicinity of a vehicle, which in particular ensures robust detection of such objects or structures. The present invention also relates to a corresponding device.

In the meantime, a large number of assistance systems are used in motor vehicles in order to enable improved comfort and increased safety, in particular for the driver, when driving the vehicle in traffic. For this purpose, among other things, sensors for monitoring the vehicle environment, for example one or more video cameras or sensors based on ultrasound, radar or laser, are installed in the vehicle.

Simple driver assistance systems inform and warn the driver about particularly dangerous situations, while further developed systems also automatically intervene in the vehicle control. Adaptive cruise control can thus take place with an adaptive cruise control, often also referred to as ACC for short (abbreviation of the English expression “Adaptive Cruise Control”). Here, the position and speed of a vehicle ahead is determined with a sensor, often a radar sensor located in the front area, and the speed and the distance of the following vehicle equipped with adaptive cruise control is automatically controlled with an engine and brake intervention. This automated longitudinal control takes the strain off the driver, so that long motorway journeys, for example, can be carried out more comfortably. This, if necessary in conjunction with an emergency brake assistant, can also contribute to driving safety and enable a more energy-efficient driving style. Lane keeping assistants, which recognize side markings based on cameras, can ensure that the vehicle is kept within the lane and, if necessary, automatically counter-steers to correct it. Furthermore, a lane departure warning system and the adaptive cruise control can be used together for an emergency system in which an emergency stop is automatically initiated. Here, for example, the unconsciousness of a driver can be recognized with sensors because the driver does not show any steering, braking or acceleration activities over a longer period of time.

While optical sensor systems such as infrared, lidar or camera sensors rely on good visibility, ultrasound and radar sensors can also be operated without problems in darkness, fog or rapidly changing light conditions, such as when entering or exiting a tunnel . Even with such boundary conditions, precise distances between objects detected by the sensors and the vehicle can be measured, in particular by means of radar sensors, and several objects, such as other vehicles, guard rails or people, can be separated from one another and their movement can be tracked, whereby the relative speeds can also be precisely determined . For this reason, radar sensors are often used for driver assistance systems, either in addition to other sensors, with a joint evaluation of the generated sensor data then taking place, or else based entirely on radar sensors.

But even if objects in the vicinity of the vehicle and structural limits of the road, such as guard rails, can be reliably detected with radar or ultrasonic sensors under many environmental conditions, there are also situations in which the detection is falsified or disturbed and the detected objects are implausible.

With this in mind, the DE 10 2015 219 933 A1 Method for the plausibility check of measured values that have been generated, for example, by the sensors of a vehicle. To increase the security and reliability of the sensor system, an ego vehicle equipped with V2X technology uses the V2X information from V2X vehicles in the local area, among other things, to check the plausibility of its own sensor data.

It is an object of the invention to provide an improved method for detecting objects and / or structures in the vicinity of a vehicle and a corresponding device.

This object is achieved by a method with the features of claim 1 and by a device according to claim 11. Preferred embodiments of the invention are the subject of the dependent claims.

In a method according to the invention for detecting objects and / or structures in the vicinity of a vehicle, the position of the vehicle is determined. Localization data for the determined position are transmitted to a central server. Swarm data for this position and / or a subsequent future position when driving on a route are received by the central server, the swarm data being based on a statistical evaluation of sensor data that was obtained at an earlier point in time at this and / or the future position from a large number of other vehicles were recorded. The swarm data received are used to check whether sensory detection of objects and / or structures in the vicinity of the vehicle is impaired.

The swarm data can be based on sensor data that were previously determined by a large number of other vehicles when driving the same route at exactly the same positions. This is in contrast to the one in the DE 10 2015 219 933 A1 The method described is not limited to sensor data from vehicles that are located at the same point in time as the host vehicle in the vicinity of the position of the host vehicle, that is, in the current local environment of the host vehicle. Rather, the swarm data can be generated based on sensor data from other vehicles that have been recorded over a relatively long period of time, which can extend over several hours, days, weeks or months. This ensures that the swarm data for any positions of the host vehicle can be used to determine whether objects or structures in the current or upcoming vehicle environment are reliably detected with the sensors of the host vehicle and can be used, for example, by a driver assistance system, or whether the functionality of the vehicle is impaired Sensors present.

According to one embodiment of the invention, the surroundings of the vehicle are recorded with one or more sensors of the vehicle, with sensor data being generated for objects and / or structures in the surroundings of the vehicle and a plausibility check of the detection of the objects and / or structures by a comparison of the The sensor data generated by the vehicle is carried out with the swarm data.

In particular, in the event of a negative plausibility check result, the sensor data generated by the vehicle can be discarded or corrected.

It is advantageous here if, in the event of a negative plausibility check result, the sensor data generated by the vehicle and / or information about the negative plausibility check result is transmitted to the central server.

Likewise, information is preferably received from the central server about a surrounding object lying ahead on the route of the vehicle, for which a negative plausibility check result has been determined for a large number of vehicles.

Highly accurate localization data for the position of the vehicle when driving on the route are advantageously determined by means of a digital map of the surroundings and a sensor-based detection of landmarks recorded in the digital map of the surroundings.

According to one embodiment, the averaged lane of the swarm vehicles is evaluated during the plausibility check and a surrounding object detected by the vehicle's sensor is declared invalid if it is located next to the lane according to the digital map, but the sensor detection shows that the surrounding object is on this lane is located.

Likewise, according to a further embodiment, the digital map is advantageously evaluated to determine whether the vehicle will shortly be entering an environment for which the sensors are known to be impaired and, in this case, a warning about possible impairment of the sensors is output.

Furthermore, according to a further embodiment, the digital map is evaluated as to whether it has a surrounding object in the current surroundings of the vehicle that should be detected unimpaired by a sensor of the vehicle and, in the event that this surrounding object is still not detected, a warning via a possible defect, misalignment or contamination of the sensors.

Likewise, according to a further embodiment, for a surrounding object detected by a sensor of the vehicle, which according to the digital map and / or further sensor data runs essentially parallel to the direction of travel over an extended area, but for which the sensor data has a jump, the detected surrounding object is for declared invalid.

• - mindestens einen Sensor, der das Umfeld des Fahrzeugs erfasst;
• - eine Positionsermittlungseinheit zur Ermittlung der Position des Fahrzeugs;
• - eine Kommunikationseinheit zur Übertragung von Lokalisierungsdaten für die ermittelte Position des Fahrzeugs an einen zentralen Server und zum Empfang von Schwarmdaten, die generiert worden sind, indem Sensordaten, die zu einem früheren Zeitpunkt an dieser und/oder der zukünftigen Position von einer Vielzahl von weiteren Fahrzeugen erfasst wurden, statistisch ausgewertet worden sind, von dem zentralen Server; und
• - eine Auswerte- und Steuereinheit, die mittels der empfangenen Schwarmdaten prüft, ob eine Detektion von Objekten und/oder Strukturen im Umfeld des Fahrzeugs mit dem mindestens einen Sensor beeinträchtigt wird.

The device according to the invention for detecting objects and / or structures in the vicinity of a vehicle comprises

• - At least one sensor that detects the surroundings of the vehicle;
• a position determination unit for determining the position of the vehicle;
• - A communication unit for transmitting localization data for the determined position of the vehicle to a central server and for receiving swarm data that have been generated by adding sensor data that was at an earlier point in time at this and / or the future position of a large number of other vehicles have been recorded, have been statistically evaluated, by the central server; and
• an evaluation and control unit that uses the received swarm data to check whether a detection of objects and / or structures in the vicinity of the vehicle with the at least one sensor is impaired.

At least one sensor is preferably designed as a radar sensor.

• 1 zeigt ein Ablaufdiagramm eines erfindungsgemäßen Verfahrens; und
• 2 zeigt schematisch ein Fahrzeug mit einer erfindungsgemäßen Vorrichtung und einen zentralen Server, der basierend auf Lokalisierungsdaten des Fahrzeugs Schwarmdaten an das Fahrzeug überträgt.

Further features of the present invention will become apparent from the following description and the claims in conjunction with the figures.

• 1 shows a flow chart of a method according to the invention; and
• 2 shows schematically a vehicle with a device according to the invention and a central server which transmits swarm data to the vehicle based on localization data of the vehicle.

For a better understanding of the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It goes without saying that the invention is not restricted to these embodiments and that the features described can also be combined or modified without departing from the scope of protection of the invention as defined in the claims.

1 shows a flow chart of a method according to the invention. In a first method step 1, the current position data of a host vehicle are determined while driving a route, for example with a satellite-supported navigation system such as the GPS system in combination with digital map data. Particularly in connection with highly accurate map data in which positions of various landmarks such as traffic lights or traffic signs are recorded, the current position of the host vehicle can be determined very precisely by comparing it with the relative distances and positions of these landmarks measured by vehicle sensors.

The determined position is transmitted to a central server in a method step 2. This server can be provided as a backend server for the host vehicle and a large number of other vehicles on the Internet and can be part of an IT infrastructure not described further here. The communication between the respective vehicle and the server takes place via a wireless data radio connection, for example by means of mobile radio units provided in the vehicles.

The server then determines swarm data on the position transmitted by the host vehicle or a position passed by the host vehicle in the further course of the journey. For this purpose, the central server can access a database in which a comprehensive collection of swarm data for the road network in a larger geographical area is stored.

These swarm data are based on parameters that have been recorded by a large number of vehicles, the so-called vehicle swarm, at an earlier point in time when driving on the road network. For this purpose, the parameters recorded in each case were transmitted from the sponge vehicles to the central server, which statistically evaluated them with the help of suitable algorithms and determined the swarm data from them.

The swarm data can contain various metadata, such as a swarm trajectory for a road being traveled, which results from the trajectories driven by the various swarm vehicles for this road, or the lane markings, crash barriers or lane edges recognized by the swarm vehicles when driving on this road. Furthermore, recognized landmarks such as street signs can also be contained in the swarm data. If necessary, in the statistical evaluation of the sensor data, time-related parameters such as the time, the date and / or the day of the week can also be taken into account when the sensor data is recorded and thus time-dependent swarm data can be made available. Provision can also be made for the acquired sensor data to be given a time stamp and for this to be taken into account in the statistical evaluation. For example, older sensor data can be weighted less or given an expiration date based on the time stamp in order to be able to react better to changes in the road network when generating the swarm data.

The swarm data requested based on the position information of the host vehicle are then sent back from the server to the host vehicle and from the latter in a method step 3 receive. This transmission can also take place in particular via a cellular network.

The swarm data received in this way is then used in one process step 4th checked whether a sensory detection of objects and / or structures in the environment of the host vehicle is impaired in the current or future position. Such an impairment can be based in particular on faulty sensor signals, which are determined by an automatic plausibility check in one process step 5 sensor data received by sensors of the host vehicle can be detected. For example, suitable algorithms with the vehicle sensors can be used to compare objects in the vehicle environment with the swarm data for the current vehicle position.

If, for example, the guardrails next to the roadway are detected with radar and / or ultrasonic sensors in the host vehicle, the position of the guardrails determined by the host vehicle on the basis of the sensor data can be compared with information about the current position of the outer lane markings contained in the swarm data. The relative position of the guardrail and the lane marking and also the distance between the guardrail and the lane marking can then be determined.

Although the detection of the guardrails, especially with radar sensors, can usually be carried out without problems over large parts of a traveled route, undesired reflections of the radar waves emitted, for example, when driving through a tunnel or a metal bridge can lead to incorrect detection or assignment of the guardrails detected by the radar sensor. Thus, based on the detected reflections, a position of the guardrail can be incorrectly determined which is not next to the roadway being traveled on, but in the middle of this roadway.

This can be detected by comparing the sensor data with the swarm data. This can initially be done in one process step 6th it is checked whether one or more objects and / or structures in the environment of the host vehicle have been detected with sensors of the host vehicle. If this is the case, then first of all in one process step 7th checked whether these are plausible in comparison with the swarm data. In the case of plausible sensor data, this can then be done in one process step 8th can be used by a driver assistance system.

If, on the other hand, the sensor data is not plausible, for example because the distance between the guardrail and the lane marking changes abruptly due to incorrect object detection by the radar / ultrasonic sensors in a tunnel or on a metal bridge, then further action can be taken. So can initially in one process step 9 it is checked whether a correction of the corrupted data is possible, for example by shifting the position of a detected object determined on the basis of the sensor data by an offset distance known from the history. After such a correction in one process step 10 can also use the corrected data in step 8th can be used by the driver assistance system. If, on the other hand, a correction is not possible, the data of the radar or ultrasound objects concerned are set to invalid and discarded in a method step 11.

For the plausibility check of the sensor data, in addition to the information about the lane marking, further information can also be used. For example, the swarm trajectory can also be taken into account in this area in order to check whether the distance between the radar object and the swarm trajectory is plausible. Information about whether the vehicle is currently in a tunnel or a metal bridge can also be obtained by means of the swarm data or also map data and taken into account in the plausibility check.

This plausibility check significantly increases the robustness or reliability of the detected radar objects. This is particularly advantageous if an outside camera is basically also provided for the detection of corresponding objects, but this does not provide any clear data due to unfavorable lighting conditions.

Furthermore, the comparison of the radar objects with the swarm data, compared to a comparison with information from an external camera, enables a significantly further forecast, since the sponge data is not limited to the current field of view of the camera. This forecast can also be carried out for areas that are behind a curve that has yet to be driven through or within a tunnel that has not yet been driven through. This is particularly advantageous for automated vehicle functions in which the other road or lane conditions must be known at an early stage. For example, in the case of an automatic emergency stop if the driver's health is impaired, it must be ensured that the hard shoulder is sufficiently wide for an automatic change to the beach strip with the vehicle subsequently automatically stopped.

Likewise, due to the prospect of the swarm data that is possible over a greater distance, possible system limits of a driver assistance system can be indicated at an early stage. For example, before entering a tunnel, a warning can be issued that an automatic emergency stop function is only available to a limited extent in the tunnel.

Furthermore, in the event of strong deviations between the swarm data and the sensor-based object recognition, information about this can be transmitted to the central server, stored there and then used for further functional improvements. For example, information about the fact that the guardrail in a certain tunnel is apparently incorrectly recognized by radar sensors, since this is located according to the radar signals within the lane marking known from the swarm data, can be stored and displayed Vehicles are transmitted before entering this tunnel.

As described above, the swarm data can be transmitted in the form of a pull model if the swarm data is required by a vehicle and is therefore requested from the server. However, the transmission can also take place in advance in the form of a push model with subsequent storage of the swarm data in a memory of the vehicle as soon as new swarm data is available for a route section. In this case, the transmission can also take place to a large number of vehicles at the same time.

Instead of improving the robustness or reliability of detected radar objects through the plausibility check described above, the functionality of one or more sensors can also be checked by comparing the sensor data with the swarm data. For this purpose, the swarm data, possibly in combination with a high-precision digital map present in the vehicle, can be evaluated to determine whether there is an object in the surroundings in the current surroundings of the vehicle that should be detected by the vehicle sensors. If such a surrounding object is present, but no corresponding sensor data are output by a sensor of the vehicle, then this sensor may be defective, misaligned or soiled. This can then be done in one process step 12th the driver of the vehicle can be informed by means of a warning notice. The warning can be output here, for example, by a corresponding visual display in the instrument cluster or head-up display or as a voice output via the loudspeakers of the vehicle.

In 2 is schematically a vehicle F. with a device according to the invention and a central server S. shown. With the vehicle F. it can in particular be a passenger car or truck.

The vehicle has different units for the inventive detection of objects and / or structures in the vicinity of the vehicle. In particular, there is an evaluation and control unit AS provided for carrying out the method according to the invention, for example as a control unit of the vehicle F. can be configured and can have one or more microcontrollers or microprocessors.

A rough localization of the vehicle can first be made with a navigation unit N based on received GPS signals. The vehicle can then be localized very precisely in conjunction with high-precision map data and sensor-supported recording of landmarks in the vehicle's surroundings. The vehicle can have a vehicle camera for this K which includes the vehicle environment in front of and possibly also next to the vehicle. The image or video data generated by the vehicle camera can be transmitted via a digital data bus DB in the vehicle of the evaluation and control unit AS which evaluates the image data in order to identify the landmarks in the vehicle surroundings using suitable image processing methods and thus to generate localization data for the current position of the vehicle.

The vehicle also has at least one radar sensor RS with which the vehicle environment is recorded and sensor data RSD for objects and / or structures in the vicinity of the vehicle are generated. The sensor data RSD of the radar sensor are also used by the evaluation and control unit AS fed.

The vehicle also has a cellular radio transmission unit C. on, with which the localization data is transmitted to the server and swarm data SD can be received.

The evaluation and control unit AS can then based on the received swarm data SD carry out the plausibility check of the RSD sensor data described above. The data can be compared continuously in order to be able to react quickly to faulty sensor data at any time.

In the vehicle F. is still a driver assistance system FA provided, which based on the sensor data, if recognized as valid during the plausibility check, provides assistance functions such as an automatic emergency stop of the vehicle.

In addition, the vehicle can have other components that are not shown. For example, a memory can be provided for storing the swarm data.

The invention can be used in any vehicles that require high-precision localization for integrated assistance systems or automatic driving, but is not limited thereto, but can also be used, for example, in drivable robots.

List of reference symbols

1 - 12

Procedural steps

F.

vehicle

RS

Radar sensor

K

camera

AS

Evaluation and control unit

N

Navigation unit

FA

Driver assistance system

C.

Communication unit

DB

Data bus

LD

Localization data

SD

Swarm data

S.

server

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102015219933 A1 [0006, 0010]

Claims (12)

Method for detecting objects and / or structures in the vicinity of a vehicle, wherein - The position of the vehicle (F) is determined (1); - Localization data (LD) for the determined position are transmitted to a central server (S) (2); - Swarm data (SD) for this position and / or a subsequent future position when driving on a route from the central server (S) are received (3), the swarm data being based on a statistical evaluation of sensor data that was earlier on this and / or the future position of a large number of other vehicles were detected; and - By means of the received swarm data (SD) it is checked (4) whether a sensory detection of objects and / or structures in the vicinity of the vehicle is impaired. Procedure according to Claim 1 , the surroundings of the vehicle being recorded with one or more sensors (RS) of the vehicle and sensor data (RSD) being generated for objects and / or structures in the surroundings of the vehicle and a plausibility check of the detection of the objects and / or structures by a comparison of the sensor data (RSD) generated by the vehicle with the swarm data (SD) takes place. Procedure according to Claim 2 In the event of a negative plausibility check result, the sensor data (RSD) generated by the vehicle are discarded or corrected. Procedure according to Claim 2 or 3 , in the case of a negative plausibility check result, the sensor data (RSD) generated by the vehicle and / or information about the negative plausibility check result being transmitted to the central server (S). Method according to one of the preceding claims, information being received from the central server (S) about a surrounding object lying ahead on the route of the vehicle for which a negative plausibility check result has been determined for a plurality of vehicles. Method according to one of the preceding claims, wherein highly precise localization data for the position of the vehicle when driving on the route are determined by means of a digital map of the surroundings and a sensory detection of landmarks recorded in the digital map of the surroundings. Procedure according to Claim 6 , whereby the averaged lane of the swarm vehicles is evaluated during the plausibility check and a surrounding object detected with a sensor (RSD) of the vehicle is declared invalid if it is located next to the lane according to the digital map, but the sensor detection shows that this is the case Surrounding object is located in this lane. Procedure according to Claim 6 , the digital map being evaluated to determine whether the vehicle will shortly be entering an environment for which the sensors are known to be impaired and, in this case, a warning about possible impairment of the sensors is output. Procedure according to Claim 6 , with the digital map being evaluated to determine whether it has an environmental object in the current surroundings of the vehicle that should be detected unimpaired by a sensor (RSD) of the vehicle and, in the event that this environmental object is still not detected, a warning via a possible defect, misalignment or contamination of the sensors. Procedure according to Claim 6 For a surrounding object detected by a sensor (RSD) of the vehicle, which according to the digital map and / or further sensor data runs essentially parallel to the direction of travel over an extended area, but for which the sensor data has a jump, the detected surrounding object is invalid is explained. Device for detecting objects and / or structures in the vicinity of a vehicle, with - At least one sensor (S) which detects the surroundings of the vehicle (F); - A position determination unit (N) for determining the position of the vehicle; - A communication unit (C) for transmitting localization data (LD) for the determined position of the vehicle to a central server (S) and for receiving swarm data (SD) that have been generated by sensor data that was sent to this at an earlier point in time and / or the future position of a large number of other vehicles has been recorded, has been statistically evaluated, by the central server; and - An evaluation and control unit (AS), which uses the received swarm data (SD) to check whether a detection of objects and / or structures in the vicinity of the vehicle with the at least one sensor (S) is impaired. Device according to Claim 11 , wherein at least one sensor (S) is designed as a radar sensor.

Images