DE102019205469A1 - Object detection in the vicinity of a vehicle using a primary sensor device and a secondary sensor device - Google Patents

Abstract

A method for object recognition in the surroundings (60) of a vehicle, comprising receiving first surroundings data which a primary sensor device (20) generates when detecting the surroundings (60) of the vehicle; Obtaining second environment data which a secondary sensor device (30) generates when detecting the environment (60) of the vehicle; Extracting particle information from the second environment data; and processing the first environment data based on the extracted particle information.

Description

TECHNICAL AREA

The present invention relates to the field of vehicles, in particular autonomous or partially autonomous vehicles. Specifically, the present invention relates to a device, a method, a computer program product, a computer-readable storage medium and a data carrier signal for object recognition in the surroundings of a vehicle.

TECHNICAL BACKGROUND

Object recognition plays a central role in automated driving. Objects that are in the vicinity of a self-driving or partially self-driving vehicle must be detected with a high degree of accuracy and timeliness. This is used to take measures to avoid or at least reduce a collision with such objects, such as other vehicles, road users, people, animals, buildings, plantings, public and private transport facilities ... etc.

Numerous approaches to a solution have been proposed in the past for the purpose of more reliable and faster object recognition. For example disclosed DE102018115265A1 a device for transmitting a light pulse with a first time duration and for receiving a reflected light pulse with a second time duration and for determining an orientation of a target surface as a function of a difference between the first and second time periods.

However, there is a need to improve the speed and accuracy of object recognition. The known approaches to a solution are inadequate in that the data obtained by means of the environment sensors are not adequately utilized by eliminating the false positives and a large amount of information is filtered out unused. The degree of confidence that detected sensor signals can be assigned to a certain object or a certain object class is also limited in the known approaches due to the type and manner of data processing and the physical conditions underlying the sensor system used.

The present invention is therefore based on the object of improving the object recognition in the vicinity of a fully or partially autonomous vehicle with regard to speed and reliability.

The object is achieved by a device, a method, a computer program product, a computer-readable storage medium and a data carrier signal according to the independent claims.

The device according to the invention for object recognition can be, for example, an electronic control or regulating unit (ECU), an electronic control or regulating module (ECM = Electronic Control Module) or a control / regulating unit for autonomous driving ( eg an “autopilot”). The device can be located on the vehicle, or outside or partially outside the vehicle. For example, the device according to the invention can be part of a central monitoring device in road traffic, in particular in city traffic, in motorway traffic, on an at least partially closed industrial, commercial site and / or on an agricultural area. The communication between the device according to the invention and the vehicle or between the central monitoring device and the vehicle can be wireless, for example via Bluetooth, infrared, near-field communication (NFC), radio, Internet, intranet, cloud systems, blockchain systems , 4G communication networks, 5G communication networks and / or wired systems.

The vehicle can be a passenger vehicle and / or a utility vehicle, for example a land vehicle, an industrial vehicle, an industrial machine, a vehicle for a swap body, a mobile robot and / or an automated driverless transport system.

The primary and / or secondary sensor device are preferably attached to the vehicle so that an ongoing detection of the surroundings is made possible when the vehicle is moving. Alternatively, the primary and / or the secondary sensor device can be arranged in a mobile or stationary manner at another location. The communication between the primary and / or secondary sensor device and the device according to the invention can be wireless, for example via Bluetooth, infrared, near-field communication (NFC), radio, Internet, intranet, cloud systems, blockchain systems, 4G communication networks , 5G communication networks and / or wired systems.

The primary sensor device is, for example, a distance sensor, for example a radar sensor, a lidar sensor and / or an ultrasonic sensor. The first environment data generated by the primary sensor device typically contain position data of one or more of the primary Sensor device in the vicinity of the vehicle detected objects.

For example, in the event that the primary sensor device comprises a radar sensor, a radio frequency signal (hereinafter: RF signal) is sent from a transmitter of the radar sensor. The RF signal is scattered back or reflected on an object in the vicinity of the vehicle and propagates back to a receiver of the radar sensor. The period in which the RF signal is propagated between the time of transmission and reception is generally known in the technical field as the time of flight (ToF). The position of the object reflecting the RF signal can be determined from the flight time, typically in the form of an elevation angle, a lateral angle and a distance.

Alternatively, if the primary sensor device comprises a lidar sensor, a light signal can be sent from a transmitter of the lidar sensor. The light signal is scattered or reflected back on an object located in the vicinity of the vehicle and propagates back to a receiver of the lidar sensor. The period of time in which the light signal is propagated between the time of transmission and reception is also known as the time of flight. The position data of detected objects can also be determined from the flight time.

Such position data are an essential part of the first environment data. In addition, at least for the case that a radar sensor is used, the speed at which the detected object is moving can be calculated based on the Doppler effect. In the event that a lidar sensor is used, additional information (attributes) in the first environment data can be taken into account due to the intensity distribution of the reflected light signal.

The secondary sensor device is, for example, a camera, for example an RGB camera, a stereo camera and / or a surround view camera. The secondary sensor device is preferably designed to detect particles or air particles, for example rain particles, snow particles, fog particles, smoke particles, PM2.5 particles, dust particles, ... etc. in the vicinity of the vehicle. These particles can be in the air or on a surface, for example that of a housing of the first and / or second sensor device. The housing can have a transparent or semi-transparent cover side for the (partial) passage of light.

The evaluation unit is designed to extract particle information or air particle information from the second environment data. The secondary sensor device detects the particles and generates particle information based on which the first environment data are processed. This means that the first environment data are evaluated taking into account the particle information. In this way, the evaluation of the first environment data is particularly accurate and reliable, since it is adapted to the particle information and can therefore actively use it as background information on the environment of the vehicle, instead of disregarding it as general noise.

Advantageous refinements and developments are specified in the subclaims.

According to one embodiment, the method further comprises determining a frequency distribution from the first environment data.

The frequency distribution refers to a distribution of the signal strength (e.g. intensity) as a function of spatial distance. In the event that the primary sensor device is a radar and / or lidar sensor, the frequency distribution can include a histogram.

According to a further embodiment, the frequency distribution is evaluated with the aid of a fitting parameter that is adapted to the extracted particle information.

In the event that the primary sensor device comprises a radar and / or lidar sensor, the frequency distribution or histogram can be described approximately functionally by means of a fitting analysis in order to determine the spatial distance with the highest frequency value. This fitting analysis uses one or more fitting parameters, which in the present case are adapted to the extracted particle information. The extracted particle information is thus taken into account by the fitting parameters. The first environment data can therefore be evaluated particularly precisely.

According to a further embodiment, the fitting parameter relates to a Gaussian distribution for the approximate description of the frequency distribution.

The Gaussian distribution is particularly suitable for describing a histogram in order to determine the distance of an object detected by a radar and / or lidar sensor with high accuracy. The Gaussian distribution can be defined by means of a fitting parameter, which can be, for example, a half width, a main maximum or a secondary maximum of the Gaussian distribution.

The computer program product according to the invention is designed to be loaded into a memory of a computer and comprises software code sections with which the method steps of the method according to the invention are carried out when the computer program product is running on the computer.

A program belongs to the software of a data processing system, for example an evaluation device or a computer. Software is a collective term for programs and associated data. The complement to software is hardware. Hardware describes the mechanical and electronic alignment of a data processing system. A computer is an evaluation device.

Computer program products generally comprise a sequence of instructions which, when the program is loaded, cause the hardware to carry out a specific method that leads to a specific result. When the program in question is used on a computer, the computer program product produces the inventive technical effect described above.

The computer program product according to the invention is platform independent. That means it can run on any computing platform. The computer program product is preferably executed on an evaluation device according to the invention for detecting the surroundings of the vehicle.

The software code sections are written in any programming language, for example in Python, Java, JavaScript, C, C ++, C #, Matlab, LabView, Objective C.

The computer-readable storage medium is, for example, an electronic, magnetic, optical or magneto-optical storage medium.

The data carrier signal is a signal which the computer program product from a storage medium on which the computer program product is stored to another entity, for example another storage medium, a server, a cloud system, a wireless communication system 4G / 5G or a data processing facility, transmits.

• 1 eine schematische Darstellung einer Vorrichtung zur Objekterkennung in einem Umfeld eines Fahrzeugs gemäß einer Ausführungsform; und
• 2 eine schematische Darstellung eines Histogramms;
• 3 eine schematische Darstellung einer Sensoreinrichtung in einer Draufsicht; und
• 4 eine weitere schematische Darstellung der Sensoreinrichtung aus 3 in einer perspektivischen Ansicht.

Embodiments will now be described by way of example and with reference to the accompanying drawings. Show it:

• 1 a schematic representation of a device for object recognition in an environment of a vehicle according to an embodiment; and
• 2 a schematic representation of a histogram;
• 3 a schematic representation of a sensor device in a plan view; and
• 4th a further schematic representation of the sensor device 3 in a perspective view.

In the figures, the same reference symbols relate to the same or functionally similar reference parts. The relevant reference parts are identified in the individual figures.

1 shows a schematic representation of a device 10 for object recognition in the surroundings of a vehicle according to one embodiment.

The device 10 comprises a first data input 12 for obtaining first environment data which a primary sensor device 20th generated when detecting the surroundings of the vehicle. The primary sensor device 20th is preferably a radar sensor, a lidar sensor or a combined sensor system consisting of a radar part and a lidar part. The primary sensor device 20th can be arranged on the vehicle. The primary sensor device 20th has a first detection area which, for example, has a conical shape.

The device 10 also includes a second data input 14th to obtain second environment data, which a secondary sensor device 30th generated when detecting the surroundings of the vehicle. The secondary sensor device 30th is preferably a camera, in particular an RGB camera. The secondary sensor device 30th can also be vehicle 50 be arranged. The secondary sensor device 30th has a second detection area.

The device 10 further comprises an evaluation unit 16 for extracting material class data from the second environment data. The evaluation unit 16 is also designed to combine the first environment data with the extracted material class data as described in more detail above.

The device 10 finally includes a signal output 18th for forwarding a control signal based on the evaluation by the evaluation unit 16 is generated. The control signal can be output to control the vehicle n the vehicle, in particular to an actuator of the vehicle.

2A-B each shows a schematic representation of a histogram which is derived from the first environment data is obtained. The primary sensor device 20th is preferably a lidar sensor. The secondary sensor device 30th is preferably an RGB camera. In 2A the histogram refers to an environment under good weather conditions. In 2 B the histogram refers to an environment under foggy weather conditions. Compared to the histogram in 2A shows the histogram in 2 B a main maximum 62a , b which is surrounded by secondary maxims with higher detection values. These higher detection values of the secondary maxims are due to the fog particles, since the fog particles backscatter or reflect the light signal emitted by the lidar sensor, which increases the number of light signals received.

In order to take account of the increased fog particle concentration, it is advantageous to adapt the computational model on which the evaluation of the first environment data is based to the secondary maxims. According to the invention, this is implemented in that the fog particles are recorded by the RGB camera, so that the second environment contains data relating to fog particles. Based on the second environment data, a Gaussian distribution can thus be selected, for example, which the histogram in 2 B describes approximately with higher accuracy.

3 shows a schematic representation of the primary sensor device 20th in a top view. The sensor device 20th is a lidar sensor and includes a transmitter unit 26th for emitting a laser light signal and a receiving unit 28 for receiving the laser light signal after it has been scattered back from an object in the vicinity of the vehicle. The lidar sensor has a housing 22nd on, which has a transparent front cover 24 includes. The sender and receiver unit 26th , 28 are to the front cover 24 aligned towards. Furthermore, in this example there is the secondary sensor device 30th , which is an RGB camera, also in the housing 22nd arranged. In particular, the RGB camera is on the front cover 24 attached so that the RGB camera is over the transparent material of the front cover 24 can capture the surroundings of the vehicle. In 4th is the sensor device 20th , 30th shown in a perspective view.

If there are particles on the front cover in poor weather or air conditions 24 precipitate, these are recorded by the RGB camera. The resulting (second) environment data therefore contain the particle information. As described above, these can be used to evaluate the (first) environment data of the lidar sensor. The evaluation of the first environment data and the associated object recognition are therefore particularly reliable.

List of reference symbols

10

contraption

12

first data entry

14th

second data input

16

Evaluation unit

18th

Signal output

20th

primary sensor device

22nd

casing

24

Front cover

26th

Sending unit

28

Receiving unit

30th

secondary sensor device

32

second detection area

60

environment

62a, b

Main maximum

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102018115265 A1 [0003]

Claims (10)

A method for object recognition in the surroundings (60) of a vehicle, comprising: - Receiving first environment data which a primary sensor device (20) generates when detecting the environment (60) of the vehicle; - Receiving second environment data which a secondary sensor device (30) generates when detecting the environment (60) of the vehicle; Extracting particle information from the second environment data; and - Processing of the first environment data based on the extracted particle information. Procedure according to Claim 1 , further comprising determining a frequency distribution from the first environment data. Procedure according to Claim 2 , the frequency distribution being evaluated with the aid of a fitting parameter that is adapted to the extracted particle information. Procedure according to Claim 3 , whereby the fitting parameter relates to a Gaussian distribution for the approximate description of the frequency distribution. Procedure according to Claim 4 , wherein the fitting parameter comprises a half width, a main maximum or a secondary maximum of the Gaussian distribution. Device (10) for object recognition in the surroundings (60) of a vehicle, comprising: - A first data input (12) for receiving first environment data, which a primary sensor device (20) generates when detecting the environment (60) of the vehicle; - A second data input (14) for receiving second environment data which a secondary sensor device (30) generates when detecting the environment (60) of the vehicle; and - An evaluation unit (16) for extracting particle information from the second environment data and for processing the first environment data based on the extracted particle information. Use of the device (10) after Claim 6 in a vehicle. Computer program product, comprising instructions which, when the program is executed by a computer, cause the computer to perform the method according to Claim 1 execute. Computer-readable storage medium on which the computer program product is based Claim 8 is stored. Data carrier signal that the computer program product according to Claim 8 transmits.

Images