DE102018221449A1 - Sensor system for object detection - Google Patents

Abstract

Sensor system (1) for object detection, in particular for a vehicle (2), comprising a sensor for object detection that generates sensor data using electromagnetic and / or optical signals, a device for acoustic data acquisition of audio data, and a data device (7) for joint data acquisition and / or data processing of the sensor data of the sensor for object detection and the audio data.

Description

The present invention relates to a sensor system for object detection, in particular for a vehicle, and a vehicle with such a sensor system.

Technological background

Generic vehicles or means of transportation, such as. B. motor vehicles or motorcycles are increasingly equipped with driver assistance systems, which use sensor systems to detect the surroundings, recognize traffic situations and support the driver, e.g. B. by braking or steering intervention or by issuing an optical or acoustic warning. Radar sensors, lidar sensors, camera sensors or the like are regularly used as sensor systems for environmental detection. Conclusions about the environment can then be drawn from the sensor data recorded by such sensors.

The environmental detection by means of radar sensors is based on the emission of bundled electromagnetic waves and their reflection, e.g. B. by other road users, obstacles on the road or the edge of the road. The signals received by the radar sensor can then be used to draw conclusions about the objects which have reflected the radar signal. These objects can then be classified, for example into moving objects or targets, such as. B. other road users or static, immovable objects such. B. the edge development of the roadway, crash barriers or the like. The classification of objects is possible using various parameters, such as B. backscatter coefficient / RCS (Radar Cross Section), micro-Doppler signature, motion profile or the like. However, conventional radar sensors cannot be classified arbitrarily. For example, an ambulance vehicle (with an active Martinshorn) cannot be distinguished from a 3.5 t truck. However, this can be of crucial importance for different traffic situations, in particular with (partially) autonomous vehicle guidance, for example when planning overtaking and evasive maneuvers or, for example, when creating an emergency lane. Accordingly, the distinction of such obstacles or the correct object classification is of particular importance in order to determine when e.g. B. a braking or steering intervention should be initiated.

Printed state of the art

From the EP 2 833 336 A1 a driver assistance device is known which has a separate microphone which is mounted on the vehicle in order to record sound information in the surroundings of the vehicle. In addition, a driver assistance device is provided that carries out driver assistance that is based on sound information recorded by the microphone. In addition to the microphone, another means of environmental detection can also be used, e.g. B. a camera or radar, which may include the separate microphone. This arrangement is intended to improve the accuracy in the detection of the surroundings.

The US 8,525,654 B2 discloses blind spot detection for a vehicle. For this purpose, the blind spot detection comprises a microphone that detects noises while driving. In the event that another vehicle is now in the blind spot of the vehicle, all driving noises are initially recognized by the microphone. The vehicle noise of the vehicle located in the blind spot is then extracted by means of a vehicle noise extraction unit. The extracted vehicle noise can then be used to determine whether or not a vehicle is in the blind spot.

Object of the present invention

The object of the present invention is now to provide a sensor system by means of which the object recognition is improved in a simple and inexpensive manner based on the prior art.

Solution of the task

The above object is achieved by the entire teaching of claim 1 and the independent claim. Advantageous embodiments of the invention are claimed in the subclaims.

The sensor system according to the invention for object detection is especially prepared for the object detection of a vehicle and serves z. B. for sensor data acquisition of a driver assistance system. The sensor system comprises a sensor for object detection, which generates sensor data using electromagnetic and / or optical signals, and at least one device for acoustic data acquisition of audio data. Furthermore, a data device is provided for common data acquisition and / or data processing of the sensor data of the sensor for object detection and the audio data, ie within a unit. This can create a merger and common The sensor and audio data are evaluated and used to classify objects. This has the advantage that, in addition to object detection by means of electromagnetic and / or optical signals, acoustic signals or audio data can also be used to detect the objects and then classify them. For example, a better classification of an ambulance with an active warning signal can be carried out by recognizing the ambulance or its body using the sensor data and the associated warning signal based on the audio data. The merger can particularly improve the security of the object classification. Compared to the prior art, the present sensor system can thus be used to implement a novel integration of a device for acoustic data acquisition in a sensor for object detection, ie a highly integrated combination at the hardware and software level, e.g. B. by a common arrangement and a common data acquisition and / or data processing. The detection and classification of the objects is particularly improved since the audio data record an additional, redundant parameter of the respective object, which can also be evaluated with the sensor data for object detection. Another advantage arises for such a combined sensor when used in electric vehicles, eg. B. in urban scenarios, due to the comparatively low engine noise or due to the characteristics of the easily compensated engine noise of an electric vehicle.

The sensor for object detection is preferably a radar sensor, so that the sensor system represents a combined radar-acoustic sensor. Alternatively or additionally, however, a lidar sensor or camera sensor or another sensor known from the prior art for object detection can also be provided.

According to a preferred embodiment of the sensor system, the device for acoustic data acquisition comprises a microphone. In an advantageous manner, however, at least two microphones are provided for acoustic data acquisition in order to thus generate a stereo recording or stereo data acquisition.

The distance between the eardrums in the human ear canal is in the range of about 20 cm. The distance between the microphones is expediently 3 cm to 25 cm, preferably 4 cm to 20 cm, preferably 5 cm to 15 cm, in a particularly preferred manner 6 cm to 14 cm. Regarding the acoustic location property, i.e. H. the source angle determination as a function of frequency, the human stereo hearing can thus be used for orientation.

The sensor for object detection expediently comprises a housing. For example, the housing can be the sensor housing of a conventional radar sensor. The device for acoustic data acquisition or the microphones can be arranged in or on the housing in a simple manner. The invention also expressly includes arrangements in which, for. B. the microphones are only partially arranged within the housing. The manufacturing and material costs can be reduced to a particular extent.

The data device for common data acquisition preferably has a multiplexer which is arranged in terms of switching technology between the sensor for object recognition and the device for acoustic data acquisition, i. H. between the radar sensor and the microphone or the microphones, and in particular is designed to switch them alternately, so that the sensor data and the audio data are recorded alternately.

Alternatively or additionally, the data device can also acquire the audio data during a transmission and / or reception pause when the radar signals are transmitted. This results in the advantage that an oversampling of the audio signal occurs (e.g. sampling with e.g. 20 MHz instead of at 20 kHz). B. the phase noise can be determined. Furthermore, an increase in the dynamic range can be achieved, which results in a higher sensitivity.

According to a special embodiment of the invention, the audio data and the radar data can be sampled together by the data device.

For common (i.e. time-synchronous) sampling, the data device and / or the device for acoustic data acquisition can comprise a coupler that pre-filters the audio data. In this case, the audio data are preferably low-pass filtered so that they can then be scanned together with the radar data. In the same way, high-pass filtering of the radar data can also be provided.

The data device can expediently separate the sensor data or the radar data or the radar signal and the audio data or the audio signal within the digital domain. It has proven to be particularly advantageous to carry out the separation using filtering, the sensor data or radar signals using a high-pass filter and the Audio data are filtered using a low-pass filter to separate them.

For this purpose, a microcontroller or microprocessor is preferably provided for the common radar data and audio data processing. The microcontroller can be e.g. B. the microcontroller of a radar sensor, which is usually used for radar signal processing. The invention can thus be implemented in a particularly simple and cost-effective manner and can be retrofitted in existing systems.

Furthermore, the audio signals from the microphones can be used to detect mechanical defects or malfunctions on the vehicle. For example, mechanical defects on the vehicle often cause additional noise that can be picked up by the microphones. By reading or listening to and analyzing the vehicle noise, perforated exhaust systems, rattling bearings, other conspicuous engine noises, defective tires or the like can be identified.

In addition, by listening to tire noises or tire rolling noises of the vehicle's tires, the device for acoustic audio signal detection (microphones) can be used to identify the roadway properties, since the tire noise or the soundtrack, depending on the surface condition of the roadway or the roadway condition, are characteristic audio features exhibit. For example, it can be recognized whether the roadway z. B. is wet, dry, snowy or icy. In addition, it can also be recognized whether it is z. B. is asphalt, cobblestone, concrete, forest floor, gravel or the like. The lane properties or the condition of the lane can then either be displayed to the driver or used as parameters for controlling (partially / autonomously) driving functions and assistance systems. The safety of the respective assistance system or the driving function can be improved to a particular degree.

The radar data or radar signals and / or audio data or audio signals of several sensors or radar sensors with integrated microphones can expediently also be processed together. This enables an improved localization of sound sources to be achieved. The common processing can be done in a simple manner, for. B. on one of the radar sensors, the data device or a central processing unit.

Furthermore, the present invention claims a vehicle with a sensor system according to the invention, which z. B. is part of a driver assistance system or a controller for (partially) autonomous vehicle guidance.

Figure list

• 1 eine vereinfachte schematische Darstellung eines erfindungsgemäßen Sensorsystems mit einem Radarsensor und zwei integrierten Mikrofonen;
• 2 eine vereinfachte schematische Darstellung verschiedener Ansichten des Sensorsystems aus 1;
• 3 eine vereinfachte schematische Darstellung einer Radar-Datenerfassung;
• 4 eine vereinfachte schematische Darstellung eines Ausführungsbeispiels eines Schaltbildes zur Integration der Audio-Datenerfassung, sowie
• 5 eine vereinfachte schematische Darstellung einer Implementierungsvariante zur kombinierten Radar-Audio-Datenerfassung.

The invention is explained in more detail below on the basis of practical exemplary embodiments. Show it:

• 1 a simplified schematic representation of a sensor system according to the invention with a radar sensor and two integrated microphones;
• 2nd a simplified schematic representation of different views of the sensor system 1 ;
• 3rd a simplified schematic representation of a radar data acquisition;
• 4th a simplified schematic representation of an embodiment of a circuit diagram for integrating the audio data acquisition, and
• 5 a simplified schematic representation of an implementation variant for combined radar audio data acquisition.

Reference number 1 in 1 describes a sensor system in a vehicle 2nd is arranged. The sensor system is preferably 1 in the front area of the vehicle 2nd arranged such. B. in front of the engine compartment, behind the radome, in the area of the bumper, behind the windshield or the like (using two sensors as an example in 1 shown). According to 2nd points the sensor system 1 a radar sensor 3rd for object detection using radar data generated from electromagnetic signals and two microphones 4th , 5 for acoustic data acquisition. With the microphones 4th , 5 can it be z. B. MEMS (Microelectromechanical Systems) microphones that z. B. be used in the field of mobile communications. The microphones 4th , 5 are preferably on the sides of the case 6 of the radar sensor 3rd arranged as in 2nd shown, these either outside or inside the housing 6 can be arranged. The acoustic damping when installed inside the housing 6 , can be particularly useful for suppressing noise. In a practical way it can be the case 6 the radar sensor housing of a conventional radar sensor so that the microphones 4th , 5 are a kind of extension of an existing radar sensor system. With such an arrangement, the microphones are installed 4th , 5 e.g. B. in the area of the pressure compensation membrane of the radar sensor 3rd . Furthermore, a chip (MMIC) for acoustic data acquisition can be provided.

The microphones preferably comprise 4th , 5 mechanical low-pass filter to avoid noise, e.g. B. from the vehicle 2nd themselves or induced by flow acoustics. Surprisingly, it has been shown that, in particular when radar sensors are installed behind the bumper / radome, there is the advantage that they can (depending on the material) already serve as an acoustic filter (in particular for suppressing influences by flow acoustics). The distance between the two microphones 4th , 5 , depending on the design of the radar sensor (e.g. long / short range radar), is approximately 6 cm to 14 cm.

According to the invention, the electronics of an existing radar sensor are proposed for data acquisition 3rd which is usually an analog-digital converter ( ADC or AD converter) and possibly an amplifier, to use or to expand additional elements. With the radar sensor 3rd the data are stored in so-called radar cycles with a cycle time of e.g. B. 20 ms to 100 ms. A cycle is made up of several send-receive processes, so-called “chirps” with a duration of 50 µs, for example. For each chirp or radar chirp z. B. with an AD converter after a transmitter ( TX ) Settling phase (without active data acquisition) acquires a certain number of "radar samples", especially once per radar receiver ( RX ) Channel (e.g. 512 samples), as in 3rd using the radar chirps as an example R1 , R2 , R3 shown.

The acoustic data acquisition can take place via different implementation variants. According to the invention, this is a data device 7 for common data acquisition and / or data processing of the sensor data and the audio data. 4th shows a first implementation variant of acoustic data acquisition, with the help of a multiplexer MUX between the sources, RX channel (radar receiver channel) and microphone, can be switched back and forth. Amplifiers can be used for (signal) amplification 8th be arranged. An amplifier can also expediently be used 8th can be used for both paths, ie radar and audio, in which case the multiplexer MUX then in front of the amplifier 8th is arranged. A microcontroller is used for radar signal processing 9 arranged the z. B. also the AD converter ADC may include. The acoustic data acquisition (ie the scanning of the microphones) is also integrated in the radar data acquisition, ie is also carried out by the microcontroller 9 accepted. As in 5 shown, the data acquisition is carried out by switching two of the ADC converter inputs to the microphones via a multiplexer during a phase of inactive radar data acquisition (e.g. during the TX settling phase), typically a single audio sample AS1 , AS2 , AS3 (ie one audio sample per microphone) is recorded. The audio sampling of the implementation variant 5 is preferably carried out at 50 µs per channel, ie a chirp repetition rate of 20 kHz.

As an alternative to the first implementation variant according to 4th The acoustic data acquisition can also be carried out by switching on the acquisition of the audio data or the audio recording during the pause in transmission of the radar signals. A radar measurement cycle is z. B. 20 to 25 ms. With a cycle time of 50 ms, this corresponds to a duty cycle of 50%. The remaining 50% are made available for audio recording. The resulting oversampling of the audio signal (sampling with e.g. 20 MHz instead of 20 kHz) offers additional advantages, such as. For example, the phase noise can be determined or the dynamic range (DR) can be increased, resulting in a higher sensitivity. For example, with an oversampling factor of 1000 (corresponding to 20 MHz instead of 20 kHz), the dynamic range can be increased by approximately 10 bits, which improves the signal-to-noise ratio (SNR, signal-to-noise ratio) is about 30 dB.

Another implementation variant for simultaneous data acquisition is the use of an analog, steep high-pass filter 10th in the radar RF signals. All signals up to z. B. 50 or 100 kHz filtered out (self-coupling). By using a coupler instead of the multiplexer MUX in 4th , the audio data (low-pass filtered) are fitted into a lower frequency range (e.g. 50 to 100 kHz) and sampled together with the radar signals. The separation of the signals in the digital domain is then implemented using appropriate digital filters. For example, a high pass filter 10th for the radar signal and a low pass filter (in 4th not shown) can be used for the audio signal. Alternatively, bandpass filters can also be used. In addition to the implementation variants described, additional components known from the prior art can also be incorporated into the radar sensor for data acquisition 3rd and / or the microcontroller 9 or chip (MMIC) for acoustic data acquisition.

To process the audio signals, e.g. B. the classification or the angle determination, known methods from audio signal processing are used, such as. B. Voice recognition. The angle determination of a loud and clearly identifiable audio source, such as B. Martinshorn, dog axes, kicks or the like, can be done for example via a cross correlation. The audio signal processing can either also on the microcontroller 9 for radar signal processing be carried out or it can be done on a central control unit if this z. B. has only a low audio bandwidth.

In addition to classic signal processing, algorithms and approaches from the area of machine learning (in particular from the area of voice recognition) can also be used for object detection and object classification.

When installing several radar microphone sensor units on one vehicle 2nd ("Multi-sensor setup"), where the sensor units 1 For example, arranged centrally forward and diagonally forward, the respective audio signals or audio data can also be merged centrally on a control unit. The advantages here are improved localization of the noise source and better noise suppression. For example, four short-range radar sensors and one long-range radar sensor with two microphones each could be installed, so that a total of ten microphones are available for improved source location, thereby improving the source location and object classification.

Using the microphones 4th , 5 can also the tire noises or tire rolling noises of the tires 2a of the vehicle 2nd are detected in order to carry out a recognition of the road properties. The tire noise or the sound track of this depends on the surface condition of the road or the condition of the road, so that it is recognized whether the respective road z. B. is wet, dry, snowy or icy. This information is then displayed to the driver or as a parameter for controlling assistance systems in the vehicle 2nd used

Furthermore, the invention may e.g. B. can also be used in the field of monitoring and security technology. For example, when monitoring building access. Advantages of the invention are offered by a compact design and the possibility of optically hiding the sensor system behind a cover. It is particularly advantageous in this embodiment that the sensor system does not have to have a direct “view” of the room to be monitored, as is required, for example, in the case of a camera, so that the sensor system can be arranged so that it is protected from view and destruction.

In summary, a synchronized detection of radar signals and audio signals can be realized by the present invention. As a result, audio objects (angular movement, volume change via movement or the like) can be associated with radar objects for improved object classification over several cycles (for example by tracking). This association can be on a central control unit or on the radar sensor 3rd yourself. The detected radar objects can thus be expanded or supplemented by audio features. By such an inexpensive extension of a radar sensor 3rd new possibilities for object classification are created. In addition, sound sources can be reliably recognized and associated with radar objects. As a result, an already synchronized radar and audio data acquisition is made available, which is also particularly suitable for electric vehicles, since these cause only slight or characteristic motor noises. In addition, non-deterministic sources of interference from the vehicle interior, such as. B. radio, mobile phone, calls or the like, only a very small influence when the microphones 4th , 5 already in the radar sensor 3rd or on / in its housing 6 are implemented.

Reference symbol list

1

Sensor system

2nd

vehicle

2a

tires

3rd

Radar sensor

4th

microphone

5

microphone

6

casing

7

Data setup

8th

amplifier

9

Microcontroller

10th

High pass filter

MUX

multiplexer

ADC

Analog-to-digital converter

R

Radar signal recording

A

Audio signal recording

AS1

Audio sample

AS2

Audio sample

AS3

Audio sample

R1

Radar chirp

R2

Radar chirp

R3

Radar chirp

TX

Sender channel (transmitter)

RX

Receiver channel

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 2833336 A1 [0004]
• US 8525654 B2 [0005]

Claims (15)

Sensor system (1) for object detection, in particular for a vehicle (2), comprising a sensor for object detection, which generates sensor data using electromagnetic and / or optical signals, a device for acoustic data acquisition of audio data, and a data device (7) for the joint data acquisition and / or data processing of the sensor data and the audio data. Sensor system (1) after Claim 1 , characterized in that the sensor for object detection is a radar sensor (3), lidar sensor or camera sensor. Sensor system (1) after Claim 1 or 2nd , characterized in that at least two microphones (4, 5) are provided as the device for acoustic data acquisition. Sensor system (1) after Claim 3 , characterized in that the distance between the microphones (4, 5) is 3 cm to 25 cm, preferably 4 cm to 20 cm, preferably 5 cm to 15 cm, particularly preferably 6 cm to 14 cm. Sensor system (1) according to at least one of the preceding claims, characterized in that the sensor for object detection has a housing (6) and the device for acoustic data acquisition is arranged in or on the housing (6). Sensor system (1) according to at least one of the preceding claims, characterized in that the data device (7) has a multiplexer (MUX), which is arranged between the sensor for object detection and the device for acoustic data acquisition, and in particular is arranged between them to be switched alternately so that the sensor data and the audio data are recorded alternately. Sensor system (1) according to at least one of the preceding claims, characterized in that the acquisition of the audio data by the data device (7) takes place during a transmission and / or reception pause of the radar signals. Sensor system (1) according to at least one of the preceding claims, characterized in that the audio data and the radar data are sampled together by the data device (7). Sensor system (1) according to at least one of the preceding claims, characterized in that the data device (7) and / or the device for acoustic data acquisition comprises a coupler which filters the audio data. Sensor system (1) according to at least one of the preceding claims, characterized in that the data device (7) carries out a separation of the sensor data and audio data within the digital domain, in particular by the sensor data being high-pass filtered and the audio data being low-pass filtered. Sensor system (1) according to at least one of the preceding claims, characterized in that a microcontroller (9) is provided for the common radar data and audio data processing. Sensor system (1) according to at least one of the preceding claims, characterized in that the audio data are used to detect defects or malfunctions on the vehicle (2). Sensor system (1) according to at least one of the preceding claims, characterized in that tire sounds of the tires of the vehicle (2) are recorded on the basis of the audio data in order to determine the condition of the road surface or properties of the road surface. Sensor system (1) according to at least one of the preceding claims, characterized in that a plurality of sensors for object detection are provided, in particular with integrated devices for acoustic data acquisition, and the sensor data and audio data are processed together by the data device (7). Vehicle (2) with a sensor system (1), characterized in that the vehicle (2) has a sensor system (1) according to at least one of the preceding claims.

Images