DE102022212819A1 - Method for compressing sensor data of at least one sensor of a vehicle - Google Patents

Abstract

The invention relates to a method (100) for compressing sensor data (300) of at least one sensor (40) of a vehicle (1), comprising the following steps:- determining the sensor data (300) which are specific to an at least one-dimensional spectrum (300) of a sensor signal received from the at least one sensor (40),- determining relevant areas (302) in the sensor data (300) which are intended for object evaluation in an environment (2) of the vehicle (1), wherein the relevant areas (302) partially overlap,- detecting the overlap of the relevant areas (302) for compressing the sensor data.

Description

The present invention relates to a method for compressing sensor data from at least one sensor of a vehicle. The invention also relates to a computer program and a device for this purpose.

State of the art

Scene recognition for autonomous vehicles requires precise detection and classification of objects and other road users. It is known from the state of the art to use a radar device on the vehicle for this purpose.

Vehicle radar has shown great potential as a sensor for driver assistance systems due to its robustness to weather and lighting conditions. However, reliable detection and classification of radar object types in real time has proven to be very difficult.

To enable autonomous driving, it is therefore desirable to improve the detection and classification performance of radar sensors, e.g. to be able to detect and classify cars, pedestrians, cyclists, motorcyclists and small objects such as cans or manhole covers.

Disclosure of the invention

The subject matter of the invention is a method with the features of claim 1, a computer program with the features of claim 9 and a device with the features of claim 10. Further features and details of the invention emerge from the respective subclaims, the description and the drawings. Features and details that are described in connection with the method according to the invention naturally also apply in connection with the computer program according to the invention and the device according to the invention, and vice versa, so that with regard to the disclosure of the individual aspects of the invention, reference is or can always be made to each other.

• - Ermitteln der Sensordaten, welche für ein mindestens ein- oder mindestens zweidimensionales oder mindestens dreidimensionales oder mindestens vierdimensionales Spektrum eines von dem wenigstens einen Sensor empfangenen Sensorsignals spezifisch sind,
• - Bestimmen von relevanten Bereichen in den Sensordaten, welche für eine Objektauswertung, wie eine Objektdetektion und/oder Objektklassifikation und/oder Segmentierung und/oder einer anderen Anwendung, vorzugsweise von mindestens einem oder mehreren Objekten, in einer Umgebung des Fahrzeuges vorgesehen sind, wobei die relevanten Bereiche teilweise überlappen (d. h. eine Überlappung aufweisen),
• - Detektieren der Überlappung der relevanten Bereiche zur Komprimierung der Sensordaten.

A method according to the invention is used in particular for compressing sensor data from at least one sensor of a vehicle. The following steps can be provided, which are preferably carried out one after the other or in any order and/or automatically:

• - Determining the sensor data which are specific to an at least one- or at least two-dimensional or at least three-dimensional or at least four-dimensional spectrum of a sensor signal received by the at least one sensor,
• - Determining relevant areas in the sensor data which are intended for an object evaluation, such as object detection and/or object classification and/or segmentation and/or another application, preferably of at least one or more objects, in an environment of the vehicle, wherein the relevant areas partially overlap (ie have an overlap),
• - Detecting the overlap of the relevant areas to compress the sensor data.

This has the advantage that the need for resources such as a data storage device when processing the sensor data can be significantly reduced. In addition, this can enable improved processing of the sensor data, e.g. for the detection and/or classification of objects. The sensor data can be implemented, for example, as radar data, preferably as a radar spectrum in which objects in the environment can be evaluated, i.e., detected and/or classified, depending on a spatial position and/or relative speed and/or distance and/or the like. Accordingly, the sensor can be designed as a radar sensor or the like. It is also possible for the sensor to be attached to the vehicle in such a way that the sensor can detect the environment of the vehicle. For example, the sensor is aligned in the direction of travel for this purpose. The relevant areas can be provided, for example, for object evaluation and in particular detection in such a way that at least one object in the environment can be evaluated, preferably detected and/or classified, by evaluating the relevant areas. For this purpose, the relevant areas include, for example, data values that indicate a distance and/or a position and/or a relative speed and/or the like of the object. This is particularly the case for objects that are close to one another and also when If several reflections are measured from an object, such as radar reflections, these areas can also overlap.

It is possible that for compression, in particular for storage and/or transmission, the sensor data are not retained in full, but (only) the relevant areas are selected, preferably stored and/or transmitted. If each area were to be selected in full, this could lead to redundancy of the data values if the areas overlap. Therefore, according to the invention, it can be provided that the overlaps are detected in order to at least reduce the redundancy of the information. For example, the overlapping areas are only selected once, e.g. stored and/or transmitted.

For the evaluation, preferably detection and/or classification, of objects using radar data, methods can generally be used that use either radar reflections or radar spectra or a combination of these, i.e. both radar spectra and radar reflections, as input data. To calculate the radar reflections, the radar can send out a sensor signal, receive it and calculate the radar spectra from it (e.g. distance, Doppler and/or angle spectra). Possible radar reflections can be detected in the spectrum as peaks in the radar spectra, for example by peak detection such as a so-called "constant false alarm detector". Various attributes such as distance, relative radial velocity, azimuth angle, elevation angle and/or radar cross section (RCS) can be calculated for these reflections. The set of these radar reflections is also referred to as a radar point cloud.

The radar spectra generally contain even more information than the detected radar reflections. Therefore, it has proven advantageous to use not only the radar point cloud, but the radar spectra or the combination of radar spectra and radar point cloud as input data for an algorithm, e.g. an artificial neural network (NN). An approach using radar spectra as input data is described in [1] (the references mentioned are given at the end of the description). The relevant areas of the spectrum (region of interest, ROI) are extracted and used as input for a NN.

A combination of radar spectra and radar point clouds as input data for a NN is used in [2]. The relevant areas of the radar spectrum are identified and used together with the radar point cloud.

Using radar spectra as input for a neural network (NN) can generally be more computationally and memory intensive than radar point cloud based approaches. For example, the radar spectrum must be stored even though it is very sparse and only a small part of the spectrum is used for further processing. Therefore, it may be useful to use a more memory efficient method to efficiently store the radar spectrum. This is particularly advantageous when the radar spectrum is to be transmitted from the radar sensor to a central processing unit, as only a limited bandwidth may be available for this. A compressed radar spectrum can be transmitted with a lower bandwidth.

Using radar spectra as input for a neural network (NN) is therefore often more computationally expensive and requires more memory than radar point cloud based approaches.

The method proposed according to the invention has the advantage that it can be identified in advance which part of the radar spectra is relevant for the evaluation, such as detection and/or classification of an object. Only this part therefore needs to be stored. A method is therefore proposed in particular for storing this part more efficiently and in this way reducing the amount of data to be stored.

Furthermore, it is optionally possible within the scope of the invention that the detected overlap of the relevant areas is used for compression and preferably for redundancy reduction in that, on the basis of the detection, the compressed sensor data are obtained in which the amount of overlap is at least reduced or completely eliminated. The overlapping parts can comprise redundant information, so that the detection is carried out for redundancy reduction.

Furthermore, it is conceivable that the sensor data are designed as the at least one- or at least two-dimensional (or multi-dimensional) spectrum of the received sensor signal and consist of a processing of the received sensor signal, wherein the sensor data can be present in an at least one- or at least two-dimensional or multi-dimensional form, wherein the relevant regions each identify an at least one- or at least two- or multi-dimensional region of the sensor data which contains information about at least one detected point of at least or exactly one (detected) object in the surroundings of the vehicle. In particular, peak value detection can be used to determine those points in the sensor data which identify at least one (detected) object. The relevant regions can be determined, for example, by selecting an at least one- or at least two-dimensional region around the respective points. This can result in overlaps if points are arranged close to one another.

• - Detektieren von mehreren relevanten Punkten, also Points of interest (POI), in den Sensordaten, welche für die Objektauswertung und vorzugsweise - detektion in der Umgebung des Fahrzeuges vorgesehen sind, vorzugsweise durch eine Spitzenwerterkennung,
• - Bestimmten der relevanten Bereiche als mindestens ein- oder mindestens zweidimensionale Bereiche um die detektierten relevanten Punkte, sodass vorzugsweise jeder der relevanten Punkte von einem der relevanten Bereiche umgeben ist.

It may also be possible that determining the relevant areas includes the following steps:

• - Detecting several relevant points, i.e. points of interest (POI), in the sensor data, which are intended for object evaluation and preferably detection in the surroundings of the vehicle, preferably by peak value detection,
• - Determining the relevant regions as at least one- or at least two-dimensional regions around the detected relevant points, so that preferably each of the relevant points is surrounded by one of the relevant regions.

This has the advantage that not only the POI, e.g. the radar reflections, but also part of the sensor data, preferably the radar spectrum, can be used for object evaluation or detection.

In a further possibility, it can be provided that the sensor data, and thus also the relevant areas of the sensor data, have several data values in an at least one- or at least two-dimensional data structure, wherein the data values can each be assigned coordinates, wherein the detection of the overlap takes place by an evaluation and in particular a comparison of the coordinates of the data values of the relevant areas. The data structure is, for example, a matrix or the like.

It is also optionally possible for the detection to be carried out during data transmission and/or data storage of the sensor data, whereby, on the basis of the detection, multiple transmission and/or storage of the overlapping parts is avoided, and preferably, after a first transmission and/or storage, a renewed transmission and/or storage of the overlapping parts is skipped. For this purpose, for example, before each transmission and/or storage of data values of a relevant area, it can be checked whether the associated coordinates have already been transmitted and/or saved. The overlaps and preferably the overlapping parts can accordingly designate those data values of the relevant areas that have the same coordinates in the data structure.

• - Dekomprimieren der komprimierten Sensordaten, vorzugsweise derart, dass die Überlappungen wiederhergestellt werden,
• - Auswerten der relevanten Bereiche aus den dekomprimierten Sensordaten, vorzugsweise durch ein künstliches neuronales Netz, um auf Basis des Auswertens das Fahrzeug zu steuern.

Furthermore, within the scope of the invention, it can be provided that the following steps are carried out:

• - Decompressing the compressed sensor data, preferably in such a way that the overlaps are restored,
• - Evaluating the relevant areas from the decompressed sensor data, preferably using an artificial neural network, in order to control the vehicle based on the evaluation.

The compressed sensor data can, for example, be transmitted to a control unit of the vehicle and/or to a server outside the vehicle, where it can be decompressed and evaluated.

The vehicle can be designed, for example, as a motor vehicle and/or passenger vehicle and/or autonomously driving vehicle. Furthermore, the control of the vehicle can be carried out, for example, by a driver assistance system and/or by an autonomous driving function and/or by a braking function such as emergency braking based on the evaluation of the relevant areas and/or the object evaluation and/or the object detection.

Preferably, it can be provided that the sensor data are implemented as a radar spectrum or an ultrasound spectrum or a lidar spectrum.

The invention also relates to a computer program, in particular a computer program product, comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to the invention. The computer program according to the invention therefore brings with it the same advantages as have been described in detail with reference to a method according to the invention.

The invention also relates to a device for data processing which is set up to carry out the method according to the invention. The device can be, for example, a computer which executes the computer program according to the invention. The computer can have at least one processor for executing the computer program. A non-volatile data memory can also be provided in which the computer program is stored and from which the computer program can be read by the processor for execution.

The invention can also relate to a computer-readable storage medium which comprises the computer program according to the invention. The storage medium is designed, for example, as a data storage device such as a hard disk and/or a non-volatile memory and/or a memory card. The storage medium can, for example, be integrated into the computer.

Furthermore, the method according to the invention can also be implemented as a computer-implemented method.

• 1 eine schematische Darstellung der Sensordaten gemäß Ausführungsvarianten der Erfindung,
• 2+3 weitere beispielhafte Sensordaten in der Form von Radarspektren,
• 4 gemessene Sensordaten,
• 5 eine Auswertung des Verfahrens gemäß Ausführungsvarianten der Erfindung,
• 6 das Verfahren gemäß Ausführungsvarianten der Erfindung.

Further advantages, features and details of the invention emerge from the following description, in which embodiments of the invention are described in detail with reference to the drawings. The features mentioned in the claims and in the description can each be essential to the invention individually or in any combination. They show:

• 1 a schematic representation of the sensor data according to embodiments of the invention,
• 2 + 3 further exemplary sensor data in the form of radar spectra,
• 4 measured sensor data,
• 5 an evaluation of the method according to embodiments of the invention,
• 6 the method according to embodiments of the invention.

In the following figures, identical reference numerals are used for the same technical features even in different embodiments.

In 6 a method 100 for compressing sensor data 300 of at least one sensor 40 of a vehicle 1 is shown schematically according to embodiments of the invention. According to a first method step 101, the sensor data 300 can be determined, for example by receiving the sensor data 300 by a processing function which processes a sensor signal received by the at least one sensor 40 in order to output an at least one-dimensional or at least two-dimensional spectrum 300 of the sensor signal. Subsequently, according to a second method step 102, relevant areas 302 in the sensor data 300 can be determined, wherein the relevant areas 302 can be provided for object evaluation and in particular detection of one or more objects in an environment 2 of the vehicle 1. For example, these relevant areas 302 should be fed as relevant parts of the spectrum 300 to an algorithm for object evaluation and preferably detection. The relevant areas 302 can partially overlap. In a third method step 103, therefore, detection of the overlap of the relevant areas 302 is provided for compressing the sensor data 300. For example, the detected overlapping parts of the relevant areas 302 are only stored and/or transmitted once. The method 100 can thus provide an efficient compression algorithm and preferably storage algorithm.

Furthermore, a computer program 15 and a data processing device 10 according to embodiments of the invention are shown in 6 shown.

The proposed method 100 can use the special structure of the sensor data 300 and in particular of radar spectra and radar point clouds to efficiently store the relevant parts of the sparse sensor data 300. Furthermore, the special structure can be used to efficiently process the data. The proposed method 100 is also applicable not only for radar, but also for other sensors where peaks in a spectrum can be detected, such as Lidar. or ultrasound. The method 100 is described in more detail below using the example of a radar spectrum.

In 1 the sensor data 300 are shown schematically in the form of a radar spectrum 300. The detected radar reflections 301 are shown in the "sparse" radar spectrum 300, which can also be referred to as "points of interest" (POI for short) or, within the scope of the invention, as relevant points 301. Furthermore, the relevant areas 302 of the spectrum 300 around the radar reflections 301 are shown, which can also be referred to as a "patch". The remaining area of the spectrum 300 may not be relevant for further evaluation.

In embodiments of the invention, it can be provided that the positions of the detected radar reflections 301, i.e. the POls 301, as well as the other relevant areas 302 of the spectrum 300 are stored. Storage can be carried out in such a way that the points at which the areas 302 overlap are only stored once. It has been found that this procedure is often more storage-efficient than storing the entire spectrum 300. It can also be more storage-efficient than storing the spectrum 300 like a sparse matrix in which, for example, the coordinate and the value of the pixel are stored for each pixel that is not equal to 0. This is due to the fact that the spectra 300 are generally not sparse enough for such a storage algorithm to be advantageous. Other conventional storage algorithms that require a block structure of the matrix are often unsuitable either. This is because such a block structure is generally not applicable to radar spectra. With the proposed storage algorithm, however, only the coordinate (position in the spectrum 300) needs to be stored for each patch 302.

Even if the proposed method 100 is described in connection with one- or at least two-dimensional radar data, it is also applicable to data of higher dimensions (e.g. 4D for range, Doppler, azimuth, elevation spectra). It is also applicable to sensors of other types than radar, e.g. for lidar or ultrasound. In particular, the spectrum 300 results from a measurement of an environment 2, preferably of a vehicle 1, preferably by a sensor 40 such as a radar, lidar or ultrasound sensor 40. The method 100 can also be used for object evaluations such as classification tasks or a detection of objects or a semantic segmentation based on the sensor data 300 such as the spectra, e.g. with regard to pedestrians, other vehicles 1 or smaller objects. It is also possible that a measurement of a distance, a speed and/or an acceleration is carried out based on the sensor data 300 such as the spectra. Tracking of objects can also be provided if necessary. In addition, a control signal, e.g. for a vehicle 1, can be generated based on the evaluation of the spectra. The control signal can be used, e.g. for braking and/or steering and/or accelerating the vehicle 1, e.g. for emergency braking and/or for a driver assistance system and/or for an autonomous driving function. Furthermore, the method 100 according to embodiments of the invention can be used to generate training data for training a neural network. In particular, the effort for storing and/or processing the training data can be reduced by the method 100. Embodiments of the invention can also be used to construct radar hardware that uses the memory and the available bandwidth efficiently.

It can be provided that sensor data 300 from several sensors are combined and processed in a central control unit. The bandwidth with which the sensor data 300 can be sent from a sensor 40 to the central control unit is limited. It is therefore advantageous to transmit the sensor data 300 in a compressed form and thereby save bandwidth or manage with a limited bandwidth. It is therefore an advantage of embodiments of the invention that the relevant areas 302 of the radar spectrum can be efficiently compressed and stored. A further application for embodiments of the invention can be provided by temporarily storing radar spectra in a radar sensor when algorithms access them at a later point in time. If a new radar measurement is carried out in parallel during this time, the new spectra 300 must also be stored. It is therefore advantageous if the radar spectra are stored in a compressed form in order to save storage space. When a spectrum 300 is needed, it can be decompressed again.

Even if the embodiments are explained using 2-dimensional sensor data 300 of a radar sensor, the explanations also apply to use with other sensors and/or higher (or only one) dimensions, for example for a radar in 4 dimensions. The method 100 according to embodiments is also applicable to different radar spectra 300, e.g. non-cohesive rent-integrated spectra, so-called beamformed spectra, or the complex spectra of the various receiving channels. For complex spectra, the real and imaginary parts or the magnitude and phase of the complex number for the value under consideration can be stored. The method can also be used for data from other sensors 40.

 def compress_spectrum(pois, spectrum):
  # pois: Liste der Koordinaten der POls
  # spectrum: Das zu komprimierende Spektrum

 
  spectrum_compressed = empty_list_of_values 0
  #spectrum_compressed: Variable, in welcher das komprimierte Spektrum
  gespeichert wird
 
  #Erzeugen einer Maske, um zu wissen, wenn ein bestimmter Punkt des
  Spektrums bereits gespeichert wurde:
  mask = boolean_array_initialized_to_False()

 
  #gehe durch die Liste der Punkte:
  for p in pois:
      # erhalte Koordinaten des Patches, welcher zu einem aktuellen POI p
 gehört: 
      x_coordinates, y_coordinates = compute_patch_coordinates (p)
 
      # gehe durch alle Koordinaten des Patches:
     for x in x_coordinates:
       for y in y_coordinates:
       #prüfe, ob der Pixel gespeichert werden soll
       #falls ja, hinterlege dies in der Maske, damit der Pixel nicht ein weiteres
       Mal gespeichert wird
       if mask[x,y] is False:
            mask[x,y] = True
            spectrum_compressed.append(spectrum[x,y]])
 return pois, spectrum_compressed

In the following, a rectangular patch as shown in 1 shown, used as an example as relevant area 302. The storage algorithm, preferably a compression algorithm, can also be easily extended to other shapes of the patch, e.g. an ellipse or a trapezoid. The storage algorithm is shown below as an example as pseudocode:

 def compress_spectrum(pois, spectrum):
  # pois: List of coordinates of the POls
  # spectrum: The spectrum to be compressed

 
  spectrum_compressed = empty_list_of_values 0
  #spectrum_compressed: Variable in which the compressed spectrum
  is stored
 
  #Creating a mask to know when a certain point of the
  spectrum has already been saved:
  mask = boolean_array_initialized_to_False()

 
  #go through the list of points:
  for p in pois:
      # get coordinates of the patch which corresponds to a current POI p
 heard: 
      x_coordinates, y_coordinates = compute_patch_coordinates (p)
 
      # go through all coordinates of the patch:
     x_coordinates
       for y in y_coordinates:
       #check if the pixel should be saved
       #if yes, store this in the mask so that the pixel does not have another
       Times saved
       if mask[x,y] is False:
            mask[x,y] = True
            spectrum_compressed.append(spectrum[x,y]])
 return pois, spectrum_compressed

The function receives as input variables the one or two-dimensional coordinates of the POI 301 in the spectrum 300 as well as the spectrum 300 that is being compressed. For each POI 301 specified with "p", the corresponding patch coordinates are iterated over. If a pixel value has not yet been saved in the respective patch, it is saved. A mask stores which pixel values have already been saved so that no value is saved twice.

The information with which the spectrum 300 can be decompressed again is contained in the list with the coordinates of the POI 301 "pois", the compressed spectrum "spectrum_compressed" and the size of the spectrum "spectrum_size". It is advantageous that the mask does not need to be saved, as it can be calculated on-the-fly during decompression.

 def compress_spectrum_no_mask(pois, spectrum):
 # pois: Liste der Koordinaten der POls
 # spectrum: Das zu komprimierende Spektrum
 spectrum_compressed = empty_list_of_values 0
 invalid value = -1.0
 for p in pois:
     x_coordinates, y_coordinates = compute_patch_coordinates (p)
     for x in x_coordinates:
       for y in y_coordinates:
       #prüfe, ob der Pixel gespeichert werden soll
       if spectrum[x,y] is not invalid_value:
              spectrum_compressed.append(spectrum[x,y]])
              spectrum[x,y] = invalid_value
 return pois, spectrum_compressed

If the memory on the radar sensor is very limited, the compression algorithm can be executed more efficiently during compression. The compressed result is still the same. Instead of storing the mask in an extra array, the mask can be omitted. This reduces the amount of data storage required during the calculation. Instead, as soon as a value from the spectrum has been stored, this value in the spectrum is replaced by an invalid value. This information can be used to avoid storing these values twice in the future. To do this, it can be compared whether the current pixel value being considered is equal to invalid_value. It must be noted that the invalid _value is a value that cannot occur in real measurements. The pseudocode for this is shown below:

 def compress_spectrum_no_mask(pois, spectrum):
 # pois: List of coordinates of the POls
 # spectrum: The spectrum to be compressed
 spectrum_compressed = empty_list_of_values 0
 invalid value = -1.0
 for p in pois:
     x_coordinates, y_coordinates = compute_patch_coordinates (p)
     x_coordinates
       for y in y_coordinates:
       #check if the pixel should be saved
       if spectrum[x,y] is not invalid_value:
              spectrum_compressed.append(spectrum[x,y]])
              spectrum[x,y] = invalid_value
 return pois, spectrum_compressed

 def decompress_spectrum(pois, spectrum_compressed, spectrum_size):
 # pois: Liste der Koordinaten der POls
 # spectrum_compressed: Das komprimierte Spektrum
 # spectrum_size: Größe des Original-Spektrums
 spectrum_decompressed = array_of_zeros(spectrum_size)
 mask = boolean_array_initialized_to_False()
 current_spectrum_index = 0
 for p in pois: 





     x_coordinates, y_coordinates = compute_patch_coordinates (p)
     for x in x_coordinates:
       for y in y_coordinates:
       #prüfe, ob der Pixel in spectrum_decompressed geschrieben werden
 soll
       if mask[x,y] is False:
              mask[x,y] = True
              spectrum_decompressed[x,y] =
              spectrum_compressed[current_roi_index]])
              current_roi_index += 1
 return pois, spectrum_decompressed

The decompression algorithm can work analogously to the compression algorithm and is simplified below using pseudocode:

 def decompress_spectrum(pois, spectrum_compressed, spectrum_size):
 # pois: List of coordinates of the POls
 # spectrum_compressed: The compressed spectrum
 # spectrum_size: Size of the original spectrum
 spectrum_decompressed = array_of_zeros(spectrum_size)
 mask = boolean_array_initialized_to_False()
 current_spectrum_index = 0
 for p in pois: 





     x_coordinates, y_coordinates = compute_patch_coordinates (p)
     x_coordinates
       for y in y_coordinates:
       #check if the pixel is written to spectrum_decompressed
 should
       if mask[x,y] is False:
              mask[x,y] = True
              spectrum_decompressed[x,y] =
              spectrum_compressed[current_roi_index]])
              current_roi_index += 1
 return pois, spectrum_decompressed

The spectrum is initialized as an array with 0 in a given size. As with the compression algorithm, it is iterated over the POls and the respective patch coordinates x,y. For each pixel value that has not yet been saved in "spectrum_decompressed", the next value from spectrum_compressed is saved to the current coordinate x,y.

def decompress_spectrum_no_mask(pois, spectrum_compressed,

 spectrum_size):
 # pois: Liste der Koordinaten der POls
 # spectrum_compressed: Das komprimierte Spektrum
 # spectrum_size: Größe des Original-Spektrums
 spectrum_decompressed = create_array_and_set_values_to_invalid_value Q
 # Funktion, um ein Array zu erstellen und alle Wert auf „invalid_value" zu setzen
 current_spectrum_index = 0
 for p in pois: 



     x_coordinates, y_coordinates = compute_patch_coordinates (p)
     for x in x_coordinates:
       for y in y_coordinates:
       #prüfe, ob der Pixel in spectrum_decompressed geschrieben werden
 soll
       if spectrum_decompressed[x,y] == invalid _value:
              spectrum_decompressed[x,y] =
              spectrum_compressed[current_roi_index]])
              current_roi_index += 1
              set_all_pixel_values_which_have_invalid_value to_zero(spectrum_decompresse
              d, invalid_value)
              # Funktion, um alle Wert mit„invalid_value" auf 0 zu setzen
 return pois, spectrum_decompressed

Analogous to the compression algorithm, the decompression algorithm can also dispense with the calculation of the mask by initializing the spectrum_decompressed at each point with an invalid_value. Only values that have not been overwritten before, i.e. those that have the value == invalid_value, are overwritten. Finally, all remaining pixels that have the value invalid_value are set to 0. The pseudocode for this is shown below:

 def decompress_spectrum_no_mask(pois, spectrum_compressed,

 spectrum_size):
 # pois: List of coordinates of the POls
 # spectrum_compressed: The compressed spectrum
 # spectrum_size: Size of the original spectrum
 spectrum_decompressed = create_array_and_set_values_to_invalid_value Q
 # Function to create an array and set all values to "invalid_value"
 current_spectrum_index = 0
 for p in pois: 



     x_coordinates, y_coordinates = compute_patch_coordinates (p)
     x_coordinates
       for y in y_coordinates:
       #check if the pixel is written to spectrum_decompressed
 should
       if spectrum_decompressed[x,y] == invalid _value:
              spectrum_decompressed[x,y] =
              spectrum_compressed[current_roi_index]])
              current_roi_index += 1
              set_all_pixel_values_which_have_invalid_value to_zero(spectrum_decompresse
              d, invalid_value)
              # Function to set all values with "invalid_value" to 0
 return pois, spectrum_decompressed

In 2 An example spectrum with typical parameters of a radar sensor is shown. The size of the spectrum is 32x256, with 250 POls and a patch size of 5x9 pixels. The values in the patches were randomly generated. For this example, the storage space required for various storage methods is compared below: "uncompressed" refers to uncompressed storage, "saving as sparse matrix" refers to a method in which the position of the pixel and its value are stored for each value not equal to 0. "compressed" is the proposed method according to embodiments of the invention. Other known compression methods are listed for further comparison: Memory requirements: 32 bit values and 16 bit for indices Memory requirements: 8 bit values and 16 bit for indices uncompressed 32KB 8KB saving as sparse matrix 46.5KB 29KB compressed 24.2KB 6.8KB Block Sparse Row 30.8KB 8.4KB Compressed Sparse Row 34.9KB 17.5KB Compressed Sparse Column 35.4KB 17.9KB

The table shows the storage space required for 8 bits or 32 bits per stored value and 16 bits per stored index. It can be seen that for this example, "Saving as sparse matrix" is not advantageous and actually requires more storage space than the uncompressed method. This is because the data is not sparse enough. The other methods Block Sparse Row, Compressed Sparse Row, Compressed Sparse Column are also not suitable. The proposed method "compressed" requires the least storage space, only about 76% of the storage space for 32 bit values and 85% of the storage space for 8 bit values compared to the uncompressed method.

For another example with only 100 POls, see 3 , the following values can be obtained: Memory requirements: 32 bit values and 16 bit for indices Memory requirements: 8 bit values and 16 bit for indices uncompressed 32KB 8KB saving as sparse matrix 25.6KB 10.1KB compressed 13.2KB 3.6KB Block Sparse Row 17.6KB 5.9KB Compressed Sparse Row 19.3KB 9.7KB Compressed Sparse Column 19.7KB 10.1KB

“Saving as sparse matrix” requires less memory than the uncompressed method for 32-bit values, but it is still unsuitable for 8-bit values and requires more memory than the uncompressed method. The other methods also all require more memory than the proposed method. Here too, the proposed method requires the least memory, at around 41% and 45% compared to the uncompressed methods.

Another example is 4 a spectrum from a real measurement in a city is shown. The spectrum size is 256x32 bins and a patch size of 5×9 bins. The patches are shown in lightly shaded points, the POls in darker points in the center of the lightly shaded patches. It can be seen that the spectrum is sparse and many of the patches overlap, analogous to the simulated radar spectra above. This means that the proposed method can also be advantageously applied to real measurements.

For those in 5 The evaluation shown uses real measurements and the memory required for these measurements is calculated for different methods. The abscissa indicates the data size in "bytes" and the ordinate the number. The methods a) no compression, b) saving the POls and their patches without taking the overlap of the patches into account and c) efficient compression are compared. It is clear that a) always requires the same amount of memory for all measurements. b) requires less memory than a). c) requires less memory than a) and b). The histogram shows that most measurements require less than 4kB, which is about 50% of the size of the uncompressed version a).

The following table shows the maximum required memory and the 0.98 quantile: Without compression Localization + 5×9 Patch Efficient compression, proposed storage algorithm 0.98 quantile 8KB 8.4KB 4.0KB Maximum data size 8KB 10.9KB 5.2KB

It can be seen that for the proposed method, in 98% of the measurements, 49% of the memory requirement is enough to store the relevant part of the spectrum. Thus, real world measurements can be efficiently compressed using the proposed method.

It is often necessary to store additional attributes for the POls in order to use them in later processing. For radar reflections, these are, for example, the distance, Doppler velocity, azimuth angle, elevation angle. In general, the indices of the bins in the spectrum under consideration in which the POls are located can be calculated from these attributes. In this case, it is not necessary to store the indices again, i.e. the memory requirements of the proposed method can be further reduced. During decompression, the required indices can be calculated on-the-fly.

1. [1] Kanil Patel and Kilian Rambach and Tristan Visentin and Daniel Rusev and Michael Pfeiffer and Bin Yang, „Deep Learning-based Object Classification on Automotive Radar Spectra“, IEEE Radar Conference 2019
2. [2] Adriana-Eliza Cozma, Lisa Morgan, Martin Stolz, David Stoeckel, Kilian Rambach, „DeepHybrid: Deep Learning on Automotive Radar Spectra and Reflections for Object Classification“ 2022, https://arxiv.org/abs/2202.08519

In a further optimization, the proposed method can be combined with a “run length encoding”. In this case, the sequence of spectrum values is not stored, but rather saves how often a value occurs in a row and then the value itself. Depending on the spectrum being considered, this can lead to further storage space savings.

1. [1] Kanil Patel and Kilian Rambach and Tristan Visentin and Daniel Rusev and Michael Pfeiffer and Bin Yang, “Deep Learning-based Object Classification on Automotive Radar Spectra”, IEEE Radar Conference 2019
2. [2] Adriana-Eliza Cozma, Lisa Morgan, Martin Stolz, David Stoeckel, Kilian Rambach, “DeepHybrid: Deep Learning on Automotive Radar Spectra and Reflections for Object Classification” 2022, https://arxiv.org/abs/2202.08519

The above explanation of the embodiments describes the present invention exclusively in the context of examples. Of course, individual features of the embodiments can be freely combined with one another, provided they are technically expedient, without departing from the scope of the present invention.

Claims (10)

Method (100) for compressing sensor data (300) of at least one sensor (40) of a vehicle (1), comprising the following steps: - determining (101) the sensor data (300) which are specific to an at least one-dimensional spectrum (300) of a sensor signal received by the at least one sensor (40), - determining (102) relevant areas (302) in the sensor data (300) which are intended for an object evaluation in an environment (2) of the vehicle (1), wherein the relevant areas (302) partially overlap, - detecting (103) the overlap of the relevant areas (302) for compressing the sensor data (300). Procedure (100) according to Claim 1 , characterized in that the detected overlap of the relevant areas (302) is used for compression and preferably for redundancy reduction in that, on the basis of the detection (103), the compressed sensor data (300) are obtained in which the amount of overlap is at least reduced. Method (100) according to one of the preceding claims, characterized in that the sensor data (300) are designed as the at least one-dimensional spectrum (300) or an at least two-dimensional spectrum (300) of the received sensor signal and result from processing of the received sensor signal, wherein the sensor data (300) are present in an at least one- or two-dimensional form, wherein the relevant regions (302) each characterize an at least one- or two-dimensional region of the sensor data (300) which contains information about at least one detected point of at least one object in the environment (2) of the vehicle (1). Method (100) according to one of the preceding claims, characterized in that determining the relevant areas (302) comprises the following steps: - detecting a plurality of relevant points (301) in the sensor data (300) which are provided for object evaluation, preferably object detection and/or object classification and/or semantic segmentation, in the environment (2) of the vehicle (1), preferably by peak value detection, - determining the relevant areas (302) as at least one- or at least two-dimensional areas around the detected relevant points (301), so that preferably each of the relevant points (301) is surrounded by one of the relevant areas (302). Method (100) according to one of the preceding claims, characterized in that the sensor data (300), and thus also the relevant areas (302) of the sensor data (300), have a plurality of data values in an at least one- or at least two-dimensional data structure, wherein the data values are each assigned coordinates, wherein the detection of the overlap takes place by an evaluation and in particular a comparison of the coordinates of the data values of the relevant areas (302). Method (100) according to one of the preceding claims, characterized in that the detection is carried out during a data transmission and/or data storage of the sensor data (300), wherein on the basis of the detection a multiple transmission and/or storage of the overlapping the parts is avoided, and preferably after an initial transfer and/or saving, a re-transfer and/or saving of the overlapping parts is skipped. Method (100) according to one of the preceding claims, characterized in that the following steps are carried out: - decompressing the compressed sensor data (300), preferably in such a way that the overlaps are restored, - evaluating the relevant areas (302) from the decompressed sensor data (300), preferably by an artificial neural network, in order to control the vehicle (1) on the basis of the evaluation. Method (100) according to one of the preceding claims, characterized in that the sensor data (300) are embodied as a radar spectrum or an ultrasound spectrum or a lidar spectrum (300). Computer program (15) comprising instructions which, when the computer program (15) is executed by a computer, cause the computer to carry out the method (100) according to one of the preceding claims. Device (10) for data processing, which is arranged to carry out the method (100) according to one of the Claims 1 until 8th to execute.