DE102019204384A1 - Method and device for compressing sensor data - Google Patents

Abstract

Method for compressing sensor data (I), wherein the sensor data (I) are fed to an auto-encoder (100), and wherein the compressed sensor data is dependent on output data (Z) which is then sent to an output of an encoder (120) of the autoencoder) 100) are present and, depending on a compression (K) of a reconstruction error (R, S) of the autoencoder, (100) can be determined.

Description

The invention relates to a method for compressing or decompressing sensor data, a computer program, a machine-readable storage medium and a control device.

State of the art

• - optisches Erfassen mindestens einer Szene aus einem Umfeld des Fahrzeugs,
• - Bewerten der erfassten Szene und daraus Ableiten einer Betriebs- und insbesondere Fahrsituation des Fahrzeugs,
• - in Abhängigkeit von der Betriebssituation gesteuertes Eingreifen in den Betrieb des Fahrzeugs und dadurch Anpassen des Betriebs und insbesondere des Bewegungs- oder des Fahrverhaltens des Fahrzeugs, bei welchem:
• - das Bewerten auf der Grundlage eines Bewertungsprozesses durchgeführt wird, der eine erscheinungsformbasierte Objektdetektion und -klassifikation umfasst.

From the DE 10 2017 206 396 A1 a control method for an environment detection system for a vehicle is known, with the steps:

• - Optical recording of at least one scene from the surroundings of the vehicle,
• - Evaluating the captured scene and deriving an operating and, in particular, driving situation of the vehicle from it,
• - Depending on the operating situation, controlled intervention in the operation of the vehicle and thereby adaptation of the operation and in particular the movement or driving behavior of the vehicle, in which:
• the evaluation is carried out on the basis of an evaluation process which comprises an appearance-based object detection and classification.

Advantages of the invention

In systems of this type, it may be necessary to store the captured scenes from the surroundings of the vehicle for a predefinable period of time, for example in order to be able to understand the cause of a malfunction at a later point in time.

The method with the features of claim 1 provides a simple possibility of efficiently compressing the captured scenes in order to be able to save them in a memory-saving manner.

Further aspects of the invention are the subject of the independent claims. Advantageous further developments are the subject of the dependent claims.

Disclosure of the invention

In a first aspect, the invention relates to a method for compressing sensor data, the sensor data being fed to an autoencoder, and the compressed sensor data depending on output data that are then present at an output of an encoder of the autoencoder, and depending on a compression of a reconstruction error of the autoencoder, that is to say a difference between sensor data and then reconstructed sensor data present at an output of the autoencoder. In particular, the compressed sensor data can comprise the output data and the compression of the reconstruction error, e.g. be given by them.

This is based on the knowledge that the reconstruction error can in turn be compressed considerably more than the original uncompressed sensor data.

The sensor data can in particular be image data, that is to say in particular those that have been recorded with an optical sensor, a radar sensor, a LiDAR sensor or an ultrasonic sensor.

In particular, it is possible for the compression of the reconstruction error to be lossless compression.

This allows the surprising effect to be achieved of overall loss-free compression, although the compression using the autoencoder is usually is a lossy compression.

Alternatively or additionally, it is possible for the reconstruction error to be de-noised before the compression is carried out.

This enables an even further improved compression to be achieved, with only insignificant loss of information having to be accepted.

It is particularly advantageous here if the noise removal is carried out in such a way that an amount of a gradient of the reconstruction error, which characterizes a difference between adjacent features of the reconstruction error, is minimized in an area surrounding the reconstruction error that has not been denoise.

Such a method offers in a particularly simple manner the possibility of further increasing the compression rate with minimal loss of information.

In a further alternative or additional development, it is possible for the sensor to be mounted in a stationary manner on a vehicle that is controlled as a function of these sensor data.

Due to the fixed installation on the vehicle, the sensor receives similar data, for example similar signals from a road or from a horizon. If the autoencoder has been trained to reconstruct such sensor data, particularly efficient compression rates can be achieved.

In a further advantageous development, it can be provided that sensor data are compressed which cannot be assigned to a class with sufficient certainty by a classifier, in particular a neural network, the compressed sensor data being transmitted to a remote computer and by this or a Another remote computer receives a classification of the compressed sensor data.

I.e. It is checked whether the sensor data can be assigned to a class by the classifier with sufficient certainty (whereby the classification can also be a semantic segmentation). If this assignment is not sufficiently reliable, the sensor data are compressed and transmitted to the remote computer, which is responsible for the can decompress compressed sensor data again. This or another (e.g., particularly powerful) computer or a human assessor can determine an associated classification for such sensor data. This enables the automated control of the vehicle to be further improved. For example, the classifier can be given by a machine learning system, in particular an artificial neural network. Parameters that characterize this classifier can be retrained and adapted using the associated classification determined. The parameters determined in this way can be sent back to the vehicle in order to replace the parameters stored there. This can be done, for example, during a service interval or by radio transmission.

Whether or not the classification result was reliably determined can be determined, for example, by using a Bayesian artificial neural network as the classifier. Alternatively, it is also possible, in the case of a classifier that determines the classification result by means of an argmax operation from a large number of intermediate variables each assigned to a class, to estimate how reliable the classification result is as a function of these intermediate variables.

• 1 schematisch einen Aufbau einer Ausführungsform der Erfindung;
• 2 in einem Flussdiagramm einen möglichen Ablauf eines Verfahrens zum Speichern von Sensordaten;
• 3 schematisch einen beispielhaften Aufbau eines Autoencoders;
• 4 in einem Flussdiagramm einen möglichen Ablauf eines Verfahrens zum Komprimieren von Sensordaten.

Embodiments of the invention are explained in more detail below with reference to the accompanying drawings. In the drawings show:

• 1 schematically a structure of an embodiment of the invention;
• 2 in a flowchart a possible sequence of a method for storing sensor data;
• 3 schematically an exemplary structure of an autoencoder;
• 4th in a flow chart a possible sequence of a method for compressing sensor data.

Description of the exemplary embodiments

1 shows an exemplary vehicle 10 in which the invention can be used. One sensor 30th , in the exemplary embodiment a video sensor, supplies recorded frames of a recorded environment 20th of the vehicle 10 to a control unit 40 . This can be set up to semantically segment the frames, for example by means of an artificial neural network, and the vehicle 10 according to the determined semantic segmentation.

With the in 3 illustrated method compresses the vehicle 10 the recorded frames and saves them to memory 50 . This memory can be inside the vehicle 10 be arranged, but it can also be removed from the vehicle 10 be arranged. In the latter case it can be provided that the compressed data is transmitted by means of a radio transmitter 10 , for example a mobile radio device, to the memory 50 to submit. Control unit 40 comprises a machine-readable storage medium 46 , are stored on the computer programs that include instructions to cause the control unit to operate in 2 or. 4th to perform illustrated methods when the respective computer program is executed.

2 shows an example of a sequence of the method for storing sensor data in a flowchart. First ( 1000 ) are sensor data from the sensor 30th received, for example a video frame. Then ( 1100 ) it is optionally determined whether the sensor data should be saved. This can be done, for example, by determining whether these sensor data have been classified reliably. If the classification is not reliably possible, it is decided that the sensor data should be stored in compressed form (1200), otherwise the process ends ( 1300 ). No step 1100 , it can also be provided that the sensor data are stored in each case, i.e. step 1200 definitely follows.

In step 1200 the sensor data with the in 3 illustrated procedure compressed. Subsequently ( 1400 ) the compressed sensor data are sent directly or, for example, indirectly via a radio transmitter 10 of memory 50 transmitted, where they can also be made available for a more precise classification, for example. This ends the procedure.

3 shows schematically an exemplary structure of an autoencoder 100 , who at the in 4th illustrated procedures can be used. This comprises an encoder (English: encoder) 110 and a decoder (English: decoder) 120. A high-dimensional input signal I is sent to the encoder 110 fed. This determines a so-called latent intermediate variable Z present at its output. Intermediate variable Z has a smaller dimensionality than input signal I. Decoder 120 determines an output variable A from intermediate variable Z, which has the same dimensionality as input variable I. Encoder 110 and decoders 120 can be given, for example, by artificial neural networks. In a training, the autoencoder 100 a variety of exemplary input quantities I supplied and parameters from encoder 110 and decoders 120 adjusted so that the output variable A corresponds as closely as possible to the input variable I.

4th shows an example of a sequence of the method for compressing the sensor data / in a flowchart. First ( 2000 ) the sensor data I are sent to the autoencoder 100 supplied, and intermediate size Z and output size A are determined. Then ( 2100 ) the reconstruction error R is determined as R = I-Optionally, the reconstruction error R can now be de-noised (2200), for example by selecting an optimal de-noised reconstruction error S, which has a minimum integral of an absolute value of its gradient within a ball of a predetermined size around the reconstruction error R . This reconstruction error R or, alternatively, the noise-reduced reconstruction error S is now (2300) fed to a standard compression method, for example a GIF or PNG compression method, in order to determine a compressed reconstruction error K in this way. Compressed reconstruction error K and intermediate variable Z are now (2400) stored as compressed sensor data. This ends the procedure.

The decompression of the compressed sensor data (K, Z) can be carried out by decompressing the compressed decompression error K, supplying intermediate variable Z to the decoder 120 and adding the decompressed decompression error K to the output variable A can be determined.

The term “computer” encompasses any device for processing specifiable arithmetic rules. These calculation rules can be in the form of software, or in the form of hardware, or also in a mixed form of software and hardware.

It is further understood that the methods can not only be implemented completely in software as described. They can also be implemented in hardware, or in a mixture of software and hardware.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102017206396 A1 [0002]

Claims (10)

Method for compressing sensor data (I), wherein the sensor data (I) are fed to an auto-encoder (100), and wherein the compressed sensor data is dependent on output data (Z), which is then sent to an output of an encoder (120) of the autoencoder (100) are present and can be determined as a function of a compression (K) of a reconstruction error (R, S) of the autoencoder (100). Procedure according to Claim 1 , whereby the compression (K) of the reconstruction error (R, S) is a lossless compression. Procedure according to Claim 1 or 2 , the reconstruction error (R, S) being de-noised before the compression is carried out. Procedure according to Claim 3 wherein the noise removal is carried out in such a way that an amount of a gradient of the reconstruction error, which characterizes a difference between adjacent features of the reconstruction error, is minimized in the vicinity of the reconstruction error that has not been denoise. Method according to one of the preceding claims, wherein the sensor (30) is fixedly mounted on a vehicle (10). Method according to one of the preceding claims, wherein sensor data (I) are compressed which cannot be assigned to a class with sufficient certainty by a classifier, in particular a neural network, the compressed sensor data (K, Z) being sent to a remote computer (50 ) be transmitted. Method for decompressing sensor data (I) generated with the method according to one of the Claims 1 to 6th were compressed, the compressed reconstruction error (K) being decompressed, the output data (Z), which were present at the output of the encoder (110), to a decoder (120) of the auto-encoder (100), and the sensor data (I) depending on the the output signal (A) present at the output of the decoder (120) and the decompressed compressed reconstruction error (K) are reconstructed. Computer program which is set up, which comprises instructions which cause a computer (40) to carry out the method according to one of the Claims 1 to 7th when run on a computer (40). Machine-readable storage medium (46) on which the computer program is based Claim 8 is stored. Control device (40) which is set up, the method according to one of the Claims 1 to 7th execute.

Images