DE102023204555A1 - Computer-implemented method and computer program for preparing data fields for data compression - Google Patents

Abstract

Computer-implemented method for preparing data fields (10) for data compression, the method comprising the steps: obtaining temporally successive data fields (10), wherein a data field (10) is first, data field (10) specific, metadata (11) and data packets (12), wherein the data packets (12) are arranged in an orderly manner in the data fields (10) and the data packets (12) each comprise measurement data (18) of a sensor (19) and second metadata (13) specific to the data packet (12). (V1); Determining a difference data field (DF) between one of the data fields (10) and the subsequent data field (10), whereby by forming the difference, an information content of the difference data field (DF) is lower relative to the information content of the respective data fields (10) ( V2); Providing the difference data field (DF) for a compression algorithm (15) which compresses the difference data field (DF) for storing and/or transmitting the compressed difference data field (DF) (V3).

Description

The invention relates to a computer-implemented method and a computer program for preparing data fields for data compression.

The following definitions, descriptions and statements retain their respective meanings and apply to the entire subject matter disclosed.

The development of ADAS (autonomous driving assistance system) and AD (autonomous driving) functions relies heavily on data recorded/collected and stored specifically for this use case. To enable the reproduction of driving system behaviors in HiL (Hardware in the Loop) and SiL (Software in the Loop) applications, the data must be recorded as raw as possible, including additional metadata such as timestamps or sensor-specific information. For camera data, this means a relatively large data volume of raw data, as no lossy compression can be tolerated. This leads to high storage and processing requirements for recording, storage infrastructure, HiL and SiL applications.

The US 2007/0025630 A1 discloses a method and apparatus for compressing digital images. The method includes calculating differential values of adjacent pixels within a group of pixels, reordering the difference values of adjacent pixels according to the size of the differential pixel values of an adjacent row, and compressing the reordered differential values of the adjacent pixels.

Typically, raw data is stored in files as a collection of independent frames. A frame is a representation of any file along with metadata. The file can be, for example, an image file, a video file or an audio/sound file. The metadata can be, for example, timestamps.

In typical image applications of a peripheral camera, for example a camera sensor of an automated driving system, the raw data of the peripheral camera is transmitted via a MIPI-CSI interface, i.e. Mobile Industry Processor Interface - Camera Serial Interface, for example via a MIPI-CSI-1, 2 or 3 protocol. transmitted. A CSI transmitter encodes the raw image. The compressed raw image is received and decoded/decompressed via a CSI receiver. An input driver receives the physical raw images via the MIPI-CSI interface.

The protocol defines several image data types with which raw images are received. Alternatively, the raw images can be obtained via an FPD link, i.e. Flat Panel Display Link, or via a GMSL link, i.e. Gigabit Multimedia Serial Link, in the MIPI-CSI protocol. In general, the signals/data are recorded via a MIPI-CSI interface. FPD-Link or GMSL are protocols that serialize MIPI CSI signals/data. This means that the signals/data can optionally be acquired via the FPD or GMSL interface in order to then be deserialized and available and used as MIPI signals/data.

The input driver forwards the raw images, for example, to signal processing modules, including, for example, perception algorithms. The input driver usually runs driver software that generates image data, for example, from raw signals. The Inupt Driver can, and is sometimes used during measurement drives after additional hardware for data acquisition from the FPD-Link/GMSL.

The raw signals include, for example, data packets arranged line by line, with the input driver in the example of image processing generating corresponding image lines of the image from the data packets arranged line by line. The pixel values of an image are contained in these data packets. The raw signals are transmitted according to certain protocols. For example, the MIPI-CSI protocol includes a byte-oriented low level protocol. This protocol initiates a transmission with an ST (Start of Transmission) signal followed by an SP (Short Packet) signal and an ET (End of Transmission) signal. This is followed by an LPS (Low Power State) signal and another ST signal followed by a PH (Packet Header) signal. The PH signal is followed by the DATA signal, which includes the respective data packet, for example as an 8-bit file. The DATA signal is followed by a PF (Packet Footer) signal and another ET signal. PH, DATA and PF signals form a so-called LP (Long Packet). The typical signal sequence in the low level protocol is ST-SP-ET-LPS-ST-LP-ET-LPS-ST-LP-ET-LPS-ST-SP-ET etc.

The signal processing module then reads the image data. The raw signals can also be serialized and deserialized before they enter the input driver. Serialization and deserialization are optional and depending on the signal/data source.

The invention was based on the object of how the information content in data can be reduced before known compression algorithms are applied to the data.

The subjects of the subordinate claims each solve this task. Advantageous embodiments of the invention result from the definitions, the subclaims, the drawings and the description of preferred exemplary embodiments.

In one aspect, the invention provides a computer-implemented method for preparing data fields for data compression. A data field is, for example, an ordered collection of data, for example digital data, for example values, for example pixel values. Data compression or data compression is a process in which the amount of data is condensed or reduced. This reduces the storage space required and shortens the data transfer time. According to one aspect, the method is used in the data processing of raw data from an environment detection sensor of an automated driving system. The environment detection sensor can be, for example, a camera sensor, a radar, lidar or an acoustic sensor, for example a microphone. The procedure enables the data fields to be prepared “live” / in “real time”. Differential methods only work with current or past data. The data preparation is therefore not carried out subsequently with information from the entire measurement. This means that the reverse processing during decompression can be performed live at any time and without processing future data. Even a data loss at the end of a recording does not lead to a problem in reprocessing the data at the beginning of the recording.

One step of the method includes obtaining temporally consecutive data fields. A data field includes first, data field-specific, metadata and data packets. The data packets are arranged in an orderly manner in the data fields. The data packets each include measurement data from a sensor and second, data packet-specific metadata. The measurement data is raw data from the sensor. The data field is, for example, a frame of a camera sensor.

A further step of the method includes determining a difference data field between one of the data fields and the subsequent data field. By forming the difference, the information content of the difference data field is lower relative to the information content of the respective data fields. The difference is determined, for example, byte by byte. Since all data fields are structured in the same way, the data packets and values of a data field are in the same place as in the respective previous data field. The values of the data packets, for example pixels, and the metadata are often not changed or only changed to a small extent. The amount of data does not change up to this point, but the information content is reduced because no or only small byte values change. In this context, the information content is also called entropy. In general, a data set in which data repeats has lower entropy than a data set in which data repeats relatively less often. For example, the entropy of the data set [1,1,1,1,2,3] is less than the entropy of the data set [1,2,3,1,2,3]. Data sets with lower entropy can be compressed better than data sets with relatively higher entropy.

A further step of the method includes providing the difference data field for a compression algorithm. The compression algorithm compresses the difference data field to store and/or transmit the compressed difference data field. The compression algorithm can be, for example, a ZIP, JPEG, H.264 or Huffman compression algorithm, ANS entropy encoding, arithmetic encoding, Burrows-Wheeler transformation, Lempel-Ziv compression, Deflate, Lempel-Ziv-Welch, prediction by partial matching or run-length encoding. The difference data field has a relatively lower entropy and can therefore be compressed better. Without determining the difference data field, a ZIP compressor, for example, compresses 4.3 GB of raw camera data to a data volume of 2.8 GB. For example, with difference data field determination, the difference data field has a data volume of 4.3 GB, but with lower entropy. The ZIP compressor compresses this difference data field to a data volume of 2.6 GB. The data compressed to 2.6 GB requires less storage space and can be transferred faster than the 2.8 GB data. Depending on the camera, image sequence and other parameters, other values may result.

According to a further aspect, the data packets are arranged line by line in the data fields. Each row or line includes second metadata and the associated data packets. The data packets include measurement data from a camera. The measurement data is received via a MIPI-CSI interface. The second metadata includes timestamps.

According to a further aspect, the difference data field is determined using differential coding. Differential coding ensures clear signal reception.

According to a further aspect, the difference data field is provided for lossless compression. In contrast to lossy compression, lossless compression ensures that the compressed file is as which can be transformed back into the original file.

According to a further aspect, the invention provides a computer program for preparing data fields for data compression. The computer program includes program instructions that cause a computer to carry out the steps of the method disclosed above when the computer program is loaded or executed by the computer.

The computer program may be provided as interpretable code that is loaded onto the computer, for example directly into an internal memory of the computer, or as compiled code that is executed by the computer. The provision can be made, for example, on a machine-readable medium on which the computer program is stored, or via a data carrier signal. The program instructions can be machine instructions, source code or object codes written in assembly language, an object-oriented programming language, for example C++, or in a procedural programming language, for example C.

• 1 ein Ausführungsbeispiel eines Datenfeldes,
• 2 ein Ausführungsbeispiel eines low level protocols,
• 3 ein Ausführungsbeispiel differentieller Kodierung mit Dekodierung,
• 4 ein schematisches Ausführungsbeispiel eines hier offenbarten Verfahrens und
• 5 ein Ausschnitt aus einem Ausführungsbeispiel eines Datenverarbeitungsprozesses.

The invention is explained by way of example in the following figures. Show it:

• 1 an exemplary embodiment of a data field,
• 2 an exemplary embodiment of a low level protocol,
• 3 an embodiment of differential coding with decoding,
• 4 a schematic embodiment of a method disclosed here and
• 5 a section from an exemplary embodiment of a data processing process.

In the figures, the same reference numbers designate the same or functionally similar parts. For the sake of clarity, the reference parts relevant to the respective understanding are indicated in the respective figures.

This in 1 Data field 10 shown comprises a block of first metadata 11. The data field 10 is, for example, a frame of a camera sensor 19. Data packets 12 are arranged in lines or rows in the data field 10. The data packets 12 include measurement data from a sensor 19, for example the camera sensor 19. Each line includes a data packet 12 and second metadata 13 specific to this data packet 12. The second metadata 13 are, for example, time stamps and as such indicate the time at which the data packet 12 was received. Data fields that are consecutive in time are obtained.

2 shows an exemplary embodiment of a low level protocol according to the MIPI-CSI standard for transmitting the data fields 10, more precisely the data packets 12 of the data fields 10. The low level protocol is byte-oriented. This protocol initiates a transmission with a Start of Transmission ST signal followed by a Short Packet SP signal and an End of Transmission ET signal. This is followed by a low power state LPS signal and another ST signal followed by a packet header PH signal. The PH signal is followed by data packet 12 as an 8-bit file. The data packet 12 is followed by a packet footer PF signal and another ET signal.

In 3 the entropy in the measurement data 18, for example raw data from a camera sensor 19, is reduced by means of differential coding 14. A corresponding software and/or hardware module can be provided for the differential coding 14. After the differential coding 14, the measurement data 18 are available as difference data fields DF.

In one branch of data processing, the difference data fields DF are compressed using a compression algorithm 15, for example ZIP. The compression algorithm 15 can be an encoder that encodes pixel values, for example. In one aspect, the compression algorithm 15 is loaded and/or executed by a hardware module. This hardware module is designed to transmit the coded difference data fields KDF received. The transmission can be wired or wireless. In this context, the hardware module is called a transmitter. The coded difference data fields KDF are received by a decompression module for decompression 16. The decompression module is also called a receiver. After decompression 16, decompressed difference data fields DDF are obtained. With lossless compression, the decompressed difference data fields DDF are identical to the difference data fields DF. As compression schemes X-Y-Z, where will be: 12-10-12, 12-8-12, 12-7-12, 12-6-12, 10-8-10, 10-7-10, 10-6-10. The decompressed difference data fields DDF are decoded using differential decoding 17. After the differential decoding 17, the measurement data 18 is available again.

In a second branch of data processing, the difference data fields DF are decoded without compression 15 and decompression using differential decoding 17. To reduce the risk of working with corrupted or incorrect data after decompression 16, the compression algorithm may add a checksum of the original difference data fields DF. The checksum is then taken into account, for example, during differential decompression 17.

4 shows schematically the computer-implemented method for preparing the data fields 10 for data compression. In a step V1, the time-successive data fields 10 are obtained. In a step V2, the difference data field DF between one of the data fields 10 and the subsequent data field 10 is determined. In a step V3, the difference data field DF is provided to the compression algorithm 15.

Reference symbols

10

Data field

11

first metadata

12

Data package

13

second metadata

14

differential coding

DF

Difference data field

15

Compression algorithm

KDF

compressed difference data field

16

Decompression

DDF

decompressed difference data field

17

differential decoding

18

Measurement data

19

sensor

20

MIPI CSI interface

V1-V3

Procedural steps

ST

Start of Transmission

SP

Short package

ET

End of Transmission

LPS

Low power state

Ph

Package header

PF

Packet footer

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2007/0025630 A1 [0004]

Claims (5)

Computer-implemented method for preparing data fields (10) for data compression, the method comprising the steps: obtaining temporally successive data fields (10), wherein a data field (10) is first, data field (10) specific, metadata (11) and data packets (12), wherein the data packets (12) are arranged in an orderly manner in the data fields (10) and the data packets (12) each comprise measurement data (18) of a sensor (19) and second metadata (13) specific to the data packet (12). (V1); Determining a difference data field (DF) between one of the data fields (10) and the subsequent data field (10), whereby by forming the difference, an information content of the difference data field (DF) is lower relative to the information content of the respective data fields (10) ( V2); Providing the difference data field (DF) for a compression algorithm (15) which compresses the difference data field (DF) for storing and/or transmitting the compressed difference data field (DF) (V3). Procedure according to Claim 1 , wherein the data packets (12) are arranged line by line in the data fields (10), include measurement data (18) from a camera (19), the measurement data (18) being obtained via a MIPI-CSI interface (20), and the second Metadata (13) includes timestamps. Method according to one of the preceding claims, wherein the difference data field (DF) is determined using differential coding (14). Method according to one of the preceding claims, wherein the difference data field (DF) is provided for lossless compression. Computer program for preparing data fields for data compression, the computer program comprising program instructions that cause a computer to carry out the steps of the method of the preceding claims when the computer program is loaded or executed by the computer.

Images