GB2378868A

GB2378868A - Data Compression

Info

Publication number: GB2378868A
Application number: GB0212613A
Authority: GB
Inventors: Arnd A Braun
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2001-06-30
Filing date: 2002-05-31
Publication date: 2003-02-19
Also published as: DE10131801B4; FR2826804B1; JP4191438B2; GB0212613D0; DE10131801A1; FR2826804A1; JP2003046392A

Abstract

Data compression/decompression based on the Lempel Ziv (LZSS) method. The data is stored on a CD ROM and accessed by an electronic system, preferably a navigation system and the data can be loaded via a computer network. This system gives a rapid rate of decompression with an effective compression rate where the data stream is divided into control codes and literal sequences. It is dependent on the established maximum lengths and the frequency of the occurrence of the different literal sequences, the lengths of the literal sequences to be copied and the lengths of the back references. Control codes and literal sequences are separated into two data streams and are recombined. A pointer is placed at the start of the data stream to point to the first character of the literal sequence. Hoffman encoding may also be used to achieve further compression and sequences may be encoded in advance by a run length encoding method.

Description

Method of data compression and navigation system 5 The invention relates

to a method of data compression and / or data decompression according to a method based upon a method of the LZSS-type, and to a corresponding electronic system, in particular a navigation system.

Methods of the LZSS-type are disclosed in US-A-487 6541 and by T.C. Bell in "Better 10 OPM/L Text Compression, "IEEE Trans. On Corurnunications", Vol. COM-34' No. 12. Dec., 1986. The LZSS-method is a further development of the Lempel Ziv method.

In applying the LZSS-method, a character string is sought in the characters last transmitted within a data window of a specific length, which character string corresponds to the characters to be transmitted subsequently. If this type of character string is found, then it is replaced by a back-reference.

20 Two different control codes are used for the corresponding encoding. The control code "L" indicates that a number of "real" characters, socalled literals is transmitted next. In contrast, the control code "C" indicates that a character string is to be copied from the characters already transmitted.

25 F (s) - data window, in which identical character strings are sought. It includes a number of s characters in front of the current read position in the input data stream. L (n) - control characters to indicate that subsequently a number of n literals, i.e. a 30 literal sequence of the length n is transmitted.

C (p, n) control characters to identify a preceding literal sequence to be copied, i.e. go p characters back and copy n characters therefrom.

Figure 1 shows an example of encoding a character string 1 according to the LZSS-method 5 known from the prior art. The result of the encoding is the character string 2 in Figure 1,

wherein the characters in bold are literals.

Furthermore, different forms of the LZSS-method are known from the prior art, e.g.. LZSS

with adaptive arithmetic encoding and LZSS with adaptive Hoffman-encoding. An overview I o of this can be found in the introductory seminar "Redundancy". Lecture 5, Maximilian Hrabowski (hi t >,.;l -oc: s:. i 1se r i a; c,l t di ndanzJ i = -gi f.if;l,ZASS) Father illustrations of the LZSS-method can be found at ii,',i'ttr i,:. - 'rn s.l; is 3!s rsi. 9(i,'l o'n', ess o 'sat - c' th a[le - fuoo 't>-- ó, and k ti ^li'- - ol. il worms.deise / 's. Y6,' ores,sions Hanoi iti!ir ne ln i (,.l t 3.

US-A-5 502 439 discloses a method of compressing binary data according to the LZSS method. A buffer is used in a memory having optional access for the temporary storage of so-called flag-bits which are generated during implementation of the LZSS-method. Further methods of the LZSS-type are disclosed in US-A-5 701 125, US-A-5 673 042 and US-A-5 20 867 1 14.

It is an object of the invention to provide an improved method of the LZSS-type and a corresponding improved computer program product and an electronic system.

25 The object of the invention is achieved in each case by the features of the independent claims.

Preferred developments of the invention are provided in the dependent claims.

The LZSS-type method in accordance with the invention enables particularly rapid data decompression whilst at the same time achieving an effective compression rate. In one so preferred embodiment ofthe invention, the control codes for implementing the LZSS-method are established for this purpose in dependence upon the frequency of occurrence of different

lengths of literal sequences, lengths of literal sequences to be copied and the lengths of back references. According to a further preferred embodiment, quantities of control codes are formed in each 5 case which for their part can be e.g. Hoffman-encoded for the purpose of achieving further compression. According to a further embodiment of the indention, the back-references are only provided in one byte raster which is specified by the width of the data bus or processor used. As a result, lo during decompression the processing speed is increased once again. Likewise, the compression rate is consequently also increased.

It is particularly advantageous to use the method according to the invention for an electronic system, for example a navigation system. In the case of known navigation systems, CDs are 5 generally used for storing the navigation data bases. In order to accommodate as much navigation data as possible on one CD, it is advantageous to compress the navigation data by a method in accordance with the invention. The rate of data compression is virtually of secondary importance, as this is only performed once and not during active operation.

do However, the decompression rate is of great importance for the practical usage of the navigation system, since navigation data must be decompressed constantly during operation of the navigation system for the purpose of planning routes and determining location. In this respect, the method in accordance with the invention is also particularly advantageous, since it enables particularly rapid data decompression.

The invention is described further hereinafter, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows the encoding of a character sequence according to the prior art,

Figure 2 shows a flow diagram of an embodiment of the method in accordance with the invention, Figure 3 shows the percentage distribution of literal sequences and the length of back 5 references in a sample data record, Figure 4 shows one embodiment for determining quantities of control codes, Figure 5 shows the encoding of a character string by means of the control codes of lo Figure 4, Figure 6 shows the recoding of the encoded character string of Figure 5 by means of a further control code, 5 Figure 7 shows a block diagram of an electronic system in accordance with the invention. The method of Figure 2 serves to determine control codes for use in one embodiment of the method in accordance with the invention. For this purpose, in step 20 a sample data record is no initially input which in step 21 is subjected to encoding by means of an LZSS-method which is known per se from the prior art. A typical data record or even an actual data record can be

used as the sample data record.

In step 22, the compression result which is obtained by executing step 21 is subjected to Us statistical analysis. To this end, e.g. the frequency distribution of the different lengths of literal sequences occurring in the compression result is established as well as the frequency distributions of the lengths of back-references and the lengths of literal sequences copied during the execution of step 21.

30 In order to optimise the decompression rate, maximum lengths are determined subsequently.

For this purpose, an upper limit Sit is initially determined in step 23 for the length of the

s literal sequences, so that X% of the literals contained in the compression result of step 21 have a length < So. For example, X% can be assumed to be 95%.

Accordingly, in step 24 an upper limit S2 is determined for the length of the back-references, 5 so that Y% of the back-references have a length which is <the upper limit S2. It is also possible in this case for Y% to be 95%.

Finally, in step 25 a further upper limit S3 is determined for the length of the copied literals in the compression result of step 21, so that Z% of the copied literal sequences have a length < 0 the upper limit S3. Z% can again be selected to be 95%.

In step 26, the bit numbers required in each case for encoding the different lengths are determined, i.e. the number of bits B' for encoding S' different lengths of literal sequences, the number of bits B2 for encoding S2 different lengths of back-references and the number of l 5 bits B3 for encoding S3 different lengths of literal sequences to be copied are determined.

On the basis of the results of step 26, the control codes are established in step 27. The distinction between an L and a C control code is made by the first bit position - 0 for the control code L and l for the control code C in the example considered.

In the control code L, a number of Be bit positions X then follows for the purpose of encoding the length n of the subsequent literal sequence. In the control code C, the leading 1 is then initially followed by a number B2 of bit positions X for encoding the different lengths of backreferences, followed by a number of B3 of bit positions Y for encoding the different 25 character lengths of the literal sequences which are to be copied.

For example, for a sample data record the following values were determined: S. =128, S2 = 4096 and S3 = 32. As a result, Be = 7, B2 = 12 and B3 - 5.

30 The Tables in Figure 3 show that a high percentage of the data utilizes only a small portion of the possible control codes.

In the case of the examined sample data records literal sequences of a length 1 made up a proportion of 50% of the occurring control characters L; literal sequences of a length of 2 to 8 made up a proportion of 25% and literal sequences of > 8 up to the upper limit S. made up a proportion of 25%.

Accordingly, back-references having literal sequences which are to be copied of a length of 1 to 8 made up a proportion of 70% of the control codes C. Furtherrnore7 back-references having a length of the pointer p between l and 32 positions made up a proportion of 50% of the control codes C7 back-references of a length between 33 and 512 positions made up a I o proportion of 25% and back-references of a length of > 512 up to the upper limit made up a proportion of 25%.

Accordingly, as shown in the illustration of Figure 4 two different quantities of control codes L and C are formed For the control codes L these are the codes Lo, L2 and L3 respectively for a length range of the literal sequences of 1, 2 to 9 and lO to 265. The number of bits B' required for the control codes Lo L2 and L3 respectively amounts to 0, 3 and 8 respectively. In the example considered in this case, the control code Lit is encoded as 001, the control code L2 is encoded as 010 and the control code L3 is encoded as 01 1; the respective length for encoding a control code thus amounts to three bits in this case.

The illustration in Figure 4 also includes the encoding for the control codes C In the example considered, six control codes Cal to C6 are formed according to the distribution of the back-references in Figure 3. The control code Cal is encoded as 1001, the control code C2 is encoded as 1010 etc The number of bits used for encoding each of the control codes C is always four; however, it is alternatively also possible for the control codes L and C to be encoded e.g. according to a Huffman-method, wherein the probability of a specific code occurring is taken into consideration in accordance with the table of Figure 3.

After Table 3 has been used to determine the number of codes and their size, the frequency of

the individual codes is determined with respect to all of the occurring codes and the Huffman code is allocated according to this frequency.

If the literal codes constitute 40% of all codes and the copy codes having a short character 5 string constitute 70% of all copy codes, the following distribution is produced using Table 3: Code Frequency Ll 20% L2 10%

loL3 10% C, 21%

C2 10.5%

3 10.5%

C4 9% _ _ 5C5 4.5%

C6 4.5%

In this case, different code lengths are produced, wherein the code having the highest frequency receives the shortest encoding. In the example considered, this is code Cal.

The control code Cal is used for a back-reference with a pointer in the value range of 2 to 33 characters to a literal sequence of a length of 2 to 5 characters. It is necessary to take into consideration that a backreference is only performed, if the length of the back-reference is at least two characters and the length of the literal sequence, which is to be copied and to which 25 a back-reference is to be made, is at least two. Accordingly, the number of bits for encoding the value range of the pointer amounts to five and the number of bits for encoding the value range of 2 to of the length of the literal sequences to be copied is two bits. Corresponding allocations can also be found in the Table of Figure 4 for the control codes C2 to C6.

JO If the characters in the sequence to be compressed are arranged in a byte raster, for example a

width of two or four bytes, the data compression can be further optimised, in that only those pointer lengths which actually occur are mapped in the control codes C. For example, the bit number for encoding the pointer length in the control code Cat for data in a two byte raster can be reduced from 5 to four bits, since by definition odd numbers of backreferences cannot 5 occur. In the case of a raster of four bytes in length, it is accordingly possible to achieve a reduction by an additional bit. The presence of data in a byte raster is also defined as alignment. The alignment of the data is transferred accordingly to the back-references.

Figure 5 illustrates the encoding of the sequence 1 (cf. Figure 1) in accordance with a method 0 according to the invention by means of the control codes of Figure 4. This gives the compression result 3.

A disadvantage of the compression result 3 is that due to the bitoriented encoding of the co nrnands, the literal sequences included in the compression result 3 are no longer oriented 15 to byte limits and therefore must be shifted accordingly.

In order to overcome this disadvantage, the control commands and the literal sequences are initially separated into two data streams during encoding. The data stream of the literal sequences is byte-oriented. The data skeam of the control codes is bit-oriented.

Once the two data streams are complete, they can then be combined to form one single data stream, in which e.g. the two data streams are attached to each other. The separation of the two data streams is characterized in the data stream, which is produced by attachment, by means of a further control code which can be placed approximately at the beginning of the 25 resulting data stream, in order to reference from this point the separation between the data streams. Figure 6 shows a corresponding example, in which the compression result 3 of Figure 5 is re encoded. The compression result 3 is initially divided into a data stream 4 of control codes 30 and into a data skeam 5 of literal sequences.

Attaching the data streams 4 and 5 together produces the resulting data stream 6, in front of which is placed a pointer Z(n) which points to the first character of the data stream 5.

Figure 7 shows a block diagram of a navigation system 7 which includes a CD-ROM player 5 8. The navigation system 7 also has a microprocessor 9 and memory regions 10, 11 and 12.

The CD-ROM of the CD-ROM player 9 contains navigation data compressed according to a method in accordance with the invention.

Sequences of such navigation data are interrogated by the CD-ROM player 8 of the 10 navigation system and are transmitted to the navigation system 7. Upon reception of a data stream corresponding to the data stream 6 of Figure 6, the microprocessor 9 divides the received data stream in to a first data stream of control codes and into a second data stream of literal sequences, wherein this is performed using the pointer Z(n) placed in front.

5 The control code data sequence is stored in the memory region 10, the literal sequences are stored in the memory region 11. For decoding purposes, the microprocessor 9 must then merely process the control codes in the memory region 10 and access the literal sequences in the memory region 11. Decompression results which are determined after execution of a control code are then stored consecutively in the memory region 12 without the need for 20 shift-operations. This renders it possible to achieve extremely rapid decoding in the navigation system 7, so that during the journey it is possible to react very quickly to e.g. any changes in the route and the like.

It is possible to increase further the rate of decompression if during compression only back 25 references of a pointer length greater than the length of the literal sequence to be copied are permitted. For example, a back-reference C4 (17, 20) is then split into C4 (17, 17) C4 (17, 3).

This provides economy of processor performance.

If the data to be compressed includes particular structures, further supplementary methods 30 and, where appropriate, further control codes render it possible once again to improve the compression rate or decompression time:

some data structures have regions, in which a long sequence of identical characters occurs; these sequences can also be encoded in advance by a RUN-LENGTH-ENCODING method.

s - if it is apparent that control code sequences are repeated on a number of occasions consecutively, then they can be encoded by a repetition command. The advantage is that the corresponding control code sequence must only be decoded once.

Claims

1. Method of data compression and / or data decompression according to a method of the 5 LZSS-type, using the following control codes: a first control code for literal sequences of a first maximum length, a second control code for a pointer for a back-reference to a literal sequence to be compressed, wherein the back-reference has a second maximum length and 0 the literal sequence to be copied has a third maximum length.

2. Method as claimed in claim 1, wherein in order to establish the first maximum length the frequency distribution of the lengths of the literal sequences is used in a sample data record which is encoded by means of LZSS-method performed control codes without any length I s restriction.

3. Method as claimed in claim 1 or 2, wherein in order to establish the second maximum length, the frequency distribution of the lengths of the pointers is used in a compressed data record determined by means of an LZSS-method without any length restriction.

4. Method as claimed in claim 1, 2 or 3, wherein in order to establish the third maximum length, the frequency distribution of the lengths of literal sequences to be copied is used in a compressed data record of a sample data record as determined by means of an LZSS-method without any length restriction.

5. Method as claimed in any of the preceding claims 1 to 4, wherein in order to perform the LZSS-method, a first quantity of first control codes and a second quantity of second control codes are used.

30

6. Method as claimed in claim 5, wherein the first quantity includes in each case a first control code for literal sequences within a specific value range.

7. Method as claimed in claim 6, wherein the first and the second control codes are encoded according to the frequency of occurrence of literal sequences or back-references in the relevant value ranges according to a Huffman-method.

s

8. Method as claimed in any of the preceding claims 5, 6 or 7, wherein the second quantity includes in each case a control code for a value range of the pointer and a value range of the literal sequence to be copied.

9. Method as claimed in claim 8, wherein the second control codes are Huffmann-encoded.

1 O. Method as claimed in claim 8 or 9, wherein the value range of the pointer is divided byte - r,se or in multiples of one byte length.

1 1. Method as claimed in any of the preceding claims 1 to 10, wherein the first and the I S second control codes and the literals are stored in two mutually separate portions of the compression result and a third control code serves to identify the separation.

12. Method of decompression of a character string, which is compressed according to a method of the preceding claims 1 to 11, comprising the steps of: - separating the control codes from the literals by means of the third control code, - storing the control codes in a first memory portion, - storing the literals in a second memory portion, 25 - accessing literal sequences, which are to be copied, in the second memory and storing the literal sequences, which are to be copied, in a third memory.

13. Computer program product on a computer-readable medium or a data file which can be loaded via a computer network and comprises program means for performing a method as 30 claimed in any of the preceding claims 1 to 12, when the computer program is executed on an electronic system.

14. Electronic system, comprising means for performing the method steps as claimed in any of the preceding claims 1 to 13.

15. Electronic system, according to claim 13, comprising a first memory region (10) for s storing control codes, a second memory region (11) for storing literal sequences and a third memory region (12) for storing copied literal sequences.

16. Computer program product or electronic system as claimed in claims 13, 14 or 15, in which the electronic system is in the form of a navigation system.

17. A method of data compression, substantially as hereinbefore described, with reference to the accompanying drawings.