JP2536422B2 - Data compression device and data decompression device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distortionless data compression / decompression technique using a dictionary in which an index is associated with a compression-processed data sequence generated from encoded or decompression-processed data. The device of the present invention is particularly effective for data having a record structure including fixed length fields.

[0002]

2. Description of the Related Art A data compression technique using a dictionary is well known. This technology is roughly classified into two types, LZ77 system and LZ78 system.

In the LZ77 system, encoded data is stored in a buffer, and an arbitrary data string having a fixed length or less starting from this is made a referable data string. It has a position in the buffer and its length as an index.

The LZ78 system has a dictionary in which input data is cut out according to a certain rule and numbers are sequentially associated with it. The starting position is limited, but the length is usually not limited.

It is known that these techniques show an excellent compression rate and coding restoration speed particularly in the compression processing of text data.

As a data compression technique using a plurality of dictionaries generated from encoded or decompressed data, Japanese Patent Laid-Open No. 4-256192 “Data compression / decompression method” is known.
It has been known. FIG. 5 shows a flowchart of the algorithm of this data compression / decompression technique.

The apparatus of FIG. 5 is characterized in that a plurality of dictionaries are searched, a dictionary that can represent the longest character string is selected, and the dictionary is designated by a reference number and encoded. To do.

[0008]

The data compression technique using the dictionary exhibits a good compression ratio when a long matching string exists between the data string registered in the dictionary and the input data string.

However, although it is more advantageous when a large number of data strings are registered in the dictionary, when a large number of useless data strings are registered, the number of indexes increases remarkably compared with the matching length, The number of bits required for one output increases, which causes deterioration of the compression rate.

One technique for solving the disadvantage of the deterioration of the compression ratio is Japanese Patent Laid-Open No. 4-256192 "Data compression / restoration system".
Is disclosed in.

Another technique is to expect that the data strings appearing differ depending on the state, set some states, and change the dictionary to be referred to according to the state at the time of encoding. Yes, this technique allows asymptotic evaluation of the data length of the compression rate when the state setting matches the state of the information source (IEEE Trans.
Vol. IT-24, 1992, Jan. , 79-8
1).

[0012] The problem in Japanese Unexamined Patent Publication No. 4-256192 "Data compression / restoration method" is how to set the state. If an appropriate state is set, it is disclosed in JP-A-4-256.
It should not be necessary to search through all dictionaries to find the maximum match length as described in 192 "Data Compression and Restoration".

The method usually used to determine the state is to determine the state by one or several symbols immediately before the data string to be encoded (Study of Technical Bulletin IT92-134, 1993, 3). . In this technical report, especially the case of the first-order Markov is described.

However, when the nature of the data is known to some extent, a more effective state determination technique may exist. Now consider data having a record structure consisting of fixed length fields. For example, it is assumed that the data has a record structure composed of two fields, one field consists of four alphabetic characters, and the other field consists of four numerical characters.

Regarding this data, the data string starting after the first character of the English character field and the data string starting from the last fourth character are obviously different, but the status is classified by the immediately preceding character. 4-256192
The “data compression / decompression method” cannot catch this well.

[0016]

The gist of the means for solving the above problems resides in the following two items.

[1] A dictionary to which a number corresponding to a compression processing data string obtained by compression of data to be compressed is added is reconfigured while being updated in parallel with the compression processing, and is registered in the dictionary. In the data compression device that outputs the number added to the compression-processed data string when the same data string as the compression-processed data string appears again in the compressed data string, Position detecting means for detecting the position of the beginning of the compressed data string in
Dictionary selection means for selecting a dictionary to be used for compression processing from among the plurality of dictionaries based on the position of the beginning of the compression processing data string detected by the position detection means and the data structure of the compressed data; The compressed data is generated by compressing the compressed data string in the compressed data using the dictionary, and the dictionary used for the compression process is reconfigured while being updated in parallel with the compression process. A data compression apparatus comprising a compression updating means.

[2] A dictionary to which a number corresponding to the decompression-processed data sequence obtained by decompressing the compression-processed data sequence is added is reconstructed in parallel with the decompression process, and received as decompressed data. In the data decompression device that outputs the decompression-processed data string corresponding to the number in the dictionary with respect to the selected number, a position detection unit that detects the position of the head of the decompression-processed data string in the compressed data. And based on the position of the beginning of the decompression processing data string detected by the position detection means and the data structure of the compressed data,
A dictionary selection unit that determines a dictionary to be used for the decompression process from among the plurality of dictionaries, and decompresses the decompression process data string in the compressed data using the selected dictionary to generate decompression data. A data reconstructing device comprising reconstructing and updating means for reconstructing the dictionary used for the reconstructing process while updating the dictionary in parallel with the reconstructing process.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing the structure of a compression apparatus according to an embodiment of the present invention.

In FIG. 1, LZ is used as the compression updating means 1.
Any of the 77 series and LZ78 series can be used.

As the LZ77 system, "Journal of ACm.
For example, the LZSS method described on pages 928 to 951 of "VOL 29" can be used, and the LZW method described on pages 8 to 17 of "IEEE Compputer. V0L 17" can be used as the LZ78 system.

The dictionary selecting means 3 uses a signal from the position detecting means 2 for detecting the start position of the input data string to select an appropriate dictionary.
Select (4.1 to 4.K).

FIG. 2 is a block diagram showing the construction of a restoration apparatus which is an embodiment of the present invention.

The position detecting means 6 detects the position of the head of the data string to be restored next in the compressed data.

FIG. 3 is a flow chart showing the algorithm of the compression apparatus shown in FIG.

FIG. 4 shows an example of an apparatus for determining the state based on the positions detected by the position detecting means 2 and 6 in the embodiment shown in FIGS. FIG. 3A shows compressed data, which has a record structure composed of fixed length fields, and has a record structure composed of two fields.
One field consists of 4 bytes of English characters and the other field consists of 4 bytes of numbers. For this data, for example, eight dictionaries are prepared, and the position determining means 2 and 6 determine the position n from the beginning of the data as shown in FIG. 7 selects a dictionary (0 to 7) by the remainder modulo 8 of n. The data string starting from 4 bytes of English characters will be registered in the 0th dictionary, and the data string starting from 3 bytes of English characters will be registered in the 1st dictionary. Now, it is assumed that the beginning of the data string to be encoded is the m-th byte from the beginning of the data. To encode this data string, the (m mod 8) th dictionary is used.

When the LZW method is used as the compression algorithm, the data string registered in the (m mod 8) th dictionary is matched with the input data string to find the longest matching string. The index representing the longest matching sequence in the (m mod 8) th dictionary is output as a code word. Then, a new data string in which the next one symbol is connected to the longest matching string is newly registered in the (m mod 8) th dictionary. At the time of restoration, it is not necessary to add an index indicating which dictionary was used in order to know which dictionary was used at the time of encoding from the position of the beginning of the restored data string.

[0028]

EFFECTS OF THE INVENTION Even when a large number of useless data strings are registered in the dictionary and the number of indexes is significantly increased compared to the matching length, it is necessary for one output as compared with the conventional data compression / decompression technique. It is possible to suppress an increase in the number of bits and prevent deterioration of the compression rate.

Further, when the nature of the data is known to some extent, a more effective state determining device can be realized.

For data in which the statistical properties of the compressed data string differ greatly depending on its head position, it is possible to reduce unnecessary registration of the data string in the dictionary, so that it is possible to reduce the number of bits required to represent the index. Since the compression processing data string that is almost the same as when only one dictionary is used is registered, the size of the matching length with the input data string is almost the same when there is one dictionary, improving the compression rate. Can be made.

Consider the case where this device is combined with the LZ78 compression technology. Assume an ideal case where the number of dictionaries is k and the data strings registered in each dictionary are different.

When the number of times each dictionary is referred to is n1, n2, ..., nk, N = n1 + n2 + ... + nk, respectively, in comparison with the case where only one dictionary is provided, The compressed data length can be reduced by about N * H (n1 / N, ..., nk / N) bits. here

[0033]

[Equation 1]

It is

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a compression device that is an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a restoration device that is an embodiment of the present invention.

FIG. 3 is a flowchart showing an algorithm of the compression apparatus shown in FIG.

FIG. 4 is a diagram showing an example of a method of determining a state based on a position in compressed data. FIG. 4A shows compressed data, and here, a record structure composed of two 4-byte fixed length fields. FIG. 11B shows an example of a dictionary selection device for the data of FIG. 9A, and shows the functions of the position detection means and the dictionary selection means.

FIG. 5 is a flowchart showing a conventional technique.

[Explanation of symbols]

 1 compression update means 2 position detection means 3 dictionary selection means 4 dictionary 5 decompression update means

Claims (2)

(57) [Claims] 1. A dictionary to which a number corresponding to a compression processing data string obtained by compression of compressed data is added is reconfigured while being updated in parallel with the compression processing, and the compression registered in the dictionary is performed. In a data compression device that outputs the number added to a compressed data string when the same data string as the processed data string appears again in the compressed data string, a plurality of dictionaries and Position detection means for detecting the position of the beginning of the compressed data string, and based on the position of the head of the compressed data string and the data structure of the compressed data detected by the position detection means, from among the plurality of dictionaries Dictionary selection means for selecting a dictionary to be used for compression processing, and compression processing of the compression processing data string in the compressed data using the selected dictionary to generate compressed data, Data compression apparatus characterized by a dictionary used for the compression process comprising a compression updating means for reconstructing while updating in parallel with the compression process was. 2. A dictionary to which a number corresponding to the decompression processing data string obtained by decompressing a compression processing data string is added is reconstructed in parallel with the decompression processing and received as decompressed data. A data decompression device that outputs a decompression-processed data string corresponding to a number in the dictionary for a number; Dictionary selection means for deciding a dictionary to be used for the decompression processing from among the plurality of dictionaries based on the position of the head of the decompression processing data string detected by the position detection means and the data structure of the compressed data. Decompressed data is generated by decompressing the decompressed data string in the compressed data using the dictionary, and the dictionary used for the decompression process is parallel to the decompression process. Data recovery apparatus characterized by comprising a restoring updating means for reconstructing while updating.