JP3346626B2 - Data compression device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression apparatus and method which eliminate information loss by removing redundant components in data. In particular, character codes, vector graphics,
The present invention relates to a universal compression encoding method that can obtain a good compression effect without depending on a data format such as image data.

[0002]

2. Description of the Related Art Conventionally, Lempel has been used as a method capable of compressing data of various formats by a single encoding method.
-Ziv encoding method (LZ code) is known.

For the LZ code, two algorithms, a slide dictionary method and a dynamic dictionary method, have been proposed (for details, see Interface 1992, Vol. 8, No. 1).
83 "Data Compression Algorithms and Implementations").
It is generally said that the dynamic dictionary method has a lower compression ratio than the slide dictionary method, but has a simple algorithm and can perform high-speed processing. As an improvement of the dynamic dictionary system, a Lempel-Ziv coding method (LZW code) has been proposed (TA Welch, "ATe").
chnique for High-Performa
nce DataCompression ”, Comp
Uter, June, 1984). The LZW code performs incremental decomposition of an input data sequence (Incremental Pars).
ing) is an algorithm for decomposing the data into partial data strings, registering the partial data strings in a dictionary, and proceeding with the encoding process of the input data while referring to the dictionary.

Hereinafter, the LZW code will be described in detail with reference to the drawings. FIG. 4 shows a processing flow of the LZW encoding method.

First, the overall flow of the LZW encoding process will be described with reference to FIG.

First, in step S401, a dictionary is initialized. In the dictionary, all symbols that can appear in the input data as initial values are registered as one-character data strings. For example, as a simplest example, considering a binary data sequence having only 0 and 1 symbols, initial values 0 and 1 are registered in an unregistered address of the dictionary, and each can be uniquely identified. Index. In S402, a counter n for reading the input data string from the first symbol is set to one. Here, the n-th symbol of the input data sequence is represented as sn, the first symbol sn (n = 1) is read in S403, and a dictionary matching this symbol sn is searched from the dictionary. The index i is obtained. Then, the value of the counter n of the input data string is increased by one. Next, in S404, a longest match search process is performed. Here, in the input data sequence, a dictionary is searched from the dictionary for a symbol sequence that is the longest match with the partial data sequence starting from the symbol sn. Then, the index value of the longest matching symbol string is newly returned to i, and the value of the symbol next to the longest matching partial data string in the data string is newly returned to sn. The flow of the longest match search process will be described later in detail. Next, the index i of the longest matching symbol sequence is S40
5, and is converted to a code c (i) and output. In S406, a new symbol string is registered in the dictionary. First, sn is connected to the end of the symbol sequence indicated by the index i to generate a symbol sequence isn. Since the symbol sequence indicated by index i is the longest matching partial data sequence,
At the end thereof, sn which is the next symbol in the input data sequence
Are not present in the dictionary. Therefore, this symbol string isn is registered on an unregistered address in the dictionary, an index that can be uniquely identified from the symbol strings registered so far is added, and S4 is again executed.
Returning to step 03, the encoding process is performed in the same procedure.

Next, the flow of the longest match search process will be described with reference to FIG. First, in S411, it is determined whether or not symbols to be processed still remain in the input data in order to determine the end of the processing. If the symbol sn does not exist, the flow advances to S412 to encode the contents of the index i obtained before and output the code c (i), and ends the processing. If the symbol sn exists, the symbol sn is read, and the process proceeds to S413. next,
In step S413, a newly read symbol sn is connected to the tail of the symbol sequence indicated by the index i obtained earlier to generate a symbol sequence isn. Then, a search is made as to whether or not there is a dictionary that matches this symbol sequence isn. If there is a match in the dictionary with the symbol sequence isn, the index of the dictionary corresponding to the symbol sequence isn is obtained in step S415, and the value of i is updated to the value of the index. The value of 1
Just increase. Then, the process returns to step S411 again, and repeats the process in the same procedure until a symbol sequence that matches the symbol sequence isn cannot be found in the dictionary, and searches the dictionary for the longest matching symbol sequence among the partial data sequences in the input data. Do that. By this iterative process, S4
If there is no symbol string that matches the symbol string isn in the dictionary, the symbol string registered in the dictionary that has the longest match with the partial data string of the input data is indicated by the index i. It becomes a symbol sequence.

As described above, in the LZW code, the contents of the dictionary are updated inside the encoding process by decomposing the past data string into partial data strings and sequentially registering each partial data string in the dictionary. It is characterized by going. When the dictionary created here is schematically represented, it has a tree structure as shown in FIG. Here, a branch of a tree represents a symbol, and a symbol sequence represented from the root of the tree to a node is a symbol sequence registered in the dictionary. The number assigned to the node is an index for identifying the symbol string. Due to the nature of the LZW code, a branch including a symbol sequence that frequently appears in an input data sequence is easily elongated,
Further, the symbol string length indicated by such a long branch is longer than the code length when an index for identifying each branch is coded, so that data can be compressed.

[0009]

However, in the conventional data compression method as described above, in steps S403 to S40 in FIG.
The branch of the dictionary that is additionally updated by the repetition processing of No. 6 is created by connecting one symbol to the end of any of the symbol strings registered in the dictionary up to that point, and thus extends only by one symbol. In the LZW code, in order to perform effective data compression, it is desired that a branch having a high frequency of appearance in a dictionary grows as quickly as possible. It has become. For this reason, the data compression performance is deteriorated particularly at the initial stage when the dictionary is created.

[0010] In the present invention, in view of the above points, by extending the branches registered in the dictionary as quickly as possible,
The purpose is to realize higher performance data compression.

[0011]

SUMMARY OF THE INVENTION The present invention has been made to solve these problems, and is intended to eliminate compressed data by removing redundant components contained in input data composed of discrete information comprising a plurality of types of symbols. A longest match search unit for searching a longest symbol sequence that matches the partial data sequence of the input data from the symbol sequences registered in the dictionary, and an index assigned to the symbol sequence. Encoding means for generating a code based on the first longest-matching partial data string encoded by the encoding means, and a second longest-matching partial data string encoded immediately after the first longest-matching partial data string. From the longest-matching partial data string, to the end of the first longest-matching partial data string, sequentially from the first symbol of the second longest-matching partial data string. A generating means for generating a plurality of new symbol, to provide a data compression device configured symbol sequence generated by the generating means and a dictionary registration means for registering the dictionary. 2. The data compression apparatus according to claim 1, wherein said dictionary registration means has a preset maximum value of the number of symbol strings newly registered in one process.

[0012]

In the present invention, by having the above means, the symbol sequence registered in the dictionary becomes longer as quickly as possible, and the dictionary grows faster than the conventional LZW code. As a result, especially in the data compression process, the dictionary grows sufficiently even at the initial stage of the process, and the data compression effect is enhanced.

Further, by limiting the maximum number of symbol strings to be registered in one dictionary registration process, it is possible to avoid unnecessary registration of a long symbol string while growing the dictionary at high speed. The structure has a high compression effect.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows the configuration of a data compression device according to an embodiment of the present invention.

The input data sequence 101 is the data compression device 10
2, the longest match search unit 104 outputs the dictionary 105
Is searched for a partial data string of the longest input data string 101 that matches the symbol string registered in the. Each symbol sequence registered in the dictionary is provided with an index that can be uniquely identified from other symbol sequences, and the index of the symbol sequence that is the longest match with the partial data sequence of the input data 101 is transmitted to the encoding unit 106. The data is then encoded, converted to compressed data 103, and output from the data compression device 102. The symbol string searched by the longest match search unit 104 is sent to the dictionary registration unit 107. here,
A new symbol sequence is created from the currently encoded longest-match symbol sequence and the longest-match symbol sequence encoded immediately before it, and newly registered in the dictionary 105. At this time, it is assumed that the number of symbol strings registered in one dictionary registration process does not exceed a predetermined maximum registration number setting value 108.

FIG. 2 shows a flow of processing of the data compression system according to the embodiment of the present invention. Also in this embodiment, for simplicity, it is assumed that the input data string is a binary data string composed of two symbols 0 and 1.

First, the overall flow of the data compression processing will be described with reference to FIG. First, in step S201, the dictionary is initialized as in the case of the conventional LZW code. In the present embodiment, since the data sequence is a binary data sequence, 0 and 1 symbols are uniquely indexed and registered as initial values. In S202, the counter n for reading the input data string from the first symbol is set to one. Next, in step S203, the n-th symbol sn is input, a search for a symbol that matches the symbol sn is made in the dictionary, and the index is set to iL. Further, the value of the counter n is increased by one.

The index iL obtained in S203 is S2
04 and output as code c (iL). Next, in S205, a symbol sn is searched from the dictionary for a symbol sn, and the index is set as iP. Further, the value of the counter n is increased by one. S2
At 06, the longest match search process is performed. Here, in the input data sequence, a dictionary is searched from the dictionary for a symbol sequence that is the longest match with the partial data sequence starting from the symbol sn. Then, the index value of the longest matching symbol string is newly returned to iP, and the value of the symbol next to the longest matching partial data string in the data string is newly returned to sn. The flow of the longest match search process will be described later in detail. The index iP is encoded in S207 and output as a code c (iP).

Next, in S208, a dictionary registration process for registering a new symbol sequence in the dictionary is performed. Here, a plurality of new symbol strings are registered in the dictionary for each dictionary registration process. Here, it is determined in advance how many new symbol sequences are to be registered in the dictionary in the dictionary registration process S208, and the value is set as MaxEnt. That is, by one dictionary registration process, one to MaxEnt new symbol strings are registered in the dictionary. For this reason, in the dictionary registration process S208, a long partial data sequence having a high appearance frequency in the input data sequence is registered as a symbol sequence in the dictionary, and the learning effect of the dictionary is improved. Will be improved. Dictionary registration processing S
Details of the registration process by 208 will be described later.
Finally, in step S209, the index iL and the index i
P is compared, and index iL and index i
If the value of P is different, the value of index iP is assigned to index iL, and the process returns to S205. If the values of the index iL and the index iP are equal, the process returns to S205. The description of the branching process will be made at the same time as the detailed description of the dictionary registration process S208.

Next, the flow of the longest match search process will be described with reference to FIG.

First, it is checked in step S221 whether a symbol sn exists in the input data. When the symbol sn does not exist, it is determined that all the processing of the input data has been completed,
The index iP is encoded in S222, and the code c (i
P) and the data compression process ends. If the symbol sn exists, the process is continued, and the process proceeds to S223.

At S223, the symbol sequence indicated by the index iP and the symbol sn are connected. The concatenation is achieved by adding a symbol sn to the end of the symbol sequence indicated by the index iP. Then, the symbol sequence newly generated by the concatenation process is set to iPsn. In S224, a search for the symbol sequence iPsn is performed. In other words, the dictionary is searched for a match with the symbol sequence iPsn. If there is no match, the symbol sequence indicated by the index iP becomes the longest match symbol sequence, and the indexes iP and The value of the symbol sn that appears next to the longest matching symbol sequence is returned to S208 in the overall flow of the data compression process, and the longest matching search process ends. If there is a dictionary that matches the symbol sequence iPsn in S224, the process returns to S221 to continue the longest match search process. As described above, the partial data string is expanded by one symbol at a time from the input data by the concatenation process, and by sequentially searching the dictionary, it is possible to search for the partial data string that has the longest match with the symbol string registered in the dictionary.

Next, the flow of the dictionary registration process is shown in FIG.
This will be described with reference to FIG. When registering a dictionary, as described above, the maximum number MaxEnt of symbol strings that can be registered in one dictionary registration process is determined in advance. In addition, indexes iL and iP are passed to the dictionary registration process from the overall flow of the data compression process. Then, an index i is added to the symbol sequence indicated by the index iL.
By linking some symbols at the head of the symbol sequence indicated by P, a new symbol sequence is generated, and the generated symbol sequence is registered in the dictionary.
First, in step S231, a counter value j for performing an iterative process required to register a plurality of (maximum MaxEnt) symbol strings is initialized to zero. Then, in S232, the j-th symbol in the symbol sequence indicated by the index iP is indicated by iP [j], and it is determined whether or not the symbol iP [j] exists in the symbol sequence indicated by the index iP. Here, it is assumed that the first symbol of the symbol sequence is the 0th symbol.

If the symbol iP [j] does not exist, the dictionary registration processing is terminated, and the flow returns to the entire data compression processing. If the symbol iP [j] is present, S23
Proceeding to 3, the symbol iP [j] is set to t. Next, S2
At 34, a symbol string is registered in the dictionary. First, a symbol t is connected to the end of the symbol sequence indicated by the index iL, and a symbol sequence newly generated by the connection is defined as iLt. The symbol string indicated by the index iL is the symbol string that is the longest match with the partial data string of the input data among the symbol strings currently registered in the dictionary. Further, since the symbol sequence indicated by the index iP is a symbol sequence following the symbol sequence indicated by the index iL in the input data, the symbol sequence iLt is not registered in the current dictionary. Therefore, a new symbol sequence iLt is registered in the dictionary, and an index that can be uniquely identified from other symbol sequences registered in the dictionary is added. After registering the symbol string in the dictionary as described above, the process proceeds to S235, where the index of the symbol string iLt is newly set to iL, and the counter value j is increased by one. Finally, the counter value j is compared with the value of MaxEnt, and if they are equal, the dictionary registration processing is terminated, and the flow returns to the entire flow of the data compression processing.
If the counter value j is not equal to MaxEnt, S
Returning to H.232, the dictionary registration process is continued.

Here, a case where the values of the indexes iL and iP are equal will be described. For example, (a) in FIG.
It is assumed that a dictionary as shown in (e) having a tree structure as shown in FIG. It is also assumed that the value of the index iL at this time is 2 (the symbol sequence indicated by iL is 00). Then, the input data string following this is 00001. . . , The longest matching partial data string is 00, and the value of the index iP is also 2. At this time, MaxEnt
Is 2, two symbol strings 000 and 0000 are newly registered in the dictionary, and the dictionary has a tree structure (b) (f).
Becomes Here, the index i
When the value of L is updated with the value of iP and the process proceeds, the value of index iL is 2 (the symbol sequence indicated by iL is 00), and
The next longest matching partial data string is 01, and the value of the index iP is 4. Therefore, the symbol sequence registered in the dictionary is 000,0001, and the dictionary has a tree structure (c).
Is updated as shown in (g). Then, the symbol strings indicated by index 6 and index 8 are both 000
And the components of the dictionary are duplicated. This means that unnecessary indexes increase in the dictionary, so that the code length for coding so that the indexes can be uniquely identified increases, and the coding efficiency decreases. Therefore, when the value of the index iL is equal to the value of iP, the value of the index iL is updated with the value of iP, and the processing is not advanced. It is assumed that the value of the index iL is updated with the index of the most recently registered symbol string generated by concatenation, and the process proceeds. Then, in the example of FIG. 2, the value of the index iL is 7, and the symbol sequence indicated by it is 000.
0, and the next longest matching partial data string is 01,
The value of the index iP is 4, and the dictionary is eventually updated as shown in (i) having the tree structure (d).

[0026]

As described above, according to the present invention, when performing dictionary registration processing, a plurality of new symbols are connected to the end of the longest matching partial data sequence up to a plurality of symbols following the input data sequence. By creating a column and registering it in the dictionary, the dictionary grows faster than the conventional data compression processing, and the compression effect is enhanced.

By limiting the maximum number of symbol strings linked to the longest matching partial data string, it is possible to register a long symbol string whose appearance frequency in the input data string is extremely low. The compression efficiency is not reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a data compression device of the present invention.

FIG. 2 is a flowchart illustrating control of the data compression device of the present invention.

FIG. 3 is an explanatory diagram of a dictionary registration process used in the data compression device of the present invention.

FIG. 4 is a flowchart illustrating control of a conventional data compression apparatus method.

FIG. 5 is a diagram illustrating an expression using a tree structure of a dictionary.

[Explanation of symbols]

 Reference Signs List 101 input data 102 data compression device 103 compressed data 104 longest matching data search unit 105 dictionary 106 encoding unit 107 dictionary registration unit 108 longest registration number setting value

Claims (2)

(57) [Claims] 1. A data compression apparatus for generating compressed data by removing redundant components included in input data composed of discrete information composed of a plurality of types of symbols, comprising: A longest match search unit for searching the symbol sequence registered in the dictionary for a symbol sequence, a coding unit for creating a code based on an index given to the symbol sequence, and a coding process performed by the coding unit. The first longest done
A matching partial data string and the first longest matching partial data string
Or the second longest matching partial data string coded immediately after
At the end of the first longest matching partial data string.
2 in sequence from the first symbol of the longest matching partial data string
Binding to, a generation unit for generating a plurality of new symbol, the data compression apparatus characterized by being configured symbol sequence generated by the generating means and a dictionary registration means for registering the dictionary. 2. The data compression apparatus according to claim 1, wherein the dictionary registration means sets in advance a maximum value of the number of symbol strings newly registered in one process.