JPH07152533A - Data compressing device - Google Patents

Abstract

PURPOSE:To provide a data compressing device and its method which can quickly grow a dictionary and can improve the data compressing effect by registering plural partial data strings in a single dictionary registration processing when the partial input data strings are sequentially registered in the dictionary and then the input data are compressed and coded by referring to the dictionary. CONSTITUTION:A data compressing device 102 consists of a longest string coincidence retrieving part 104 which retrieves the longest one of symbol strings registered in a dictionary 105 that is coincident with a partial data string of an input data string 101, a coding part 106 which codes the index of the longest coincident symbol string, and a dictionary register part 107 which registers a symbol that connects the symbols including the 1st symbol of the coded longest coincident symbol string through the MaxEnt-th symbol decided by the largest register number set value 108 to the longest coincident symbol string coded just before the relevant time point into the dictionary 105.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression apparatus and method which eliminates redundant components in data without causing information loss. In particular, character codes, vector graphics,
The present invention relates to a universal compression encoding method that can obtain a good compression effect without depending on a data format such as image data.

[0002]

2. Description of the Related Art Conventionally, as a method capable of compressing data of various formats with a single encoding method, Lempel
A Ziv encoding method (LZ code) is known.

Two algorithms, a slide dictionary method and a dynamic dictionary method, have been proposed for the LZ code (specifically, interface 1992, vol. 8 No. 1).
83 "Data compression algorithms and their implementation").
Generally, the dynamic dictionary method has a lower compression rate than the slide dictionary method, but it is said that the algorithm is simple and high-speed processing is possible. Further, as an improvement of the dynamic dictionary system, a Lempel-Ziv encoding method (LZW code) has been proposed (TA. Welch, “ATe”).
chnique for High-Performa
nce DataCompression ”, Comp
uter, June, 1984). The LZW code is an incremental parsing of the input data string (Incremental Pars).
ing) is a method of decomposing into partial data strings by a method called ing), registering the partial data strings in a dictionary, and proceeding with encoding processing of input data while referring to the dictionary.

The LZW code will be described in detail below with reference to the drawings. FIG. 4 shows the flow of processing of the LZW encoding method.

First, the overall flow of the LZW encoding process will be described with reference to FIG.

First, in S401, the dictionary is initialized. All symbols that can appear in the input data are registered as a single-character data string in the dictionary as initial values. For example, considering the binary data series in which only 0 and 1 symbols exist as the simplest example, initial values 0 and 1 are registered in the dictionary at unregistered addresses in the dictionary, and each can be uniquely identified. The index is attached. In S402, the counter n for reading the input data string from the first symbol is set to 1. Here, the n-th symbol of the input data string is represented by sn, the first symbol sn (n = 1) is read in S403, and the dictionary that matches this symbol sn is searched for, The index i is calculated. Then, the value of the counter n of the input data string is incremented by 1. Next, in S404, the longest match search process is performed. Here, in the input data string, the longest matching symbol string with the partial data string starting from the symbol sn is searched from the dictionary. Then, the index value of the longest matching symbol string is newly returned to i, and the value of the symbol next to the longest matching partial data string in the data string is newly returned to sn. The flow of the longest match search process will be described later in detail. Next, the index i of the longest matching symbol string is S40.
It is converted into code c (i) in 5 and output. In S406, a new symbol string is registered in the dictionary. First, sn is connected to the end of the symbol string indicated by the index i to generate the symbol string isn. Since the symbol string indicated by index i is the longest matching partial data string,
At the end, sn which is the next symbol in the input data string
The symbol string isn, which is a concatenation of, does not exist in the dictionary. Therefore, this symbol string isn is registered on an unregistered address in the dictionary, an index that can be uniquely identified from the symbol strings registered up to that point is added, and S4 is again set.
Returning to 03, the encoding process is performed in the same procedure.

Next, the flow of the longest match search process will be described with reference to FIG. First, in S411, in order to determine the end of processing, it is determined whether or not there are still symbols to be processed in the input data. When the symbol sn does not exist, the process proceeds to S412, the content of the index i obtained before is encoded, the code c (i) is output, and the process is ended. If the symbol sn exists, the symbol sn is read and the process proceeds to S413. next,
In step S413, the newly read symbol sn is connected to the end of the symbol string indicated by the index i previously obtained to generate the symbol string isn. Then, it is searched whether or not there is a match with this symbol string isn in the dictionary. If there is a match in the dictionary with the symbol string isn, the index of the dictionary corresponding to the symbol string isn is obtained in S415, the value of i is updated to the value of the index, and the counter n The value of 1
Only increase. Then, the process returns to S411 again, and the same procedure is repeated until no symbol string matching the symbol string isn can be found in the dictionary, and the longest matching symbol string is searched from the dictionary among the partial data strings in the input data. Do things. By this iterative process, S4
If there is no match in the dictionary with the symbol string isn in 14, the longest match with the above-mentioned partial data string of the input data among the symbol strings registered in the dictionary is indicated by the index i. It becomes a symbol string.

As described above, in the LZW code, the contents of the dictionary are updated inside the encoding process by dividing the past data string into partial data strings and sequentially registering each partial data string in the dictionary. It is characterized by going. A schematic structure of the dictionary created here has a tree structure as shown in FIG. Here, a tree branch represents a symbol, and a symbol string represented from the root of the tree to a node is a symbol string registered in the dictionary. Further, the number attached to the node serves as an index for identifying the symbol string. By the nature of the LZW code, a branch including a symbol string that frequently appears in the input data string is likely to extend,
Further, since the symbol string length indicated by such a long extended branch is longer than the code length when the index for identifying each branch is coded, data can be compressed.

[0009]

However, in the conventional data compression method as described above, S403 to S40 in FIG.
The branch of the dictionary that is additionally updated by the iterative processing of 6 is created by connecting one symbol to the end of any of the symbol strings registered in the dictionary up to that point, and thus extends for only one symbol. In the LZW code, in order to perform effective data compression, it is desirable that the branch with the high frequency of appearance in the dictionary grows as quickly as possible, but the speed at which the branch grows is extremely limited due to the above configuration. Has become. For this reason, the data compression performance deteriorates especially in the initial stage of creating the dictionary.

In the present invention, in view of the above point, by extending the branch registered in the dictionary as quickly as possible,
The purpose is to achieve higher performance data compression.

[0011]

In order to solve the above problems, according to the present invention, the conventional data compression processing shown in FIG.
In the dictionary registration process of S406 of (a), a plurality of symbols are concatenated to the end of the longest matching partial data string in one process so that a plurality of newly generated symbol strings are registered in the dictionary. The dictionary registration method was improved. For more information,
In the conventional dictionary registration process, only one new symbol string generated by connecting one symbol following the longest matching partial data string of input data to the end of the longest matching partial data string of input data is newly added to the dictionary. While I was registered,
According to the present invention, a plurality of new symbol strings generated by connecting a plurality of symbols following the longest matching partial data string of input data to the end of the longest matching partial data string of input data are newly registered in the dictionary. A dictionary registration means is provided. Furthermore, by limiting the number of symbols linked to the longest matching partial data string, it is possible to register in advance a dictionary in order to prevent registration of a large number of large symbol strings that appear in the data string with a low appearance frequency. The maximum number of symbol strings registered in the processing means is set in advance, and means for limiting the number of symbol strings registered in the dictionary in one processing is provided.

[0012]

According to the present invention, by providing the above means, the symbol string registered in the dictionary becomes longer as quickly as possible, and the dictionary grows faster than the conventional LZW code. As a result, especially in the data compression process, the dictionary grows sufficiently even in the initial stage of the process, so that the data compression effect is enhanced.

Further, by limiting the maximum number of symbol strings to be registered in one dictionary registration process, it is possible to grow a dictionary at high speed and avoid registering an unnecessarily long symbol string. It has a high compression effect.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows the configuration of a data compression apparatus according to an embodiment of the present invention.

The input data string 101 is the data compression device 10.
2 is input, the longest match search unit 104 causes the dictionary 105
The partial data string of the longest input data string 101 that matches the symbol string registered in is searched. The symbol sequences registered in the dictionary are each provided with an index that can be uniquely identified from other symbol sequences, and the index of the symbol sequence that has the longest match with the partial data sequence of the input data 101 is sent to the encoding unit 106. It is encoded, encoded, converted into compressed data 103, and output from the data compression device 102. The symbol string searched by the longest match search unit 104 is sent to the dictionary registration unit 107. here,
A new symbol string is created from the currently encoded longest matching symbol string and the immediately preceding longest matching symbol string, and is newly registered in the dictionary 105. At this time, the number of symbol strings registered in one dictionary registration process does not exceed a predetermined maximum registration number setting value 108.

FIG. 2 shows a processing flow of the data compression method according to the embodiment of the present invention. Also in this embodiment, for simplicity, the input data string is assumed to be a binary data string composed of two symbols 0 and 1.

First, the overall flow of data compression processing will be described with reference to FIG. First, in S201, the dictionary is initialized as in the conventional LZW code. In this embodiment, since it is a binary data string, symbols of 0 and 1 are registered as initial values with indexes that can be uniquely identified from each other. In S202, a counter n for reading the input data string from the first symbol is set to 1. Next, in step S203, the n-th symbol sn is input, the one matching the symbol sn is searched from the dictionary, and its index is set to iL. Also, the value of the counter n is incremented by 1.

The index iL obtained in S203 is S2
It is encoded in 04 and output as a code c (iL). Next, in step S205, the symbol sn is searched for in the dictionary for a match, and its index is set to iP. Also, the value of the counter n is incremented by 1. S2
In 06, the longest match search process is performed. Here, in the input data string, the longest matching symbol string with the partial data string starting from the symbol sn is searched from the dictionary. Then, the index value of the longest matching symbol string is newly returned to iP, and the value of the symbol next to the longest matching partial data string in the data string is newly returned to sn. The flow of the longest match search process will be described later in detail. The index iP is encoded in S207 and output as a code c (iP).

Next, in S208, a dictionary registration process for registering a new symbol string in the dictionary is performed. Here, a plurality of new symbol strings are registered in the dictionary for each dictionary registration process. Here, the maximum number of new symbol strings to be registered in the dictionary is determined in advance in the dictionary registration processing S208, and the value is set as MaxEnt. That is, one to MaxEnt new symbol strings are registered in the dictionary by one dictionary registration process. Therefore, in the dictionary registration processing S208, a long partial data string having a high appearance frequency in the input data string is registered as a symbol string in the dictionary, and the learning effect of the dictionary is improved, so that the data compression rate is increased. It will be improved. Dictionary registration process S
Details of the registration processing by 208 will be described later.
Finally, in step S209, the index iL and the index i
P is compared and index iL and index i
When the value of P is different, the value of the index iP is substituted into the index iL, and then the process returns to S205. When the values of the index iL and the index iP are equal, the process returns to S205. The description of this branching process will be given at the same time as the detailed description of the dictionary registration process S208.

Next, the flow of the longest match search process will be described with reference to FIG.

First, in S221, it is checked whether or not the symbol sn exists in the input data. When the symbol sn does not exist, it is determined that the processing of the input data is completed,
The index iP is encoded in S222, and the code c (i
P) to end the data compression process. If the symbol sn is present, the process is continued and the process proceeds to S223.

At S223, the symbol string indicated by the index iP and the symbol sn are connected. The concatenation is achieved by adding the symbol sn to the end of the symbol string indicated by the index iP. Then, the symbol string newly generated by this concatenation processing is set to iPsn. In S224, the symbol string iPsn is searched. That is, the dictionary is searched for a match with the symbol string iPsn, and if there is no match, the symbol string indicated by the index iP becomes the longest matching symbol string, and the index iP of the longest matching symbol string and The value of the symbol sn appearing next to the longest match symbol string is returned to S208 of the overall flow of the data compression process, and the longest match search process is ended. If there is a match in the dictionary with the symbol string iPsn in S224, the process returns to S221 to continue the longest match search process. In this way, the partial data string is extended from the input data by one symbol by the concatenation process and sequentially searched from the dictionary, so that it is possible to search for the partial data string that has the longest match with the symbol string registered in the dictionary.

Next, the flow of the dictionary registration process is shown in FIG.
An explanation will be given using (c). When registering a dictionary, the maximum number MaxEnt of symbol strings that can be registered in one dictionary registration process is set in advance as described above. Further, the indexes iL and iP are passed to the dictionary registration processing from the overall flow of the data compression processing. Then, the index i is added to the symbol string indicated by the index iL.
A new symbol string is generated by concatenating some symbols at the beginning of the symbol string indicated by P, and the generated symbol string is registered in the dictionary.
First, in step S231, a counter value j for performing an iterative process required to register a plurality of (maximum MaxEnt) symbol strings is initialized to zero. Then, in S232, the j-th symbol in the symbol string indicated by the index iP is indicated by iP [j], and it is determined whether or not the symbol iP [j] exists in the symbol string indicated by the index iP. Here, it is assumed that the leading symbol of the symbol string is the 0th symbol.

When the symbol iP [j] does not exist, the dictionary registration processing is terminated and the overall flow of the data compression processing is returned. When the symbol iP [j] exists, S23
3, the symbol iP [j] is set to t. Next, S2
At 34, the symbol string is registered in the dictionary. First, the symbol t is connected to the end of the symbol string indicated by the index iL, and the symbol string newly generated by the connection is set to iLt. The symbol string indicated by the index iL is the longest match with the partial data string of the input data among the symbol strings currently registered in the dictionary. Further, since the symbol string indicated by the index iP is a symbol string following the symbol string indicated by the index iL in the input data, the symbol string iLt is not registered in the current dictionary. Therefore, the symbol string iLt is newly registered in the dictionary, and an index that can be uniquely identified from other symbol strings registered in the dictionary is added. When the symbol string is registered in the dictionary in this way, the process proceeds to S235, the index of the symbol string iLt is newly set to iL, and the counter value j is incremented by 1. Finally, the counter value j and the MaxEnt value are compared, and if they are equal, the dictionary registration process is terminated and the overall flow of the data compression process is returned to.
If the counter value j and MaxEnt are not equal, S
Returning to 232, the dictionary registration processing is continued.

Here, the case where the values of the indexes iL and iP are equal will be described. For example, (a) in FIG.
It is assumed that a dictionary such as (e) having a tree structure like this has been created. The value of the index iL at this time is 2 (the symbol string indicated by iL is 00). Then, the subsequent input data string is 00011. ． ． Then, the longest matching partial data string is 00, and the value of the index iP is also 2. At this time, MaxEnt
Let 2 be 2, two symbol strings 000 and 0000 are newly registered in the dictionary, and the dictionary has a tree structure (b) (f).
Becomes Here, the index i
When the value of L is updated with the value of iP and the processing is advanced, the value of the index iL is 2 (the symbol string indicated by iL is 00),
The next longest matching partial data string is 01, and the index iP value is 4. Therefore, since the symbol string registered in the dictionary is 0000001, the dictionary has a tree structure (c).
Is updated as in (g). Then, the symbol strings indicated by index 6 and index 8 are both 000.
Therefore, the constituent elements of the dictionary are duplicated. This results in an increase in unnecessary indexes in the dictionary, resulting in a large code length for encoding the index so that it can be uniquely identified, resulting in a decrease in encoding efficiency. Therefore, when the values of the index iL are equal to the values of the iP, the value of the index iL is updated with the value of the iP and the process is not advanced. It is assumed that the value of the index iL is updated with the index of the latest registered symbol string generated by concatenating and the processing is advanced. Then, in the example of FIG. 2, the value of the index iL is 7, and the symbol sequence indicated by it is 000.
Since it becomes 0, and the next longest matching partial data string is 01,
The value of the index iP becomes 4, and the dictionary is eventually updated to (i) having the tree structure (d).

[0026]

As described above, according to the present invention, when a dictionary registration process is performed, a plurality of new symbols are connected to the end of the longest matching partial data string up to a plurality of symbols following the input data string. By creating a column and registering it in the dictionary, the dictionary grows faster and the compression effect is higher than in the conventional data compression process.

By limiting the maximum number of symbol strings concatenated to the longest matching partial data string, it is possible to register a large symbol string whose appearance frequency in the input data string is extremely low. This prevents the compression efficiency from deteriorating.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a data compression device of the present invention.

FIG. 2 is a flowchart showing control of the data compression apparatus of the present invention.

FIG. 3 is an explanatory diagram of dictionary registration processing used in the data compression apparatus of the present invention.

FIG. 4 is a flowchart showing control of a conventional data compression method.

FIG. 5 is a diagram illustrating an expression based on a tree structure of a dictionary.

[Explanation of symbols]

 101 Input Data 102 Data Compressor 103 Compressed Data 104 Longest Match Data Search Unit 105 Dictionary 106 Encoding Unit 107 Dictionary Registration Unit 108 Longest Registration Number Set Value

Claims (2)

[Claims] 1. A data compression apparatus for generating compressed data by removing a redundant component contained in input data composed of discrete information composed of a plurality of types of symbols, wherein the longest data matching a partial data string of the input data. Longest matching search means for searching the symbol string registered in the dictionary for the symbol string, an encoding means for creating a code based on the index given to the symbol string, and an encoding by the encoding means. Generating means for newly generating a plurality of symbol sequences from the generated partial data sequence and the partial data sequence encoded immediately before, and a dictionary registration for registering the symbol sequence generated by the generating means in the dictionary And a data compression device. 2. The data compression apparatus according to claim 1, wherein the dictionary registration means is preset with a maximum value of the number of symbol strings newly registered in one processing.