JP3012679B2 - Data compression method - Google Patents

Description

[Contents] Outline Industrial application field Conventional technology (FIGS. 7 to 13) Problems to be Solved by the Invention Means for Solving the Problems (FIG. 1) Action Embodiment (A) Description of one embodiment (FIGS. 2 to 6) (b) Description of another embodiment [Summary] Regarding a data compression method for compressing data using LZW codes, a registered part in a dictionary For the purpose of searching columns at high speed and shortening the encoding time, the encoded data is divided into different sub-sequences, the sub-sequences are registered in a dictionary, and the search order of the concatenated sub-sequences is changed. A data compression method for comparing and searching input data and sub-strings in the dictionary according to the search list shown, and designating and encoding the input data by the reference number of the sub-string in the dictionary that matches the maximum length , One of the searched substrings in the search list The search list is rewritten to replace the previous subsequence.

[Industrial applications]

The present invention relates to a data compression method for compressing data using an LZW code.

In recent years, various types of data such as character codes, vector information, and images have been handled by computers.
The amount of data handled is also rapidly increasing. When dealing with a large amount of data, by compressing the amount of data by omitting redundant portions in the data, the storage capacity can be reduced or the data can be transmitted faster.

Universal coding has been proposed as a method that can compress various data in one system. Here, the field of the present invention is not limited to character code compression, and can be applied to various types of data.However, in the following, following the name used in information logic, a 1-word unit of data is called a character, Are connected to an arbitrary word.

As a typical method of universal code, Ziv-Lempe
There is an l (Jib-Lempel) code (for example, for example,
Munakata "Ziv-Lempel Data Compression", Information Processing, Vol.2
6, No. 1, 1985).

For the Ziv-Lempel code, two algorithms of a universal type and an incremental decomposition type (Incremental parsing) have been proposed. Furthermore, as an improvement of the universal algorithm, there is an LZSS code (TCBell, “Better O
PM / L Text Compression ", IEEE Trans.on Commun., Vo
l.COM-34, No. 12, Dec. 1986). Further, as an improvement of the incremental decomposition type algorithm, LZW (Lempel-Ziv-Welc
h) Signed (TAWelch, “A Technique for High-
Performance Data Compression ", Computer, June 1984).

Among these codes, the LZW code is used for file compression of a storage device or the like because of its high speed processing and the simplicity of the algorithm.

[Conventional technology]

First, the LZW code will be described with reference to FIGS. 7 to 9.

FIG. 7 is a flowchart of the LZW encoding process, FIG. 8 is a flowchart of the LZW decoding process, and FIG. 9 is an explanatory diagram of the LZW encoding and decoding.

FIG. 9 shows a case where data consisting of a combination of three characters of abc is compressed and decompressed for the sake of simplicity.

LZW encoding has a rewritable dictionary, divides the input character code and data into different character strings, adds numbers to these character examples in the order they appear, and registers them in the dictionary.
The currently input character string is represented by the number of the longest matching character string registered in the dictionary and encoded.

 The encoding process will be described with reference to the flowchart of FIG.

First, in step S1 (hereinafter "step" is omitted), a character string consisting of one character for all characters is registered in advance as an initial value, and then encoding is started. In the encoding of S1, a dictionary is searched by the first character K input to obtain a reference number ω, which is used as a prefix string.

Next, the next character of the input data is read in S2, and it is checked whether or not the character input is completed in S3.
A search is made to see if the dictionary has a character (ωK) obtained by adding the character K read in S2 to the initial character string ω obtained in S5 or ω in S5.

If the character string (ωK) is not in the dictionary in S4, the process proceeds to S6 and S1
The reference number ω of the character K obtained in step (1) is output as a code word code (ω), a new reference number is added to the character string (ωK), and the character string (ωK) is registered in the dictionary. And increment the dictionary address and return to S2 to read the next character K.

On the other hand, if the character string (ωK) is found in the dictionary in S4, the character string (ωK) is replaced with the reference number ω in S5, and the process returns to S2 to search for the maximum matching length until the character string (ωK) cannot be found in the dictionary. Continue.

The encoding is specifically described below with reference to FIGS. 9 (A) and 9 (B).

First, the input data of FIG. 9A is read from left to right. When the first character a is input, there is no matching character string other than a in the dictionary 10, so the output code (reference number ω) is output as a code word. And the expanded string
ab is assigned a reference number 4 and registered in the dictionary 10. The actual registration is in the form of a character string (1b). Subsequently, the second b becomes the head of the character string. Since there is no matching character string other than b in the dictionary 10, reference number 2 is output as a code word, and the expanded character string ba is actually registered in the dictionary 10 with reference number 5 in the form of 2a. . The third "a" is the beginning of the next character string. Hereinafter, this process is similarly continued.

The decoding process in FIG. 8 performs the reverse operation of the encoding in FIG.

In the decoding of FIG. 8, similarly to the encoding, the decoding is started after a character string consisting of one character for every character is registered in the dictionary as an initial value in advance.

First, the first code (reference number) is read in S1, and the current code is read.
CODE is OLDcode, and since the first code corresponds to one of the reference numbers of one character already registered in the dictionary, a character code (K) that matches the input code CODE is searched for, and the character K
Is output. The output character (K) is set in FINchar for the exception processing in S8 described later.

Then, the process proceeds to S2, where the next code is read and set as CODE in INCODE.

Check whether there is a new code in S3, that is, check whether the code input is completed, proceed to S4, and enter the code input in S3.
Check whether the CODE is defined (registered) in the dictionary.

Normally, since the input code word is registered in the dictionary in the previous processing, the process proceeds to S5 and the character string corresponding to the code CODE is processed.
The code (ωK) is read from the dictionary, the character string K is temporarily stacked in S6, the reference number code (ω) is set as a new CODE, and the process returns to S5 again. Is repeated until one character is reached. Finally, the process proceeds to S7, and the characters stacked in S6 are popped up and output in LIFO (Last In Fast Out) format. At the same time, in S7, a new reference number is added to the character string represented as a combination (ω, K) of the code ω used last time and the first character K of the character string restored this time, and registered in the dictionary.

The decoding process is specifically described below with reference to FIGS. 9 (C) and 9 (D).

First, in FIG. 9 (D), the first input character is 1, and one character a, b, c is already registered in the dictionary 10 as reference numbers 1, 2, 3 as shown in FIG. 9 (C). Therefore, by referring to the dictionary 10, it is replaced with the character string a of the reference number that matches the code 1, and is output. Similarly, the next code 2 is replaced with the character b and output. At this time, a new reference number 4 is added to (1b), which is a combination of the previously processed code and the first character b decoded this time, and registered in the dictionary 10.

The third code 4 replaces 1b with ab by searching the dictionary 10 and outputs a character string ab. Code 2 processed at the same time
The character string 2a (= ba) obtained by combining the character string 2a (= ba) with the first character a of the character string decoded this time
Register with.

 Hereinafter, similarly, this processing is repeated.

 9 (d) has the following exception processing.

This exception processing occurs when the sixth input code 8 is decoded. The code 8 is not defined in the dictionary at the time of decoding and cannot be decoded. In this case, a character string 5b is obtained by adding the first character b of the previously decoded character string ba to the previously processed code 5 and further replaced with 2ab and bab for output. Then, a reference number 8 is added to a character string 5b obtained by adding the character b of the character string decoded this time to the previous code word 5 to the output word of the character string and registered in the dictionary.

This out-of-column processing is performed through the processing of S4 and S8 in the decoding processing flow of FIG. 8, and finally the output of the character string in S7 and the registration in the dictionary in which the reference number is added to the new character string. Will be

The encoding / decoding processing shown in FIGS. 7 and 8 is performed while creating the same dictionary.

Encoding according to the procedure shown in the flow chart of FIG. 7 takes time because, at worst, every time a character string is searched for in a dictionary, the entire dictionary must be searched. Therefore, in the past, external hashing (open hashing or ch
aining) to increase the processing speed (see, for example, Ohmsha Publishing, Information Processing Society of Japan, Information Processing Bandbook).

 FIG. 10 is an explanatory diagram of the external hash method.

When considering a set S composed of character strings, a high-speed search can be performed if a position where a character string x of S is located is directly calculated from the character string x.
The hash method realizes this.

Assuming that addresses from 0 to m-1 are assigned to the storage case (hash table), one function h: S → [0,1,..., M-1] is defined by the hash method. , S, the address of the character string x is determined by h (x). The function h is called a hash function, and the value h (x) is called a hash address of x.

The hash method is usually used when the size of S is much larger than m. Therefore, no matter how h is selected, h is set for different character strings x ₁ and x _{2 of} S.
(X ₁ ) = h (x ₂ ). This is called a collision, and an external hashing method (open hashing or chainig) is used as one of the measures against the collision.

In the external hash method, as shown in FIG. 10, a list is prepared for each hash address i, and x (h (x) = i)
Are ordered from the top of the list. Each list with the same hash address is called a bucket.

11 to 13 are explanatory diagrams of the prior art, FIG. 11 is an example of a search tree, FIG. 12 is a state of a character string storage table and an external hash table, and FIG. 6 is a flowchart of an LZW code using the method.
(For details, see Shouisha Publishing, edited by AP-Labo, Hard Disk Cookbook).

In FIG. 12, the character string storage table 10b stores the code and the character ext [i] for the index i, and the array first100 stores the first concatenated index first [i] for the index (address) i in the array next1.
01 stores the next connected index next [i] for the index (address) i.

The array first100 is the index dictin of the external hash method of Fig. 10.
The array next101 corresponds to the linked list in FIG. 10 corresponding to ary.

In the external hash method, when a new character K is input,
Reference number (hash address) i of the previous character string
The reference number of the character string in which the character K is added to is obtained by the external hash method.

According to the external hash method, a character string obtained by adding one character to the character string of the reference number i is subtracted as i as a hash address (index). In the linked list, the character added to the character string i is stored in the name, and the matching between the character of the name and the character K is checked. One character additional character string can be searched. If there is no character string to which the character K is added in the bucket, the linked address 0
Is obtained, and it can be known that the corresponding character string is not registered.

Next, as shown in FIG. 11, the character strings “ab”, “ah”,
“Az”, “abf”, “abx”, “ahd”, “ahf”, “azc”, “a
When "zg" and "azw" are registered in the dictionary, the character string "a
A conventional search method will be described using an example of searching for "zw".

The state of the character string storage table 10b in the initial state and the state of the external hash table 10a (100, 101) are as shown in FIG.

The first character "a" has already been registered, and when searching for the second character "z", P1 is found by subtracting the array fisr using "a" as the hash address. Therefore, the character string "ab" following "a" stored in P1 is found. But,
Since it does not match this, P1 becomes P2 when the array next is subtracted as a hash address. Here, "ah" of the character string stored in P2 is found, but this does not match, so P3 is found by further subtracting the array next using P2 as the hash address. Here, it matches the second character “z”.

Next, the process proceeds to the search for the third character “w”. In the search for the third character, first, an array first with P3 as a hash address is subtracted. Then, it becomes P8 and the third character "d" is searched. However, because they do not match, the array next with P8 as the hash address is now P9. However, this does not match, so if the array next with P9 as the hash address is further subtracted, it becomes P10, and the target third character "w" is found. The search is performed through the route shown in FIG.

[Problems to be solved by the invention]

Conventionally, once a character string is registered, the external hash table is fixed and is not rewritten.

Therefore, even if a character string with a high search frequency is stored in a long place on the search path, it is searched every time through the long path, which is inefficient and registration is delayed. If it is registered at the end of the list with the same hash address, there is a problem that it takes time to search each time.

Therefore, an object of the present invention is to provide a data compression method capable of searching a registered sub-sequence in a dictionary at high speed and shortening the encoding time.

[Means for solving the problem]

 FIG. 1 is a diagram illustrating the principle of the present invention.

In the present invention, as shown in FIG. 1, the encoded data is divided into different sub-sequences, the sub-sequences are registered in the dictionary 10, and a search list 10a indicating the search order of the connected sub-sequences. In the data compression method, the input data is compared with a subsequence in the dictionary 10 in accordance with the following, and the input data is specified and designated by the reference number of the subsequence in the dictionary 10 that matches the maximum length. The search list 10 is replaced with the subsequence in the search list 10a.
is to rewrite a.

Further, in the present invention, the search list 10a indicates a search order from a small number of characters in the partial string to a large number of characters in the partial string, and the searched partial string is a partial string having the same number of characters in the search list 10a. Is to rewrite the search list 10a so that it comes immediately before.

[Action]

According to the present invention, in consideration of the frequency of use of a character string, the character string once searched is replaced with the character string having the same hash address immediately before the character string. Then, after the second search, the number of times of searching for the same character string is reduced by one per character. Each time a search is performed, the search is replaced by the previous search.
The character string having the same hash address will be registered at the beginning.

Therefore, encoding using the dictionary can be speeded up, and the encoding time can be reduced.

〔Example〕

(A) Description of one embodiment FIG. 2 is a processing flowchart of one embodiment of the present invention, and FIGS. 3 to 6 are explanatory diagrams of the operation thereof.

 Incidentally, S1 to S7 are the same as S1 to S7 in FIG.

S1) The processor (see FIG. 1) initializes the dictionary 10 to include the first character. That is, the character code 1 is registered at the address m (= 1) of the dictionary 10.

Also, the number n of characters currently registered in the dictionary 10 is set as the number of characters.

Further, the first character K input using the character string storage table 10b is referred to as a first character string as a reference number (index) i.
Convert to

Next, the first [NMAX] and nex of the dictionary search array first100
next [NMAX] of t101, ext [NMAX] of character string table 10b
Is initialized to 0.

S2) The processor reads the next input character K.

S3) The processor checks whether there is a next character K.

S7) If there is no next character K, the code K (ω) of the character K is output and the processing ends.

S4, S5) On the other hand, if there is the next character K, the process proceeds to the dictionary search step.

First, the processor sets the search index i to the character string ω
And the registration index j is set to 0 and the position detection index lk is set to 0.

Next, the concatenated index first [i] of the index i is obtained with reference to the array first100, and set to the index i.

The index, i, is checked to see if it is "0". If not, the character string ext of the index i in the character string table 10b
It is checked whether [i] is the input character K.

If it is not the input character K, the registration index j is saved in the position detection index lk, and the search index i is saved in the registration index j.

Further, the array next101 is referred to by the index i, the concatenated index next [i] is obtained, set to the index i, and the process returns to the determination of whether the index i = 0.

S8) On the other hand, if the character string ext [i] is the input character “K”, it is determined whether the position detection index lk is “0”.

If lk is not "0", it means that the input character K has been found at the third and subsequent concatenated indices, and the previous subsequence is replaced.

That is, the index i is set to the index [lk] next [lk].
And the next [i] of the index i of the array next101 is moved to the next [j] of the index j, and the index j is
Move to next [i] and return to step.

On the other hand, if lk = 0, it means that the input character K has been found at the second or earlier connected index.

Therefore, it is determined whether the registration index j is “0”.

When j = 0, it means that the input character K was found at the first [i], that is, the first connected index of the connection, and there is no need to replace the input character K, so the process returns to step S2.

On the other hand, if j ≠ 0, the input character K is found at the second and subsequent concatenated indices, and the input character K is replaced with the leading character.

That is, the next [i] of the index i of the array next101 is moved to the next [j] of the previous index j, and the array firs
The first [ω] of the index ω of t100 is moved to next [i], and the index i is moved to first [ω].

As a result, the index i of the input character K is rewritten to the head.

 Then, the process returns to step S2.

S6) In step S4, if i = 0, it is not in the dictionary 10, so the registration step is started.

 First, code (ω) is output as a code word.

Next, the number of character strings n is set in the index i, the number of character strings n is incremented to n + 1, and the character K is stored in the index i of the character string table 10b as next [i].

 Next, it is checked whether the registration index j is “0”.

If j = 0, since there is only one character string at that stage, it is set to the first [ω] of the index i array first.

On the other hand, if j ≠ 0, the index i is set in the array next [ω] because there are two or more character strings at that stage.

Then, the character K is set in the index i, and the process returns to the step.

 A specific example will be described with reference to FIGS.

As shown in FIG. 3A, the same example as in FIG. 11 will be used to search for the character string "azw".

In the case of FIG. 3A, the character string table 10b, the array first
100 and the array next101 are as shown in FIG. 3 (B).

When the input character K = Z is input and the array first and the array next are traced, next [i = P2] = P3 is obtained as the index i.

At this time, since ext [P3] = Z, j = P2, lk = P1
It is.

Therefore, in step S8, lk ≠ 0, and FIG.
And replace "Z" with the previous "h".

That is, as shown in FIG. 4, i = P3 is changed to next [lk = P1].
, Next [i = P3] = 0 to next [j = P2], and j = P2
Set next [i = P3].

As a result, the connected state becomes as shown in FIG.
“Z” and “h” in FIG. 3A are interchanged, and the table state is as shown in FIG. 4B.

Next, returning to step S2, when the next character "w" is input, the array first100 and the array next101 are traced as shown in FIG. 5B, and next [P9] = P10 is obtained as the index i.

At this time, ext [P10] = w, j = P9, and lk = P8.

Therefore, in step S8, lk ≠ 0, and FIG.
"W" is replaced with "g" immediately before.

That is, i = P10 is replaced with next [lk = P8], and next [i = P10].
= 0 is set to next [j = P9], and j = P9 is set to next [i = P10].

As a result, as shown in FIG. 6A, "g" and "w" in FIG. 5A are interchanged, and the table state becomes as shown in FIG. 6B.

As described above, every time a search character is found, the contents of the hash table are rewritten so as to be replaced by the contents of the character string having the same hash address, so that the same character string is searched next time. In this case, the search is performed on a shorter route than the previous one, and the search process can be sped up. In this method, the position in the linked list only goes up one by one even if it is searched once, so if a character string that is used less frequently comes, it will not go up easily, and if a character string that is used frequently comes, Each time the search is performed once, the position is surely increased by one, so that a linked list structure corresponding to the search frequency can be constructed.

If the code and character registration processing, hash table array first and array next registration processing, linked list rewriting and search continuation processing for the next character are performed in the respective pipelines, higher-speed encoding processing can be performed. .

(B) Description of Other Embodiments In addition to the above-described embodiments, the present invention can be modified as follows.

The format of the hash table is not limited to the format of the array first and array next, and may be another format.

The code (ω) may be further compressed using run-length coding or the like.

 Not limited to a character string, it may be an encoded data string.

Although the present invention has been described with reference to the embodiments, the present invention can be variously modified in accordance with the gist of the present invention, and these are not excluded from the present invention.

〔The invention's effect〕

As described above, the present invention has the following effects.

In LZW encoding, when performing a dictionary search, the search is performed while rewriting the external hash table one by one so that the searched character string is always searched on the shortest path, so the search path for frequently used character strings becomes shorter. High-speed search processing can be performed, and encoding can be speeded up.

Since it only replaces the previous one, it does not change the search tree so rapidly, so that the efficiency of searching for a random code string is relatively improved.

[Brief description of the drawings]

 FIG. 1 is a principle diagram of the present invention, FIG. 2 is a processing flowchart of an embodiment of the present invention, FIGS. 3 to 6 are explanatory diagrams of an operation of the embodiment of the present invention, and FIG. FIG. 8 is an LZW decoding process flow diagram, FIG. 9 is an explanatory diagram of LZW encoding and decoding, FIG. 10 is an explanatory diagram of an external hashing method, and FIGS. FIG. In the figure, 10: dictionary, 10a: search list.

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Hirotaka Chiba 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Inside Fujitsu Limited (56) References JP-A-60-116228 (JP, A) JP-A-63-209228 (JP, A) (58) Field surveyed (Int. Cl. ⁷ , DB name) H03M 7/42

Claims (2)

(57) [Claims] An encoded data is divided into different sub-sequences, the sub-sequences are registered in a dictionary (10), and input in accordance with a search list (10a) indicating a search order of a connected sub-sequence. A data compression method for comparing and searching data and subsequences in the dictionary (10), and designating and encoding input data by a reference number of a subsequence in the dictionary (10) that matches the maximum length, A data compression method characterized by rewriting the search list (10a) so that the searched subsequence is replaced with the previous subsequence in the search list (10a). 2. The search list (10a) indicates a search order from the smallest number of characters in the partial string to the largest number of characters in the partial string, and the searched partial string is represented by the same number of characters in the search list (10a). The data compression method according to claim 1, wherein the search list (10a) is rewritten so as to be immediately preceding in the subsequence.