JP2957801B2 - Data compression method and its restoration method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression system using a universal code, particularly an LZW (Lempel-Ziv-Welch) code, and a decompression system thereof.

In recent years, various types of data such as character codes, vector information, and image information have been handled by computers, and the amount of data handled has rapidly increased. When dealing with a large amount of data, by compressing the amount of data by omitting redundant portions in the data, the storage capacity can be reduced or transmission can be performed faster.

[0003] Data compression / compression of various data by one method
As a method that can be restored, there is universal coding. The universal coding is a technique in which, when the statistical properties of an information source are unknown, the statistical properties are checked and adapted while performing the coding. Note that the universal code is not limited to a character code, and can be applied to various data. However, in the following description, a word unit of data is called a “character” regardless of what the data is, An arbitrary number of these characters is called a “character string”.

As a typical example of the universal code,
There is a Ziv-Lempel (Jib-Lempel) code.
For the Ziv-Lempel code, two algorithms of a universal type and an incremental decomposition type have been proposed. Further, as an improvement of the universal algorithm, LZ
SS codes have been proposed, and as an improvement of the incremental decomposition type algorithm, LZW (Lempel-Ziv-Welc) has been proposed.
h) Codes have been proposed. In a field such as a file compression process of a storage device, an LZW code is often used because of a high speed process and a simple algorithm. The present invention relates to data compression and decompression using the LZW code.

[0005]

2. Description of the Related Art FIG. 5 shows a configuration of a conventional data compression / decompression device. The compression unit in FIG. 5A includes an LZW encoding unit 3, a dictionary unit 4, and a variable length encoding unit 5 each including a dictionary search unit 1 and a dictionary registration unit 2, and the LZW encoding unit 3 Is divided into different character strings, the character strings are numbered in the order in which they appear, and registered in the dictionary in the dictionary unit 4, and the currently input character string is compared with the character string already registered in the dictionary. It is represented by the number of the longest matching character string and output in the order of input, which is variable-length coded by the variable-length coding unit 5 and output.

The restoring unit shown in FIG. 5B includes a variable length coding unit 6, an LZW decoding unit 9 including a dictionary search unit 7 and a dictionary registration unit 8, and a dictionary unit 10. After the converted coded data is restored by the variable length decoding unit 6,
The ZW decoding unit 9 restores the original input data by using the dictionary in the dictionary unit 10 and repeating reference and registration.

FIG. 6 is a flowchart of a conventional LZW encoding process, and FIG. 7 is a flowchart of a conventional LZW decoding process.

In the encoding process of FIG. 6, first, in step S1, a character string consisting of one character is registered as an initial value for all characters to be used in advance, a reference number is given to each character, and the first character input is entered. Set K as prefix string ω.

Next, in step S2, the next character K to be input is read, and it is checked in step S3 whether or not the character input has been completed. Then, the process proceeds to step S4, where the beginning of the word obtained in step S1 is obtained. A search is performed to determine whether or not the dictionary has a character string (ωK) obtained by adding the character K read in step S2 to the character string ω.

If the character string (ωK) is not found in the dictionary in step S4, the process proceeds to step S6, where the reference number ω of the obtained character K is output as a code word code (ω), and the character string (ωK) is newly added. With a unique reference number and register it in the dictionary,
Further, while replacing the input character K in step S2 with the reference number ω, the dictionary address n is incremented,
After checking the dictionary registration space in steps S7 to S11, the process returns to step S2 to read the next character K again.

On the other hand, if the character string (ωK) is found in the dictionary in step S4, the character string (ωK) is replaced with the reference number ω in step S5, the dictionary registration space is checked in steps S7 to S11, and Returning to step S2,
The search for the character string having the maximum matching length is continued until the character string (ωK) cannot be searched from the dictionary.

A specific example of the LZW encoding process will be described with reference to FIG. In addition, in order to simplify explanation, "a"
The case of input data composed of a combination of three characters “b” and “c” is taken as an example. These three characters “a”, “b”, and “c” are registered in a dictionary in advance as shown in FIG. 9A, and reference numbers = 1, 2, 3
Is given.

It is assumed that input data (INPUT SYMBOLS) in FIG. 8 is read from left to right. When the first character “a” is input, this character “a” is set as the prefix character string ω. Next, when the second character K = “b” is input, the character string (ωK) =
"Ab" is set, and a search is performed to determine whether or not the character string "ab" exists in the dictionary.

Since there is no character string "ab" in the dictionary, FIG.
As shown in (A), this character string “ab” is registered in the dictionary and a new reference number = 4 is assigned. Then, the reference number = 1 is read out for the prefix character string ω = “a” in the character string (ωK), and as shown in FIG.
This reference number = 1 is replaced with the code word of the character string ω = “a” (OUTPUT
CODES). When the encoding of the character “a” ends, the remaining character “b” in the character string (ωK) = “ab” becomes the next prefix character string ω.

Next, when the third character K = “a” is input, the character string (ωK) = “ba” is combined with the new prefix character string ω = “b”, and this character string “ "ba" is found in the dictionary. Since there is no character string "ba" in the dictionary, as shown in FIG.
Is registered in the dictionary, and a new reference number = 5 is given. Then, the prefix character string ω in the character string (ωK) =
The reference number = 2 is read out for “b”, and this reference number = 2 is output as a code word (OUTPUT CODES) of the character string ω = “b”, as shown in FIG. When the encoding of the character “b” is completed, the remaining character “a” in the character string (ωK) = “ba” becomes the next prefix character string ω.

Next, when the fourth character K = “b” is input, the character string (ωK) = “ab” in combination with the new prefix character string ω = “b”, and this character string “ Search for "ab" in the dictionary. The character string "ab" is already registered in the dictionary as reference number = 4. Therefore, in this case, the character string “ab” is replaced with a new prefix character string ω
Then, the next fifth character K = “c” is input, and the fifth character K = “c” is combined with the new prefix character string ω = “ab” to form a character string (ωK). = “Abc”
And whether or not this character string “abc” exists in the dictionary is searched.

Since there is no character string "abc" in the dictionary, as shown in FIG. 9A, this character string "abc" is registered in the dictionary and a new reference number = 6 is given. Then, the prefix character string ω = “ab” in the character string (ωK)
, The reference number = 4 is read out, and as shown in FIG. 8, this reference number = 4 is output as the code word (OUTPUT CODES) of the character string ω = “ab”. When the encoding of the character string “ab” ends, the remaining character “a” in the character string (ωK) = “abc” becomes the next prefix character string ω.

By repeating the above processing, the input data "ababcbababaaaaaaa" is converted into encoded data "1,2,4,3,5,8,1,10,11" and output. At this time, a character string as shown in FIG. 9A and its reference number are registered in the dictionary. In the actual dictionary, in order to reduce the memory capacity, as shown in FIG. 9 (B), the registered character string is replaced with another registered character string already registered on the upper side except for the lower one digit. The dictionary is replaced with the reference number.

The decoding process of FIG. 7 performs the reverse operation of the encoding process of FIG. That is, in the decoding process of FIG.
As in the case of encoding, a character string consisting of one character is registered in advance as an initial value for all the characters to be used, and decoding is started after a reference number is assigned to each character string.

In step S11, the first input code (= reference number) is read and is set as OLDcode. Since the first code word corresponds to one of the reference numbers of one character already registered in the dictionary, a character code (K) that matches the input code is searched for, and the character “K” is output. Note that the output character "K" is FINchar for later processing.
Set to.

Next, the process proceeds to step S12, where the next input code is read and set as INcode. Step S
In step S13, it is checked whether or not there is a new code, that is, whether or not the input of the code has been completed. The process proceeds to step S14 to determine whether or not the code is a dictionary clear code. Of the input codeword
Check whether code is defined (registered) in the dictionary.

Normally, since the input code word has already been registered in the dictionary in the previous decoding processing, step S1 is executed.
7 and the character string code (ωK) corresponding to the input codeword
Is read from the dictionary, the character string K is temporarily stacked in step S18, the reference number code (ω) is set as a new character string, and the process returns to step S17 again.
Steps S17 and S18 are recursively repeated until the reference number ω becomes one character. Finally, the process proceeds to step S19.

Then, in step S19, the characters stuck in step S18 are transferred to the LIFO (Last In Fast).
Out) pop-up and output in step S20
After checking the dictionary registration area in step S21,
, A character string (ω) combining the code ω (OLDcode) used last time and the first character K of the character string restored this time
K) is assigned a new reference number and registered in the dictionary. Then, the process returns to step S12, and the above process is repeated until there is no more input code.

A specific example of the LZW decoding process will be described with reference to FIG. First, although the first input code is “1”, the characters “a”, “b”, and “c” are already registered in the dictionary as shown in FIG. , Reads and outputs the character string “a” having the reference number that matches the first input code “1”.

For the next input code "2", a character string "b" having a reference number corresponding to the code "2" is similarly read out and output. At this time, a new reference number = 4 is added to the character string “1b” obtained by combining the code “1” decoded last time and the first character “b” of the character string decoded this time, and registered in the dictionary. Hereinafter, similar processing is repeated to execute decoding of each input code.

The following exception processing is performed in the decoding of FIG. This exception processing is performed by the sixth input code “8”.
Occurs in That is, the code “8” is not yet defined in the dictionary and cannot be decoded immediately. In such a case, the character string “5” obtained by adding the first character “b” of the previously decoded character string “ba” to the previously processed code “5”
b ”is obtained, and the code“ 5 ”in the character string is searched from the dictionary and replaced with the character string“ 2a ”to obtain“ 2ab ”. Further, the code“ 2 ”in“ 2ab ”is searched from the dictionary and the character“ 2ab ”is searched. By substituting with “b”, the character string “bab” is finally decoded. Since the decoded character string “5b” (= “bab”) is not defined in the dictionary, it is registered with a new reference number = 8 as shown in FIG. 9B.

This exception processing is performed through the processing of steps S4 and S16 of the LZW decoding processing shown in FIG. 7. In step S19, the character string "bab" is output and registered in the dictionary. Therefore, even during the decoding process, decoding is performed while creating the same dictionary as in FIG. 9B. Thus, finally, the input code “1,2,4,
3,5,8,1,10,11 "is the original character string" ababcbabab
aaaaaa ".

Next, initialization (clearing) of the dictionary in the encoding process of FIG. 6 and the decoding process of FIG. 7 will be described.
In the case of the LZW encoding process in FIG. 6, when the character string is registered in the dictionary in step S6, it is determined in step S7 whether the current dictionary registration address n has exceeded the maximum address NMAX of the dictionary, that is, the dictionary is full. Check if it is. If it is determined in step S7 that the registration in the dictionary is full, the process proceeds to step S8, where the registration in the dictionary is stopped, and the compression ratio is checked in units of several hundred bytes.

If it is determined in step S9 that the compression ratio tends to be worse than that of the previous check,
Judge that the dictionary is out of alignment with the statistical properties of the data,
After outputting the dictionary clear code in step S10, the process proceeds to step S11, and the other character strings are initialized (cleared) except for one character "a", "b", and "c", and again,
Returning to step S2, encoding is executed while registering in the dictionary.

In the case of the decoding process of FIG. 7, step S20
Then, it is determined whether or not there is a dictionary space that can be registered. If registration is possible, registration to the dictionary is performed in step S21.
If a dictionary clear code is output on the encoding side, the determination is made in step S14, and the process is continued again from step S11.

[0031]

As described above, in the conventional LZW code, while a dictionary is created in an area secured on a storage device, encoding and decoding are performed using the dictionary. Therefore, these processes are established under the precondition that data on the storage device is not destroyed by external factors or the like.

However, when actually configured as an apparatus,
Failures such as destruction of the contents of the storage device and garbled data also occur. In this case, in the conventional LZW code,
Even if erroneous data is read and used for processing, it has not been possible to determine whether it is correct data or erroneous data.

The present invention has been made in view of the above circumstances, and has as its object to provide a data compression method capable of accurately determining whether restored data is valid data without errors. And its restoration method.

[0034]

According to a first data compression method of the present invention, encoded data is divided into different partial columns, and different reference numbers are added to the respective partial columns and registered in a dictionary. In a data compression method in which input data is specified and encoded by a reference number of a subsequence in the dictionary that matches the maximum length, a reference number for checking is set in the reference number, and the encoded input It is characterized in that the check reference number is inserted for each predetermined number of data reference numbers.

According to a second data compression method of the present invention, a plurality of reference numbers for checking are set as reference numbers, and a plurality of reference numbers for checking are set for each predetermined number of reference numbers of the encoded input data. The reference numbers are inserted in a predetermined order.

A first data decompression method according to the present invention is a decompression method for data compressed by the first data compression method, wherein encoded data is divided into different sub-sequences, and Register the input data in different dictionaries and check the input data against the reference number in the data restoration method that restores the original character string from the code word specified by the reference number of the subsequence of the substring that matches the maximum length By setting a reference number for the input data and detecting whether or not the reference number for the check is inserted for each predetermined number of the reference numbers of the input encoded data, It is characterized in that valid / invalid is determined.

The second data decompression method of the present invention is a decompression method for data compressed by the second data compression method, wherein a plurality of reference numbers for checking are set as reference numbers, and By detecting whether a plurality of reference numbers for checking are inserted in a predetermined order for each predetermined number of reference numbers of coded data to be received, the validity / invalidity of the restored data is determined. It is characterized by determining.

[0038]

According to the present invention, at the time of data compression, one or more reference numbers for checking are inserted for each specified number of reference numbers of encoded input data, and are inputted at the time of decompression. The validity / invalidity of the restored data is determined by detecting whether or not the reference number for checking is inserted for each predetermined number of reference numbers of the encoded data. Therefore, even when a failure such as the content of the storage device is destroyed or the data is garbled, it can be accurately determined whether or not the restored data is valid.

[0039]

FIG. 1 shows an embodiment of a data compression / decompression device constructed by applying the method of the present invention, and FIGS. 2 and 3 show LZW encoding processing and decoding of the present invention for this embodiment. Each of the flowcharts of the processing is shown. Since the LZW encoding process and the decoding process are the same as those of the conventional example described above, the description of the LZW encoding process and the decoding process itself is omitted below, and the processing added in the present invention will be described. Only the explanation will be focused. The processing flow part added in the present invention is a step S1 surrounded by a dotted line in FIG.
3 and S14, and steps S22 and S23 and S24 surrounded by a dotted line in FIG. The processing of other parts other than these is the same as that of the conventional example of FIGS.

First, the configuration and operation of the compression unit shown in FIG. In FIG. 1A, 1 is a dictionary search unit, 2 dictionary registration units, 3 is an LZW encoding unit, 4 is a dictionary unit, 5 is a variable length encoding unit, 11 is a reference number counting unit, and 12 is a check code transmission. Department. The same parts as those in the conventional example (FIG. 5A) are denoted by the same reference numerals.

The input data to be encoded is input to the LZW encoding unit 3. The LZW encoding unit 3 uses the dictionary search unit 1 and the dictionary registration unit 2 to repeatedly refer to and register a character string in a dictionary in the dictionary unit 2 and perform LZW encoding in the same manner as in the above-described conventional example. Then, the obtained reference number (dictionary registration number) is output to the reference number counting unit 11. When the dictionary space required for encoding is exhausted during the encoding process, the dictionary of the dictionary unit 4 is initialized (cleared) by the dictionary clear code each time, and encoding is continued.

The reference number counting section 11 is provided with the LZW encoding section 3
Is counted (step S13 in FIG. 2), and the counted value is notified to the check code sending unit 12. The check code sending unit 12 sends out a set reference number for checking (hereinafter referred to as “check code”) for each number of sending out predetermined reference numbers (step S14 in FIG. 2). Then, the variable length coding unit 5 performs variable length processing on the reference number including the check code and outputs the reference number.

The number of check codes is not limited to one, and a plurality of check codes may be used as one set. When a plurality of check codes are used, the combination order can also be checked, so that the reliability can be further increased.

FIG. 4 shows a specific example of data encoded according to the present invention. In this example, the present invention is applied to the encoded data illustrated in FIG. 8, and a check code is inserted for every four reference numbers of the encoded character string. FIG. 4A shows a case where one reference number “0” is used as a check code, and FIG. 4B shows a case where three reference numbers “0”, “−1” and “−2” are used as a check code. Are respectively shown.

Next, the configuration and operation of the restoration unit shown in FIG. 1B will be described. In FIG. 1B, reference numeral 6 denotes a variable length decoding unit;
7 is a dictionary search unit, 8 is a dictionary registration unit, 9 is an LZW decoding unit, 10 is a dictionary unit, 14 is a check code determination unit, 15
Is a reference number counting unit. Conventional example (FIG. 5B)
The same parts as those described above are denoted by the same reference numerals.

The coded data is input to the variable length decoding section 6, subjected to variable length decoding and returned to the reference number of the original data length, and then sent to the check code determination section 14. In the encoded data, a reference number giving a character string other than the check code is sent from the check code determination unit 14 to the encoding unit 9 and described above using the dictionary search unit 7, the dictionary registration unit 8, and the dictionary unit 10. The original input data is decoded as in the conventional example.

On the other hand, the reference number counting section 15 counts the number of reference numbers of the encoded data input to the check code determination section 14 (step S22 in FIG. 3). Then, the check code determination unit 14 detects a check code in the input code (step S23 in FIG. 3). If the check code is a check code, whether the check code is inserted for each prescribed input code is determined. Is determined (step S24 in FIG. 3).

If a check code is inserted for each prescribed input code, the process ends normally as valid decoding without error. On the other hand, when the check code is not inserted for each prescribed input code, the check code determination unit 1
No. 4 judges that an error has occurred and determines the error signal (ERRO).
R) is output, and the operation ends abnormally.

For example, taking the case of FIG. 4 as an example, whether a check code "0" is inserted for every four reference numbers of the encoded data to be inputted (in the case of FIG. 4A) Alternatively, whether three check codes “0”, “−1”, and “−2” are inserted in this order for every four reference numbers of the input encoded data (in the case of FIG. 4B). Examine
If a predetermined check code is inserted at a predetermined position, it is determined that the data is valid without error.

[0050]

As is apparent from the above description,
According to the present invention, at the time of data compression, a reference number for checking is inserted for each specified number of encoded reference numbers, and at the time of decompression, a reference for checking is inserted for each predetermined number of input reference numbers. By judging whether or not the number is inserted, the validity / invalidity of the restored data is determined. Therefore, even if a failure such as the content of the storage device is destroyed or the data is garbled occurs. ,
Whether the restored data is valid or not can be accurately determined, and highly reliable data compression and restoration can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing a flowchart of an LZW encoding process of the present invention.

FIG. 3 is a diagram showing a flowchart of an LZW decoding process of the present invention.

FIG. 4 is a diagram showing a specific example of encoded data according to the present invention.

FIG. 5 is a block diagram showing a configuration of a conventional data compression / decompression device.

FIG. 6 is a flowchart of a conventional LZW encoding process.

FIG. 7 is a flowchart of a conventional LZW decoding process.

FIG. 8 is an explanatory diagram of a specific example of LZW encoding.

FIG. 9 is an explanatory diagram of a specific example of a dictionary configuration.

FIG. 10 is an explanatory diagram of a specific example of LZW decoding.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Dictionary search part 2 Dictionary registration part 3 LZW encoding part 4 Dictionary part 5 Variable length encoding part 6 Variable length decoding part 7 Dictionary search part 8 Dictionary registration part 9 LZW decoding part 11 Reference number counting part 12 Check code transmission Unit 14 Check code judgment unit 15 Reference number counting unit

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Shigeru Yoshida 1015 Kamikodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Inside Fujitsu Limited (56) References JP-A-61-270955 (JP, A) JP-A-49-29929 (JP, A) JP-A-61-147690 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06F 5/00 H03M 7/40

Claims (4)

(57) [Claims] An encoded data is divided into different sub-sequences, a different reference number is added to each sub-sequence and registered in a dictionary, and input data is stored in a maximum length of the sub-sequences in the dictionary. In a data compression method of designating and encoding by matching reference numbers, a reference number for a check is set as a reference number, and the check number is set for each predetermined number of reference numbers of the encoded input data. A data compression method characterized by inserting a reference number. 2. The encoded data is divided into different sub-sequences, and a different reference number is added to each sub-sequence and registered in a dictionary. In a data compression method of coding by specifying a reference number of a match, a plurality of reference numbers for checking are set as reference numbers, and the check is performed for each predetermined number of reference numbers of the encoded input data. A plurality of reference numbers are inserted in a predetermined order. 3. A method for restoring data compressed by the data compression method according to claim 1, wherein the encoded data is divided into different partial strings and registered in different dictionaries for each partial string. Input data is extracted from the subsequences in the dictionary.
In a data restoration method for restoring an original character string from a code word specified by a reference number of a maximum matching length, a reference number for checking is set to the reference number, and a reference number of the input encoded data is By detecting whether or not the check reference number is inserted for each predetermined number, the validity / invalidity of the restored data is determined.
A data restoration method characterized by determining invalidity. 4. A method for restoring data compressed by the data compression method according to claim 2, wherein the encoded data is divided into different partial sequences and registered in different dictionaries for each partial sequence. Input data is extracted from the subsequences in the dictionary.
In a data restoration method for restoring an original character string from a code word specified by a reference number of a maximum length match, a plurality of reference numbers for checking are set as reference numbers, and the input encoded data is referred to. The method is characterized in that the validity / invalidity of the restored data is determined by detecting whether or not the plurality of check reference numbers are inserted in a predetermined order for each predetermined number of numbers. Data restoration method.