JP2002135128A - Data-compression method, data compression/expansion method, data-compression device, and data compression/ expansion device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a dictionary with capable of conducting processing in single path processing ability, and moreover can obtain sufficient compressibility. SOLUTION: A compression/expansion section 12, which comprises a slide dictionary part 121, a base pointer part 122, an interval calculation part 123, and a gamma-encoding part 124, stores read data as the contents of a dictionary, reads the contents of the dictionary used for encoding from a location of a base pointer, finds longest agreement character string out of the contents, encodes an interval from the base pointer to the first agreement character, and moves the base pointer to a location of the agreement character prior to next character encoding, to perform data compression by a slide dictionary method.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression method for compressing data such as text data, a data compression / decompression method, a data compression device, and a data compression / decompression device.

[0002]

2. Description of the Related Art In recent years, with the development of information processing technology, a huge amount of various types of data such as character codes, vector information, and images have been processed by computers. Conventionally, data compression has been known as a technique for reducing the capacity of the secondary storage device and shortening the communication time in data communication. This is realized by removing redundant information included in the data, converting the data string to another short data string, and the like.

[0003] Data compression techniques are classified into two types: an irreversible compression method in which compressed data cannot be completely restored to original data, and a lossless compression method in which compressed data can be completely restored to original data. Since the irreversible compression method cannot completely restore data, it is mainly used for compression of data such as audio and images. For a secondary storage device or the like that requires complete storage of data, a reversible compression method that can completely restore the original data is used.

[0004] As a method of lossless compression, a statistical coding method such as a Huffman code or an arithmetic code that generates individual codes from the appearance of characters in data and a repetitive appearance of a character string in data are used. Slide dictionary method (LZ77
Code) and a dynamic dictionary method (LZ78 code).

Here, the slide dictionary method (LZ77 code)
Is a data compression algorithm released in 1977 by J.Ziv and A.Lempel, which registers already read data as a dictionary, finds the longest matching character string from it, and matches it with the matching position. The original code is replaced with the length that has been set and output. In the slide dictionary method (LZ77 code), a local dictionary having a limited adaptation range is provided, and a character string is extracted from the dictionary using a window called a sliding window that moves in the character string.

That is, in the slide dictionary method (LZ77 code), a character string including a character currently being encoded is called a slide window, and the character string in the slide window is to be referred to. The codeword is: m: the position (offset) of the longest matching character string from the top of the sliding window l: the length of the longest matching character string in the sliding window x: the first character that did not match (the longest matching character string) Is represented by a binary number consisting of m, l, x. If the character string does not exist in the sliding window, m = 0 and l = 0. Also, every time the sliding window outputs a codeword, l + 1
Shift to the next character string.

As described above, in the slide dictionary method (LZ77 code), the longest matching character string is found from the past text, and its start position and length are used as code words.

[0008] This type of technology is disclosed in, for example, Japanese Patent Application Laid-Open No. 60-1162.
No. 28 gazette.

There are several variations in the slide dictionary method (LZ77 code) depending on how to express the start position and length and how to search for the longest matching character string.

[0010]

By the way, data compression employing Huffman code, arithmetic code, and the like is performed in two passes and is therefore heavy, impractical in hardware, and must be performed in software. .

Also, data compression employing a dynamic dictionary method (LZ78 code) of a dictionary type encoding method utilizing the repetitive appearance of character strings in data can be processed in one pass. , Requires a large memory for the dictionary.

Data compression using the slide dictionary method (LZ77 code) of the dictionary type encoding method can be performed in one pass, and can be executed with a relatively small dictionary. There is a need for further miniaturization of the dictionary and improvement of the compression ratio.

Accordingly, an object of the present invention is to provide a data compression method, a data compression method and a data compression method capable of performing processing in one pass and obtaining a sufficient compression ratio with a small dictionary in view of the above-mentioned conventional situation. An object of the present invention is to provide a decompression method, a data compression device, and a data compression / decompression device.

[0014]

According to the present invention, the read data is registered as a dictionary, the longest matching character string is searched from the dictionary, and the original code is replaced with the matching position and length. This is a data compression method based on a slide dictionary method for outputting, in which a reference to a dictionary in encoding is performed from the position of a base pointer, an interval from the base pointer to the first matching character is encoded, and the next character is encoded. Previously, the base pointer is moved to a position where characters match.

The present invention also provides a slide dictionary which registers read data as a dictionary, finds the longest matching character string from the dictionary, replaces the original code with a length matching the matching position, and outputs the result. A data compression / expansion method based on the method. At the time of compression, a dictionary in encoding is referenced from the position of a base pointer, an interval from the base pointer to the first matching character is encoded, and the next character is encoded. Before performing the encoding, the encoding is performed by repeating the process of moving the base pointer to the position where the character matches, and at the time of decompression, for the encoded character, reference to the dictionary in decoding is performed from the position of the base pointer in decoding. From the first character to the first matched character, and before decoding the next character, move the base pointer to one character. Characterized by decoding by repeating the process of moving to the position.

The present invention also provides a slide dictionary that registers read data as a dictionary, finds the longest matching character string from the dictionary, replaces the original code with a length matching the matching position, and outputs the slide dictionary. A data compression apparatus for performing data compression processing by a method, comprising: a slide dictionary section in which read data is registered as a dictionary; and a base for designating a position at which a previous character matched as a reference position of the slide dictionary section in encoding. It is characterized by comprising a pointer, an interval calculating section for calculating an interval from the base pointer to the first matching character, and an encoding section for encoding the interval calculated by the interval calculating section.

Further, according to the present invention, a slide dictionary in which read data is registered as a dictionary, a longest matching character string is searched from the dictionary, and an original code is replaced by a length matching a matching position and output. A data compression / decompression device based on the method, wherein a reference to a dictionary in encoding is performed from a position of a base pointer, an interval from the base pointer to a first matched character is encoded, and before encoding a next character, Compression processing means for performing encoding by repeating the process of moving the base pointer to the position where the character matches, and referring to the dictionary in decoding from the position of the base pointer for decoding the encoded character, Decode the interval up to the character that matches the character, and before decoding the next character, move the base pointer to the position where the character matches. Characterized in that it comprises a decompression processing means for decoding by repeating the process of moving to.

[0018]

Embodiments of the present invention will be described below in detail with reference to the drawings.

The present invention is applied to, for example, a tape streamer drive 10 having a configuration as shown in FIG.

The tape streamer drive 10 has a SCSI (Smoll Compu- ter) as an interface for exchanging data with an external host computer 20.
terSystem Interface). A compression / expansion processing unit 12 is connected to the SCSI interface 11. An IF / ECC controller 13 is connected to the compression / decompression processing unit 12. The buffer memory 14, the recording RF processing unit 15, and the reproduction RF processing unit 16 are connected to the IF / ECC controller 13. The recording heads 18A and 18B and the reproducing heads 19A and 19B are connected to the recording RF processing unit 15 and the reproducing RF processing unit 16 via a rotary transformer 17.

The tape streamer drive 10
Performs recording / reproducing on the magnetic tape 1 by a helical scan method. The recording heads 18A, 1
8B and reproduction heads 19A and 19B are provided on a rotating drum (not shown). The recording heads 18A and 18B have a structure in which two gaps having different azimuth angles are arranged very close to each other. Reproducing head 19A, 1
9B are also heads having different azimuth angles, but are arranged, for example, 90 degrees apart.

The tape streamer drive 10 transmits the data from the host computer 20 at the time of data recording by using a transmission data unit called a fixed-length record.
Data is sequentially input via the CSI interface 11 and supplied to the compression / decompression unit 12. In such a tape streamer drive system, the host computer 20 is controlled by a set unit of variable-length data.
There are also modes in which more data is transmitted.

The compression / expansion unit 12 performs a compression / expansion process on the input data according to an instruction from a system controller (not shown) by a data compression / expansion method according to the present invention as needed.

The data subjected to the compression processing by the compression / expansion unit 12 is supplied to an IF / ECC controller 13. The IF / ECC controller 13 temporarily stores the data supplied from the compression / expansion unit 12 in the buffer memory 14, and finally stores the data as a fixed length unit corresponding to 40 tracks of a magnetic tape called a group (Group). And performs ECC format processing on this data.

E in the IF / ECC controller 13
In the CC format processing, an error correction code is added to recording data, and the data is subjected to modulation processing so as to be compatible with magnetic recording. And this ECC
The recording data subjected to the format processing is supplied to the recording RF processing unit 15. The recording RF processing unit 15 performs processing such as amplification and recording equalization on the supplied recording data to generate a recording signal. The recording signal generated by the recording RF processing unit 15 is supplied to the recording heads 18A and 18B via the rotary transformer 17, and is recorded on the magnetic tape 1.

The data reproducing operation will be briefly described. The data recorded on the magnetic tape 1 is read from the reproducing head 19.
A and 19B read out as an RF reproduction signal, and this RF reproduction signal is supplied to the reproduction RF processing unit 16 via the rotary transformer 17. This RF processing unit for reproduction 1
In 6, the supplied RF reproduction signal is subjected to processing such as reproduction equalization, reproduction clock generation, binarization, decoding (for example, Viterbi decoding). The data read in this way is supplied to the IF / ECC controller 13, where it is temporarily stored in the buffer memory 14 and subjected to error correction processing and the like. The data subjected to the error correction processing and the like by the IF / ECC controller 13 is read from the buffer memory 14 at a predetermined time and supplied to the compression / decompression unit 12.

The compression / decompression unit 12 performs decompression processing if the data has been compressed by the compression / decompression unit 12 at the time of the recording, and decompresses the data if it is uncompressed data, based on the judgment by the system controller (not shown). Outputs the data as it is without performing decompression processing.

The output data of the compression / expansion unit 12 is SCSI
The data is output to the host computer 20 as reproduction data via the interface 11.

Here, the tape streamer drive 1
As shown in FIG. 2, the compression / decompression unit 12 at 0 includes a slide dictionary unit 121, a base pointer unit 122, an interval calculation unit 123, and a gamma encoding unit 124. The compression process is performed according to the procedure shown in the flowchart of FIG. 4, and the decompression process is performed according to the procedure shown in the flowchart of FIG.

That is, in the compression processing by the compression / decompression unit 12, as shown in the flowchart of FIG. 3, when the compression processing is started, first, in step S1, the slide dictionary is initialized by padding with zeros, and the write pointer and the base are initialized. Set the pointer and run length to 0.

In the next step S2, one character is input. In step S3, a dictionary search is performed for "the previous character" + "the current character", and in the next step S4, it is determined whether or not they match.

If the decision result in the step S4 is YES, that is, if "the previous character" + "the current character" coincides with the character string on the slide dictionary, the process shifts to a step S7, where interval = match address-base pointer. Calculate the interval.

If the result of the determination in step S4 is NO
That is, if "previous character" + "current character" does not match the character string on the slide dictionary, the process proceeds to step S5, where a dictionary search is performed for "current character", and whether or not it matches in the next step S6. Determine whether or not.

The result of the determination in step S6 is Y
If the ES, that is, the "current character" matches the character string on the slide dictionary, the process proceeds to step S7, and the interval is calculated by the interval = match address-base pointer.

In the next step S8, it is determined whether or not the interval calculated in step S7 is one.

If the decision result in the step S8 is YES, that is, if the interval is 1, the process proceeds to a step S9, where the run length = run length + 1, that is, the run length is incremented, and the process proceeds to a step S12, where the base counter = match address That is, after moving the base pointer to the position of the match address,
Move to 16.

If the result of the determination in step S6 is NO
That is, if the interval is not 1, step S
Then, the process proceeds to step S1 where the flash operation is performed.
At step 1, the interval is coded according to the following Table 1, and is output. Further, at step S12, the base counter = match address, that is, the base pointer is moved to the position of the match address, and then the process proceeds to step S16.

[0038]

[Table 1]

The result of the determination in step S8 is NO.
That is, if the "current character" does not match the character string on the slide dictionary, the process proceeds to step S13 to perform a flash operation, and in the next step S14, "1" + "current character" is output. In S15, the base pointer = base pointer + 1, that is, the base pointer is incremented, and then the process proceeds to step S16.

In step S16, the previous character = character, that is, the "previous character" is updated, and the character "current character" is written at the address indicated by the write pointer. Further, the write pointer = write pointer + 1, that is, the write pointer is incremented.

In the next step S17, it is determined whether or not to continue the compression processing. If the decision result in the step S17 is YES, that is, if the compression process is continued, the process returns to the step S2, and the processes in the steps S2 to S17 are repeated. When the result of the determination in step S17 is NO, that is, when the compression process is completed, the process proceeds to the next step S18, where the flash operation is performed.
The compression processing ends.

Here, steps S10, S13, S18
If the run length is 1 or more, the run length is encoded according to Table 1 and output, and the run length is set to "0".

In the decompression processing by the compression / decompression unit 12, as shown in the flowchart of FIG. 4, when the decompression processing is started, first, in step S21, the slide dictionary is initialized by padding with zeros, and the write pointer and the base are initialized. The pointer and the run length are set to 0, and compressed data is input in the next step S22.

In the next step S23, it is determined whether or not the MSB of the input compressed data is 1.

If the decision result in the step S23 is YES, that is, if the MSB of the input compressed data is 1, the next 8 bits are outputted as a character in a step S24, and in a step S25, the base pointer = the base pointer + 1, that is, the base After incrementing the pointer, the process moves to step S30.

The result of the determination in step S23 is N
If O, that is, if the MSB of the input compressed data is not 1, the process proceeds to step S26, where decoding is performed in accordance with Table 1, and the result of the decoding is determined in step S27.
It is determined whether or not it is an interval.

If the decision result in the step S27 is YES, that is, if the decoded result is the interval, the character of the base pointer + interval is outputted in the next step S28, and in the step S29, the base pointer = the base pointer + 1, that is, the base pointer Then, the process proceeds to step S30. In this step S30, characters are written to the address indicated by the write pointer. Further, the write pointer = write pointer + 1, that is, the write pointer is incremented,
Move to step S35.

The result of step S27 is N
If O, that is, if the decoded result is not an interval, the process proceeds to the next step S31, where the base pointer +1
Is output, and in step S32, the base pointer = base pointer + 1, ie, the base pointer is incremented, and the run length = run length−1, ie, the run length is decremented. In the next step S33, characters are written to the address indicated by the write pointer,
Further, the write pointer = write pointer + 1, that is, the write pointer is incremented.

In the next step S34, it is determined whether or not the run length is 0.

If the decision result in the step S34 is YES, that is, if the run length is 0, the flow advances to the next step S35.

The result of the determination in step S34 is Y
If the ES, that is, the run length is not 0, the process returns to step S31, and the processing of steps S31 to S34 is repeated until the run length becomes 0.

In step S35, it is determined whether or not to continue the decompression processing. If the determination result of step S35 is Y
If the ES, that is, the decompression process is continued, the process returns to step S22, and the processes of steps S2 to S35 are repeated, and the decompression process is sequentially performed on the next compressed data. Then, if the decision result in the step S35 is NO, the decompression process ends.

Here, the tape streamer drive 1
A specific example of the compression processing by the compression / decompression unit 12 at 0 will be described. In the description of this specific example, it is assumed that the slide dictionary and the pointer are in the state shown in FIG. That is, the character string “ABCDEFGHIJ” is written in the slide dictionary, and the base pointer indicates the address 0 of the slide dictionary. Then, the character string “CDEJ” is encoded.

First, encoding using intervals will be described.

First, the first character "C" of the character string "CDEJ" is input in step S2, and the above steps S3 to S3 are performed.
In step S6, the character "C" is searched in the slide dictionary. The search using the slide dictionary is performed for all addresses of the slide dictionary so as to make one round from the right side of the address indicated by the base pointer. The letter "C" is found at address 2 of the slide dictionary. Since the base pointer is at the address 0, the interval 2 is calculated in step S7 by 2-0 = 2. Interval 2 is step S
In step S11, the data is encoded into "0010" according to Table 1. Then, as shown in FIG. 5B, the base pointer is moved to address 2 in step S12.

Next, the second character "D" of the character string "CDEJ" is input in step S2,
In step S6, the character "D" is searched in the slide dictionary. The letter "D" is found at address 3 in the slide dictionary. Since the base pointer is at address 2, in step S7, interval 1 is calculated by 3-2 = 1. Interval 1 is encoded to “01” in accordance with Table 1 in steps S10 and S11. Then, as shown in FIG. 5C, the base pointer is moved to address 3 in step S12.

Next, the third character "E" of the character string "CDEJ" is input in step S2,
In step S6, the character "E" is searched in the slide dictionary. The letter "E" is found at address 4 in the slide dictionary. Since the base pointer is at address 3, in step S7, interval 1 is calculated by 4-3 = 1. Interval 1 is encoded to “01” in accordance with Table 1 in steps S10 and S11. Then, as shown in FIG. 5D, the base pointer is moved to address 4 in step S12.

Further, the fourth character "J" of the character string "CDEJ" is inputted in step S2, and the above-mentioned step S3
Step S6 searches the slide dictionary for the character "J". The letter "J" is found at address 9 in the slide dictionary. Since the base pointer is at address 4, in step S7, interval 5 is calculated from 9-4 = 5. Interval 5 is encoded into “000101” in accordance with Table 1 in steps S10 and S11. And
As shown in FIG. 5E, the base pointer is moved to address 9 in step S12.

As described above, the character string “CDEJ” is represented by “2, 1, 1, 5” using intervals, and is encoded into “0010, 01, 01, 00001” according to Table 1.

In the encoding using the interval, since the interval 1 may be continuous, it is replaced with a run length, and then the encoding is performed according to Table 1 in steps S10 and S11.

That is, since the character string "CDEJ" is represented by "2, 1, 1, 5" using intervals, it is defined as "2, run length 2, 5" and according to Table 1. "0010,0101,000101"
To be encoded. As shown in Table 1, run length 2-5
Is represented using interval 1.

The code “0010,
“0101,000101” is 14 bits, and if the code length of the character is 8 bits, 14 bits / (8 bits ×
4) It means that it has been compressed to 31.25%.

Here, as shown in Table 1, only up to 15 intervals are defined. If the interval is 16 or more, a character is output as a code word instead of encoding the interval. One is added to distinguish the interval from the run length. For example, the letter "Z"
Then, the ASCII code of Z is 0x5A (01010110)
0), "1010111010" is used as a codeword.

When a character not in the dictionary is output as a code word, the overall compression ratio can be increased by converting the code word into a BSTW code or a Huffman code.

In this tape streamer drive 10,
Data compression processing and decompression processing can be performed by one-pass processing, respectively, and a sufficient compression rate could be obtained with a slide dictionary having a small capacity of about 512 bytes. The compression ratio was about 56% for text files and about 46% for program execution files. Therefore, the capacity per tape winding can be increased in appearance. In addition, since processing can be performed in one pass, software can be used and can be easily realized by hardware.

[0066]

As described above, according to the present invention, a data compression method which can be processed in one pass, can obtain a sufficient compression ratio with a small dictionary, and can be easily implemented in hardware,
A data compression / expansion method, a data compression device, and a data compression / expansion device can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a tape streamer drive to which the present invention has been applied.

FIG. 2 shows compression / compression in the tape streamer drive.
FIG. 3 is a block diagram illustrating a functional configuration of an extension unit.

FIG. 3 is a flowchart illustrating a procedure of a compression process performed by the compression / decompression unit.

FIG. 4 is a flowchart showing a procedure of a decompression process executed by the compression / decompression unit.

FIG. 5 is a diagram provided for describing a specific example of a compression process performed by the compression / decompression unit.

[Explanation of symbols]

10 tape streamer drive, 11 SCSI interface, 12 compression / decompression processing unit, 13 IF / E
CC controller, 14 buffer memory, 15 RF processing unit for recording, 16 RF processing unit for reproduction, 17 rotary transformer, 18A, 18B recording head, 19A,
19B playback head, 20 host computer, 12
1 slide dictionary unit, 122 base pointer unit, 12
3 Interval calculator, 124 gamma encoder

Claims (13)

[Claims] 1. Data compression by a slide dictionary method in which read data is registered as a dictionary, a longest matching character string is searched from the dictionary, and an original code is replaced by a length matching a matching position and output. A dictionary lookup in the encoding from the position of the base pointer, encoding the interval from the base pointer to the first matched character, and setting the base pointer to the character before encoding the next character. A data compression method characterized by moving to a coincident position. 2. The data compression method according to claim 1, wherein when encoding said interval, a small interval is encoded into a short code word. 3. The data compression method according to claim 1, wherein a short code is output by comparing a word length when encoding the interval with a word length of a character. 4. When encoding and outputting a character, selectively switching between a method of encoding and compressing an interval from the base pointer to the first matching character and another compression method. The data compression method according to claim 3. 5. The data compression method according to claim 1, wherein when interval 1 is continuous, the interval 1 is encoded as run length. 6. The data compression method according to claim 1, wherein when the base pointer is moved, the character entered immediately before and the character to be encoded are moved to a position where the two characters match. 7. A data compression by a slide dictionary method in which read data is registered as a dictionary, a longest matching character string is searched from the dictionary, and an original code is replaced by a length matching a matching position and output. In a decompression method, at the time of compression, dictionary reference in encoding is performed from the position of the base pointer, the interval from the base pointer to the first matching character is encoded, and before encoding the next character, Encoding is performed by repeating the process of moving the base pointer to the position where the character matches, and at the time of decompression, for the encoded character, reference is made to the dictionary in decoding from the position of the base pointer for decoding, and matching is performed first from the base pointer. The base pointer to the position where the character matched before decoding the next character. Data compression and decompression method characterized by decoding by repeating the process of. 8. A data compression method using a slide dictionary method in which the read data is registered as a dictionary, the longest matching character string is searched from the dictionary, and the original code is replaced with a length matching the matching position and output. A data compression device for performing processing, a slide dictionary unit in which read data is registered as a dictionary; a base pointer for designating a position where a previous character matched as a reference position of the slide dictionary unit in encoding; A data compression device comprising: an interval calculation unit that calculates an interval from a base pointer to a first matching character; and an encoding unit that encodes the interval calculated by the interval calculation unit. 9. The data compression apparatus according to claim 8, wherein said encoding section encodes a small interval into a short codeword when encoding said interval. 10. The data compression apparatus according to claim 8, wherein the encoding unit compares a word length when encoding the interval with a word length of a character and outputs a short code. 11. The data compression apparatus according to claim 8, wherein, when the interval 1 is continuous, the encoding unit encodes the interval 1 as a run length. 12. When moving the base pointer,
9. The data compression apparatus according to claim 8, wherein the character compression unit moves to a position where the immediately preceding character and the character to be encoded match. 13. Data compression by a slide dictionary method in which read data is registered as a dictionary, a longest matching character string is searched from the dictionary, and an original code is replaced with a length matching a matching position and output. A decompression device, which references a dictionary in encoding from the position of a base pointer, encodes an interval from the base pointer to the first matching character, and encodes the base pointer before encoding the next character. Compression processing means for performing encoding by repeating a process of moving to a position where a character is matched; and referring to a dictionary for decoding of the encoded character from the position of the base pointer, and the first matched character from the base pointer. Before decoding the next character, and moving the base pointer to the position where the character matched before decoding the next character. Data compression and decompression device, characterized in that it comprises a decompression processing means for decoding by repeating.