JPH05241777A - Data compression system - Google Patents

Abstract

PURPOSE:To improve the compressibility without spoiling the easiness of slide dictionary type algorithm by registering an initial value character string consisting of a special kind of data which is high in appearance frequency in a dictionary. CONSTITUTION:An initial value generation part 14 encodes representative sample data according to the dynamic dictionary type algorithm. Then a counter counts the frequency of use of a reference number, indicating a character string registered in a dictionary generated by this encoding, at the time of the encoding as an appearance frequency, and registered character strings in the dictionary 24 which have appearance frequencies larger than a specific threshold value are extracted and arrayed when the encoding of the sample data ends to generate an initialization character string. The initialization value character string generated by the initialization value generation part 14 is used for data compression using a slide dictionary at a data compression and restoration part 30. Thus, the character strings which are high in use frequency are previously registered in the dictionary, so the probability that long input data match a character string to the longest length becomes high to improve the efficiency of the encoding.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression method by the Jib-Lempel coding using a slide dictionary, and more particularly, a data compression method using the coding using a dynamic dictionary for the coding using the slide dictionary. Regarding In recent years, various types of data such as character codes, vector information, and images have been handled by computers, and the amount of data handled has been increasing rapidly.

When handling a large amount of data, the redundant portion of the data is omitted and the data amount is compressed, so that the storage capacity can be reduced or the data can be transmitted at high speed. Universal encoding has been proposed as a method of compressing various data by one method. Here, the field of the present invention is not limited to compression of character codes and can be applied to various data, but in the following, the word used in information theory is followed, and one word unit of data is called a character, A string in which data is connected to arbitrary words is called a character string.

As a typical method of the universal code,
There is a Ziv-Lempel code (specifically, for example, Munakata "Ziv-Lempel Data Compression Method", Information Processing, Vol. 26, No. 1, 1).
985). For the Jib-Lempel code, two algorithms have been proposed: a slide dictionary type (also called universal type) and a dynamic dictionary type (also called incremental decomposition type). A method for improving the practical use of these methods has been announced, and it has come to be used for file compression of an auxiliary storage device and data transmission by personal computer communication.

[0004]

2. Description of the Related Art First, conventional slide dictionary type algorithms and dynamic dictionary type algorithms will be described. (1) Slide dictionary type algorithm This algorithm is a method that can obtain a high compression rate although the amount of calculation is large. That is, it is a method of dividing encoded data from an arbitrary position of a past data sequence into a sequence of the maximum length that matches (subsequence) and encoding as a copy of a past character string. [Jib-Lempel Code] FIG. 10 shows the principle of a slide dictionary type encoder of the Jib-Lempel code.

In FIG. 10, P as a dictionary buffer
Encoded input data is stored in the buffer 12, and data to be encoded is input to the Q buffer 10 as an input buffer. Q buffer 10
Is compared with the character string in the P buffer 12 to obtain a matching character substring of the maximum length in the P buffer 12. Then, in order to specify this maximum length character string in the P buffer 12, the next set of information is encoded.

[0006]

[Table 1]

Next, the encoded character string in the Q buffer 10 is transferred to the P buffer 12 to obtain new data. Hereinafter, the same operation is repeated to decompose the data into subsequences and encode the subsequences. That is, in the Jib-Lempel code, the current character code sequence is encoded as a duplicate of the encoded past sequence. When the Jib-Lempel code is used, the document information of the character code can be compressed to about 1/2.

Further, as an improvement of the slide dictionary type algorithm, there is LZSS code (TCBell, "Better OPM / LT
ext Compression ", IEEE Trans.on Commun., Vol.COM-34,
No. 12, Dec. 1986). In the LZSS code, the combination of the [start position of the maximum matching sequence in the P buffer] and the [matching length] and the [next symbol] are distinguished by flags, and the code with the smaller code amount is coded.

Further, as an improvement of the slide dictionary type algorithm, there is QIC-122 code which is a standard compression system of 1/4 inch cartridge magnetic tape. The LZSS code will be described next. [LZSS Code] FIG. 11 shows a processing flow of encoding by the LZSS code, and its principle diagram is shown in FIGS. 12 and 13.

Encoding by the LZSS code is shown in FIG.
As shown in FIG. 12B, for example, a Q buffer 10 that can store 16-bit index information that stores a character string to be encoded and that can store 16 character numbers corresponding to 4-bit index information, as shown in FIG. 4) with 12-bit index information, as shown in FIG.
P buffer 12 for storing 096 encoded character strings
And is provided.

In the encoding process, as shown in the flowchart of FIG. 11, in step S1, the P buffer 12 is emptied and Q is set.
After packing the input data in the buffer 10, step S2
At step S3, the character string in the Q buffer 10 and the character string in the P buffer 12 are collated to find the longest matching character substring.
On the condition that the number of characters is 2 or more, the process proceeds to step S5, and in order to specify the obtained character substring, encoding is performed with a combination of [appearance position of character string S] [match length].

Subsequently, in step S6, the coded character string in the Q buffer 10 is moved to the P buffer 12, and a new character string for the coded character string is input into the Q buffer 10 to code the character string. Execute When the longest matching character substring is 1 byte, it is more advantageous to encode it with raw data, so in step S4, [raw data 1 byte] is output as it is.

Further, as shown in FIG. 13, eight pieces of coded data or raw data are collected as one set of data, and whether each of the eight pieces of collected data is coded data or raw data is shown. The 8-bit identification data composed of the flag bits obtained in steps S4 and S5 is added to the head and output as a set of data. (2) Dynamic decomposition type (incremental decomposition) algorithm This algorithm is inferior to the universal type in compression rate, but is known to be simple and easy to calculate.

In the incremental decomposition type Dibulenpel code, assuming that the sequence of input symbols is X = aabababaa ..., The incremental decomposition into the component sequence X = X ₀ X ₁ X ₂ ... First, let X _{1 be} the longest column in which the rightmost symbol of the existing component has been removed, and X = a · ab · aba · b · aa. Therefore, X ₀ = λ (empty column) X ₁ = X ₀ a X ₂ = X ₁ b X ₃ = X ₂ a X ₄ = X ₀ b X ₅ = X ₁ a. Each incrementally decomposed component series is encoded by the following set using the existing component series.

[0015]

[Table 2]

That is, the incremental decomposition type algorithm finds a coding pattern having a maximum length match among the previously decomposed partial sequences and encodes it as a duplicate of the previously decomposed partial sequences. As an improvement of the dynamic dictionary type algorithm, LZW (Lempel-Ziv-Welch) code (TAWelch, "A Tech
nique for High-Performance Data Compression ", ComPu
ter, June 1984) LZJ code (M. Jakobsson, "Comperssion of Character
Strings by An Adaptive Dictionar, BIT, No. 25, 198
5 years, see pages 593 to 603). Next, the LZW code will be described. [LZW Code] FIG. 14 shows a flow of processing for encoding the LZW code. That is, the LZW encoding has a rewritable dictionary, divides the data of the input character code into different character strings, numbers the character strings in the order in which they appear, registers them in the dictionary, and is currently inputting them. The character string is represented by only the number of the longest matching character string registered in the dictionary and is encoded. The technique of the dynamic dictionary type code and the LZW code is disclosed in JP-A-59-231683, US Pat.
No. 8,302. The encoding process of FIG. 14 is as follows.

S1: Encoding is started after a character string consisting of one character for all characters is registered in advance as an initial value. The registered number n of the dictionary is set as the character type number A. Place the cursor at the beginning of the data. S2: Find the longest character string S in the dictionary registration that matches the character string from the cursor position.

S3: The dictionary number of the character string S is [log ₂
[n] bits and output. However, [log ₂ n]
Is the smallest integer greater than or equal to log ₂ n. The dictionary registration number n is incremented by one. S4: The character string SC in which the first character C of the cursor is added to the character string S is registered in the dictionary. The cursor moves to the character after the character string S. Return to S2.

FIG. 15 is a flowchart showing the decoding of the LZW code, which is the reverse process of the coding. The dynamic dictionary algorithm has a high processing speed because the sequence in the dictionary is selected only from the sequences coded (sampled) in the past. However, there is a drawback that the compression ratio cannot be high because only a part of the series of data that has appeared in the past is included.

As an improved version of the dynamic dictionary algorithm,
There is an LZJ code in which the learning amount for the dictionary is increased so that it can be coded only by the index. [LZJ Code] FIG. 1 shows a processing flow of encoding the LZJ code.
6 and the decoding process flow is shown in FIG.

Here, the notation of the dictionary and the character string is defined as follows. Let A be a set of character types, and S be a character string formed by combining the characters of the set A. The i-th character of the character string S is S (i). Further, a plurality of partial character strings S
(I), S (i + 1), ..., S (j) are converted into S (i,
j). The dictionary is represented by D _h (S), and the dictionary tree (t
All the partial character strings having a constant length h in the character string S are registered as the path from the root of the lee to the leaf.

The LZJ encoding process of FIG. 16 is as follows. S1: Encoding is started after registering one character of all character types as an initial value in the dictionary. The number n of registered characters in the dictionary is set as the number of character types A. Set the cursor k = 0. S2 to S5: All partial character strings of the character string S (1, k) have already been registered in the dictionary D _h (S (1, k)), assuming that the encoding has been completed up to the kth input character. S (k +
1), encoding from the character string.

The details will be described below. S2: S (k + 1), ... From the dictionary D _h (S (1,
k)) the longest matching substring S (k +
1, k + z). S3: dictionary number a _{x of the} partial character string S (k + 1, K + z)
Is output with [log ₂ n] bits. However, n
Is the current number of registrations in the dictionary, and [log ₂ n] is log
It is a minimum integer of ₂ n or more. Here, the code word a _x represents the partial character string S (i _x , j _x ). Each a _x is a dictionary D
_h (S (1, j _x-1 )), (i _x ≤j _x ≤i _x + h, i
It is a dictionary number of _x = jx _-1 + 1).

S4: Partial character string S (k-h + 2, k +
1), ..., S (k + z-h + 1, k + z) is added to the dictionary by adding a dictionary number while incrementing n.
Construct the dictionary D _h (S (1, k + z)). S5: The cursor k = k + z is set. S6: S1 to S5 are repeated until all characters are processed.

The dictionary registration of the character string in step S4 is illustrated in FIG. Next, LZ in FIG.
The J decoding process is as follows. S1: Similar to S1 of FIG. 16, one character of all character types is registered as an initial value in the dictionary. The number n of registered characters in the dictionary is set as the number of character types A. Set the cursor k = 0.

S2 to S4: The dictionary number a _w is decoded,
Up to the character string S (1, j _w ) can be used, and the dictionary D
_{h (S (1, j w} )) has been re-configured. Next, the codeword a _{w + 1} is decoded. The detailed description is as follows. S2: Dictionary D _h from the dictionary number obtained by decoding the code word a _{w + 1}
The subsequence S (i _{w + 1} , j _{w + 1} ) in (S (1, j _w )) is restored. The subsequence S (i _{w + 1} , j _{w + 1} ) is a character string represented by a node at the address a _{w + 1} from the root in the dictionary.

S3: After decoding the character string S (1, j _{w + 1} ), the dictionary D _h (S (1, j _{w + 1} )) is constructed in the same manner as S4 in FIG. S4: The cursor k = j _{w + 1} is set. S5: S1 to S4 are repeated until all the codes are processed.

[0028]

However, the LZSS code of the conventional slide dictionary type algorithm and the LZW code of the dynamic dictionary type algorithm are premised on perfect universality, and encoding is started from a state in which the dictionary is blank. I am trying. Therefore, the conventional encoding method has a drawback that the compression rate is low at the beginning of the input data when the learning amount is small (the content of the dictionary is small).

Although universality is also important in the LZW code, the dictionary does not necessarily need to be coded from a blank state when a particular type of data appears a lot in the input data. From this point of view, the inventors of the present application have proposed a method for obtaining a high compression rate by using a dictionary that holds only character strings that appear frequently in the dynamic dictionary type algorithm, as shown in FIG.

In FIG. 19, the sample data is subjected to LZW encoding to create a dictionary, and the usage frequency is simultaneously counted in the dictionary. When the coding of the sample data is completed, a character string whose appearance frequency is equal to or higher than the threshold value T is extracted from the dictionary and registered as an initial value in the dictionary used for actual coding, and then encoded or decoded. I do. However, the dynamic dictionary algorithm balances the processing speed of encoding and decoding. On the other hand, the slide dictionary type algorithm is slow in encoding but is much faster in decoding process, which is advantageous in applications such as databases in which restoration process is mainly performed.

The present invention has been made in view of the above circumstances, and provides a data compression method that enables efficient coding by the slide dictionary type algorithm by registering an initial value using the dynamic dictionary type algorithm. The purpose is to provide.

[0032]

FIG. 1 illustrates the principle of the present invention. First, the present invention is based on the input buffer (Q buffer).
Input data in 10 is dictionary buffer (P buffer) 12
Among the subsequences of the encoded data in the middle, the longest matching one is designated by the storage position and the matching length and encoded, and the encoded input data is transferred to the dictionary buffer 12 as the new encoded data A data compression method is applied according to a slide dictionary type algorithm that encodes input data.

According to the present invention regarding such a data compression method, when typical sample data is divided into different subsequences, subsequences having an appearance frequency of a predetermined threshold value T or more are extracted and extracted. Initial value creating means 14 for arranging substrings in a row to create an initial value character string in advance, and the initial value character string created by the initial value creating means 14 are first fixed in the dictionary buffer 12 prior to encoding or decoding. Of the initial value character string and the newly set coded or decoded data that is the longest match with the input data of the input buffer 10 from the initial value character string and the newly set coded or decoded data. And a coding / decoding means 16 for searching for and designating a storage position and a matching length for coding or decoding.

Further, according to the present invention, the initial value character string created by the initial value creating means 14 is first fixedly set in the dictionary buffer 12 prior to the encoding and regarded as encoded or decoded data. Input buffer 1 only from the initial value character string
It is also possible to search for a substring that has the longest match with the input data of 0, specify the storage position and the match length for encoding, and not register new input data in the dictionary buffer 12.

Here, the initial value creating means 14 creates the initial value character string according to the encoding process of the LZW code which is a dynamic dictionary type algorithm. That is, it has a dictionary for registering coded character strings with reference numbers, searches for coded substrings in the dictionary that have the longest match with the character strings of typical sample data, and specifies them by reference numbers. Each time the encoded character string registered in the dictionary is searched, the substring in which the next sample character is added to the reference number after the encoding is added to the dictionary Count the frequency of use,
When the coding of the sample data is completed, a character string whose usage frequency is equal to or higher than a predetermined threshold is extracted, and the extracted character strings are arranged in the order of appearance to create an initial value character string in advance.

The initial value creating means 14 may create the initial value character string according to the encoding process of the LZJ code which is a dynamic dictionary type algorithm. That is, the initial value creating means 14 has a dictionary for registering coded character strings with reference numbers, and searches for a coded substring in the dictionary that longest matches the character string of typical sample data. Code with the reference number. Each character of the input character string encoded after this encoding is sequentially used as a prefix subsequence, and a subsequence in the dictionary is added to this prefix subsequence to create multiple subsequences of fixed length, and all are registered in the dictionary. The usage frequency is counted each time a coded subsequence registered in the dictionary is searched, and when the coding of the sample data is completed, the subsequence whose usage frequency is equal to or higher than a predetermined threshold is extracted, and the extracted subsequence is extracted. The initial value character strings are created in advance by arranging them in the order of appearance.

Further, the initial value creating means (14) encodes the sample data by a slide dictionary type algorithm to obtain a code string, and applies the dynamic dictionary type algorithm such as LAW coding or LZJ coding to this code string. An initial value character string that is frequently used may be generated. That is, a character string of representative sample data in the input buffer is coded by specifying the storage position and the matching length of the longest matching substring of the coded data in the dictionary buffer, and coding this coded data. When divided into different partial strings, partial strings having an appearance frequency of a predetermined threshold value or more are extracted, and the extracted partial strings are arranged in the order of appearance to create an initial value character string in advance.

Registration of the initial value character string, which is frequently used by learning, into the dictionary buffer 12 is performed by the dictionary buffer 1
2 is composed of a read-only first memory (ROM) and a readable and writable second memory (RAM), and the initial value character string created by the initial value creating means 14 is fixed to the first memory (ROM). And the encoded input data of the input buffer 10 is transferred to and stored in the second memory (RAM).

The dictionary buffer 12 is composed of a readable and writable memory (RAM) in which a rewrite prohibited area is set, and the initial value character string created by the initial value creating means 14 at the start of encoding is rewritten in the dictionary buffer 12. The encoding may be started after loading in the prohibited area.

[0040]

According to the data compression method of the present invention having such a configuration, the following effects can be obtained. First, the sample data corresponding to the type of data to be compression-encoded is encoded according to the dynamic dictionary algorithm, that is, the algorithm of LZW code or LZJ code, and the dictionary used for this encoding is provided with a counter. Is provided and the number of times the reference number is used for encoding is counted as the frequency of use.

After encoding the sample data,
A character string with a small counter count value, which indicates the frequency of use provided at the contact point of the registered character string in the dictionary, is deleted from the dictionary, and a dictionary in which only the character strings that appear with high frequency are left is obtained.
The initial value character string is generated by arranging this character string in a line.
The encoding by the slide dictionary type algorithm using the initial value character string generated by learning is performed as follows.

At the time of encoding, a high-frequency initial value character string that has been fetched in advance in a storage device is loaded as an initial value into a dictionary and then encoded. A high-frequency character string created in advance is set as an initial value at the beginning of the dictionary as a fixed part that is not rewritten, and is encoded. Slide dictionary algorithm, eg LZS
Even in the case of encoding by S code, since a frequently used character string is registered in advance in the dictionary buffer as an initial value, the character string in the dictionary that longest matches the character string of input data that is long from the beginning is searched. Therefore, the slide dictionary type encoding can be performed at a higher speed.

[0043]

FIG. 2 is a block diagram of an embodiment showing one embodiment of the present invention. In FIG. 2, 14 is an initial value creation unit,
It was used to encode the representative sample data according to the dynamic dictionary type algorithm, and to encode the reference number that indicates the character string (substring) registered in the dictionary created by this encoding. Count the number of times as a frequency of appearance with a counter,
When the coding of the sample data is completed, the registered character strings (partial strings) of the dictionary 24 having the appearance frequency equal to or higher than the predetermined threshold value T are taken out and arranged in one line to create the initialization character string.

Specifically, the initial value creation unit 14 includes a sample data storage unit 20, a dynamic dictionary type coding unit 22, and a dictionary 24.
And an initial value character string generation unit 26. In the sample data storage unit 20, representative sample data corresponding to the type of data to be compressed is stored as a learning target. The dynamic dictionary type encoding unit 22 performs encoding while creating the dictionary 24 for the sample data of the sample data storage unit 20 according to the dynamic dictionary type algorithm.

As this dynamic dictionary type algorithm, for example, the LZW coding algorithm shown in FIG. 13 or the LZJ coding algorithm shown in FIG. 15 can be used. The dictionary 24 registers a partial string as an encoded character string corresponding to the reference number, and further has a counter for counting the number of times the reference number of the encoded partial string has been used at the time of encoding. I am trying to count as. The initial value character string generation unit 26 is a dynamic dictionary type encoding unit 22.
At the stage when the coding process of the sample data by is finished, a subsequence having a frequency of occurrence of a predetermined threshold value T, for example, T = 2 or more is taken out from the dictionary 24, and the subsequences are arranged in a line and are used frequently in the initial stage. Generate a value string.

The initial value character string created by the initial value creating unit 14 is used for data compression using the slide dictionary in the data compression / decompression unit 30. The data compression / decompression unit 30 includes a Q buffer 10 as an input buffer, a P buffer 12 as a dictionary buffer, and a slide dictionary type encoding / decoding unit 16. An initial value character string created in advance by the initial value creating unit 14 is registered in the P buffer 12 prior to encoding and decoding, and an input character string is shifted from the Q buffer 10 and stored in the initial value registration area. Even if it is not discarded, it is fixedly held. That is, the initial value character string registered in the P buffer 12 is regarded as encoded data, and encoding and decoding according to the slide dictionary type algorithm are performed.

The slide dictionary type encoding / decoding unit 16 performs encoding or decoding in accordance with the slide dictionary type algorithm. Specifically, the slide dictionary type encoding / decoding unit 16 includes the Jib-Lempel encoding algorithm and the LZSS encoding shown in FIG. Run the algorithm. FIG. 3 is a flow chart showing the processing of the initial value creation unit 14 of FIG. The initial value creation process in FIG. 3 is as follows.

S1: A data sample that often appears as input data is input, and sample data is encoded according to a dynamic dictionary algorithm such as LZW code or LZJ code. In this encoding, a tree-structured dictionary is created. At the same time, each node in the tree structure of the dictionary created by encoding represents a character string, but at each node that passed when the longest matching character string was searched at the time of encoding by adding a counter to each node. The counter is incremented by one and the number of times of use is counted.

That is, when the longest matching character string is searched, all the counters of the nodes included in the searched character string are counted up. S2: A character string in which the count value of the counter provided at a node among the character strings formed by the chain of each node of the dictionary at the time when the encoding of the sample data is completed is frequently used at a predetermined threshold value T or more. Take out.

S3: The character strings extracted in S2 are arranged in the form of a single character string to generate an initial value character string. At this time, search whether there is the same character string as the newly extracted character string in the already arranged character strings, and if there is the same character string, it will be duplicated, so do not include it in the initial value character string. To
The initial value character string created through the above steps S1 to S3 is used for encoding and decoding according to the slide dictionary type algorithm, so it is desirable to take it out to an external auxiliary storage device or the like.

FIG. 4 shows the tree structure of the dictionary created when the sample data is encoded according to the dynamic dictionary type algorithm in step S1 of the initialization process of FIG. 3 and the frequency of use by the counter provided at the node. It is explanatory drawing which showed the count. In FIG. 4, first, for example, each character of abcd is initially registered in the dictionary as shown by reference numbers 1 to 3, and then the coding of the sample data is started. The tree structure is shown in the state where the character string of is registered.

For example, the encoding of the input data abc is performed at the stage where the character strings indicated by reference numbers 1 to 4 have been registered, and the character string ab matches the character string indicated by the reference number by searching the dictionary 24. Therefore, the output code is "
Then, the character string in which the next one character c is added to the reference number is newly added to the reference number and registered in the dictionary 24.

The next character string abd also matches the character string of the reference number at the longest in the search of the dictionary 24, and therefore the output code is output as "d", and for the dictionary 24, the next one character is added to the reference number. A new reference number is added to the character string added with d and registered. This character string abc and character string a
In the bd registration after encoding, the same character string ab is 2
Since it has been used twice, the reference number nodes and the counters of the reference number nodes are respectively counted up twice to become 5 and 3, respectively.

The count value of each node in the dictionary 24 having such a tree structure is 1 as the sum of the count values of the children of that node.
Is the value added. For example, the count value of the node of the character a of the reference number is 5 as a value obtained by adding 1 to the sum 1 + 3 = 4 of the count values of the reference numbers which are the children of the node. Figure 5
Is a character string used at a high frequency of a threshold value T or higher from the dictionary obtained by the coding of the sample data shown in steps S2 and S3 of FIG. 3 and converted into the shape of the character string used as the initial value. It is explanatory drawing which showed the process.

FIG. 5A shows the tree structure of the dictionary obtained after the coding of the sample data is completed. The count value in the counter provided at the node of each character indicates the frequency of use. There is. When a character string having a count value of threshold value T = 2 or more is extracted from the tree structure of the dictionary of FIG.
As shown in (b). As shown in FIG. 5C, the character strings having the count value of 2 or more are rearranged so that the character strings on the left side are grouped into one character string in order, and the character string P of the slide dictionary algorithm is used. An initial value string to be registered as an initial value in the buffer is created.

FIG. 6 is a flowchart showing the encoding process using the slide dictionary in the data compression / decompression unit 30 of FIG. 2, which is as follows. S1: The initial value character string (N characters) created by the initial value creation process of FIG. 3 is stored in the first half of the P buffer. S2: Similar to the normal slide dictionary type algorithm, the character string in the input Q buffer 10 is searched for the longest matching character string from the P buffer 12 storing the initial value character string and the encoded character string, and the start position is searched. And a set of matching lengths. In this case, the encoded character string is encoded in the following two modes depending on whether two characters match.

Encoding mode [identification bit 0] [position of longest matching character string] [match length] Raw data mode [identification bit 1] [1 character] S3: Excluding initial value character string as dictionary deletion and registration processing The part in the P buffer 10 is slid. That is, P
The characters at positions 0 to n-1 indicating the initial value character string in the buffer 10 are left as they are, and the characters are shifted to the left from the character position n by the number of encoded characters in the Q buffer 10 and deleted. The encoded character string of the Q buffer is newly added by shifting it from the left side to the right side of the P buffer 12.

When the above dictionary deletion and registration processing is completed, the coded characters are shifted to the left in the Q buffer 10 and a new character string is input. Similarly, the following steps S
The input character string is encoded by repeating the processing of 2 and S3.
FIG. 7 is an explanatory diagram showing the configuration of the P buffer in the encoding using the slide dictionary type algorithm of FIG.

FIG. 7A shows a case in which a rewritable memory such as a RAM is used as the P buffer 12, and P
An initial value character string load area is provided in the area of 0 to n-1 indicated by the shaded area in the first half of the buffer 12, the previously created initial value character string is stored therein, and the rest is used as a registration area for encoded character strings. There is. This initial value character string load area 0 to n-1
Is prohibited from being rewritten thereafter, and at the time of registration in the new P buffer 12, the registered character is discharged as a processed character string with the shift from the position of n to the left by the number of registered characters. And delete it.

FIG. 7B shows another configuration of the P buffer 12. In this embodiment, a ROM is used to realize fixed storage as an area for registering the initial value character string. A ROM area is provided, and a RAM area using a rewritable RAM is provided for the remaining area. In the RAM area of the P buffer 12, every time one encoding of the input character string in the Q buffer 10 is added, characters corresponding to the number of encoded characters are discarded from the left end of the RAM area, and the encoded character string of the Q buffer 10 is stored in the RAM. It is stored by shifting from the right edge of the area.

FIG. 8 is a flowchart showing the decoding process by the data compression / decompression unit 30 of FIG. 2, which is performed as follows. S1: Similar to step S1 of FIG. 6, an initial value character string (n characters) previously created is stored in the first half of the P buffer 12. S2: Input a code word, and in the copy mode, refer to the P buffer 12 to restore the character string.

S3: Similar to step S3 of FIG. 6, the P buffer 12 is deleted and registered. Similarly, the processes of steps S2 and S3 are repeated to restore the encoded character string. FIG. 9 is an explanatory view showing another embodiment of the initial value character string creation processing of the present invention. In the process of creating the initial value character string of FIG. 9, first, the sample data is encoded according to the slide dictionary type algorithm, and the code string obtained from the sample data is encoded according to the dynamic dictionary type algorithm. , A counter is provided at each node in the tree structure of the dictionary created at the time of encoding to count the frequency of use, and when the encoding is completed, a character string having a frequency of use equal to or higher than a predetermined threshold value T is taken out. Create initial value strings by arranging them in columns.

That is, first, the sample data is transferred to the Q buffer 1
0a, the registered character string in the P buffer 12b is searched to find the longest matching character string, and the character string is encoded with the start position pi and the matching length qi. Of course, this encoding is performed when there are two or more characters, and raw data is output when there is one character. When the output code strings S1, S2, ... Si, ... Are obtained by the encoding by such a slide dictionary type algorithm, the code strings S1, S2 ,. A tree-structured dictionary 24 that follows a dynamic dictionary algorithm
To create.

In the dictionary 24, a counter is provided for each setting of the code string Si, and the number of times used for encoding the character string is counted. LZ of code strings S1, S2, ...
If W encoding is completed, for example, the threshold value T in the dictionary 24
= 2 or more code strings are taken out and rearranged into one code string, and as initial value code strings, for example, S1, S2, S3, ...
To generate. Here, the initial value code sequences S1, S2, S3
Since each character string of ... Is known in advance at the first encoding, the initial value character string is generated by restoring the original character string.

In this way, when a code string is first created from the sample data by using the slide dictionary type algorithm and initialization creation is performed for this code string, L
Compared to the case where a dictionary is created only with the ZW encoding algorithm, the only difference is whether or not there is a restriction on the maximum length of the character string registered in the dictionary. Basically, the initial based on the dictionary created by the LZJ encoding. It is almost the same as creating the value string.

Further, as another embodiment of the present invention, when the type of input data to be subjected to data compression is known in advance, the P buffer 12 as a slide type dictionary is initially created by the initial value creating unit 14. Only the value character string may be used. As described above, when the P buffer 12 is registered only with the initial value character string, the compression rate is slightly inferior, but the decoding process is simplified and the conventional slide dictionary type algorithm takes time to decode. Solve the problem,
It is possible to realize extremely high-speed processing as compared with the conventional method.

Further, as another embodiment of the present invention, when the type of data that appears can be roughly predicted in advance, several types of initial value character strings are prepared according to the type of data, and the initial value character string is prepared. By replacing the part of, it is possible to realize a method in which a strong compression effect is obtained for the expected specific type of data.

[0068]

As described above, according to the present invention, by registering an initial value character string consisting of characteristic type data having a high appearance frequency in a dictionary, a slide dictionary type algorithm such as LZSS code can be easily used. The compression ratio can be increased with almost no change.

For data of a type having a low appearance frequency that is not included in the initial value character string, a new character string is registered each time the initial value character string is set in the empty space of the dictionary. Even low-data can be compressed without losing universality.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of the principle of the present invention.

FIG. 2 is a block diagram of an embodiment of the present invention.

FIG. 3 is a flowchart showing an initial value creation process of the present invention.

FIG. 4 is an explanatory diagram showing dynamic dictionary type encoding for sample data of the present invention and counting the number of times a character string is used in the dictionary.

FIG. 5 is an explanatory diagram showing generation of an initial value character string based on the frequency of use of the encoding dictionary of the present invention.

FIG. 6 is a flowchart of slide dictionary type encoding using an initial value character string of the present invention.

FIG. 7 is an explanatory diagram showing the configuration of a P buffer used in the slide dictionary type encoding of the present invention.

FIG. 8 is a flowchart of slide dictionary type decoding according to the present invention.

FIG. 9 is an explanatory view showing another embodiment of the present invention in which sample data is converted into a code string by slide dictionary type coding, and then a tree-structure dynamic dictionary is created to generate an initial value character string.

FIG. 10: Principle diagram of slide dictionary type encoding

FIG. 11 is a flowchart showing a conventional LZSS encoding algorithm.

FIG. 12 is a configuration diagram of a buffer used for LZSS encoding.

FIG. 13 is an explanatory diagram of an output format of encoded data of LZSS encoding.

FIG. 14 is a flowchart showing a conventional LZW encoding algorithm.

FIG. 15 is a flowchart showing a conventional LZW decoding algorithm.

FIG. 16 is a flowchart showing a conventional LZJ encoding algorithm.

FIG. 17 is a flowchart showing a conventional LZJ decoding algorithm.

FIG. 18 is an explanatory diagram showing registration of a character string in LZJ encoding.

FIG. 19 is an explanatory diagram of initial registration of a dictionary in data compression using the LZW code that the inventor of the present application has already proposed.

[Explanation of symbols]

10: Input buffer (Q buffer) 12: Dictionary buffer (P buffer) 14: Initial value creating means (initial value creating section) 16: Encoding / decoding means (slide dictionary type encoding / decoding section) 20: Sample data storage Part 22: Dynamic dictionary type encoding part 24: Dictionary 26: Initial value character string generation part 30: Data compression / decompression part

 ─────────────────────────────────────────────────── --- Continuation of the front page (72) Inventor Hirotaka Chiba 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Fujitsu Limited

Claims (7)

[Claims] 1. The input data in an input buffer (10) is converted into a subsequence of encoded data in a dictionary buffer (12),
The longest matching one is designated by the storage position and the matching length and encoded, and the encoded input data is transferred to the dictionary buffer (12) and the next input data is encoded as new encoded data. In this case, when typical sample data is divided into different substrings, substrings having an appearance frequency of a predetermined threshold value or more are extracted, and the extracted substrings are arranged in a row to create an initial value character string in advance. The value creating means (14) and the initial value character string created by the initial value creating means (14)
Prior to encoding or decoding, it is first fixedly set in the dictionary buffer (12) and regarded as encoded or decoded data, and the initial value character string and new setting are encoded every time. Alternatively, a coding / decoding means (16) that searches the decoded data for a subsequence that has the longest match with the input data of the input buffer (10) and specifies the storage position and the matching length to perform coding or decoding. And a data compression method. 2. The input data in the input buffer (10) is converted into a subsequence of encoded data in the dictionary buffer (12),
The longest matching one is designated by the storage position and the matching length and encoded, and the encoded input data is transferred to the dictionary buffer (12) and the next input data is encoded as new encoded data. In this case, when typical sample data is divided into different substrings, substrings having an appearance frequency of a predetermined threshold value or more are extracted, and the extracted substrings are arranged in a row to create an initial value character string in advance. The value creating means (14) and the initial value character string created by the initial value creating means (14)
Prior to encoding, it is first fixedly set in the dictionary buffer (12) and regarded as encoded data, and the longest match with the input data of the input buffer (10) only from the initial value character string. Encoding / decoding means (16) for retrieving a subsequence to be encoded, specifying the storage position with a matching length, and encoding or decoding
And a data compression method. 3. A data compression method according to claim 1, wherein said initial value creating means (14) has a dictionary for registering coded character strings with reference numbers, A coded substring in the dictionary that matches the character string of the sample data at the longest is searched for, designated by a reference number and coded, and after the coding, a substring in which the next sample character is added to the reference number is newly added. The reference number is added to the dictionary, the number of times of use is counted every time the encoded character string registered in the dictionary is searched, and the number of times of use is determined when the encoding of the sample data is completed. A data compression method, wherein character strings having a threshold value or more are extracted, and the extracted character strings are arranged in the order of appearance to create an initial value character string in advance. 4. The data compression method according to claim 1, wherein said initial value creating means (14) has a dictionary for registering coded character strings with reference numbers, The coded substring in the dictionary that matches the character string of the sample data at the longest is searched, designated by the reference number, coded, and each character of the input character string coded after the coding is sequentially used as a prefix substring. , Create a plurality of substrings of fixed length by adding the substring in the dictionary to the prefix substring, register all in the dictionary, and count the number of times of use every time the coded substring registered in the dictionary is searched Then, when the coding of the sample data is completed, a partial string whose usage count is equal to or more than a predetermined threshold value is extracted, and the extracted partial strings are arranged in the order of appearance to create an initial value character string in advance. Compression method. 5. The data compression method according to claim 1, wherein said initial value creating means (14) converts typical sample data in the input buffer into a partial sequence of encoded data in the dictionary buffer. Of these, when the longest matching one is encoded by specifying the storage position and the matching length and the encoded data is divided into different subsequences, subsequences having an appearance frequency of a predetermined threshold value or more are extracted, and the extracted portions are extracted. A data compression method characterized in that an initial value character string is created in advance by arranging columns in the order of appearance. 6. The data compression method according to claim 1, wherein said dictionary buffer (12) is composed of a read-only first memory and a readable and writable second memory, and said initial value The initial value character string created by the creating means (14) is the first
A data compression method, wherein the data is stored in a fixed row in a memory, and the encoded input data of the input buffer (10) is moved to a second memory and stored. 7. The data compression method according to claim 1, wherein the dictionary buffer (12) is composed of a readable and writable memory in which a rewrite prohibited area is set, and the initial value is set at the start of encoding. A data compression method characterized in that the initial value character string created by the creating means (14) is loaded into the rewrite prohibited area of the dictionary buffer (12) and then the encoding is started.