JP2823917B2 - Data compression method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Summary] The present invention relates to a data compression method for compressing and encoding an input data sequence such as a character as a copy of an already encoded sequence registered in a reference text. The reference text is divided into a plurality of regions, and the encoded data series is registered sequentially.The reference text is searched from the newly registered divided region to the old registered divided region, and further, the reference text is searched. When is full, the oldest registered divided area is updated.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression method for compressing and encoding an input data sequence such as a character as a copy of an already encoded sequence registered in a reference text.

When transmitting and storing code string information such as characters, code information is compressed and coded to reduce the amount of data to reduce transmission time and storage capacity. Universal coding is performed to extract a substring of the maximum length that matches the input code information from an arbitrary position in the reference text in which the data series has been registered, and encode it as a copy of the past series. It is necessary to obtain a high compression ratio.

[Prior art] Generally, when the capacity of data to be stored and transmitted is large, in order to effectively use the capacity of a communication line or a storage device,
A method of compressing a data string for transmission or storage and restoring the original data string when the data is used again is often used.

Conventionally, Ziv was used as a method to efficiently compress character codes.
-Lempel code (hereinafter referred to as "ZL code") is known (for example, "Ziv-Lempel data compression method" by Seiji Munakata,
Information processing, pp. 2-6, vol. 26, NO. 1, 1985).

Two algorithms have been proposed for ZL code: universal type and incremental persing. Between these two algorithms, the compression ratio is superior in the universal type, and the processing speed is faster in the incremental decomposition type.

Among these two algorithms, the universal ZL code with a high compression ratio searches the input character string for the maximum length matching character string from the previously input encoded character string,
The character string that matches the maximum length is encoded as a duplicate.

Here, data compression can be applied not only to character codes but also to general data. In the following description, one word of data is called a character, and the set is called a character string, following the names used in information theory and the like. I will.

 FIG. 5 shows the principle of the universal ZL encoder.

In FIG. 5, a coded character string is stored in a P buffer 12, and a character string to be coded is stored in a Q buffer 10. The input character string of the Q buffer 10 and all the registered character strings (substrings) of the P buffer 12 are searched and collated, and a matching maximum length substring in the P buffer 12 is obtained. Then, in order to specify the subsequence having the maximum matching length in the P buffer 12, the following information set is encoded.

Next, the encoded character string in the Q buffer 10 is
Move to step 12 to register new characters for the number of encoded characters. Hereinafter, the same operation is repeated to decompose the input data into sub-sequences and sequentially encode them.

FIG. 6 shows an example of the conventional system. In the case where a symbol is represented by 2 bytes, for example, the P buffer 12 is represented by 12 bits and the Q buffer 10 is represented by 4 bits. The search on the P buffer 12 is performed from the left side of the P buffer 12, and if there is no matching character string, an input data sequence is newly registered at the INPUT pointer.

[Problems to be Solved by the Invention] In order to improve the compression ratio in such a universal ZL coding method, as many reference texts as possible
Ideally, it is necessary to register a buffer, and the matching length determined by the matching determined by the bit width of the Q buffer can express the size without limitation.

However, in practice, when encoding / decoding by software, if the P buffer 12 and the Q buffer 10 are simply made large, the code word data determined by the buffer address becomes large, and as a result, the compression ratio deteriorates. Also, since the number of reference texts increases, it takes time to perform a match search, and the processing speed is reduced.

The present invention has been made in view of such a conventional problem, and an object of the present invention is to provide a data compression method that speeds up a match search and an update when a reference text is enlarged.

[Means for Solving the Problems] FIG. 1 is an explanatory view of the principle of the present invention.

First, according to the present invention, a data sequence is input to a first buffer (Q buffer) 10 and a second buffer (P buffer) 12 as a reference text in which an already encoded data sequence is registered is searched for input data. A data compression method for obtaining a maximum length matching subsequence of an encoded data sequence matching a sequence, outputting a set of a start position of the maximum length matching portion and a matching length as a codeword, and compressing and encoding the data sequence. I do.

In the present invention regarding such a data compression method,
The second buffer 12 is divided into a plurality of regions 12-1 to 12n, and the encoded data is sequentially registered. From the newly registered divided regions in the divided regions 12-1 to 12-n, the regions are registered from the newly registered region to the old registered region. A search for a match with the input data series is performed, and if all the divided areas 12-1 to 12-n are filled with the registered data series, the divided area with the oldest registration is updated.

Further, in the present invention, in order to increase the second buffer 12 as a reference text while reducing the code word and improving the compression ratio, the start position of the searched maximum length matching portion is set in the second buffer 12. The region numbers of the divided regions 12-1 to 12-n are divided into the region number and the position in the region, and the position in the region is numbered as the start position of the maximum length matching portion of the code word, while the region number is different from the code word. By having the second buffer 12 as identification data, the search index of the second buffer 12 is reduced in bit width to a bit width suitable for a match length search, and the reduced bits are allocated to the first buffer 10. It is configured to extend the allowable length of the match length search.

[Operation] According to the data compression system of the present invention having such a configuration, when all the divided areas 12-1 to 12-n of the second buffer 12 become full during encoding, By deleting the oldest divided area and newly registering it,
Conventionally, the troublesome work of updating the second buffer 12 as a whole is greatly reduced, and the processing can be speeded up.

In addition, by searching from the newly-registered divided area of the second buffer 12, it is possible to search from new information that has just been registered, so that the search can be completed in a short time.

Embodiment FIG. 2 is a block diagram showing an embodiment of the present invention, taking an encoder as an example, and shows an LZSS encoding method (TCBell: Bet
ter OPM / L Text Compression, IEEE Trans.on Commom., V
ol. 34, No. 12, 1986) in which codeword data is represented by 2 bytes.

In FIG. 2, reference numeral 14 denotes a data compression device which includes a Q buffer 10 as a first buffer and a P buffer 12 as a second buffer. In this embodiment, the P buffer 12 has three divided areas. It is divided into 12-1, 12-3, 12-3.

It should be noted that the divided areas of the P buffer 12 indicate that 12-1 and 12-2 have been registered, and the registration to the remaining area 12-3 is currently being performed.

A file / transmission device 16 stores or transmits a code word or raw data (input character string itself) having a data structure of a set of a start position of a maximum length matching portion and a matching length output from the data compression device 14. I do.

Here, the processing of the present invention will be described in comparison with the conventional method shown in FIG.

First, in the conventional method shown in FIG. 6, when the code word is represented by 2 bytes, for example, the size of the P buffer 12 is represented by 12 bits, and the size of the Q buffer 10 is represented by 4 bits. From the left side, and if there is no matching character string, it is newly registered at the INPUT pointer in the figure. In this method, the character comparison is
The search starts with an old registered character that is considered to have little relation to the contents of the buffer 10 and takes a long time to search. Also, P
As an update method when the buffer 12 becomes full, a method of shifting the P buffer 12 to the left as much as the encoding on the Q buffer 10 is completed, or a method of clearing a continuous P buffer 12 and starting from the beginning There was a way to redo the registration.

However, in the partial shift method, once the P-buffer 12 becomes full, it is necessary to be able to update each time, and the processing becomes complicated.
In the all clear method, the possibility that the learned P buffer 12 up to now is lost at a time and uncoded raw data is output increases, resulting in poor efficiency.

On the other hand, in the present invention, the P buffer 12 which is conventionally continuous is divided into a plurality of areas, for example, three areas 12-1 to 12-3.
Which of the buffer divided areas in the P buffer 12 is used is incorporated into the identification code in the buffer.

The search of the P buffer 12 starts from the newest registered sub-region 12-3 in order to complete the search in a short time. Further, all the divided areas 12-1 to 12-12 of the P buffer 12 are
When -3 is full, the registration that seems to be less relevant clears only the oldest divided area 12-1 of the P buffer 12,
By newly registering in the cleared divided area 12-1, search efficiency can be improved, and the other two divided areas 12-2, 12-
3 can be used as it is.

In the embodiment shown in FIG. 2, the P buffer 12 has 12 bits, the Q buffer and 10 have 4 bits in the conventional system shown in FIG. 6, but 11 bits and 5 bits, respectively. As a result, the amount of the reference text is reduced by half as much as the P buffer 12 is reduced by one bit.
By having the reference text divided into 12-1 to 12-3, the amount of the reference text is equalized to 3/2. Further, since the Q buffer is increased by one bit, the matching matching length can be lengthened and the compression ratio can be improved.

More specifically, in the conventional system shown in FIG. 6, the P buffer 12 is represented by 12 bits and the Q buffer 10 is represented by 4 bits, and for every 8 data, an identification for identifying codeword data or raw data is performed. Data is stored. That is, it is determined whether the eight pieces of data in which the identification data are successively arranged one bit at a time are codeword data or raw data.

Here, the P buffer 121 and Q
Assuming that the bit width of the buffer 10 is increased by one bit, the code word data which is a set of the start position and the matching length is not a multiple of 8, and a complicated process of packing bits is required when transferring data. Also, the bit widths of the P buffer 12 and the Q buffer 10 are set to, for example, 18 so as to be a multiple of 8.
If the number of bits is increased to 6 bits, the code word consisting of the start position of the maximum match length and the match length becomes 3 bytes, and if the maximum match length is 2 bytes or 3 bytes, both are copied. If it is represented, it will not be compressed, and the encoding efficiency will be reduced.

On the other hand, in the present invention, the continuous P buffer 12 is
As shown in the figure, for example, it is divided into three divided areas 12-1 to 12-3, and which divided area is used is determined by a P buffer.
By providing 12 pieces of identification data as 2 bits per data, it is possible to increase the compression ratio without changing the length of one set of code word data at 2 bytes.

Next, the processing operation of the present invention will be described with reference to the operation flowchart of FIG.

First, an input character string is read into a Q buffer in step S1 (hereinafter "step" is omitted). Next, if it is not the end of the input character string in S2, the process proceeds to S3, and if it is, the process ends. .

In S3, i = New. Here, New is an index indicating the newest P buffer divided area.

Next, in S4, the divided area P [i] of the P buffer 12 is scanned to find a matching character. If they match, the process proceeds to S5, where the match start position and the match length in the divided area P [i] are registered and updated in a register or the like, and the process proceeds to S6. If the matching character is not in the divided area P [i], Proceed directly to S6.

In S6, i is updated according to the function f (i). Function f
In the update of i according to (i), i = 1,2,3 is updated at first, and when it is full and clear update of old registered i = 1 is performed, then i = 2,3,1 is updated. It is updated, and in the third update, it is updated as i = 3, 1, 2, and so on.

Next, the process proceeds to S7 to determine whether New = i, that is, whether or not the search for all the divided areas has been completed.
If not, return to S3.

When the search of all the P buffer divided areas is completed and there is no matching character in the P buffer in S8, the raw data itself is output as a codeword data string in S9, and the process proceeds to the P buffer registration processing in S11. If it is determined in S8 that a matching character is present in the P buffer, codeword data consisting of a set of a matching start position (region position in a specific P buffer divided region) registered in S5 and a matching length is output. , S11, and thereafter, the same processing is repeated until the end of the character string is determined in S2.

FIG. 4 is an operation flowchart showing a P buffer registration process in S11 of FIG. 3 as a subroutine.

In FIG. 4, it is first checked whether or not S1 is full and the newest divided area P [New] is registered.
Proceed to. On the other hand, if there is still room for registration, the process proceeds to S5.

In S2, the index of the oldest P buffer divided area
Old is calculated, the oldest registered divided area P [old] is cleared in S3, and after clearing in S4, it is replaced with New = Old and the process proceeds to S5. In S5, the raw data is registered in the divided area P [New] in the clear state.

In the above embodiment, the case where the P buffer 12 is divided into three areas is taken as an example. However, the P buffer 12 can be divided into an arbitrary number as needed.

[Effects of the Invention] As described above, according to the present invention, even when the allowable length of the reference text and the matching length is increased, the reference text can be searched and updated at high speed.

[Brief description of the drawings]

 FIG. 1 is a diagram for explaining the principle of the present invention; FIG. 2 is a block diagram of an embodiment of the present invention; FIG. 3 is an operational flow diagram of the present invention; FIG. 5 is a diagram for explaining the principle of universal ZL encoding; FIG. 6 is a diagram for explaining a conventional system. In the figure, 10: first buffer (Q buffer) 12: second buffer (P buffer) 12-1 to 12-n: divided area 14: search device 16: encoder

Continuation of front page (58) Field surveyed (Int.Cl. ⁶ , DB name) H03M 7/30

Claims (2)

(57) [Claims] A first buffer for storing a data series such as a character;
And searches the second buffer (12) in which the already encoded past data sequence is registered, and searches for the maximum length of the encoded data sequence that matches the data sequence of the first buffer (10). In a data compression method in which a matching part is obtained and output as a code word of a set of a start position of the maximum length matching part and a matching length and compression encoding is performed, the second buffer (12) includes a plurality of areas (12- 1-12-n)
, The encoded data sequence is sequentially registered, and the matching search is performed from the new divided region of the encoded data sequence in the divided regions (12-1 to 12-n) to the old divided region. A data compression method characterized by updating the oldest divided area when all the divided areas are filled with the registered data sequence. 2. A start position of the maximum length matching portion is divided into an area number and an area position of a divided area (12-1 to 12-n) of the second buffer (12). It is encoded as the start position of the maximum match length portion of the code word, and the area number is stored in the second buffer (12) as identification data separately from the code word, so that the bit width suitable for match length search is obtained. The bit width of the search index of the second buffer (12) is reduced, and the reduced bit is allocated to the first buffer (10) to increase the allowable length of the match length search. Item 2. The data compression method according to Item 1.