JPH08116269A - Data processing unit and data processing method - Google Patents

Abstract

PURPOSE: To enhance the data compression rate by not simply sweeping out matched data or data string or the like from a head of a dictionary buffer but devising the slide method of dictionary data, increasing the number of reference dictionaries substantially and utilizing effectively the swept out dictionary data without storing dictionary data in duplicate. CONSTITUTION: This processing unit is provided with a dictionary control means 14, in which dictionary data or a dictionary data string and received compressed data or a compressed data string are compared and when the dictionary data or the dictionary data string and the received compressed data or the compressed data string are equal with each other, the equal dictionary data or dictionary string are swept out from the dictionary buffer, the data written areas of a dictionary buffer 12B from which the dictionary data or the dictionary data string are swept out are unidirectionally packed and equal compressed data or compressed data strings are written as new dictionary data in the dictionary buffer 12B with the data written areas packed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing device and a data processing method, and more specifically,
The present invention relates to an improvement in an apparatus and method for comparing dictionary data and input data, encoding matching data, and conversely decoding encoded compressed data.

In recent years, as information processing devices have become more sophisticated and diversified, devices using storage devices such as magnetic disk devices that store enormous amounts of data, and devices that transmit those data using communication lines. Is used. In such a field, in order to reduce the cost of the user by improving efficiency, in order to substantially increase the storage capacity when storing data, and to shorten the transmission time when transferring data, data is stored. A compressing device is used.

The current dictionary-based data compression method is described by Lempel Abraham and Ziv Jacob in the paper "A Universal Algorithm for Seq" published in 1977 in IEEE Transaction on Information Theory.
Found in uential Data Compression '. This is commonly called the Lempel-Ziv encoding slide dictionary method or the LZ77 method.

Further, there is the LZSS method which has two major modifications to the LZ77 method. This is Storer and S
'Data Compression' announced in 1982 by zymanski
Seen in via Textual Substitution ', commonly known as LZS
It is called the S (Lempel-Ziv- Storer-Szymanski) method,
This is a performance improvement during data retrieval. But,
According to the LZ77 method, when the coding of one continuous data string is completed, the next step is to remove the coded dictionary data of the number of bytes from the head of the dictionary buffer.

For this reason, the data compression rate is lowered due to the fact that the dictionary data is duplicated in the dictionary buffer, or even the dictionary data that has a history of coincidence in the past is always ejected from the dictionary buffer. Therefore, without simply pushing out the matched data or data string from the beginning of the dictionary buffer, the method of sliding the dictionary data is devised, the number of reference dictionaries is substantially increased, and duplicate dictionary data is not stocked. There is a demand for an apparatus and method that can effectively use the expelled dictionary data and increase the data compression rate.

[0006]

2. Description of the Related Art FIGS. 12 and 13 are explanatory views of a data compression method according to a conventional example. FIG. 12A is a configuration diagram of the data compression device, and FIG. 12B is a state diagram at the time of data compression. 13 (A) to 13 (C) are state diagrams of the encoding process for explaining the problem.

For example, a data compression device to which LZ77 (slide dictionary method) is applied comprises an original data file 1, a data conversion device 2 and a compressed data file 3 as shown in FIG. 12 (A). The data converter 2 is the input buffer 2
A, a dictionary buffer 2B, and a central processing unit (hereinafter referred to as CPU) 2C. The data structure of LZ77 is shown in Figure 12.
As shown in (B), the encoded input data string is configured by a certain memory range, and this is stored in the dictionary buffer 2B. The dictionary data stored in the dictionary buffer 2B is
The compressed data is transferred to the compressed data file 3 without being left as a dictionary after the compressed data. The dictionary buffer 2B in the initial state may be filled with a value suitable for the data structure.

The function of the device is that the original data read from the original data file 1 is written in the input buffer 2A of the data conversion device 2 as the input data Din. Next, the CPU 2C compares the data written in the buffer 2A with the dictionary data stored in the dictionary buffer 2B in order to search for data. This dictionary data is
Input data is used. Generally, data retrieval is performed from the head position of the previously stored data, and the dictionary buffer 2B
The longest matching data string is searched in.

As a result of the data comparison by the CPU 2C, when a portion matching the dictionary data and the input data is searched, the longest matching data is encoded and compressed, and this compressed data Dout is a compressed data file. 3 is stored. Thus, the previously encoded input data string can be used as dictionary data, the longest match between the dictionary data string and the continuous data portion in the input data string can be searched, and the match information can be encoded. (LZ77 method).

That is, the LZ77 method is based on the input buffer 2A and the dictionary buffer 2B defined by a certain memory range.
This is a method of searching for the same data part in the data string in and compressing this same data part. The longest match data at this time is, as shown in FIG. 12B, that both data are consecutively stored in the dictionary buffer 2B and the input buffer 2A from a match start position (address or offset) at which both data start to match. It is defined by the maximum length of the data before it no longer matches (usually expressed in bytes). As the dictionary data, the data immediately after being matched in the input buffer 2A is transferred to the dictionary buffer 2B.

Specifically, in FIG. 12B, the match start position is "2", and the longest match data = u, i, m,
a and d are "5" bytes. In the input buffer 2A, the following data = f continuous with the matching data = u, i, m, a, d exists. One continuous data string =
When the encoding of u, i, m, a, and d is completed, as shown in FIG.
5 bytes and 6 bytes of dictionary data = a, n, u, i, m, a corresponding to the next data = 1 byte are flushed from the head of the dictionary buffer 2B, and then the flushed 6 bytes Input buffer 2 to replenish the data of
6 bytes of data = u, i, m, a, d, f are supplemented as dictionary data from A to the dictionary buffer 2B. Since the window of the dictionary buffer 2B appears to have moved in this way, it is called the slide dictionary method.

[0012]

By the way, according to the conventional example, as shown in FIG. 12 (B), when the coding of one continuous data string = u, i, m, a, d ends, As the next step, the matched data string = 5 bytes, and the next data = 6 bytes of dictionary data corresponding to 1 byte =
a, n, u, i, m, a are expelled from the head of the dictionary buffer 2B (slide dictionary method of LZ77 method). Therefore, there are the following problems.

The same dictionary data may be duplicated in the dictionary buffer 2B, and the data compression rate is reduced. For example, if the data strings “a, b, c” match in the data retrieval state before encoding as shown in FIG. 13 (A), the modified LZ77 method is as shown in FIG. 13 (B). In addition, the number of bytes of the data string matched from the beginning of the dictionary buffer 2B unconditionally, for example, only 3 bytes, dictionary data =
"X, y, z" are kicked out. Therefore, Fig. 13 (C)
In the state after encoding as shown in, the data string "a, b, c" remains in the dictionary buffer 2B in an overlapping manner.

In order to increase the data compression rate, it is possible to expand the memory area of the dictionary buffer 2B and expand the search range. However, in general, when the size of the dictionary buffer 2B is increased, much search time is required. Further, if the size is increased, it is necessary to extend the data length of the position information regarding the data to be encoded. Further, in this method, even dictionary data having a past coincident record is always expelled from the dictionary buffer 2B.

The present invention was created in view of the problems of the conventional example, and devises a method for sliding dictionary data without simply pushing out the matched data or data string from the beginning of the dictionary buffer, and makes reference. A data processing device and a data processing capable of effectively increasing the number of dictionaries and effectively using the expelled dictionary data without stocking duplicate dictionary data and increasing the data compression rate. The purpose is to provide a method.

[0016]

1 to 11 show an embodiment of a data processing apparatus and a data processing method according to the present invention. The first data processing device, as shown in FIG.
Equipped with a dictionary buffer that stores dictionary data or a dictionary data string that uses compressed data, sequentially compares the input compressed data and encodes the matching dictionary data or dictionary data string, and vice versa. When decoding the compressed data that has been compressed, the dictionary data or dictionary data string is compared with the input compressed data or compressed data string and the dictionary data or dictionary data string and the compressed data or When the compressed data string matches, the dictionary data or dictionary data string of the matched part is stored in the dictionary buffer 12B.
The data write range of the dictionary buffer 12B from which the dictionary data or the dictionary data string has been flushed is packed in one direction, and the matched compressed data or compressed data string is stored in the dictionary buffer 12B filled with the data write range. Is provided as dictionary control means 14 for writing as new dictionary data.

The second data processing apparatus of the present invention, as shown in FIG. 1, is an auxiliary dictionary buffer 12C for storing the data or dictionary data string that has been expelled from the dictionary buffer 12B.
Is provided. As shown in FIG. 1, the third data processing device of the present invention is a fixed dictionary buffer in which fixed data or fixed data sequence having a high frequency of appearance, which has been investigated in advance in the compressed data, is written as dictionary data.
12D is provided.

The fourth data processing apparatus of the present invention is shown in FIG.
As shown in (A), the dictionary buffer 12B has a ring structure in which memory areas for writing compressed data as dictionary data are connected in a non-terminal loop. A first data processing method of the present invention compares dictionary data or a dictionary data string using compressed data with sequentially input compressed data and encodes a matching data or data string, and vice versa. , Step P of the process flowchart of FIG. 3 when decoding the encoded compressed data
In 3, the dictionary data or dictionary data string is compared with the input compressed data or compressed data string, and then in step P4 the dictionary data or dictionary data string and the compressed data or compressed data If the columns match, in step P8, the dictionary data or dictionary data string of the matched portion is expelled from the data writing range of the dictionary,
The data writing range of the dictionary from which the data or data string is ejected is packed in one direction, and the matching compressed data or compressed data string is written as new dictionary data in the dictionary whose data writing range is packed. Characterize.

The second data processing method of the present invention, when comparing the dictionary data or dictionary data string with the input compressed data or compressed data string, step P2 of the processing flowchart of FIG. And the dictionary buffer 12B
It is characterized in that it refers to the data or dictionary data string that has been evicted from. A third data processing method according to the present invention, when comparing the dictionary data or dictionary data string with the input compressed data or compressed data string,
In step P3 of the processing flowchart of (B), the fixed data or the fixed data sequence having a high appearance frequency that is previously investigated in the compressed data is referred to.

A fourth data processing method according to the present invention is shown in FIG. 11B when comparing the dictionary data or dictionary data string with the input compressed data or compressed data string.
As shown in, the compressed data or the compressed data string written in the input memory area consecutive to the dictionary memory area in which the dictionary data or the dictionary data string is written is regarded as the dictionary data or the dictionary data string. The above-mentioned object is achieved.

[0021]

Next, the operation of the first data processing apparatus of the present invention will be described with reference to FIG. In FIG. 1, when the compressed data is input to the input buffer 12A and the dictionary data is written to the dictionary buffer 12B within a certain memory range,
This dictionary data or dictionary data string and the input compressed data or compressed data string are compared by the dictionary control means 14, and as a result, the dictionary data or dictionary data string and the compressed data or compressed data string are If they match,
The dictionary data or the dictionary data string of the matched portion is expelled from the dictionary buffer 12B by the dictionary control means 14.

The data writing range of the dictionary buffer 12B from which this dictionary data or dictionary data string has been flushed is packed in one direction (head direction), and the dictionary buffer 12B packed with this data writing range is input from the input buffer 12A. The compressed data or the compressed data string that coincides with is written as new dictionary data (first data processing method). Therefore, since the dictionary data or the dictionary data string that overlaps the matched data or the data string is driven out of the data writing range of the dictionary buffer 12B to the outside or the like, data that is not related to the matched data is merely There is no need to push out from the beginning of the dictionary buffer 12B. That is, it is not necessary to store the duplicate dictionary data in the dictionary buffer 12B.

As a result, different types of dictionary data can always be stored in the dictionary buffer 12B, the redundancy of the dictionary buffer 12B is reduced as compared with the conventional example, and the data compression rate can be increased. . According to the second data processing apparatus of the present invention, the auxiliary dictionary buffer 12C is provided as shown in FIG. 1, and the data or dictionary data string displaced from the dictionary buffer 12B is stored in the buffer 12C.

Therefore, the number of dictionaries that can be referred to is substantially reduced by storing the dictionary data or the dictionary data string having the past matching record in the auxiliary dictionary buffer 12C without changing the memory capacity of the dictionary buffer 12B. You can increase. The auxiliary dictionary buffer 12C can be referred to when there is no dictionary data in the dictionary buffer 12B. That is, as shown in the processing flowchart of FIG. 7, the dictionary data or the dictionary data string and the input compressed data or the compressed data are referred to while referring to the data or the dictionary data string expelled from the dictionary buffer 12B in step P2. The columns can be compared with each other, and it is possible to improve the probability of compressing data that has been encoded as inconsistent as the original data (second data processing method).

As a result, the dictionary data expelled from the dictionary buffer 12B can be effectively used, and the data compression can be performed by the moving window using the dictionary buffer 12B and the auxiliary dictionary buffer 12C together. According to the third data processing device of the present invention, as shown in FIG. 1, the fixed dictionary buffer 12D is provided, and the fixed data or fixed data string having a high appearance frequency, which has been investigated in advance in the compressed data, is stored in the dictionary. The data is written in the buffer 12D.

Therefore, the dictionary data registered in the fixed dictionary buffer 12D as having a high appearance frequency is compared with the fixed data or the fixed data string in the compressed data, so that the data retrieval speed is increased. Can be achieved. That is, as shown in the processing flow chart of FIG. 9B, the dictionary data or the dictionary data string is referred to while referring to the fixed data or the fixed data string having a high appearance frequency that has been investigated in advance in the compressed data in step P3. , It is possible to compare the input compressed data or the compressed data string (the third data processing method of the present invention).

As a result, the data retrieval becomes faster than in the conventional example, and the data compression processing can be speeded up. According to the fourth data processing apparatus of the present invention, the dictionary buffer 12B having a ring structure as shown in FIG. 10 (A) is provided. Therefore, the previously encoded input data string can be written as dictionary data in the memory areas connected in a non-terminal loop, and the number of dictionaries that can be referred to can be substantially increased.

Utilizing this, the longest match with a continuous portion in the input data string can be searched and the match information can be encoded, and the moving window in which the dictionary buffer 12B of the first device is extended is searched. This makes it possible to compress the data.
According to the fourth data processing method of the present invention, as shown in FIG. 11B, the compressed data or the compressed data string written in the input memory area continuous to the dictionary memory area is the dictionary. It is regarded as data or a dictionary data string.

Therefore, the number of dictionary data that can be referred to can be substantially increased, and the dictionary data or the dictionary data string can be expanded to the input memory area to search for the matching data or data string. . As a result, it becomes possible to compare compressed data or compressed data strings in the input memory area, and the data search becomes faster than in the conventional example, so that it is possible to speed up the data compression process. Becomes

[0030]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, each embodiment of the present invention will be described with reference to the drawings. 1 to 11 are diagrams illustrating a data processing device and a data processing method according to an embodiment of the present invention. (1) Description of First Embodiment FIG. 1 is a configuration diagram of a data processing device according to each embodiment of the present invention, and FIG. 2 is an explanatory diagram of a dictionary buffer during data compression according to the first embodiment. Is. FIG. 3 is a data compression flowchart, and FIG. 4 is an explanatory diagram of encoding processing at the time of data compression according to each embodiment. FIG. 5 shows a data restoration flowchart.

For example, as shown in FIG. 1A, a data compression or decompression device in which the first to third devices of the present invention are combined is an original data file 11, a memory 12 and an EPRO.
An M13, a dictionary control means (hereinafter referred to as CPU) 14, a keyboard 15, a display 16 and a compressed data file 17. That is, the original data file 11 is a memory that stores original data at the time of compression or decompression. A magnetic disk device or a semiconductor memory device is used for the file 11. The memory 12 temporarily stores dictionary data and compressed data during compression. For example, memory 1
2 comprises an input buffer 12A, a dictionary buffer 12B, an auxiliary dictionary buffer 12C and a fixed dictionary buffer 12D. A memory that can be written / read at any time is used as the memory 12.

The input buffer 12A temporarily stores the compressed data. The dictionary buffer 12B stores dictionary data using the compressed data. The auxiliary dictionary buffer (hereinafter also referred to as the matching dictionary buffer) 12C stores the data or the dictionary data string displaced from the dictionary buffer 12B. The matching dictionary buffer 12C is used in the second embodiment of the present invention (second device).

The fixed dictionary buffer 12D is a buffer for writing fixed data or a fixed data string having a high frequency of appearance, which has been investigated in advance in the compressed data, as dictionary data. The fixed dictionary buffer 12D is used in the third embodiment of the present invention (third device). The memory 12 temporarily stores the dictionary data, the dictionary data string or the restored data, and the restored data string when the data is restored.

The EPROM 13 is a programmable read-only memory that stores the control algorithm used in each embodiment. For example, in the first embodiment, the data compression algorithm as shown in FIG. 3 and the data decompression algorithm as shown in FIG. 5 are stored. In the second embodiment, a matching dictionary creation algorithm as shown in FIG. 7 and a data restoration algorithm as shown in FIG. 8 are stored. In the third embodiment, a data compression algorithm as shown in FIG. 9B and a data decompression algorithm as shown in FIG. 9C are stored. The specific contents of this control algorithm will be described in each embodiment.

The CPU 14 is an example of dictionary control means,
The input / output of the input buffer 12A, the dictionary buffer 12B, the auxiliary dictionary buffer 12C, and the fixed dictionary buffer 12D is controlled.
For example, the CPU 14 compares the dictionary data or the dictionary data string in the dictionary buffer 12B with the compressed data or the compressed data string sequentially input to the input buffer 12A. As a result, when the dictionary data or the dictionary data string matches the compressed data or the compressed data string, the CPU 14
The dictionary data or the dictionary data string of the matched portion is expelled from the dictionary buffer 12B.

Thereafter, the CPU 14 unidirectionally packs the data writing range of the dictionary buffer 12B from which the dictionary data or the dictionary data string has been driven out, and the compressed data or the compressed data that matches the dictionary buffer 12B whose data writing range is packed. Write the data string as new dictionary data. The keyboard 15 is a tool for inputting fixed data or the like having a high appearance frequency as a control statement. The display 16 is the keyboard 1
5 and the CPU 14 are tools for assisting input / output. The compressed data file 17 is a memory that stores compressed data at the time of compression or restoration. For the file 17, a magnetic disk device or a semiconductor memory device similar to the file 11 is used.

Thus, the apparatus is configured, and the dictionary data or the dictionary data string using the compressed data is sequentially compared with the input compressed data, and the matching data or data string is encoded, and the inverse data is encoded. In addition, the encoded compressed data can be decoded. Next, regarding the data compression method according to the first embodiment of the present invention, the operation of the apparatus will be described with reference to the processing flowchart of FIG.
FIG. 3 is a data compression flowchart according to the first embodiment of the present invention, which constitutes a control algorithm stored in the EPROM 13 shown in FIG.

For example, when the dictionary data or the dictionary data string using the compressed data is compared with the sequentially input compressed data to encode the matching data or data string, in the flowchart of FIG. First, in step P1, the dictionary buffer 12B is initialized. At this time, due to the initialization, the inside of the dictionary buffer 12B becomes the state of data = “zero”.

Next, in step P2, the original data string is read from the file 11 into the input buffer 12A. At this time, the uncoded data is moved to the head position of the input buffer 12A. For example, as shown in FIG. 2 (A), several bytes of compressed data Din = a, b, c, x, y, z, r,
e, w ... Are input to the input buffer 12A. The compressed data Din from the input buffer 12A is shifted to the dictionary buffer 12B, and as a result, n bytes of dictionary data = x, y,
z, d, g, k, g ... A, b, c, u are dictionary buffers 12
Written to B.

Next, in step P3, the dictionary buffer 12B
It is searched whether there is a matching data string in. For example, the CPU 14 compares the dictionary data string = a, b, c stored in the dictionary buffer 12B with the compressed data string = a, b, c in the input buffer 12A. Then, in step P4, it is determined whether or not the dictionary data string and the compressed data string match. At this time, if there is a matching data string (YES), the process proceeds to step P5. If there is no matching data string (NO), the process proceeds to step P6, the original data is encoded as it is, and the process proceeds to step P7.

In step P4, since the dictionary data string = compressed data string = a, b, c, it is judged that there is a matching data string, and the matching information is encoded in step P5.
Specifically, the position information of the matching data, its length information, etc. are encoded. The encoding at this time will be described with reference to FIG. Then, in step P7, the encoded compressed data is stored in the file 17.

Then, the input data string encoded in step P8 is moved to the dictionary buffer 12B. At this time, CP
As shown in FIG. 2B, U14 pushes out the dictionary data string = a, b, c of the matched portion (overlapping portion) from the data writing range of the dictionary buffer 12B, and the dictionary in which this data row is pushed out. The data writing range of the buffer 12B is reduced toward the head. Specifically, in FIG. 2 (A),
The data string on the right from the storage position 0 of the dictionary buffer 12B is shifted to the left.

Further, as shown in FIG. 2B, the CPU 14 matches the compressed data string in the dictionary buffer 12B in which the data write range is filled from the input buffer 12A =
Write a, b, and c as new dictionary data. At this time, unlike the conventional example, x, y, z stored at the head position of the dictionary buffer 12B remain as dictionary data as they are, as shown in FIG. 2C, and a new dictionary from the input buffer 12A is used. As data, the compressed data string = a, b, c is stored in the last part of the dictionary buffer 12B. As a result, the first data string = x, y, z does not have to be expelled, and is used as dictionary data for the next stage.

Then, in step P9, it is determined whether or not all the original data has been compressed. At this time, when all the original data are compressed (YES), the control algorithm according to the first embodiment is ended, and when all the original data are not compressed (NO), the process returns to step P2. , The original data string is read from the file 11 into the input buffer 12A, and the following steps are continued.

Thus, the dictionary data string using the compressed data and the input compressed data can be sequentially compared and the matching data string can be encoded. Here, a specific encoding method will be described. In the embodiment of the present invention, the LZSS method is improved in order to increase the compression rate.
The data compression rate is [original data / compressed data] x 100
It is expressed in%.

When the matching data searched in the first embodiment of the present invention is converted into compressed data as shown in FIG. 4A, the position information of the searched data and the length of the matched data are introduced. To do. Generally, at the end of the dictionary buffer 12B,
Utilizing the fact that there are many cases where data match near the beginning position of the input buffer 12A (hereinafter referred to as neighborhood matching), an identifier is provided at the beginning of the compressed data, and the position information for data retrieval is added. In the position information at this time, "0" is written as a short position information value and "1" is written as a long position information value.

For example, FIG. 4A shows the dictionary buffer 12B.
Shows the data format in which the position information in the case of the size of 2 KB is added to the compressed data. In FIG. 4A, in the embodiment of the present invention, the data length is 7 bits (0 to 0).
In the case of 63), "0" is written as the short position information value, and when the data length is 12 bits (0 to 2047), "1" is written as the long position information value. In practice, it is necessary to make adjustments that are most suitable for the type of data to be compressed, such as in what range the neighborhood match is set, or by providing a plurality of position information increments.

When the number of matching bytes is encoded, a code having a high appearance frequency is represented by a bit having a short length. This is in consideration of encoding about the length of data by allocating a short code to the number of short matching data, paying attention to the fact that there are generally many matching data with a short bit length. For this purpose, a code tree as shown in FIG. 4 (B) is used. In the code tree of FIG. 4B, when “0” is written in the short position information value,
“0,0” and = “0,1” are the targets, which is shown in FIG.
As shown in (C), it means 2-byte match or 3-byte match. FIG. 4C shows the relationship between the length of compressed data composed of a code tree and the corresponding bit string.

Further, in FIG. 4B, when "1" is written in the long position information value, "= 1, 0,
0 ”, =“ 1,0,1 ”, = 1,1,0,0”,
= “1,1,0,1”, = “1,1,1,0,0” and = “1,1,1,0,1” are targets, which are as shown in FIG. And 4 to 9 bytes match, respectively.

In this method, by assigning a small number of bits to data with a high appearance probability, even if different numbers of bits are assigned to data with a low appearance probability, Shannon-Fano encoding or This is a method known as Huffman coding. It should be noted that it is necessary to make adjustments because how many bytes coincide with each other differ depending on the type of compressed data.

Next, the compressed data decompression process according to the first embodiment of the present invention will be described. For example, when decoding the matching data or data string by sequentially comparing the dictionary data or dictionary data string using compressed data with the input decoded data, in the decoding flowchart of FIG. The dictionary buffer 12B is initialized at P1. Next, in step P2, the compressed data is read from the file 17 into the input buffer 12A. At this time, the undecoded data is moved to the head position of the input buffer 12A.

Next, in step P3, the decrypted information is analyzed and returned to the original data. For example, the CPU 14 compares the dictionary data string stored in the dictionary buffer 12B with the decoded data string in the input buffer 12A. Then, in step P4, the decrypted original data is stored in the file 11. Next, the input data string decoded in step P5 is moved to the dictionary buffer 12B. At this time, the CPU 14
Removes the dictionary data string of the matched portion (overlapped portion) from the data writing range of the dictionary buffer 12B, and closes the data writing range of the dictionary buffer 12B from which this data string is ejected in the head direction.

Further, the CPU 14 transfers data from the input buffer 12A to the dictionary buffer 12B filled with the data writing range.
The matched decoded data string is written as new dictionary data. Then, in step P6, it is determined whether or not all the compressed data has been restored. At this time, if all the compressed data have been compressed (YES), the control algorithm ends, and if all the compressed data have not been compressed (NO), the process returns to step P2 and the decoded data string is stored in the file. Read from 17 into the input buffer 12A and continue the following steps.

Thus, the dictionary data string using the restored data and the input restored data can be sequentially compared and the matching data string can be decoded. In this way, according to the data processing apparatus in the first embodiment of the present invention, as shown in FIG. 1, when the dictionary data string and the compressed data string are compared and both data strings match, A CPU 14 is provided which drives out the dictionary data string of the matched portion from the dictionary buffer 12B, and writes the compressed data string matched to the dictionary buffer 12B in which the data writing range is narrowed in one direction as new dictionary data.

Therefore, the dictionary data string in the dictionary buffer 12B overlapping the input data string is expelled from the data writing range to the outside or the like. That is, assuming that the data strings "a, b, c" match in the data search state before encoding as shown in FIG. 2A, the embodiment of the present invention shows the same as shown in FIG. 2B. Thus, the dictionary data of the data writing range of the dictionary buffer 12B = “a, b, c” = 3 bytes is expelled.

As a result, unlike the conventional example, the data "x, y, z" stored at the head position of the dictionary buffer 12B, which is not related to the matching data, is not simply expelled from the dictionary buffer 12B. As a result, in the state after encoding as shown in FIG. 2C, the data strings "a, b, c" do not remain in the dictionary buffer 12B in an overlapping manner. Generally, when the dictionary data increases, the size of the dictionary buffer itself increases and the compression rate decreases. However, in the embodiment of the present invention, even if the number of dictionary data substantially increases, the memory capacity of the dictionary buffer itself, and The major feature is that the code bit after compression is not increased.

In the embodiment of the present invention, different types of dictionary data can always be stored in the dictionary buffer 12B, the redundancy of the dictionary buffer 12B is reduced as compared with the conventional example, and the data compression rate is reduced. It is possible to raise it. (2) Description of Second Embodiment FIG. 6 is an explanatory diagram of a data processing device according to a second embodiment of the present invention, and FIG. 6A is an explanatory diagram of the matching dictionary buffer thereof. FIG. 6B is an explanatory diagram of the encoding of the matching data, and FIG. 6C is an explanatory diagram of the search range of the matching data.

In the second embodiment, unlike the first embodiment,
When comparing the dictionary data sequence and the compressed data sequence, the dictionary data sequence expelled from the dictionary buffer 12B is referred to. That is, in the second data processing device of the present invention, as shown in FIG. 1, the matching dictionary buffer 12C that stores the data or the dictionary data string that is expelled from the dictionary buffer 12B is used. As the match dictionary buffer 12C, for example, as shown in FIG. 6A, a buffer having a ring structure having a certain memory capacity is used. The circular structure buffer will be described in detail in the fourth embodiment of the present invention.

The coincident dictionary buffer 12C sequentially overwrites the old data, which has been previously expelled from the dictionary buffer 12B, with the data, which has been newly expelled from the dictionary buffer 12B. For such a method, the reference frequency may be counted and the reference frequency may be overwritten. This is effective when it is desired to increase the compression rate even if the processing speed is sacrificed.

A reference pointer and a storage pointer are set in the matching dictionary buffer 12C. The reference pointer indicates the position where the matching dictionary buffer 12C is referenced. The storage pointer indicates the position for storing the current dictionary data that has been flushed from the dictionary buffer 12B. The reference pointer is placed before the storage pointer. This is to retrieve the newest dictionary data that has been expelled from the dictionary buffer 12B.

The matching dictionary buffer 12C is the dictionary buffer 12B.
Uses a larger memory capacity than the matching dictionary buffer 12
The dictionary data of C and the dictionary data of the dictionary buffer 12B are
Avoid duplication. This is because the matched data string displaced from the dictionary buffer 12B moves from the input buffer 12A, so that the matched data string does not disappear from the dictionary buffer 12B immediately.

In FIG. 6A, the shaded area indicates the range in which the dictionary data is searched in the matching dictionary buffer 12C. Although this search range is limited by the encoding of the compressed data, the embodiment of the present invention assumes a compressed data file with the largest number of 2-byte matches.
Therefore, the match dictionary buffer 12C has a structure for storing only 2-byte match data.

FIG. 6B shows a data format in which the position information when the size of the matching dictionary buffer 12C is 2 KB is added to the compressed data. In FIG. 6B, similar to the first embodiment of the present invention, when the data length is 12 bits (0 to 2047), “1” is written as the long position information value. In practice, it is necessary to make adjustments that are most suitable for the type of data to be compressed, such as in what range the neighborhood match is set, or by providing a plurality of position information increments. Under this condition, the correspondence with the matching dictionary buffer 12C is made without increasing the sign bit, which is a feature of the present invention.

Specifically, as shown in FIG. 6B, the value of the short position information in the long position information, 0 to 1 in this case.
The portion overlapping 27 is assigned. That is, 0
The value of 127 does not appear in long position information because it is encoded with short position information. This is the focus of attention. For example, the first in the matching dictionary is a 1-bit identifier = "1" and an 11-bit codeword "000 0000.
"0000" is written, and the second in the matching dictionary is 1
Bit identifier = "1" and 11-bit codeword "00
0 0000 0001 ”is written, and similarly, in the third position of the matching dictionary, 1-bit identifier =“ 1 ”and 1
The 1-bit codeword “000 00000010” is written.

As a concrete example, FIG. 12C described above is used.
Data "r, e" in the input buffer 2A after the encoding of "2" and the second "r, e" in the matching dictionary buffer 12C in FIG. 6A.
, And the code word “1000 0000 0
A bit string of "001" will be generated. In this case, since the length is fixed, the length code is not generated. That is, since the total identification bit is 13 bits including the entire identification bit, it is compressed by 3 bits rather than 2 bytes (16 bits) when the original data does not match.

As described above, in the example of FIG. 6B, the matching dictionary buffer 12C can hold 128 dictionary data. It should be noted that the size of the matching dictionary buffer 12C has 256 bits, for example, as shown in FIG. 6C, in order to search the dictionary data that extends from the dictionary buffer 12B as much as possible. When searching for this, always search earlier than 128 bits from the current storage point.

As a result, the number of dictionary data is substantially increased, and it is possible to perform data compression by the moving window together with the matching dictionary buffer 12C. Next, regarding the data processing method according to the second embodiment of the present invention, the operation of the apparatus will be described with reference to the processing flowchart of FIG. Figure 7
2 is a flowchart for generating a matching dictionary according to the second embodiment of the present invention, which constitutes a control algorithm stored in the EPROM 13 shown in FIG.

For example, in the case of creating a matching dictionary using the dictionary data expelled from the dictionary buffer 12B, in the flowchart of FIG. 7, first, in step P1, was there a matching data portion in the dictionary buffer 12B? Determine whether or not. If there is a match at this time (YES)
For that, the process proceeds to step P3. If there is no matching portion (NO), the process moves to step P2 to search the matching dictionary buffer 12C, and then the processing ends without generating the matching dictionary.

When there is a coincident portion in step P1 (Y
For ES), it is determined in step P3 whether the two bytes match. At this time, if the two bytes match (YES), the process proceeds to step P4. If the two-byte match is not found (NO), the process ends without generating the match dictionary. If there is a match of 2 bytes in step P3 (YES), it is determined in step P4 whether the same dictionary data exists in the matching dictionary buffer 12C. At this time, if the same dictionary data exists (YES), the process ends without generating the matching dictionary.

If there is no same dictionary data in the matching dictionary buffer 12C in step P4 (NO), step P5
Then, the dictionary data expelled from the dictionary buffer 12B is stored in the matching dictionary buffer 12C. After that, the process proceeds to step P6 to advance the storage pointer to the next bit,
To finish. As a result, a matching dictionary can be created using the dictionary data that has been expelled from the dictionary buffer 12B, and when the dictionary data string is compared with the input compressed data string, it will be expelled from the dictionary buffer 12B. It is possible to refer to the dictionary data string that has been created.

Next, the compressed data decompression process according to the second embodiment of the present invention will be described. FIG. 8 is a flowchart for decompressing compressed data according to the second embodiment of the present invention, which constitutes the control algorithm stored in the EPROM 13 shown in FIG. For example, when decoding the data to be decoded while referring to the dictionary data in the matching dictionary buffer 12C, first, in step P1, it is determined whether or not there is a reference code in the matching dictionary buffer 12C. At this time, if there is a reference code (YES), the process moves to step P2 to refer to the matching dictionary buffer 12C and decode the restored data. After that, the process ends without generating the matching dictionary.

If there is no reference code in step P1 (NO), it is determined in step P3 whether the two bytes match. At this time, if the two bytes match (YES), the process proceeds to step P4. If the two-byte match is not obtained (NO), the process proceeds to step P7, the normal encoding process is executed without generating the match dictionary, and then the process ends.

If two bytes match in step P3 (YE
In step S4, it is determined whether or not the same dictionary data exists in the matching dictionary buffer 12C in step P4. At this time, if the same dictionary data exists (YES), the process proceeds to step P7, the normal encoding process is executed without generating the matching dictionary, and then the process ends. If the same dictionary data does not exist in the matching dictionary buffer 12C in step P4 (NO), the process moves to step P5 and the dictionary data expelled from the dictionary buffer 12B is stored in the matching dictionary buffer 12C. Then, the process proceeds to step P6, where the storage pointer is advanced to the next bit, and further the process proceeds to step P7 where the normal encoding process is executed without generating the matching dictionary, and then the process ends.

As a result, the matching dictionary can be created by using the dictionary data expelled from the dictionary buffer 12B, and the decoded data can be decoded while referring to the dictionary data in the matching dictionary buffer 12C. it can. In this way, according to the data processing apparatus in the second embodiment of the present invention, the matching dictionary buffer 12C as shown in FIG. 1 is provided, and the dictionary data string expelled from the dictionary buffer 12B is stored in the buffer. It is stored in 12C.

Therefore, the number of dictionaries that can be referred to is substantially reduced by storing the dictionary data or the dictionary data string having a past matching record in the matching dictionary buffer 12C without changing the memory capacity of the dictionary buffer 12B. You can increase. Further, in the embodiment of the present invention, even if the number of dictionaries is substantially increased, the size of the dictionary itself left to be encoded and the code bits after compression do not increase.

The matching data string extending from the dictionary buffer 12B can refer to the matching dictionary buffer 12C as an auxiliary dictionary. That is, as shown in the processing flowchart of FIG. 7, the dictionary data string can be compared with the input compressed data string while referring to the dictionary data string that has been expelled from the dictionary buffer 12B in step P2. It is possible to improve the compression rate of data that has been encoded as inconsistent with the original data.

As a result, the dictionary data expelled from the dictionary buffer 12B can be effectively used, and the data can be compressed by the moving window using the dictionary buffer 12B and the auxiliary dictionary buffer 12C together. In addition, it is possible to perform data compression by using dictionary data that has a past record of coincidence, and it is possible to increase the data compression rate compared to the conventional example.

(3) Description of Third Embodiment FIG. 9 is an explanatory diagram of a data processing method using a fixed dictionary according to a third embodiment of the present invention, and FIG. It is a content diagram of a dictionary buffer. 9B is a flowchart for data compression, and FIG. 9C is a flowchart for data decompression.

The third embodiment differs from the first and second embodiments in that, when comparing a dictionary data string and a compressed data string, a fixed data with a high frequency of appearance that has been investigated in advance in the compressed data. It refers to data or a fixed data string. That is, in the third data processing device of the present invention, as shown in FIG. 1, the fixed dictionary buffer 12D for storing fixed data or fixed data string having a high appearance frequency which has been investigated in advance in the compressed data is used. To do. The fixed dictionary buffer 12D is
For example, as shown in FIG. 9 (A), fixed data such as characters and symbols having a high frequency of occurrence, such as Roman letters “i, f”, arithmetic symbols “+, =” and descriptive symbols “),)”, are stored. It is a memory that is stored as a fixed dictionary.

As the fixed dictionary, there are two methods: a method held in the compression / decompression control program in the EPROM 13 and a method added to the encoded compressed data. In order to have a fixed dictionary in advance, it is necessary to investigate a data type and a data string having a high appearance frequency. When determining the type of the fixed data, a method of determining from the file extension or a method of specifying by the user via the keyboard 15 is used.

For a data string having a high appearance frequency, for example, which pattern has a large number of 2-byte matches is checked beforehand by an auxiliary tool. In this case, as compared with the method of unconditionally storing regardless of the appearance frequency, a method of fixing only the number of matching bytes having a high appearance frequency is practical from the viewpoint of the memory used and the processing speed. The fixed dictionary creation tool may be connected as a preprocessor of the data compression tool.

Next, the data processing method according to the third embodiment of the present invention will be described with reference to the fixed dictionary reference flowchart at the time of compression in FIG. 9B. For example, fixed dictionary buffer
When data compression is performed using the fixed dictionary data of 12D, in the flowchart of FIG. 9B, first, in step P1, the dictionary data in the dictionary buffer 12B is searched.

Next, in step P2, it is determined whether or not there is a matching data portion. At this time, if there is a matching portion (YES), the dictionary data of the dictionary buffer 12B is used to perform data compression without referring to the fixed dictionary buffer 12D. If there is no matching data portion in step P2 (NO), the process moves to step P3 and the fixed dictionary buffer 12D is searched. As a result, the data can be compressed using the fixed dictionary data in the fixed dictionary buffer 12D.

Next, a description will be given of a fixed dictionary reference flowchart at the time of decompression in the decompression process of compressed data according to the second embodiment of the present invention. For example, when decoding the data to be decoded while referring to the fixed data in the fixed dictionary buffer 12D, the fixed code of the fixed dictionary buffer 12D is first stored in the input buffer 12A in step P1 in the flowchart of FIG. 9C. Determine if there is. At this time, if there is a fixed code in the input buffer 12A (YES),
In step P3, the fixed dictionary buffer 12D is searched to restore the restored data, and then the process is ended.

If there is no fixed code in step P1 (NO), the encoding process is executed in the dictionary buffer 12B in step P2, and then the process ends. As a result, the data to be decoded can be decoded while referring to the fixed data in the fixed dictionary buffer 12D. In this way,
According to the data processing device of the third embodiment of the present invention,
A fixed dictionary buffer 12D as shown in FIG. 9A is provided, and fixed data with a high frequency of appearance, which has been examined in advance in the compressed data, is written in the buffer 12D as dictionary data.

Therefore, the dictionary data registered in the fixed dictionary buffer 12D as having a high appearance frequency is compared with the fixed data in the compressed data, so that the data retrieval speed can be increased. It will be possible. By using this fixed dictionary buffer 12D together with the dictionary buffer 12B, the number of dictionaries is substantially increased, but the size of the dictionary itself left after encoding and the code bits after compression do not increase.

That is, as shown in the processing flow chart of FIG. 9B, the dictionary data and the compressed data are compared with each other while referring to the fixed data having a high appearance frequency which has been investigated in advance in the compressed data in step P3. Can be compared. As a result, the data search is faster than in the conventional example, and the data compression processing can be speeded up. It should be noted that, compared with the second embodiment, the data processing speed is increased in the third embodiment because the process of generating the matching dictionary buffer 12C is unnecessary. The embodiments of the present invention are suitable for applications in which the type of compressed data is specified.

(4) Description of Fourth Embodiment FIG. 10 is an explanatory diagram of a data processing device according to a fourth embodiment of the present invention, and FIG. 10 (A) is a configuration diagram of its circular dictionary buffer. Is. FIGS. 10B and 10C show search state diagrams in the circular dictionary buffer, respectively. The fourth embodiment differs from the first and third embodiments in that the dictionary buffer 12
B has a cyclic structure.

In the fourth data processing device of the present invention, as shown in FIG.
A circular dictionary buffer 12E as shown in (A) is provided. The difference from the dictionary buffer 12B of the first embodiment is that in the circular dictionary buffer 12E of the fourth embodiment, the memory areas for writing the compressed data as dictionary data are connected in a non-terminal loop. By doing this, it is possible to substantially increase the number of dictionary data, and the circular dictionary buffer 12
E can be used to search for a match between consecutive data strings in input buffer 12A. As a result, it becomes possible to perform data compression using the moving window which is an extension of the dictionary buffer 12B of the first embodiment.

That is, the dictionary data in the circular dictionary buffer 12E as shown in FIG.
When encoding the compressed data input to 12A, first, the original data string is read from the file 11 as described in FIG. 1 into the input buffer 12A. At this time, the uncoded data is moved to the head position of the input buffer 12A.

For example, as shown in FIG. 10C, several bytes of compressed data Din = u, i, m, a, d, f, r,
e, w ... Are input to the input buffer 12A. The compressed data Din from the input buffer 12A is the circular dictionary buffer 12E.
, So that n bytes of dictionary data = i,
m, a, d, g, k, g ... A, b, c, u are written in the circular dictionary buffer 12E.

Next, it is searched whether or not there is a matching data string in the circular dictionary buffer 12E. For example, the dictionary data string stored in the circular dictionary buffer 12E = u, i,
m, a, d and the compressed data sequence of the input buffer 12A =
u, i, m, a, d are compared. It is encoded because it matches. Specifically, the position information of the matching data, its length information, etc. are encoded. For example, Figure 10
In the matching data string of (B), since the long position information is shown, the identifier is "1" and the search position is the input buffer 12A.
The position information is “0” and the longest match data is u, i, m, since it is at the head position of the circular dictionary buffer 12E as seen from
Since it becomes a and d, it becomes "5" bytes.

As shown in FIG. 10C, the compressed data of several bytes Din = u, i, m, a, d, u, i,
m, a, d, u, i, m, a, d, x ... are input buffers
When the compressed data Din from the input buffer 12A is shifted to the circular dictionary buffer 12E, the 5-byte dictionary data = i, m, a, d, u is written to the circular dictionary buffer 12E. Be done.

Next, it is searched whether or not there is a matching data string in the circular dictionary buffer 12E. For example, the dictionary data string stored in the circular dictionary buffer 12E = i, m,
a, d, u and the compressed data string of the input buffer 12A =
u, i, m, a, d, u, i, m, a, d, u, i,
m, a, d are compared. This is encoded because it matches in the three data strings. That is, in the fourth embodiment, up to the 15th of the input buffer 12A can be encoded at one time, and the encoded compressed data is stored in the file 17.

Specifically, the position information of the matching data, its length information, etc. are encoded. In the matching data string in FIG. 10C, the identifier is "1" because it indicates long position information, and the position information is "1" because the search position is at the head position of the circular dictionary buffer 12E when viewed from the input buffer 12A. 0 ",
Since the longest match data is u, i, m, a, d, it is "5" bytes. The encoding at this time is shown in FIG.
Please refer to the explanation.

When decompressing the compressed data, the same circular dictionary buffer 12E can be used and the decompression can be performed according to the decompression flowchart similar to that of the first embodiment, and therefore the description thereof is omitted. Thus, the fourth aspect of the present invention
According to the data processing device of the embodiment, the circular dictionary buffer 12E as shown in FIG. 10 (A) is provided.

Therefore, the previously encoded input data string can be written as dictionary data in the memory area connected in a non-terminal loop, and the number of dictionaries that can be referred to can be substantially increased. As a result, in the first embodiment, only the method of searching the dictionary buffer 12B for the dictionary data on a straight line can be adopted, but in the second embodiment, the dictionary buffer is used.
It is possible to search dictionary data in a circular 12E. Utilizing this, the longest match with a continuous portion in the input data string can be searched and the match information can be encoded.

From this, the possibility that the data strings that did not match in the first embodiment will match in the fourth embodiment increases, and data compression by the moving window in which the dictionary buffer 12E is expanded can be performed. It will be possible. (5) Description of Fifth Embodiment FIG. 11 is an explanatory diagram of a data compression method according to a fifth embodiment of the present invention. FIG. 11A is a diagram in which a certain memory area is divided into a dictionary buffer 12B and an input buffer 12A. FIG. 11B shows a diagram in which the boundary line dividing the area is moved to the input buffer 12A side.

In the fifth embodiment, unlike the first to fourth embodiments, the compressed data string written in the input memory area continuous to the dictionary memory area is regarded as the dictionary data string. is there. That is, in the fifth data processing apparatus of the present invention, a boundary (cursor) line 12F when a certain memory area is divided into a dictionary buffer 12B and an input buffer 12A as shown in FIG. The ability to move is added. This function is
The control data is stored as a control algorithm in the CPU, and the CPU 14 reads this data for dictionary control.

In the dictionary control at this time, when comparing the dictionary data string and the compressed data string, as shown in FIGS. 11A to 11B, the boundary line 12F is set in the input memory area. Move in the direction of. The dictionary buffer 12B is the input buffer 12
The dictionary buffer 12B 'is extended to the area A. As a result, a match search can be performed between the dictionary buffer 12B 'and the input buffer 12A, and the compressed data string in the input memory area that is continuous with the dictionary memory area in which the dictionary data string is written is converted into the dictionary. It can be regarded as a data string and the matching data can be searched.

Specifically, in the matching data string of FIG. 11A, the identifier is "1" because it indicates long position information, and the search position is the dictionary buffer as seen from the input buffer 12A.
Since it is in the 3rd bit of 12A, the position information is "3", and the longest match data is u, a, b, c, which is "8" bytes. Note that the device for increasing the compression rate at the concrete encoding stage is the same as that in the first embodiment, and therefore its explanation is omitted.

Further, when decompressing the compressed data, the overlapping portion of the input buffer 12A and the dictionary buffer 12B is set to 1
Restore one by one. In this way, according to the data compression method of the fifth embodiment of the present invention, as shown in FIG. 11B, the data written in the memory area for input that is continuous with the memory area for dictionary is written. The compressed data string is regarded as the dictionary data string.

Therefore, the number of dictionary data that can be referred to can be substantially increased, and the dictionary data or dictionary data string can be expanded to the input memory area to search for the matching data or data string. . As a result, the compressed data strings in the input memory area can be compared with each other, and the possibility that unmatched data will match increases as compared with the search method of the first embodiment. In particular, in the data string in the dictionary buffer 12B, the same data = u, a, as the arrangement of data = u, a, b, c near the boundary line 12F.
When the arrangement of b and c repeatedly appears in the input buffer 12A, the data can be compressed most efficiently.

As a result, the size of the dictionary and the size of the coded information do not increase in spite of the increase in the number of dictionaries, and the data search time can be shortened as compared with the first embodiment. It is possible to speed up the data compression process. Table 1 shows the results of comparing the data compression methods according to the first, fourth and fifth embodiments. Table 1 shows 6339 bytes of binary data and 3 on the software.
This is a result when 177-byte text data is compressed, and shows the number of bytes after compression with respect to the number of original data.

[0105]

[Table 1]

According to this, in the first embodiment, since the dictionary buffer 12B is linearly searched, the number of bytes after compression of the binary data is 4524 bytes. Also,
In the text data, the number of bytes after compression became 1373 bytes. On the other hand, in the fourth embodiment, since the dictionary buffer 12D is searched in a ring shape, the number of bytes after compression of the binary data is 4,522 bytes. In the text data, the number of bytes after compression became 1373 bytes.

In the fifth embodiment, since the input buffer 12A is expanded and searched from the dictionary buffer 12B, the number of bytes after compression of the binary data is 4495 bytes.
In the case of text data, the number of bytes after compression is 1369.
It became a part-time job. As described above, it is clear that the fifth embodiment improves the data compression rate as compared with the first and fourth embodiments, and a large effect can be expected when handling a large amount of data.

[0108]

As described above, according to the data processing apparatus of the present invention, the dictionary data or the dictionary data string is compared with the compressed data or the compressed data string, and these data or the data string are matched. In this case, a dictionary control means is provided for erasing the matched data or data string from the dictionary buffer, and then writing the matched compressed data or compressed data string as new dictionary data by packing the data writing range in one direction. To be

For this reason, the duplicate dictionary data or the dictionary data string is pushed out of the data writing range of the dictionary buffer to the outside or the like, so that, unlike the conventional example, the data not related to the matching data is simply deleted from the beginning of the dictionary buffer. It is possible to store different types of dictionary data in the dictionary buffer all the time, and to reduce the redundancy of the dictionary buffer.

According to another data processing device of the present invention, an auxiliary dictionary buffer for storing the data or dictionary data string displaced from the dictionary buffer is provided. Therefore, the number of dictionaries that can be referred to can be substantially increased, data can be compressed while referencing the dictionary data in the auxiliary dictionary buffer, and the data that was encoded as the original data as a mismatch does not occur. It is possible to improve the compression rate.

According to another data processing apparatus of the present invention, a fixed dictionary buffer is provided for writing fixed data or fixed data string having a high appearance frequency, which has been investigated in advance, in the compressed data as dictionary data. Therefore, the data can be compressed while referring to the dictionary data in the fixed dictionary buffer, and the data search speed can be increased.

According to another data processing device of the present invention, a dictionary buffer having a ring structure is provided. For this reason,
It is possible to perform data compression while sliding the expanded moving window onto the dictionary buffer while referring to the dictionary data written in the non-terminal loop memory area. According to the data processing method of the present invention, the compressed data or compressed data string in the input memory area that is continuous with the dictionary memory area is used as the dictionary data or dictionary data string.

Therefore, the number of dictionary data that can be referred to can be substantially increased, and the dictionary data or the dictionary data string can be expanded to the input memory area to search for the matching data or data string. . As a result, it is possible to configure a data compression or decompression device that can search dictionary data effectively and at high speed, and substantially increase the allowable storage capacity of the magnetic disk device and the transmission time during data transmission. It greatly contributes to the shortening of.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a data processing device according to each embodiment of the present invention.

FIG. 2 is an explanatory diagram of a dictionary buffer during data compression according to the first embodiment of the present invention.

FIG. 3 is a data compression flowchart according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram of encoding processing during data compression according to each embodiment of the present invention.

FIG. 5 is a flowchart for decoding compressed data according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram of a matching dictionary, data encoding, and its search range according to the second embodiment of the present invention.

FIG. 7 is a flowchart of creating a matching dictionary according to the second embodiment of the present invention.

FIG. 8 is a decoding flowchart of compressed data according to the second embodiment of the present invention.

FIG. 9 is an explanatory diagram of a data processing method using a fixed dictionary according to a third embodiment of the present invention.

FIG. 10 is an explanatory diagram of a circular dictionary buffer according to a fourth embodiment of the present invention.

FIG. 11 is an explanatory diagram of a dictionary buffer having an input buffer continuous structure according to a fifth embodiment of the present invention.

FIG. 12 is an explanatory diagram of a data compression method according to a conventional example.

[Fig. 13] Fig. 13 is a state diagram of encoding processing for explaining a problem in the conventional example.

[Explanation of symbols]

 11 ... Original data file, 12 ... Memory, 12A ... Input buffer, 12B ... Dictionary buffer, 12C ... Auxiliary dictionary (matching dictionary) buffer, 12D ... Fixed dictionary buffer, 13 ... EPROM, 14 ... Dictionary control means (CPU), 15 ... keyboard, 16 ... display, 17 ... compressed data file, 18 ... bus.

Claims (8)

[Claims] 1. A dictionary buffer for storing dictionary data or a dictionary data string using compressed data, wherein the dictionary data or the dictionary data string that is sequentially compared with the input compressed data is encoded, Conversely, when decoding the encoded compressed data, the dictionary data or dictionary data string is compared with the dictionary data or dictionary data string and the input compressed data or compressed data string. When the compressed data or the compressed data string matches, the matching part of the dictionary data or the dictionary data string is expelled from the dictionary buffer, and the dictionary data or the dictionary data string is expelled the data writing range of the dictionary buffer In one direction and the matched compressed data or compressed data string is stored in the dictionary buffer filled with the data writing range as a new dictionary data. The data processing apparatus characterized by the dictionary control means for writing the data is provided. 2. The data processing apparatus according to claim 1, further comprising an auxiliary dictionary buffer that stores data or a dictionary data string displaced from the dictionary buffer. 3. The data processing according to claim 1, further comprising a fixed dictionary buffer in which fixed data or a fixed data string having a high frequency of occurrence, which has been investigated in advance in the compressed data, is written as dictionary data. apparatus. 4. The data processing apparatus according to claim 1, wherein the dictionary buffer has a ring structure in which memory areas for writing compressed data as dictionary data are connected in a non-terminal loop. 5. A dictionary data or a dictionary data string using compressed data and sequentially input compressed data are compared to encode a matching data or data string, and vice versa.
When decoding the encoded compressed data, the dictionary data or the dictionary data string is compared with the input compressed data or the compressed data string, and the dictionary data or the dictionary data string and the compressed object are compressed. When the data or the compressed data string matches, the matched dictionary data or dictionary data string is pushed out of the dictionary data writing range, and the data writing range of the dictionary in which the data or data string is pushed out is unidirectional. And writing the matched compressed data or compressed data string as new dictionary data in the dictionary in which the data writing range is narrowed. 6. When comparing the dictionary data or dictionary data string with the input compressed data or compressed data string, refer to the data or dictionary data string evicted from the dictionary buffer. The data processing method according to claim 5, characterized in that 7. Fixed data, which has a high frequency of appearance and has been investigated in advance in the compressed data when comparing the dictionary data or the dictionary data string with the input compressed data or the compressed data string. The data processing method according to claim 5, wherein a fixed data string is referred to. 8. When comparing the dictionary data or dictionary data string with the input compressed data or compressed data string, a dictionary memory area in which the dictionary data or dictionary data string is written is used. The data processing method according to claim 5, wherein the compressed data or the compressed data string written in the continuous memory area for input is regarded as the dictionary data or the dictionary data string.