JPH0936747A - Data compression method and data compressor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the data compression method and the data compressor by which data can be compressed at a high speed without taking mush time for data retrieval by adopting the hash method for data compression by the LZ slide dictionary method so as to retrieve data. SOLUTION: When a data input output device 1 reads compression processing object data to a memory 3, a control circuit 2 retrieves the data string the same as compression processing object data string from a hash table by the hash method and compares the data of the retrieved data string with that of the compression processing object data string one by one and provides an output of equal data length of both the data strings, repeats the retrieval and comparison till the retrieval object is not in existence and selects a data string whose equal data length is longest among data strings at the retrieved hand position and succeeding positions and provides an output of the head position of the selected data string and the equal data length as equality information and registers the head position of the compression processing object data string to a hash table as a retrieval object.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a data compression method and a data compression apparatus for compressing data such as text data.

[0002]

2. Description of the Related Art As an example of a data compression method for compressing text data or the like, there is an LZ (Lempel-Ziv) slide dictionary method. As shown in FIG. 12, this LZ slide dictionary method searches for data to be compressed from the beginning, and when a data string that appears once appears again, the beginning of the previously appeared data string is searched. This is a method of performing data compression by replacing the address and the length of the matching data in both data strings with the matching information and replacing it with the data string that has appeared again.

In the LZ slide dictionary method, various data retrieval methods are used to retrieve whether or not the data string to be compressed is data that has previously appeared. For example, in the search method using a binary tree, each character code appearing in the input data is set as a head (root node), and a search path having a tree structure that branches into two nodes in sequence is configured. Then, regarding a certain character code and the character code that follows, depending on the comparison result of the size when these character codes are expressed in hexadecimal code, for example, to which node the subsequent character code should be branched and arranged. Make a decision.

[0004]

[Problems to be Solved by the Invention] However, the above 2
In the search using the branch tree, the tree growth state is unbalanced because one character string placed at each node is compared to search one data string (only a part of the search path is long. However, it takes a very long time to search the data located at the final node (leaf node) of the search path.

In some cases, a search method using a trie tree is adopted, and in the case of this trie tree, the search can be performed faster than in the case of the binary tree. However, the above trie tree has a problem that it requires a huge memory space.

The present invention is intended to solve the above problems, and an object thereof is to perform data retrieval at high speed when compressing data by the LZ slide dictionary method, and to minimize the memory space required at the time of retrieval. (EN) Provided is a data compression method and a data compression device capable of performing the same.

[0007]

A data compression method according to the present invention is a data compression method for compressing data such as document data using the LZ slide dictionary method. A search step of searching the compression processed data for a candidate of the same data string as the compression target data string starting from the position by a hash method, and comparing the data string searched in this search step with the compression target data string. And a comparison step of obtaining a matched data length, and a search step and the comparison step are repeatedly executed until there are no search candidates for the same data string as the compression target data string from the compressed data. The data with the longest data length is selected from the matching data lengths obtained in the iterative step and the comparison step. Having a selecting step of selecting and outputting data column, characterized in was a registration step of registering the hash table the compressed data string as the search candidate by hashing.

In this case, in the repeating step, it is preferable that the search and the comparison are repeated a predetermined number of times and then terminated. In addition, the number of times the search and comparison are terminated
It is preferably set to 3 to 20 times. Further, in the registration step, it is preferable that the compression target data string is registered in the first searched position among the search candidates by the hash method. In addition, by statistically storing the number of searches when the longest matching data length is detected in the comparison step, the number of cutoffs in the repeating step may be automatically optimized and determined.

The data compression apparatus of the present invention is a data compression apparatus for compressing data such as document data using the LZ slide dictionary method, and the compression target starting from the next position of the compression processed data in the compression processing target data. A search unit that searches for candidates of the same data string as the data string from the compressed data by the hash method and the data string searched by this searching unit and the compression target data string are compared and they match. The comparison means for obtaining the data length, the repetition means for repeatedly executing the search and comparison of the candidates of the same data string as the compression target data string from the compression processed data until there are no search candidates, and the comparison means The selection means for selecting and outputting the data string having the longest data length from the matched data lengths and the data string for compression Characterized in place and a registration means for registering the hash table as a search candidate by Interview method.

In the case of this configuration, it is preferable that the repeating means is configured to terminate the subsequent search and comparison after repeating the search and comparison a predetermined number of times. Further, it is preferable to set the number of times of termination in the repeating means to any of 3 to 20 times. Further, the registration means may be configured to register the data string to be compressed at the first searched position among the search candidates by the hash method. It is also preferable that the number of times of retrieval when the longest matching data length is detected by the comparison means is statistically stored so that the number of times of termination in the repeating means is automatically optimized and determined. .

According to the above means, since the same data string as the data string to be compressed is searched from the compressed data by the hash method, the data string can be searched at high speed. Then, at the time of searching by this hash method, a hash table is created in the memory space, but in this hash table, only the position of the search candidate data string (the position in the compressed data) is registered, and the actual The hash table does not use much memory space because it does not register data. In addition, the hash chain also only registers the position of the search candidate data string in a chain,
It doesn't use much memory space because it doesn't register the actual data.

If the search and comparison are repeated a predetermined number of times and then terminated, the data search can be executed at a higher speed. Furthermore, by setting the number of times of aborting the search and comparison to 3 to 20, it is possible to sufficiently shorten the time required for the compression process while making the data compression efficiency sufficiently good. In addition, if the compression target data string is registered in the first searched position among the search candidates by the hash method, it is possible to increase the probability that the longest matching data length can be obtained quickly. Further, when the comparison step is configured to statistically store the number of searches when the longest matching data length is detected and to automatically optimize and determine the number of truncations in the repeating step, the data compression efficiency and The compression processing speed can be optimized.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION A first embodiment of the present invention will be described below with reference to FIGS. In FIG. 1 showing an electrical configuration of a data compression device, a data input / output device 1 which is a data input / output means is composed of, for example, a SCSI interface, and a data file to be compressed is stored in, for example, a hard disk or a floppy disk. It has a function of inputting / outputting data to / from an external storage device (not shown). The data input / output device 1 is connected to a control circuit 2 including a microcomputer via address and data bus lines and control signal lines. A memory 3 is connected to the control circuit 2 via address and data bus lines and control signal lines.

The control circuit 2 also stores a control program for controlling execution of data compression processing.
The control circuit 2 includes a search logic 4 as a search means, a match comparison logic 5 as a comparison means, a selection logic 6 as a selection means, a data compression logic 7, and
It is configured to have each function as a registration logic 8 which is a registration means. The control circuit 2 also has a function as a repeating unit. The functions of the above logics will be described below. First, the search logic 4
Hashing (computation by a hash function) is performed on the compression target data string in the compression target data file read on the memory 3, and the same data string as the compression target data string is shown in the search table to be described later with the head position. It has a function of searching from a data string in which position information is written and registered.

The match comparison logic 5 compares the data string to be compressed and the data string starting from the head position searched by the search logic 4 one by one (for example, one character), and the data lengths that match each other. Has the function of outputting. The selection logic 6 has a function of selecting the data string having the longest match data length output from the match comparison logic 5 from the data strings starting from the head position searched by the search logic 4. The data compression logic 7 outputs the start position of the data string selected by the selection logic 6 and the matched data length output by the match comparison logic 5 for the data string as matching information. Then, the registration logic 8 registers the start position of the compression processing target data string in the hash table as position information that is a search candidate.

Next, the operation of the above embodiment will be described with reference to FIGS. The flow charts shown in FIGS. 2 and 3 show the control contents of the control program of the control circuit 2, that is, the general control contents of the data compression processing. In addition, as the data (file) to be compressed, FIG.
Data composed of characters arranged as shown in (a) is used.
When compressing the data of this character string by the data compression device, the operator operates the input means such as a keyboard (not shown) of the data compression device to specify the file name of the data file to be compressed and execute the compression process. Enter the command. Then, the control circuit 2 reads the specified data file from the external storage device via the data input / output device 1, transfers it to the memory 3, and then starts the data compression process.

First, in the flow chart of FIG.
The processing step R1 of "initial processing" is executed. Here, the control circuit 2 initializes an area used for compression processing in the memory 3. Then, the process proceeds to the process step R2 of "search process". This "search process" is defined as a subroutine shown by the flowchart of FIG. 3, and will be described below with reference to the flowchart of FIG.

As shown in FIG. 3, the control circuit 2 initializes the area related to the search processing provided on the memory 3 (step S1), and then executes the next "data search" (step S1). S2). Here, in order to perform hashing by the search logic 4, FIG.
0th (0th address) character data “A” and 1st (1st address) character data “B” of the data string shown in
Read the two characters. In this case, the exclusive OR is selected as the hash function, and J of the character data “A” at the 0th address is selected.
The hash value 0x03 is obtained by performing the logical operation of the exclusive OR of the IS code 0x41 and the JIS code 0x42 of the character data “B” at the first address. Then, the hash table is configured to be searched by the hash value 0x03.

The hash table is a table (data table) in which the head position (address) of a character string having the hash value is registered as position information using the hash value as an index, which is expanded on the memory 3. . Incidentally, in the initialization of the step R1, for example, data (0xFFFF ...) Of all bits “1” is written in the hash table and a hash chain which will be described later, so that no data is registered. It is in a state. Therefore, at this point, if the registered data is read by performing a data search on the hash table with the hash value 0x03,
“No registered data” is detected (see FIG. 5). A combination of the hash table and the hash chain is defined as a search table (search target).

Then, it is judged whether or not there is no registered data (step S3). Since "no registered data" is detected at this point, "YES" in step S3.
Then, the process proceeds to step R3 shown in FIG. 2, and the process of "search result output" is executed. Here, the data compression logic 7 outputs the result of the search process performed in step R2 as an output data string in a format as shown in FIG. 4B. As shown in FIG. 4B, the management unit is arranged at the 0th address of the compressed output data string, and the setting of this management unit will be described later.

Further, in the step R3, it is judged whether or not the longest matching data length is a predetermined value (for example, "3") or more as a result of the search processing in the step R2, and the data is output. Specifically, at this time point, the registration data could not be detected in step S3, so that the matching information is not output, and the longest matching data length is "0" as it is initialized and less than the predetermined value "3". Therefore, the character data "A" is directly output to the first address. Then, the process of "registration in hash table" is executed (step R4). Here, the position information is registered in the hash table. This registration process is executed as in the conceptual diagrams shown in FIGS.

Specifically, in the search result at this time, since there is no data registered for the index 0x03 of the hash table, the position (address) of the first data "A" in the hash table. Position information "0" indicating
Registration is performed by overwriting the code indicating "no registration data" (see FIG. 6). Then, the pointer of the data string to be compressed is incremented and the search logic 4 next focuses on the character data "B" at the first address. To do so. Also, the loop counter is incremented.

Then, it is judged whether "looped for the number of processed data?" (Step R5). Here, by comparing the number of pieces of processed data with the count value of the loop counter, it is determined whether or not the registration processing in the hash table in step R4 has been looped for the number of pieces of processed data. Here, when the longest matching data length obtained by the selection logic 6 is "3" or more, the number of pieces of processing data is a numerical value set equal to the matching data length because compression processing is performed as described later. If the longest matching data length is “0” or “2”, the compression process is not performed,
Since one character data is output as it is, the number of processed data is a numerical value set as "1" (the longest matching data length "1" is not output).

At this point in time, the longest matching data length is "0" and the number of processed data is set to "1", which is equal to the count value "1" of the loop counter.
In judgment step R5, the process proceeds to "YES" to judge "end of data?" (Step R6). here,
It is determined whether or not the compression target data string has been processed to the end. However, at this point, the process is not yet completed, so step R6
Then, the process proceeds to "NO" and returns to step R2. The steps R4 and R5 correspond to the registration step.

Then, in step R2, the longest match data length is initialized in step S1 as in the previous time, and then the process proceeds to step S2, this time the character data "B" of the first address. And hashing is performed with the character data “C” at the second address to obtain a hash value 0x01.
Then, the hash table is searched by the index 0x01, but "no registered data" is detected, and the process proceeds to "NO" in the next step S3 and proceeds to step R3.

In step R3, the character data "B" is output as it is to the second address of the output data string, as in the previous time. Then, the process proceeds to step R4, and the position information "1" indicating the position (address) of the character data "B" is written and registered in the area of the index 0x01 of the hash table. Further, the pointer of the data string to be compressed is incremented to focus on the character data "C" at the second address, and then the process proceeds to the next step R5. Since only one character has been processed in step R5, the process proceeds to "YES" and proceeds to step R6. At this step R6, the data is not yet over, so "N
O ”, and returns to step R2.

A character string "C" of the following third and fourth addresses
D ”and the character string“ DA ”of the fourth and fifth addresses are not output the same as above, and the character data“ C ”and“ D ”are directly output to the output data string.
Position information "2" and "3" are written in the areas indicated by the indexes of the hash values in the hash table.

When the pointer of the data string to be compressed becomes "4", the fourth and fifth pointers are obtained in step S2.
Hashing is performed on the character string "AB" of the address. Then, since the hash value 0x03 is obtained, when the hash table is searched, the position information “0” of the registered data is obtained (FIG. 6).
reference). Therefore, in the next determination step S3, the process proceeds to "NO", and the process proceeds to the next "Same data string?" Determination step S4.

Since the hash method is a mapping from a large set to a small set, there are cases where different data strings have the same hash value (collision). In the present embodiment, the position information of character strings indicating the same hash value are all registered in chained data linked to the same index, that is, a so-called hash chain (hereinafter, simply referred to as a chain). , It is confirmed whether the data string of the position information “0” is the same as the current data string to be compressed. Therefore, in the determination step S4,
The match comparison logic 5 reads the data “A” at the 0th address of the data string to be compressed and compares it with the data “A” at the fourth address to obtain a match result. here,
If the first data is the same in the same hash value, the second data is also the same, so the confirmation ends. And
Then, the process proceeds to step S5 of the next "compare one data at a time".

In the processing step S5, since the third and subsequent data of the character string are compared one by one, the coincidence comparison logic 5 reads the data "C" at the second address and the data "C" at the sixth address. Compare and get a match result. Next, when the data “D” at the third address and the data “E” at the seventh address are read and compared, a non-match result is obtained. Therefore, the match data length “3” up to this point is compared with the current match data length of the memory 3. To be stored in the memory area. Note that steps S4 and S5 correspond to the comparison step.

Then, it is judged whether "the longest matching length?" (Step S6). Here, the selection logic 7 determines whether or not the current matching data length “3” obtained in step S5 is larger than the content stored in the area of the longest matching data length. In this case, since the longest match data length is initialized at step S1 and is "0", the process proceeds to "YES" at step S6 and the process of "update match position and longest match length" is executed (step S
7).

In step S7, "0" is written in the storage area of the matching position on the memory 3 and "3" is written in the storage area of the longest matching data length to be stored. Then, the process proceeds to the next "take the next chain" processing step S8. Note that steps S6 and S7 correspond to the selection step.

In the processing step S8, the next (first in this case) data of the chain of index 0x03 is read. Then, the code of "no registered data" is read (see FIG. 6), so "N" is displayed in the next step S3.
Go to "O" and move to step R3. Incidentally, step S
2 corresponds to the search step, and steps S3 and S8 correspond to the iterative step.

Then, in step R3, step R2
The longest matching data length obtained by the search process of is a predetermined value "3"
As described above, the data compression logic 7 writes the matching position “0” and the matching data length “3” as the matching information to the fifth address of the output data string, for example, as the data of the 2-byte size with each 1-byte size. Put in. Then, in order to show that the fifth address is the matching information, the management unit of the 0th address is set.

This management unit is data of the same 1-byte size as other character data, and is inserted every 8th data from the 0th address of the output data string, and its content is the data of the following 8 data. It shows what number of data is the matching information output as a result of the data compression processing. For example, in order to perform the setting in this case, when the content of the management unit at the 0th address is read out, new information is added without clearing the bit for which "1" has already been set. Logical operation is performed by exclusive OR with 4-byte (corresponding to the fifth address) 1-byte data in which "1" is set. At this point, the contents of the management section are cleared to 0 at the time of initialization in step R1, so the result of the logical operation by the exclusive OR is only the 4th bit as "1" as shown in FIG. 4 (c). Become. Then, the process proceeds to step R4.

In step R4, position information "4"
Is registered in the hash table. At this point in time, as shown in FIG. 6, since the position information "0" has already been registered in the area indicated by the index 0x03 in the hash table, the position information "0" is sent to the first area of the chain. Then, the location information “4” is displayed in the area indicated by the index 0x03 in the hash table.
To write and register for update. The state after this update registration is shown in FIG.
Shown in After that, when the pointer of the data string to be compressed is incremented to "5", the loop counter is incremented and then the process proceeds to step R5.

At step R5, the number of processed data at this time is set to "3" because the longest matching data length obtained by the selection logic 6 at step S5 is "3", and the loop counter is set to Since it is 1, it is determined to be “NO”, and after that, step R4 is executed twice by looping twice. That is, the position information “5” by hashing the character string “BC” and the position information “6” by hashing the character string “CE” are registered in the areas indicated by the respective indexes of the hash table. At the end of this processing, the pointer of the data string to be compressed is "7".
It is. Then, in step R6, since the data is not yet finished, it is determined to be "NO", and the process proceeds to step R2.

In step R2, the character string "EA" of the seventh and eighth addresses is processed by the same process as above.
Output to output data string and register in hash table.
The following description will be focused on the data string “AB”. When processing is performed on the character strings "AB" of the eighth and ninth addresses, the hash table is searched by hashing in step S2, and position information "4" is obtained (see FIG. 7). In steps S4 and S5, the character strings starting from the fourth address are compared one by one, and the obtained matching data length is "2". Therefore, in step S6, a comparison is made with the initialized longest matching data length "0" and "YE
S "is determined and the process proceeds to step S7. When the matching position “4” and the longest matching data length “2” are updated and registered in step S7, the process proceeds to step S8.

In step S8, the index 0x
The data stored in the first area of the 03 chain is read.
Then, since the position information "0" is obtained, it is determined to be "NO" in step S3 and the process proceeds to step S4, and further, "YES" is determined in step S4 and the process proceeds to step S5. In this case as well, since the matching data length obtained in step S5 is "2", the same applies to step S below.
6, S8, S9, S10, S3, step S
In "10", the "no data" code stored in the second area of the chain is obtained, so that "YES" is determined in step S3 and the process proceeds to step R3.

In step R3, when the matching data length is "2", there is no data compression effect, so the leading character data "A" is output as it is to the output data string. Then, in step R5, the position information “4” registered in the hash table is sent to the chain, and the position information “8” is registered in the hash table (see FIG. 8). Hereinafter, the same processing is performed for the character strings “BB” and “BA”.

Next, a search process is performed for the character string "AB" of the eleventh and twelfth addresses. First, when the character strings are compared for the position information “8” registered in the hash table, the matching data length “2” is obtained because the third character is different. Then, in the next search process, the position information “4” stored in the first area of the chain is obtained, the character strings are compared, and the matching data length “2” is also obtained. Further, the positional information “0” stored in the second area of the chain is obtained in the next search process, and the character strings are compared. Then the matching data length is "2"
Get. Further, in the next search process, "no data" is detected and the search process is ended, but the longest matching data length is "2", and as a result, the compression process is not executed.

Thereafter, the character string "AB", that is, the index
As shown in FIGS. 9 and 10, the 0x03 chain has the position information “11” and “16” of the input data sequentially registered in the hash table, so that the position information data is sequentially transmitted to the chain and gradually becomes longer. I'm going. The one having the longest matching data length among them is output to the output data string instead of the character string as matching information. Similarly, if the data compression processing is performed for all the compression processing target data, the process proceeds to “YES” in step R6 and proceeds to the processing step R7 of “post-processing”. In this step R7, the output data on the memory 3, which is the data after the compression process shown in FIG. 4B, is output to the external storage device by the data input / output device 1, and the data compression process ends.

As described above, according to this embodiment, the control circuit 2 compresses the candidates of the same data string as the data string to be compressed starting from the position next to the compressed data in the data to be compressed. The data is searched by the hash method, and the searched data string and the data string to be compressed are compared to obtain the matching data length. Then, the control circuit 2 is configured to repeatedly execute the above-described search and comparison until candidates for the same data string as the data string to be compressed are excluded from the compressed data. Further, the control circuit 2 selects and outputs the data string having the longest data length from the matching data lengths obtained by the comparison, and registers the compression target data string in the hash table as a search candidate by the hash method. As configured. According to this configuration, since the hash method is used to search for the same data string as the compression target data string, the data string can be searched at high speed and the memory space used can be made sufficiently small. it can.

FIG. 11 shows a second embodiment of the present invention. The same parts as those of the first embodiment are designated by the same reference numerals and the description thereof will be omitted. Only different parts will be described below. In FIG. 11, a processing step S9 of “search count counter + 1” and a determination step S10 of “search count = limit count?” Are inserted between steps S7 and S8 of the flowchart of FIG. 2 in the first embodiment. Is configured to. The other structure of the second embodiment is the same as that of the first embodiment.

Next, the operation of the second embodiment will be described.
In the second embodiment, when the process of step S7 is completed, the process proceeds to the next "search counter +1" process step S9. In this step S9, when the search table is searched once for one data string by the search logic 4,
The search count counter initialized in step S1 is incremented by 1, and the process proceeds to the next "search count = limit count?" Determination step S10.

In step S10, it is determined whether or not the count value of the search counter is equal to a predetermined value as the search limit of the search table. In this case, the limit number of times is set to "3".
When the count value of the search number counter becomes equal to the limit number in step S10, the process proceeds to "YES",
Complete the search process. In step S10,
If the count value of the search number counter is not equal to the limit number (less than the limit number), the process proceeds to "NO" and proceeds to step S8. The repeating step in the first embodiment plus steps S9 and S10 corresponds to the repeating step in the second embodiment.

Here, in the second embodiment as well, the same compression target data as in the first embodiment is subjected to the compression processing, and the character string "AB" will be similarly noted and described.
As shown in FIG. 7, until the position information “4” of the character string “AB” is registered in the hash table indicated by the index of the hash value 0x03, the count value of the search counter in step S9 is “3”. Is less than step S
In step 10, since the determination is "NO" and the process proceeds to step S8, the same process as in the first embodiment is performed.

Then, the next position information "8" shown in FIG.
When the character string “AB” at the 11th and 12th addresses is searched at the time when is registered, the position information “0” in the second area of the chain is read in step S8, and then steps S3 to S7 are performed. The same process is performed, and in the next step S9, the count value of the search number counter reaches "3" which is the limited number of times. Then, step S1
Since the process proceeds to “YES” at 0, the search process is terminated without performing further search and comparison.

Further, as shown in FIGS. 9 and 10, the data compression process proceeds thereafter, and the position information "11" and "16" are entered.
Even if the chain becomes longer due to being sequentially registered in the hash table, if the character string “AB” appears in the compression processing target data string after this, the search processing in step R2 is performed only three times. That is, position information "16", "1"
Only "1" and "8" are searched, and no further search is performed in the search table, and the one having the longest matching data length is output to the output data string as matching information. .

In the second embodiment, the longest matching data length may not be obtained because the matching data strings appearing in the search table are not searched. But,
A feature of general text data is that a data string having the longest matching length is likely to appear immediately near the position of interest. Therefore, even if the method of searching the latest matching data string only three times as in the second embodiment is used, the compression efficiency is hardly deteriorated.

As described above, according to the second embodiment, the number of times the search processing is performed is counted, and when the number of times of search processing becomes equal to the preset number of times, the search is aborted and the longest matching data length at that time is determined. Since the start position of the indicated data string and the matching data length thereof are output as matching information, the time required for the search is different from the first embodiment in which the search of the search table is repeated until there is no matching data string. Is the same every time even if the chain becomes long, and the data compression process can be executed even faster. Moreover, in this case, the data compression efficiency is hardly reduced.

The present invention is not limited to the embodiments described above and illustrated in the drawings, and the following modifications are possible. First, the hexadecimal code of the character data is the JIS code, but the EBCDIC code or ASCII
It can be a code. Further, although the hash function is the exclusive OR, the present invention is not limited to this, and an appropriate calculation (not limited to the logical calculation) that causes the occurrence of collisions as much as possible may be appropriately selected. Further, in step R3, the predetermined value of the longest match data length for outputting the match information is set to "3", but it may be set to an appropriate value of "4" or more. In addition, the number of characters to be hashed may be three or more.

Further, the matching information data (2-byte data) is composed of 1-byte data representing the matching position (address) and 1-byte data representing the matching data length, but is not limited to this. The size of the data to be compressed may be appropriately changed depending on the balance with the expected maximum matching data length. For example, the size of the matching position may be 12 bits and the size of the matching data length may be 4 bits. Further, the data size of the matching information is not limited to 2 bytes, and may be 3 bytes, for example, if the number of bits is further required according to the size of the compression processing target data. In this case, the compression efficiency may be considered according to the data size of the matching information, and the minimum matching data length for outputting the matching information may be appropriately changed, such as from “3” to “4”.

Furthermore, the data to be compressed is not limited to sentences in a language consisting of alphabets, but may be other languages such as Japanese. Also, if the hexadecimal code that represents one character of the language is 2 bytes,
The data size may be 2 bytes, and one may be added for every 16 characters.

Further, although the number of times of retrieval (the number of times of termination) in the iterative logic is set to 3, the number of times can be appropriately determined depending on which is more important in consideration of the data compression efficiency and the processing speed. good. For example, the search limit number is set to a large value when the compression efficiency is emphasized, and the search limit number is set to a small value when the processing speed is emphasized. In the case of making a decision by giving importance to an appropriate balance between the data compression efficiency and the processing speed, it is desirable to set the search limit number to any of 3 to 20 times. In this case, the most preferable limit is around 10. Further, the user may be allowed to specify the limited number of searches depending on the nature of the data to be compressed.

On the other hand, in each of the above embodiments, the latest compression target data string is registered in the first searched position of the hash table. However, the present invention is not limited to this, and the latest compression target data The row may be registered at the last searched position.

Further, in step S5 of each of the above-described embodiments, the longest matching data length defined as a limit (the physical limit of the storage area of the longest matching data length, or the matching data length longer than this due to the nature of the sentence data). If it is assumed that is not possible) is detected, the subsequent search may be terminated. With this configuration, the compression processing can be further speeded up.

Further, immediately before "return" in the flow chart of FIG. 11, a processing step S1 of "store the number of times of retrieval" is performed.
1 is provided, and the number of searches for the data string in which the longest matching length is detected is found every time the data string is searched. And
The processing routine of the “search count optimization processing” may be executed when the data compression processing has advanced to some extent. In this processing routine, an appropriate statistical process is performed on the number of searches stored in step S11 to automatically obtain and set the number of searches in which compression efficiency is sufficiently good and the search processing time can be sufficiently shortened. It is configured to be able to (optimize).

It should be noted that the output data output after being compressed according to the above-described embodiments and modifications may be further compressed by applying another compression method such as arithmetic coding. Well, with this configuration, the data compression efficiency can be further improved.

[0060]

As apparent from the above description, according to the present invention, in a data compression process for compressing data such as document data using the LZ slide dictionary method, the same data sequence as the compression target data sequence has been compressed. Since the data is searched by the hash method from the data, the data string can be searched at high speed, and the memory space required for the search can be minimized.

Further, in this configuration, if the search and comparison are repeated a predetermined number of times and then terminated, the data search and the data compression process associated therewith can be executed at a higher speed. Furthermore, by setting the number of times the search is terminated to any of 3 to 20, it is possible to achieve an appropriate balance between the data compression efficiency and the compression processing speed. Then, when the latest compression target data string is configured to be registered in the first searched position among the search candidates by the hash method,
The probability of obtaining the longest matching data length can be increased.
In addition, by statistically storing the number of searches when the longest matching data length is detected, the number of times the search is aborted is automatically optimized and determined, and the data compression efficiency and the compression processing speed are improved. Can be optimized.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of the present invention.

FIG. 2 is a flowchart of control contents.

FIG. 3 is a flowchart of the control contents of step R2 in FIG.

FIG. 4 is a diagram showing input data, output data, and contents of a management unit.

FIG. 5 is a conceptual diagram showing contents of a search table in an initialized state.

FIG. 6 is a conceptual diagram showing update registration processing of a hash table.

FIG. 7 is a diagram corresponding to FIG. 6;

FIG. 8 is a diagram corresponding to FIG. 6;

FIG. 9 is a view corresponding to FIG.

FIG. 10 is a view corresponding to FIG.

FIG. 11 is a view corresponding to FIG. 2 showing a second embodiment of the present invention.

FIG. 12 is a conceptual diagram of data compression processing by the LZ slide dictionary method.

[Explanation of symbols]

1 is a data input / output device, 4 is a search logic (search means), 5 is a match comparison logic (comparison means), 6 is a selection logic (selection means), 7 is a data compression logic, and 8 is a registration logic (registration means). Show.

Claims (10)

[Claims] 1. A data compression method for compressing data such as document data using the LZ slide dictionary method, the same data sequence as a compression target data sequence starting from the position next to the compression processed data in the compression target data. Search step for searching the candidate of the compressed data by the hash method, and comparing the data string searched in this search step with the compression target data string to obtain a matching data length A step of repeating the search step and the comparing step until a search candidate for the same data string as the compression target data string is exhausted from the compressed data, and Selection step to select and output the data string with the longest data length from the matching data length , The data compression method is characterized in that a registration step of registering the hash table the compressed data string as the search candidate by hashing. 2. The data compression according to claim 1, wherein in the repeating step, after the search step and the comparison step are repeated a predetermined number of times, the subsequent search step and the comparison step are terminated. Method. 3. The data compression method according to claim 2, wherein the number of times of termination in the repeating step is set to 3 to 20 times. 4. The registering step, wherein the compression target data string is registered at a position searched first among the search candidates by the hash method.
4. The data compression method according to any one of 3 to 3. 5. The number of times of search in the repeating step is automatically optimized and determined by statistically storing the number of times of searching when the longest matching data length is detected in the comparing step. The data compression method according to claim 2. 6. A data compression apparatus for compressing data such as document data using the LZ slide dictionary method, the same data string as a compression target data string starting from the position next to the compression processed data in the compression target data. Searching means for searching the candidate of the compressed data by the hash method, the data string searched by the searching means, and the compression target data string to obtain a matching data length Means, a repeating means for repeatedly executing a search and comparison of candidates for the same data string as the compression target data string from the compressed data until there are no search candidates, and a matching data length obtained by the comparing means. Selecting means for selecting and outputting the data string having the longest data length from the inside, and a search candidate by the hash method for the data string to be compressed Data compression apparatus characterized by comprising a registration means for registering in the hash table with. 7. The data compression apparatus according to claim 6, wherein the repeating unit is configured to terminate the subsequent search and comparison after repeating the search and comparison a predetermined number of times. 8. The number of times of termination in the repeating means is
The data compression apparatus according to claim 7, wherein the data compression apparatus is set to 3 to 20 times. 9. The data according to claim 6, wherein the registration unit registers the compression target data string at a position searched first among the search candidates by the hash method. Compressor. 10. The repeating unit statistically stores the number of searches when the comparison unit detects the longest matching data length, thereby automatically optimizing and determining the number of search and comparison aborts. 7. The method according to claim 6, wherein
Alternatively, the data compression device according to item 9.