JPWO2011105463A1 - Data compression apparatus, data compression method, and program storage medium - Google Patents

Abstract

Provided is a data compression device or the like that can maintain an efficient data compression rate. The data compression apparatus 600 includes a compression unit 601, a monitoring unit 602, a timing control unit 603, and a dictionary generation unit 604. The compression unit 601 has a function of compressing data to be compressed based on dictionary data given in advance. The monitoring unit 602 has a function of calculating the use frequency of the dictionary data when the compression unit 601 compresses the data. The timing control unit 603 has a function of issuing an instruction to update the dictionary data when the calculated use frequency falls below a preset threshold value. In response to the instruction, the dictionary generation unit 604 has a function of newly generating the dictionary data using the data compressed by the compression unit 102.

Description

  The present invention relates to a data compression apparatus, a data compression method, and a program storage medium.

Japanese Patent Laid-Open No. 2001-044850 (Patent Document 1) discloses a data compression apparatus that compresses data using a lexicographic compression method. When compressing data, this data compression apparatus repeats updating the dictionary until an optimal dictionary used for data compression is generated. And this data compression apparatus compresses data using the dictionary in the stage where the optimal dictionary was produced | generated.
FIG. 24 is a block diagram showing a configuration example of the data compression apparatus. The data compression apparatus 12 includes a dictionary generation / update unit 13, a dictionary holding unit 15, a dictionary output unit 16, a data compression unit 17, and a compressed data output unit 18. The dictionary generation / updating unit 13 has a function of generating the dictionary 14 and a function of updating the dictionary 14 based on all data strings 11 to be compressed. The dictionary holding unit 15 has a function of holding the dictionary 14. The dictionary output unit 16 has a function of outputting the dictionary 14. The data compression unit 17 has a function of compressing the entire data sequence 11 based on the dictionary 14. The compressed data output unit 18 has a function of outputting the compressed data 19 from the data compression unit 17. The dictionary 14 here is data indicating a combination of all data strings 11 and registration numbers. The data compression device 12 compresses the data by replacing the entire data string 11 with the registration number based on the dictionary 14.
The data compression device 12 can increase the data compression rate by preferentially replacing data with a large number of frequent occurrences with the registration number of the dictionary 14 when the entire data string 11 is replaced with the registration number. However, the dictionary 14 generated in this way may have a very large number of registrations. For this reason, the data compression apparatus 12 updates the dictionary by deleting registration data that is less frequently used from the dictionary 14. Then, the data compression device 12 determines that the dictionary 14 is optimized when the number of registrations in the dictionary 14 reaches a predetermined number. The data compression apparatus 12 increases the data compression rate by compressing the entire data string 11 based on the dictionary 14 obtained in this way.

JP 2001-044850 A

However, the data compression device 12 does not update the dictionary 14 when data is continuously input. For this reason, in the following cases, the data compression rate by the data compression device 12 may decrease. For example, from a state where character string data including many “A”, “B”, and “C” is input to the data compression device 12, characters that hardly include “A”, “B”, and “C”. The trend may change to a state where column data is entered. In such a case, the data compression device 12 performs data compression using the dictionary 14 including many “A”, “B”, and “C” before the trend changes. However, if the tendency of the input data changes and the input data hardly includes “A”, “B”, and “C”, the frequency at which the data compression device 12 uses the dictionary 14 decreases. Thereby, the data compression rate by the data compression apparatus 12 falls.
The present invention has been made to solve the above problems. That is, a main object of the present invention is to provide a data compression apparatus, a data compression method, and a program storage medium that can maintain efficient data compression.

The data compression apparatus of the present invention
Compression means for compressing data to be compressed based on dictionary data given in advance;
Monitoring means for calculating the frequency of use of the dictionary data when the compression means compresses the data;
Timing control means for issuing an instruction to update the dictionary data when the calculated use frequency falls below a preset threshold;
In response to the instruction, the apparatus has dictionary generation means for newly generating the dictionary data based on the data compressed by the compression means.
The data compression method of the present invention includes:
Compress data to be compressed based on dictionary data given in advance,
Calculating the frequency of use of the dictionary data when compressing the data;
When the calculated usage frequency falls below a preset threshold value, an instruction to update the dictionary data is issued,
In response to the instruction, the dictionary data is newly generated based on the compressed data.
The program storage medium of the present invention includes:
A process of compressing data to be compressed based on dictionary data given in advance;
A process of calculating the use frequency of the dictionary data when compressing the data;
A process of issuing an instruction to update the dictionary data when the calculated use frequency falls below a preset threshold;
In response to the instruction, a computer program for causing the data compression apparatus to execute processing for newly generating the dictionary data based on the compressed data is stored.
The main object of the present invention is also achieved by a data compression method corresponding to the data compression apparatus having the above-described configuration. Furthermore, the main object of the present invention is also achieved by a program storage medium storing a computer program for realizing the data compression apparatus and the data compression method by a computer.

  According to the present invention, efficient data compression can be maintained.

FIG. 1 is a block diagram showing the configuration of the data compression apparatus according to the first embodiment of the present invention.
FIG. 2 is a diagram for explaining an example of data compression by the lexicographic compression method.
FIG. 3 is a diagram illustrating a result of counting the number of appearances of two adjacent characters in the data D001 in FIG.
FIG. 4 is a diagram showing the result of counting the number of appearances of two adjacent characters in the data D002 in FIG.
FIG. 5 is a diagram illustrating a result of counting the number of appearances of two adjacent characters in the data D003 in FIG.
FIG. 6 is a diagram illustrating an example of dictionary data.
FIG. 7A is a diagram showing an image of data in which dictionary data used for compression and the number of uses thereof are paired.
FIG. 7B is a diagram illustrating an example of the total capacity of compressed data.
FIG. 8 is a diagram illustrating an example of data indicating the usage frequency of dictionary data.
FIG. 9A is a diagram illustrating an example of compressed data.
FIG. 9B is a diagram illustrating an example of dictionary data.
FIG. 10 is a diagram for explaining an example of a technique for expanding data compressed by the lexicographic compression method.
FIG. 11 is a diagram for explaining an example of a technique for expanding data compressed by the lexicographic compression method following FIG.
FIG. 12 is a diagram for explaining an example of a technique for further expanding data compressed by the lexicographic compression method.
FIG. 13 is a flowchart showing an operation example of the data compression apparatus according to the first embodiment of the present invention.
FIG. 14 is a block diagram showing the configuration of the data compression apparatus according to the second embodiment of the present invention.
FIG. 15 is a flowchart showing an operation example of the data compression apparatus according to the second embodiment of the present invention.
FIG. 16 is a flowchart showing a specific operation example of the adjustment unit constituting the data compression apparatus according to the second embodiment of the present invention.
FIG. 17 is a block diagram showing the configuration of the data compression apparatus according to the third embodiment of the present invention.
FIG. 18 is a flowchart showing an operation example of the data compression apparatus according to the third embodiment of the present invention.
FIG. 19 is a block diagram showing the configuration of the data compression apparatus according to the fourth embodiment of the present invention.
FIG. 20 is a flowchart showing an operation example of the data compression apparatus according to the fourth embodiment of the present invention.
FIG. 21 is a block diagram showing the configuration of the data compression apparatus according to the fifth embodiment of the present invention.
FIG. 22 is a flowchart showing an operation example of the data compression apparatus according to the fifth embodiment of the present invention.
FIG. 23 is a diagram for explaining another embodiment.
FIG. 24 is a block diagram showing the configuration of the data compression device described in Patent Document 1.

Embodiments according to the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing a configuration of a data compression apparatus (data compression system) according to the first embodiment of the present invention. The data compression apparatus 100 includes a control device 101, a first storage unit 103, a second storage unit 104, and a third storage unit 108. The first storage unit 103, the second storage unit 104, and the third storage unit 108 are configured by a storage device. The storage device has a storage medium (for example, a hard disk device, a random access memory (RAM), or the like) that stores (stores) data. In the first embodiment, the first storage unit 103 functions as a compressed data storage unit (compressed data storage unit) that stores compressed data to be described later. The second storage unit 104 functions as a dictionary data storage unit (dictionary data storage unit) that stores dictionary data described later. The third storage unit 108 functions as a development data storage unit (expansion data storage unit) that stores development data to be described later.
The control device 101 is a computer including a CPU (Central Processing Unit), for example. The control device 101 governs the overall operation of the data compression device 100 by appropriately executing various computer programs stored in a computer-readable storage medium.
In the first embodiment, the control device 101 has a function of compressing data generated by the data generation unit 110 using a lexicographic compression method. Furthermore, the control device 101 has a function of generating dictionary data to be referred to when compressing data and a function of updating dictionary data. That is, the control apparatus 101 implement | achieves the following functional blocks by operate | moving according to a computer program. The functional blocks include a compression unit 102, a monitoring unit (dictionary usage frequency monitoring unit) 105, a timing control unit (dictionary update timing unit) 106, a decompression unit (compressed data decompression unit) 107, and a dictionary generation unit ( Dictionary construction means) 109. The data generated by the data generation unit 110 may be any kind of data, but here it is assumed to be character string data for easy understanding of the explanation.
The compression unit (compression unit) 102 has a function (data compression function) for compressing data by a lexicographic compression method based on dictionary data as described later stored in the second storage unit 104. The data to be compressed is data input from the data generation unit 110 or decompressed data as will be described later stored in the third storage unit 108. Note that there are various types of lexical compression methods such as LZW (Lempel-Ziv and Welch) and LZ77 (Lempel-Ziv 77). The type of lexicographic compression method employed by the compression unit 102 is not particularly limited. However, in the first embodiment, in order to make the explanation easy to understand, the compression unit 102 compresses data by a lexicographic compression method called a BPE (Byte Pair Encoding) method.
The BPE method is an algorithm for compressing target data by repeatedly replacing 2-byte data with high appearance frequency with 1-byte data. FIG. 2 is a diagram for explaining an example of data compression by the BPE method. The compression unit 102 compresses data by the BPE method as follows.
For example, the second storage unit 104 holds a plurality of dictionary data as shown in FIG. The dictionary data is data representing the relationship between the character string before replacement and the character after replacement corresponding to the character string. One of the dictionary data shown in FIG. 6 is data representing the relationship between the character string “AB” before replacement and the character “G” after replacement corresponding to the character string. One of the other dictionary data shown in FIG. 6 is data representing the relationship between the character string “DE” before replacement and the character “H” after replacement corresponding to the character string. One of the other dictionary data shown in FIG. 6 is data representing the relationship between the character string “GC” before replacement and the character “I” after replacement corresponding to the character string. Such dictionary data is generated by the dictionary generation unit 109 as described later.
For example, when data D001 which is a character string as shown in FIG. 2 is input, the compression unit 102 refers to the dictionary data in the second storage unit 104 and compresses the data as data D002. That is, the compression unit 102 replaces two adjacent characters “AB” in the data D001 with one character “G” based on the dictionary data. Thus, the compression unit 102 creates data D002 obtained by compressing the data D001. Further, the compression unit 102 replaces two adjacent characters “DE” in the data D002 with one character “H” based on the dictionary data. Accordingly, the compression unit 102 creates data D003 obtained by compressing the data D002. Further, the compression unit 102 replaces two adjacent characters “GC” in the data D003 with one character “I” based on the dictionary data. Thereby, the compression unit 102 creates data D004 obtained by compressing the data D003. Further, the compression unit 102 tries to compress the data D004 in the same manner as described above. However, the data D004 does not have any of the character strings “AB”, “DE”, and “GC” corresponding to the dictionary data. For this reason, the compression unit 102 cannot compress the data D004. Thus, the compression unit 102 ends the process of compressing data (data compression process).
In addition to the data compression function as described above, the compression unit 102 further has a function of writing the finally obtained data D004 into the first storage unit 103. It should be noted that dictionary data is not stored in the second storage unit 104 in the initial operation after the data compression apparatus 100 starts operation. For this reason, the compression unit 102 does not perform the compression operation, and writes the data input from the data generation unit 110 to the first storage unit 103 as it is.
Further, as will be described later, when the dictionary data stored in the second storage unit 104 is replaced with new dictionary data, the compression unit 102 stores all the decompressed data stored in the third storage unit 108. Is compressed based on new dictionary data. Further, the compression unit 102 has a function of deleting all the decompressed data stored in the third storage unit 108 after the completion of the compression of all the decompressed data stored in the third storage unit 108. .
Further, the compression unit 102 creates data (usage status data) relating to the usage status of dictionary data in the data compression process and data (total capacity data) indicating the total capacity of the uncompressed data processed in the data compression process. It has a function. Further, the compression unit 102 has a function of temporarily storing the data and then outputting the data to the monitoring unit 105.
FIG. 7A shows an example of usage status data. FIG. 7B shows an example of total capacity data. In the example of FIG. 7A, the usage status data is data in which dictionary data and the number of uses thereof are paired. According to the usage status data of FIG. 7A, dictionary data that replaces “AB” with “G” is used ten times. The dictionary data for replacing “DE” with “H” has been used 20 times. Furthermore, dictionary data for replacing “GC” with “I” is used five times.
Further, according to FIG. 7B, the total capacity data is 153000 bytes. Note that the timing at which data is output from the compression unit 102 to the monitoring unit 105 may be every time the compression unit 102 compresses the data input from the data generation unit 110, or may be every preset time interval.
In the initial operation after the data compression apparatus 100 starts operation, the compression unit 102 does not perform the data compression process as described above, so only the total capacity of the data input from the data generation unit 110 is sent to the monitoring unit 105. Output.
The dictionary generation unit (dictionary generation unit) 109 has a function of generating dictionary data and a function of storing the generated dictionary data in the second storage unit 104. The procedure for generating dictionary data by the dictionary generation unit 109 is set according to the type of lexicographic compression method employed by the compression unit 102. That is, the type of lexicographic compression method employed in the first embodiment is the BPE method. Therefore, the lexicographic generation unit 109 generates dictionary data according to a procedure according to the BPE method.
For example, when receiving an instruction to start an operation (an operation instruction) from the expansion unit 107 described later, the lexicographic generation unit 109 generates dictionary data based on all data stored in the third storage unit 108. Generate as follows. In the initial operation after the operation of the data compression apparatus 100, the data input from the data generation unit 110 is directly passed through the compression unit 102, the first storage unit 103, and the expansion unit 107, as will be described later. And stored in the third storage unit 108.
That is, first, the lexicographic generation unit 109 counts the number of appearances of two adjacent characters (character strings) in the input data D001 which is a character string as shown in FIG. In the case of data D001, “AB” appears first as two adjacent characters. Thereby, the lexicographic generation unit 109 sets the count value of “AB” to “1”. The next two adjacent characters are “BD”. Thereby, the lexicographic generation unit 109 sets the count value of “BD” to “1”. The next two adjacent characters are “DA”. Thereby, the lexicographic generation unit 109 sets the count value of “DA” to “1”. The next two adjacent characters are “AB”. Since “AB” has already appeared, the lexicographic generation unit 109 increases the count value of “AB” by “1” to “2”. Thereafter, the lexicographic generation unit 109 repeats the same process (count process).
The table shown in FIG. 3 is a table showing the number of appearances (count value) of two adjacent characters in the data D001. The number of appearances (count value) is a value obtained by the lexicographic generation unit 109 performing the above processing. Based on the table shown in FIG. 3, in the data D001, the adjacent two characters having the highest appearance count are “AB”, and the appearance count (count value) of “AB” is four.
Next, the lexicographic generation unit 109, based on the result obtained by the counting process as described above, another character (for example, “G”) corresponding to the adjacent two characters “AB” having the highest number of appearances. Is set as the replacement character. The replacement character (replacement character) is a character not included in the input data D001. This is because if the characters included in the input data D001 are set as replacement characters, the compressed data cannot be restored to the original state. The replacement character may be a number or a symbol.
The lexicographic generation unit 109 creates (generates) data corresponding to the adjacent two characters “AB” and the replacement character “G” as dictionary data.
Further, the lexicographic generation unit 109 replaces “AB” with the replacement character “G” in the input data D001. As a result, the lexicographic generation unit 109 creates data D002.
Next, the lexicographic generation unit 109 counts the number of appearances of two adjacent characters for the data D002 as well as the input data D001. Thereby, the lexicographic generation unit 109 can obtain information as shown in FIG. The table shown in FIG. 4 is a table showing the number of appearances (count value) of two adjacent characters in the data D002. Based on the table shown in FIG. 4, in the data D002, the two adjacent characters having the highest number of appearances are “DE”, and the number of appearances (count value) of “DE” is three. Therefore, the lexicographic generation unit 109 sets a replacement character (“H”) that replaces the adjacent two characters “DE” having the highest number of appearances, as described above. Accordingly, the lexicographic generation unit 109 creates data corresponding to the adjacent two characters “DE” and the replacement character “H” as dictionary data. Further, the lexicographic generation unit 109 creates data D003 by replacing “DE” in the data D002 with a replacement character “H”.
Further, the lexicographic generation unit 109 counts the number of appearances of two adjacent characters (character strings) for the data D003 as well as the input data D001. The table shown in FIG. 5 is a table showing the number of appearances (count value) of two adjacent characters in the data D003. Based on the table shown in FIG. 5, in the data D003, the two adjacent characters having the highest number of appearances are “GC”, and the number of appearances (count value) of “GC” is two. Therefore, the lexicographic generation unit 109 sets a replacement character (“I”) that replaces the two adjacent characters “GC” having the highest number of appearances, as described above. Thereby, the lexicographic generation unit 109 creates data corresponding to the adjacent two characters “GC” and the replacement character “I” as dictionary data. Further, the lexicographic generation unit 109 creates data D004 by replacing “GC” in the data D003 with the replacement character “I”.
Further, the lexicographic generation unit 109 performs the same process as described above for the data D004. That is, the lexicographic generation unit 109 counts the number of appearances of two adjacent characters for the data D004. With this process, the lexicographic generation unit 109 detects that any two adjacent characters appear only once in the data D004. Thereby, the lexicographic generation unit 109 ends the process of counting the number of appearances of two adjacent characters as described above and the process of setting a replacement character, that is, the process of generating dictionary data. The lexicographic generation unit 109 stores the generated new dictionary data (see FIG. 6) in the second storage unit 104. When dictionary data is already stored in the second storage unit 104, the lexicographic generation unit 109 deletes the old dictionary data and then stores new dictionary data in the second storage unit 104. .
Furthermore, the lexicographic generation unit 109 has a function of notifying the compression unit 102 that new dictionary data has been stored in the second storage unit 104 after the new dictionary data has been stored in the second storage unit 104. Have.
The monitoring unit (monitoring unit) 105 has a function of calculating the usage frequency of the dictionary data based on the usage status data and the total capacity data received from the compression unit 102. For example, the monitoring unit 105 calculates the usage frequency of the dictionary data using the following formula (1).
Usage frequency = (Number of times dictionary data is used) ÷ (Total data capacity) (1)
The monitoring unit 105 further has a function of outputting data indicating the calculated use frequency (use frequency data) to the timing control unit 106. FIG. 8 shows an example of usage frequency data output from the monitoring unit 105 to the timing control unit 106. The usage frequency data shown in FIG. 8 is data indicating the relationship between dictionary data and usage frequency.
In the initial operation after the operation of the data compression apparatus 100, the data output from the compression unit 102 to the monitoring unit 105 is only the total capacity data as described above. For this reason, the monitoring unit 105 outputs the total capacity data to the timing control unit 106 as it is without calculating the usage frequency of the dictionary data.
When the timing control unit (timing control unit) 106 detects that the use frequency of the dictionary data is decreasing based on the use frequency data received from the monitoring unit 105, the timing control unit (timing control unit) 106 instructs the expansion unit 107 to start the operation. It has a function to output. Specifically, the timing control unit 106 decreases the usage frequency of the dictionary data when the value (total value) of the usage frequencies of the received usage frequency data is smaller than a preset threshold value. And instructing the development unit 107 to start the operation. Note that the threshold used here may be set appropriately by the user.
In the initial operation after the operation of the data compression apparatus 100, the timing control unit 106 receives not the usage frequency data but the total capacity data from the monitoring unit 105. In such a case, for example, when the timing control unit 106 detects that the capacity value of the total capacity data received from the monitoring unit 105 is equal to or greater than a preset value, the timing control unit 106 starts the operation of the expansion unit 107. Instruct. Alternatively, the timing control unit 106 may instruct the development unit 107 to start operation when the number of times of receiving the total capacity data from the monitoring unit 105 reaches a preset number. Alternatively, the timing control unit 106 may instruct the development unit 107 to start an operation after a preset time has elapsed after first receiving the total capacity data from the monitoring unit 105.
The expansion unit (expansion unit) 107 has a function of expanding the compressed data stored in the first storage unit 103 when receiving an instruction to start an operation from the timing control unit 106. Note that the method in which the decompressing unit 107 decompresses the compressed data is a method corresponding to the compression method employed by the compressing unit 102. In the first embodiment, since the BPE method is used, the expansion unit 107 expands data based on the dictionary data stored in the second storage unit 104 as follows.
For example, here, the expansion unit 107 expands (data expands) compressed data T012 as illustrated in FIG. 9A with reference to the dictionary data illustrated in FIG. 9B. Note that the compressed data T012 in FIG. 9A corresponds to the data D004 in FIG. The dictionary data in FIG. 9B corresponds to the dictionary data in FIG. 10 to 12 are diagrams for explaining the flow of data development.
The expansion unit 107 first prepares a buffer 111 (see FIG. 10) having an appropriate size. Then, the decompressing unit 107 inputs the top data “G” of the compressed data T012 to the top of the buffer 111 as B001 shown in FIG. The head data “G” in the buffer 111 corresponds to “AB” when the dictionary data is referred to. Therefore, the expansion unit 107 replaces “G” in the buffer 111 with “AB” as shown in B002 in FIG. Next, the two characters (character string) corresponding to the data “A” at the head of the buffer 111 are not registered in the dictionary data. In such a case, the expansion unit 107 extracts the data “A” from the buffer 111 as output data. As a result, the top data of the buffer 111 becomes “B” as shown in B003 in FIG. Further, the output data is “A” as shown by T101 in FIG.
Two characters (character string) corresponding to the data “B” at the head of the buffer 111 are not registered in the dictionary data. From this, the expansion unit 107 extracts the data “B” from the buffer 111 as output data in the same manner as described above. As a result, the output data becomes “AB” as indicated by T102 in FIG.
By taking out “B”, the buffer 111 becomes empty. As a result, the decompressing unit 107 inputs the second data “D” of the compressed data T012 to the buffer 111 as indicated by B004 in FIG. Two characters (character string) corresponding to the data “D” are not registered in the dictionary data. From this, similarly to the above, the expansion unit 107 extracts the data “D” from the buffer 111 as output data. As a result, the output data becomes “ABD” as shown by T103 in FIG.
By taking out “D”, the buffer 111 becomes empty again. As a result, the decompressing unit 107 inputs the third data “I” of the compressed data T012 to the buffer 111 as shown in B005 in FIG. The data “I” corresponds to “GC” with reference to the dictionary data. Therefore, the expansion unit 107 replaces “I” in the buffer 111 with “GC” as shown in B006 in FIG.
Further, the expansion unit 107 replaces the leading data “G” in the buffer 111 with data “AB” as shown in B007 shown in FIG. 12 based on the dictionary data, as described above. As a result, the top data of the buffer 111 is data “A”. Two characters (character string) corresponding to the data “A” are not registered in the dictionary data. For this reason, the expansion unit 107 extracts the data “A” from the buffer 111 as output data in the same manner as described above. As a result, the output data becomes “ABDA” as shown by T104 in FIG.
By extracting the data “A”, the top data in the buffer 111 becomes data “B” as shown in B008 in FIG. Two characters (character string) corresponding to the data “B” are not registered in the dictionary data. Therefore, the expansion unit 107 extracts the data “B” from the buffer 111 as output data, as described above. As a result, the output data becomes “ABDAB” as shown by T105 in FIG.
By extracting the data “B”, the top data of the buffer 111 becomes data “C” as shown in B009 shown in FIG. The expansion unit 107 expands the data T012 by repeating the above processing (operation). That is, the data (output data) extracted from the buffer 111 is the expanded data.
The expansion unit 107 has a function of storing the data (expansion data) expanded as described above in the third storage unit 108. Furthermore, the decompressing unit 107 has a function of deleting all the compressed data stored in the first storage unit 103 after decompressing all the compressed data stored in the first storage unit 103. Yes. Further, the expansion unit 107 instructs the dictionary generation unit 109 to start the operation after deleting the compressed data in the first storage unit 103 and storing the expanded data in the third storage unit 108. It has a function to output. Thereby, the dictionary generation unit 109 receives the instruction as described above, and generates new dictionary data using the expanded data stored in the third storage unit 108.
Note that, as described above, the second storage unit 104 does not hold dictionary data in the initial operation after the data compression apparatus 100 starts operation. If the dictionary data is not stored in the second storage unit 104, the compression unit 102 does not perform an operation (process) for compressing the data. In this case, the data stored in the first storage unit 103 is uncompressed data (uncompressed data). Furthermore, the expansion unit 107 cannot expand the data unless the dictionary data is stored in the second storage unit 104. Therefore, the decompressing unit 107 has a function of moving the uncompressed data stored in the first storage unit 103 to the third storage unit 108 as it is.
In the first embodiment, the data stored in the third storage unit 108 when the dictionary generation unit 109 starts the operation of generating dictionary data is the initial operation after the data compression apparatus 100 starts operation, Alternatively, it is data stored since the previous dictionary data was generated. Since the dictionary generation unit 109 generates dictionary data using such data, it is possible to generate dictionary data with the highest data compression rate when the dictionary data is generated.
Actually, it takes time to generate the dictionary data. For this reason, data may be input from the data generation unit 110 while the dictionary data is being generated. In this case, if the compression unit 102 compresses the input data based on the old dictionary data stored in the second storage unit 104, the following inconvenience occurs. That is, the compressed data compressed based on the old dictionary data while generating new dictionary data does not match the compressed data compressed based on the new dictionary data after generating new dictionary data. Therefore, in the first embodiment, the compression unit 102 is configured not to compress data while generating dictionary data. Specifically, the compression unit 102 has a buffer (not shown). In addition, the compression unit 102 accumulates data input from the data generation unit 110 in the buffer during the period in which the dictionary generation unit 109 is operating. Then, as described above, after the dictionary generation unit 109 stores the generated new dictionary data in the second storage unit 104, the compression unit 102 refers to the data stored in the buffer with the new dictionary data. Compress while.
Hereinafter, an operation example (data compression method) of the data compression apparatus 100 according to the first embodiment will be described with reference to the flowchart of FIG. The flowchart of FIG. 13 shows the processing procedure of the computer program executed by the control device (CPU) 101 in the data compression device 100. The computer program is stored in a storage medium (for example, a non-volatile storage medium) of a storage device such as a memory or a hard disk included in the data compression apparatus 100. Further, for example, the computer program is stored in a portable storage medium such as a compact disk (CD) or a memory card, and then transferred from the portable storage medium to the storage medium of the storage device of the data compression apparatus 100. May be stored.
First, when data is input from the data generation unit 110 (step S010), the compression unit 102 compresses the input data based on the dictionary data stored in the second storage unit 104 (step S020). In addition, the compression unit 102 stores compressed data, which is data after compression, in the first storage unit 103. Subsequently, the monitoring unit 105 uses the usage status data output from the compression unit 102 (data indicating the dictionary data used for compression and the number of uses thereof), and the total capacity data (input from the data generation unit 110 to the compression unit 102). The dictionary use frequency is calculated based on the data indicating the total capacity of the data (step S030). Then, the monitoring unit 105 outputs the calculation result to the timing control unit 106.
Next, the timing control unit 106 determines whether or not the value (total value) obtained by totaling the usage frequencies input from the monitoring unit 105 has decreased below a set threshold value (step S040). The timing control unit 106 does nothing if the total value is not less than or equal to the threshold value (NO in step S040). Then, the data compression apparatus 100 repeats the operations after step S020. On the other hand, when the total value is equal to or less than the threshold (YES in step S040), the timing control unit 106 instructs the development unit 107 to start the operation.
As a result, the decompression unit 107 decompresses the compressed data stored in the first storage unit 103 while referring to the dictionary data stored in the second storage unit 104, and decompresses the decompressed data that is the decompressed data. It stores in the 3rd memory | storage part 108 (step S050). Subsequently, the expansion unit 107 deletes the compressed data stored in the first storage unit 103 and instructs the dictionary generation unit 109 to start operation.
Thereby, the dictionary generation unit 109 generates dictionary data from the decompressed data stored in the third storage unit 108 according to a predetermined type of lexicographic compression method (BPE method in the first embodiment). Then, the dictionary generation unit 109 stores (updates) the generated new dictionary data in the second storage unit 104 (step S060). That is, the dictionary generation unit 109 generates new dictionary data. Then, the dictionary generation unit 109 deletes the old dictionary data stored so far in the second storage unit 104 and stores newly generated dictionary data in the second storage unit 104. Further, the dictionary generation unit 109 stores (updates) new dictionary data in the second storage unit 104 and then instructs the compression unit 102 to compress the decompressed data.
Thereby, the compression unit 102 compresses the decompressed data stored in the third storage unit 108 while referring to the new dictionary data stored in the second storage unit 104 (step S070). Then, the compression unit 102 stores compressed data, which is data after compression, in the first storage unit 103. Thereafter, the data compression apparatus 100 repeats the operations after step S020.
Note that when the data compression apparatus 100 is in the initial operation (that is, the dictionary data is not stored in the second storage unit 104), the data compression apparatus 100 operates as follows.
For example, when data is input from the data generation unit 110, the compression unit 102 stores the data in the first storage unit 103 as it is and outputs data indicating the total capacity of the data (total capacity data) to the monitoring unit 105. To do. The monitoring unit 105 outputs the total capacity data as it is to the timing control unit 106. The timing control unit 106 is a development unit at a timing based on the total capacity input from the data generation unit 110, the number of times the total capacity data is input, the elapsed time since the first input of the total capacity data, or the like. An operation start is instructed to 107. Thereby, the expansion unit 107 stores the data stored in the first storage unit 103 in the third storage unit 108 as it is. Then, the dictionary generation unit 109 generates dictionary data using the data stored in the third storage unit 108 and stores the generated new dictionary data in the second storage unit 104. Thereafter, the dictionary generation unit 109 instructs the compression unit 102 to start operation. Thus, the compression unit 102 compresses the data stored in the third storage unit 108 and stores the compressed data (compressed data) in the first storage unit 103. With this operation, after the dictionary data is generated and stored in the second storage unit 104, the data compression apparatus 100 repeats the operation as shown in the flowchart of FIG.
As described above, according to the first embodiment, the data compression apparatus 100 includes the monitoring unit 105 and the timing control unit 106. The monitoring unit 105 monitors how much the compression unit 102 is using dictionary data stored in the second storage unit 104 when the compression unit 102 compresses data input from the data generation unit 110. Then, the timing control unit 106 has decreased the frequency of use of dictionary data based on the monitoring result of the monitoring unit 105 (in other words, the effectiveness of the dictionary data stored in the second storage unit 104 has decreased). In this case, the dictionary generation unit 109 is instructed to generate new dictionary data. The new dictionary data generated by the dictionary generation unit 109 in response to this instruction is data based on all the expanded data stored in the third storage unit 108. Therefore, the new dictionary data is the dictionary data having the highest use frequency (that is, the compression effect is high) at the time when the dictionary data is generated.
As described above, the data compression apparatus 100 according to the first embodiment can generate dictionary data with increased effectiveness when the use frequency (validity) of dictionary data decreases, and can store dictionary data used for compression. The new dictionary data can be updated. Thereby, when the tendency of the data input from the data generation part 110 changes, the data compression apparatus 100 produces | generates new dictionary data based on the data after a tendency changes, and based on the said new dictionary data Data can be compressed. Therefore, the data compression apparatus 100 can always efficiently compress the data even when the tendency of the data input from the data generation unit 110 changes. In other words, the data compression apparatus 100 can maintain an efficient compression rate.
(Second Embodiment)
A second embodiment according to the present invention will be described below with reference to the drawings.
FIG. 14 is a block diagram illustrating the configuration of the data compression apparatus according to the second embodiment. In the description of the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and redundant description thereof is omitted.
In addition to the configuration of the data compression apparatus 100 of the first embodiment, the data compression apparatus (data compression system) 200 of the second embodiment further includes a resource monitoring unit (resource monitoring unit) 201 and an adjustment unit (parameter adjustment unit). 202. The data compression apparatus 200 includes a timing control unit (dictionary update timing control unit) 203 instead of the timing control unit 106 shown in the first embodiment.
The data compression apparatus 100 according to the first embodiment described above can maintain an efficient compression rate by updating the dictionary data when the usage frequency of the dictionary data decreases. Thereby, the data compression apparatus 100 can suppress the usage amount of a storage medium such as a memory or a disk (for example, a storage medium constituting the third storage unit 108 or the like). On the other hand, since the operation of updating (generating) dictionary data increases the load on the data compression apparatus 100, it is not desirable to update the dictionary data frequently. For this reason, when there is a margin in the capacity of the memory or the disk, it may be better to give priority to lowering the load on the data compression apparatus 100 than to increase the compression rate.
Considering this, the data compression apparatus 200 of the second embodiment includes the resource monitoring unit 201 and the adjustment unit 202 as described above.
The resource monitoring unit 201 has a function of monitoring (detecting) a free capacity of a resource (for example, a storage medium constituting the third storage unit 108) and outputting a monitoring result to the adjustment unit 202. The timing at which the resource monitoring unit 201 outputs the monitoring result to the adjustment unit 202 is, for example, every predetermined time interval.
The adjustment unit (adjustment unit) 202 has a function of adjusting the threshold used by the timing control unit 203 based on the monitoring result received from the resource monitoring unit 201. That is, if the monitoring result indicates that there is a sufficient free space in the resource, the resource is not immediately compressed even if the effectiveness (usage frequency) of the dictionary data is low. Thus, the adjustment unit 202 adjusts (sets) the threshold value to a small value so that the dictionary data update frequency is low. On the other hand, if the monitoring result indicates that there is no room for the free space of the resource, the resource is immediately compressed when the effectiveness of the dictionary data becomes low. Therefore, the adjustment unit 202 adjusts (sets) the threshold value to a large value so that the dictionary data update frequency increases.
More specifically, for example, the adjustment unit 202 adjusts (sets) the threshold value to “0.5” when the free capacity (remaining capacity) of the resource is 80% or more. The adjustment unit 202 adjusts the threshold value to “0.7” when the free space of the resource is less than 80% and 50% or more. Furthermore, the adjustment unit 202 sets the threshold value to “1.0” when the free capacity of the resource is less than 50%. As described above, the adjustment unit 202 may set the threshold value stepwise in accordance with the free capacity of the resource. Alternatively, the adjustment unit 202 may continuously set the threshold according to the free capacity of the resource. Note that any method may be employed as the method by which the adjustment unit 202 sets the threshold as long as the threshold is set according to the free capacity of the resource.
The timing control unit 203 has a function of determining the timing for updating the dictionary data, using the threshold value set as described above, similarly to the timing control unit 106 in the first embodiment. The timing control unit 203 has the same function as the timing control unit 106 except that the threshold set by the adjustment unit 202 is used.
As described above, the data compression apparatus 200 according to the second embodiment can change the update timing of the dictionary data according to the free capacity of the resource. Thereby, when there is a sufficient resource, the data compression apparatus 200 can reduce the number of times of updating the dictionary data (dictionary update number), and thus the load on the apparatus can be reduced. On the other hand, when there is no room for resources, the data compression apparatus 200 can increase the number of times the dictionary is updated, so that the amount of resources used (consumption) can be suppressed.
Next, operation examples of the resource monitoring unit 201 and the adjustment unit 202 will be described with reference to FIGS. 15 and 16. FIG. 15 is a flowchart illustrating exemplary operations of the resource monitoring unit 201 and the adjustment unit 202. FIG. 16 is a flowchart illustrating a more specific operation example of the adjustment unit 202. The operations shown in the flowcharts of FIGS. 15 and 16 are executed asynchronously with the operation shown in the flowchart of FIG.
15 and 16 show the processing procedure of the computer program executed by the control device (CPU) 101 in the data compression apparatus 200, similarly to the flowchart of FIG. The computer program is stored in a storage medium (for example, a non-volatile storage medium) of a storage device such as a memory or a hard disk included in the data compression apparatus 200 as described above. Further, for example, the computer program is stored in a portable storage medium such as a compact disk (CD) or a memory card, and then transferred from the portable storage medium to the storage medium of the data compression apparatus 200. May be stored.
For example, the resource monitoring unit 201 monitors the free capacity of resources such as a computer memory and a disk included in the data compression apparatus 200 at predetermined time intervals, and outputs the monitoring result to the adjustment unit 202 (see FIG. 15 step S110). Subsequently, based on the monitoring result, the adjustment unit 202 sets the threshold used by the timing control unit 203 to a small value when the free capacity of the resource is large. On the contrary, the adjustment unit 202 sets the threshold value to a large value when the resource free capacity is small (step S120).
Here, an operation example for setting the threshold value will be described with reference to FIG. The operation in step S110 shown in FIG. 16 is the same as the operation in step S110 shown in FIG.
For example, the adjustment unit 202 determines whether the free capacity of the resource is 80% or more based on the monitoring result received from the resource monitoring unit 201 (step S121). If the free space of the resource is 80% or more, the adjustment unit 202 determines that the free space of the resource has a margin, and sets the threshold used by the timing control unit 203 to “0.5” (step S123). ).
If the adjustment unit 202 determines in step S121 that the resource free capacity is not 80% or more, the adjustment unit 202 determines whether the resource free capacity is 50% or more (step S122). If the free capacity of the resource is 50% or more, the adjustment unit 202 sets the threshold value to “0.7” (step S124).
If the adjustment unit 202 determines in step S122 that the free space of the resource is not 50% or more, the adjustment unit 202 determines that there is no room in the free space of the resource, and sets the threshold to “1.0” (step 1). S125). As described above, in the second embodiment, the adjustment unit 202 adjusts (sets) the threshold according to the free capacity of the resource.
Note that the flowchart (example of operation) shown in FIG. 16 only shows an example of a method for adjusting the threshold value, and the method for adjusting the threshold value is not limited to the method described above.
For example, the step of adjusting the threshold value may be more steps than the three steps of 80% or more, less than 80%, 50% or more, and less than 50%. Further, the threshold value to be adjusted is not limited to “0.5”, “0.7”, and “1.0”.
(Third embodiment)
A third embodiment according to the present invention will be described below with reference to the drawings.
FIG. 17 is a block diagram showing the configuration of the data compression apparatus in the third embodiment. In the description of the third embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and redundant description thereof is omitted.
In addition to the configuration of the data compression apparatus 100 of the first embodiment, the data compression apparatus (data compression system) 300 of the third embodiment further includes a cost estimation unit (dictionary construction cost estimation unit) 301 and a compression rate estimation unit ( Compression rate estimating means) 302. In addition, the data compression apparatus 300 includes a timing control unit (dictionary update timing control means) 303 as described later, instead of the timing control unit 106 in the data compression apparatus 100 of the first embodiment.
As described in the second embodiment, the data compression apparatus 100 according to the first embodiment can maintain an efficient compression ratio by updating the dictionary data when the use frequency of the dictionary data is reduced. Thereby, the data compression apparatus 100 can suppress the usage amount of a storage medium such as a memory or a disk. On the other hand, the operation of updating the dictionary data increases the load on the data compression apparatus 100, so it is not desirable to update the dictionary data frequently. For this reason, when there is a margin in the capacity of the memory or the disk, it may be better to give priority to lowering the load on the data compression apparatus 100 than to increase the compression rate.
Considering this, the data compression apparatus 300 according to the third embodiment includes a cost estimation unit 301 and a compression rate estimation unit 302. And the data compression apparatus 300 changes the timing which updates dictionary data with the method different from 2nd Embodiment.
The cost estimation unit (cost estimation means) 301 has a function of estimating (estimating) a cost required for newly generating dictionary data. That is, when the cost estimation unit 301 receives an instruction to start an operation from the timing control unit (timing control unit) 303, the cost estimation unit 301 starts the operation. The timing at which the instruction is received is the timing at which the timing control unit 303 determines that the use frequency of the dictionary data has decreased due to the same operation as the timing control unit 106 in the first embodiment. The cost estimation unit 301 newly generates dictionary data based on the data amount of compressed data stored in the first storage unit 103 and the data amount of dictionary data stored in the second storage unit 104. Estimate the cost required.
As described in the first embodiment, in order to generate a dictionary, first, the decompression unit 107 stores all the compressed data stored in the first storage unit 103 in the second storage unit 104. The dictionary data is expanded with reference to existing dictionary data and stored in the third storage unit 108. Subsequently, the dictionary generation unit 109 counts the number of appearances of two adjacent characters for all the expanded data stored in the third storage unit 108, and the adjacent 2 with the largest count number. The operation (process) of replacing a character with another character is repeated. Such an operation is performed until the number of appearances of two adjacent characters in the data is all one. For this reason, the cost required for generating a dictionary is a cost proportional to the amount of data stored in the first storage unit 103 and the amount of dictionary data stored in the second storage unit 104.
For this reason, the cost estimation unit 301 calculates the cost required for generating dictionary data using the following equation (2), for example.
C = W1 × D1 + W2 × D2 (2)
In Equation (2), C represents the cost required for generating dictionary data. D1 indicates the amount of compressed data stored in the first storage unit 103. D2 indicates the amount of dictionary data stored in the second storage unit 104. Further, W1 and W2 indicate weight constants. These W1 and W2 are set to appropriate values.
When the cost estimation unit 301 finishes calculating the cost C, the cost estimation unit 301 outputs the cost C as a calculation result to the timing control unit 303.
Formula (2) is an example of a calculation formula for calculating the cost. The cost estimation unit 301 uses the data amount of compressed data stored in the first storage unit 103 and the data amount of dictionary data stored in the second storage unit 104 to calculate the cost. Any method may be adopted as long as it is.
The compression rate estimation unit 302 has a function of estimating (estimating) the compression rate after updating to new dictionary data. That is, the compression rate estimation unit 302 starts the operation when receiving an instruction to start the operation from the timing control unit 303. The timing at which the instruction is received is the timing at which the timing control unit 303 determines that the use frequency of the dictionary data has decreased as described above. The compression rate estimation unit 302 expands a part of the compressed data stored in the first storage unit 103 using the dictionary data stored in the second storage unit 104.
The data amount developed by the compression rate estimation unit 302 may be a predetermined fixed data amount, or a data amount corresponding to 10% of the total data amount stored in the first storage unit 103. The amount of data proportional to the total amount of data in the first storage unit 103 may be used.
Then, the compression rate estimation unit 302 generates new dictionary data using the expanded data, and compresses the expanded data used when generating the dictionary data using the new dictionary data. Then, the compression rate estimation unit 302 calculates the compression rate of the compressed data. Here, the compression ratio is an index indicating how much the data amount after compression is smaller than the data amount before compression. For example, when the data amount before compression is 100 MB and the data amount after compression is 50 MB, the compression rate is 50%. After calculating the compression rate, the compression rate estimation unit 302 outputs data indicating the compression rate to the timing control unit 303.
In the third embodiment, as described above, the compression rate estimation unit 302 expands only a part of the data, not all the data stored in the first storage unit 103. Furthermore, the compression rate estimation unit 302 generates dictionary data based on the expanded data, and further calculates the compression rate (dictionary data is evaluated). Thereby, the compression rate estimation unit 302 can estimate (calculate) the compression effect (compression rate) without imposing a heavy load on the data compression apparatus 300.
Unlike the first embodiment, when the timing control unit 303 determines that the use frequency of the dictionary data has been reduced in the same manner as described above, the timing control unit 303 is not the expansion unit 107 but the cost estimation unit 301. And output to the compression rate estimation unit 302.
In addition, the timing control unit 303 determines whether to update (generate) dictionary data based on the cost calculated by the cost estimation unit 301 and the compression rate calculated by the compression rate estimation unit 302.
The determination can be made using, for example, data obtained by scoring the cost and data obtained by scoring the compression rate. Here, the following expression (3) is an expression for calculating data obtained by scoring the cost. Expression (4) is an expression for calculating data obtained by scoring the compression ratio.
S1 = W3 × C (3)
S2 = W4 × R (4)
In Expression (3), S1 represents data obtained by scoring costs. W3 represents a weight constant. C represents the cost calculated by the equation (2). Moreover, in Formula (4), S2 shows the data which scored the compression rate. W4 represents a weight constant. R represents the compression rate calculated by the compression rate estimation unit 302. Note that the method of calculating the scored data using the equations (3) and (4) is an example of a method of calculating the scored data. The method for calculating the scored data is not limited to the above method.
The timing control unit 303 generates new dictionary data when the data (data obtained by scoring the compression rate) S2 has a larger value than the data (data obtained by scoring the cost) S1 (dictionary data). Is updated).
In other words, when the data (data that scores the cost) S1 is larger than the data (data that scores the compression rate) S2, the timing for newly generating dictionary data (updating dictionary data) is not. As a result, the data compression apparatus 300 does not perform a heavy load operation (processing) of updating dictionary data even though the compression rate is small. For this reason, the data compression apparatus 300 can reduce the load of the apparatus and eliminate waste.
Hereinafter, an operation example of the data compression apparatus 300 according to the third embodiment will be described with reference to the flowchart of FIG. In FIG. 18, the same reference numerals as those in FIG. 13 are given to portions showing the same operations as those shown in FIG. 13, and the duplicate description thereof is omitted.
The flowchart of FIG. 18 shows the processing procedure of the computer program executed by the control device (CPU) 101 in the data compression apparatus 300, similarly to the flowchart of FIG. As described above, the computer program is stored in a storage medium (for example, a non-volatile storage medium) of a storage device such as a memory or a hard disk included in the data compression apparatus 300. Further, the computer program is stored in a portable storage medium such as a compact disk (CD) or a memory card, for example, and then transferred from the portable storage medium to the storage medium of the data compression apparatus 300. May be stored.
In step S043 in FIG. 18, the timing control unit 303 determines whether the total value of the usage frequencies received from the monitoring unit 105 has decreased below a set threshold value. Then, the timing control unit 303 does nothing if the total value of the usage frequencies is not less than or equal to the threshold (NO in step S043). Then, the data compression apparatus 300 repeats the operations after step S020. On the other hand, when the total value of the usage frequencies is equal to or less than the threshold value (YES in step S043), the timing control unit 303 instructs the cost estimation unit 301 and the compression rate estimation unit 302 to start operations. Put out.
When receiving an instruction from the timing control unit 303, the cost estimation unit 301 calculates the cost, and the compression rate estimation unit 302 calculates the compression rate (step S210). That is, in step S210, the cost estimation unit 301, for example, based on the data amount of compressed data stored in the first storage unit 103 and the data amount of dictionary data stored in the second storage unit 104, for example The cost C is estimated (calculated) using the formula (2). Then, the cost estimation unit 301 outputs the cost C to the timing control unit 303. On the other hand, the compression rate estimation unit 302 expands a part of the compressed data stored in the first storage unit 103 using the dictionary data stored in the second storage unit 104. Then, the compression rate estimation unit 302 generates new dictionary data using the expanded data, and further uses the expanded data used when generating the new dictionary data using the new dictionary data. Compress. Further, the compression rate estimation unit 302 uses the compressed data (compressed data) and the data before compression to compress the compression rate (compression rate = (data amount after compression) / (data before compression). Amount)). Then, the compression rate estimation unit 302 outputs the calculated compression rate to the timing control unit 303.
Subsequently, the timing control unit 303 scores the cost C received from the cost estimation unit 301 using Expression (3). That is, the timing control unit 303 generates scored data S1. In addition, the timing control unit 303 scores the compression rate R received from the compression rate estimation unit 302 using Expression (4). That is, the timing control unit 303 generates scored data S2. Then, the timing control unit 303 compares the data S1 and the data S2, and determines whether or not the compression rate is greater than the cost (step S220). The timing control unit 303 does nothing when the compression rate is less than the cost (NO in step S220). Then, the data compression apparatus 300 repeats the operations after step S020. On the other hand, when the compression rate is greater than the cost (YES in step S220), the timing control unit 303 issues an instruction to start the operation to the expansion unit 107. The data compression apparatus 300 repeats the operations in and after step S020 after performing the operations in steps S050, S060, and S070 described above.
Since the data compression apparatus 300 according to the third embodiment can change the update timing of the dictionary data as described above, it performs an operation with a large load (an operation to update the dictionary data) even though the compression effect is small. Nothing will happen. Thereby, in addition to the effect of 1st Embodiment, the data compression apparatus 300 can reduce the load of an apparatus and the effect that a waste can be eliminated is acquired.
(Fourth embodiment)
A fourth embodiment according to the present invention will be described below with reference to the drawings.
FIG. 19 is a block diagram showing the configuration of the data compression apparatus according to the fourth embodiment. In the description of the fourth embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and redundant description thereof is omitted.
The data compression apparatus (data compression system) 400 according to the fourth embodiment further includes a deletion unit (dictionary data deletion unit) 403 in addition to the configuration of the first embodiment. Further, the data compression apparatus 400 includes a partial expansion unit (compressed data partial expansion unit) 402 instead of the expansion unit 107 illustrated in the first embodiment, and the timing control unit illustrated in the first embodiment. Instead of 106, a timing control unit (dictionary update timing control means) 401 is provided.
The data compression apparatus 100 according to the first embodiment described above can always have dictionary data with a high compression rate by updating the dictionary data when the frequency of use of the dictionary data has decreased. However, when updating the dictionary data, the data compression apparatus 100 recreates all the dictionary data every time and updates the dictionary data stored in the second storage unit 104. For this reason, the following problems occur. In other words, even if the usage frequency of the entire dictionary data is decreasing, the usage frequency may not be so low or the usage frequency may be increasing for some dictionary data. In such a case, it is useless to recreate all dictionary data. In addition, the operation of updating dictionary data is expensive. Considering these things, the data compression apparatus does not recreate all dictionary data every time the dictionary data is updated, but only recreates a part of the dictionary data. It is preferable to recreate only the dictionary data. As described above, when only a part of the dictionary data is recreated, the data compression apparatus 100 can reduce the load and increase the processing speed for updating the dictionary data.
Therefore, the data compression apparatus 400 of the fourth embodiment deletes only dictionary data whose usage frequency has dropped below a predetermined set value at the time of updating dictionary data, and new dictionary data is stored as described below. Generate.
For example, as described above, when it is determined that the dictionary data is to be updated, the timing control unit 401 instructs the partial expansion unit 402 to start the operation. At this time, the timing control unit 401 extracts dictionary data whose usage frequency has dropped below a set value based on the usage frequency data as shown in FIG. Then, the timing control unit 401 outputs the extracted dictionary data to the partial expansion unit 402 and the deletion unit 403. Note that the setting value used when extracting the dictionary data as described above may be a fixed value or may be arbitrarily changed during operation.
The partial expansion unit (partial expansion means) 402 has a function of expanding a part of the compressed data stored in the first storage unit 103. That is, the partial expansion unit 402 expands the compressed data stored in the first storage unit 103 using the dictionary data received from the timing control unit 401 and stores the expanded data in the third storage unit 108. . Note that the dictionary data output from the timing control unit 401 to the partial expansion unit 402 is basically a part of the dictionary data stored in the second storage unit 104. For this reason, the partial expansion unit 402 expands only a part of the compressed data stored in the first storage unit 103.
Further, the dictionary data output from the timing control unit 401 to the partial expansion unit 402 is dictionary data whose use frequency is reduced as described above. For this reason, the partial decompression unit 402 decompresses only the compressed data that is not so high in the compressed data stored in the first storage unit 103 and does not decompress the compressed data that has a high compression rate.
When the partial decompression unit 402 finishes the operation (process) for decompressing the compressed data as described above, the partial decompression unit 402 deletes all the compressed data stored in the first storage unit 103. In addition, when the partial expansion unit 402 finishes storing the expanded data in the third storage unit 108, the partial expansion unit 402 outputs an instruction to start the operation to the dictionary generation unit 109.
The deletion unit (deletion unit) 403 has a function of deleting dictionary data corresponding to the dictionary data received from the timing control unit 401 from the second storage unit 104.
When the dictionary generation unit 109 receives the instruction from the partial expansion unit 402, the dictionary generation unit 109 generates dictionary data based on the expansion data stored in the third storage unit 108 as described above. Then, the dictionary generation unit 109 stores (appends) the newly generated dictionary data in the second storage unit 104. Functions other than the above function (operation) in the dictionary generation unit 109 are the same as those in the first embodiment.
As described above, the data compression apparatus 400 according to the fourth embodiment can delete only dictionary data whose use frequency is reduced and can update only a part of the dictionary data. For this reason, the data compression apparatus 400 can reduce the load on the apparatus and increase the processing speed for updating the dictionary data as compared with the case of recreating all dictionary data.
Hereinafter, an operation example (data compression method) of the data compression system 400 of the fourth embodiment will be described with reference to the flowchart of FIG. In FIG. 20, the same reference numerals as those in FIG. 13 are given to the portions showing the same operations as those shown in FIG.
The flowchart of FIG. 20 shows the processing procedure of the computer program executed by the control device (CPU) 101 in the data compression apparatus 400, similarly to the flowchart of FIG. The computer program is stored in a storage medium (for example, a non-volatile storage medium) of a storage device such as a memory or a hard disk included in the data compression apparatus 400 as described above. Further, for example, the computer program is stored in a portable storage medium such as a compact disk (CD) or a memory card, and then transferred from the portable storage medium to the storage medium of the data compression apparatus 400. May be stored.
In step S045 of FIG. 20, the timing control unit 401 determines whether or not the total value of the usage frequencies received from the monitoring unit 105 has decreased below a set threshold value. Then, the timing control unit 401 does nothing if the total value of the usage frequencies is not less than or equal to the threshold (NO in step S045). Then, the data compression apparatus 400 repeats the operations after step S020. On the other hand, when the total value of the usage frequencies is equal to or less than the threshold value (YES in step S045), the timing control unit 401 extracts dictionary data whose usage frequency has decreased below the set value based on the usage frequency data received from the monitoring unit 105. To do. Then, the timing control unit 401 outputs the extracted dictionary data to the partial expansion unit 402 and issues an instruction to start the operation. The timing control unit 401 also outputs the extracted dictionary data to the deletion unit 403 and issues an instruction to start the operation.
Subsequently, upon receiving the instruction from the timing control unit 401, the partial expansion unit 402 expands a part of the compressed data (step S310). That is, in this step S310, the partial decompression unit 402 performs timing only on the compressed data stored in the first storage unit 103 and compressed with dictionary data whose use frequency has dropped below the set value. Expansion is performed using the dictionary data received from the control unit 401. Then, the partial expansion unit 402 stores the expanded data in the third storage unit 108.
In step S <b> 310, when the partial decompression unit 402 finishes decompressing the compressed data stored in the first storage unit 103, the partial decompression unit 402 deletes all the compressed data stored in the first storage unit 103. In addition, when the partial expansion unit 402 finishes storing the expanded data in the third storage unit 108, the partial expansion unit 402 instructs the dictionary generation unit 109 to start the operation.
On the other hand, the deletion unit 403 deletes data corresponding to the dictionary data received from the timing control unit 401 from the second storage unit 104 (step S320). Subsequently, the dictionary generation unit 109 generates dictionary data and updates the dictionary data in the second storage unit 104 (step S065). In step S065, the dictionary generation unit 109 generates dictionary data based on the expanded data stored in the third storage unit 108. When the dictionary generation unit 109 finishes generating the dictionary data, the dictionary generation unit 109 stores (adds) the generated new dictionary data in the second storage unit 104.
Subsequently, the compression unit 102 compresses the decompressed data stored in the third storage unit 108 (step S075). In step S075, the compression unit 102 compresses the decompressed data while referring to dictionary data including newly generated dictionary data. The compression unit 102 stores the compressed data (compressed data) in the first storage unit 103.
As described above, the data compression apparatus 400 according to the fourth embodiment can update only the dictionary data whose frequency of use is reduced and update the dictionary data instead of recreating all dictionary data. For this reason, the data compression apparatus 400 can reduce the load for generating dictionary data and reduce the time required to update the dictionary data as compared to the case of recreating all dictionary data from 1 (the processing speed can be reduced). Effect).
(Fifth embodiment)
The fifth embodiment according to the present invention will be described below with reference to the drawings.
FIG. 21 is a block diagram showing the configuration of the data compression apparatus according to the fifth embodiment of the present invention. Note that in the description of the fifth embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and redundant description thereof is omitted.
The control apparatus 101 which comprises the data compression apparatus (data compression system) 500 of 5th Embodiment is the compression part 102, the switching part (compression algorithm switching means) 501, and the compression cost estimation part (compression cost estimation means) 502. A compression rate estimation unit (compression rate estimation unit) 503 and an update unit (switching information update unit) 504 are provided.
In each of the above-described embodiments, the compression unit 102 compresses data using a lexicographic compression algorithm (lexicographic compression method) in any case. In contrast, in the fifth embodiment, the compression unit 102 has a function of compressing data using a compression algorithm other than the lexicographic compression algorithm. In addition, when the data is compressed by a compression algorithm other than the lexicographic compression algorithm, the compression unit 102 has a function of obtaining a compression rate of the data after compression with respect to the data before compression.
When the data is compressed by a compression algorithm other than the lexicographic compression algorithm, the compression unit 102 receives the total capacity data (data indicating the total capacity of the data received from the data generation unit 110) and the data generation unit 110. Data indicating the elapsed time since the data was received and data indicating the compression rate are output to the switching unit 501. The compression rate output to the switching unit 501 is a compression rate when the compression unit 102 compresses data received from the data generation unit 110 after the compression unit 102 previously output the compression rate to the switching unit 501. .
When the data is compressed by the lexicographic compression algorithm, the compression unit 102 uses the above-described usage status data (data indicating how much dictionary data is used) and total capacity data (from the data generation unit 110). Data indicating the total capacity of the received data) and data indicating the elapsed time since the data was received from the data generation unit 110 (elapsed time data) are output to the switching unit 501.
Based on various data received from the compression unit 102, the switching unit (switching unit) 501 calculates the data flow rate per unit time using, for example, the following equation (5).
Data flow rate = {(total capacity of current input data) − (total capacity of previous input data)} ÷ {(current elapsed time) − (previous elapsed time)} (5)
The switching unit 501 stores the total capacity data and the elapsed time data received from the compression unit 102 in order to perform the above calculation. When the switching unit 501 newly receives the total capacity data and the elapsed time data from the compression unit 102, the switching unit 501 deletes the old total capacity data and the elapsed time data. Further, when the switching unit 501 receives the total capacity data and the elapsed time data for the first time from the compression unit 102, the switching unit 501 calculates the data flow rate using the following equation (6).
Data flow rate = (Total capacity of current input data) / (Current elapsed time) (6)
The switching unit 501 then compresses the compression cost estimation unit 502 when the calculated data flow rate becomes lower than the predetermined threshold T1 or higher than the predetermined threshold T2 larger than the threshold T1. The compression rate estimation unit 503 is instructed to start the operation.
The compression cost estimation unit (compression cost estimation means) 502 has a function of estimating (estimating) a cost required for compressing data. That is, the compression cost estimation unit 502 starts operation when receiving an instruction from the switching unit 501. The compression cost estimation unit 502 compresses data using each of a plurality of preset compression algorithms. Then, the compression cost estimation unit 502 calculates a calculation amount (cost) required for the process of compressing the data. Specifically, the compression cost estimation unit 502 expands a part of the compressed data stored in the first storage unit 103 using the dictionary data stored in the second storage unit 104, thereby Generate data before compression. Note that the data amount of the data developed by the compression cost estimation unit 502 may be a data amount corresponding to a preset ratio with respect to the entire compressed data stored in the first storage unit 103, or may be a predetermined fixed amount. It may be an amount.
The compression cost estimation unit 502 calculates the calculation amount AC for each of a plurality of preset compression algorithms for the uncompressed data generated as described above, using the following equation (7). .
AC = (data amount to be compressed) ÷ (time taken for compression) (7)
In Expression (7), “data amount to be compressed” indicates the data amount of the data before compression obtained by the compression cost estimation unit 502 expanding as described above. The “time required for compression” indicates the time required to compress the data to be compressed.
Then, the compression cost estimating unit 502 scores the calculation amount AC using, for example, the equation (8) or the equation (9), and outputs the scored data to the switching unit 501.
SA1 = W11 × AC1 (8)
SA2 = W12 × AC2 (9)
In Expression (8), SA1 indicates data (compression cost score) obtained by scoring the cost required when data is compressed using the compression algorithm 1. W11 represents a weight constant. AC1 indicates the calculation amount AC corresponding to the compression algorithm 1. In Equation (9), SA2 indicates data (compression cost score) obtained by scoring the cost required when data is compressed using the compression algorithm 2. W12 represents a weight constant. AC2 indicates the calculation amount AC corresponding to the compression algorithm 2.
Note that the method of obtaining scored data using the formula (8) or the formula (9) is merely an example of a method for scoring data. The method for scoring data by the compression cost estimation unit 502 is not particularly limited as long as it is a method for calculating using the calculation amount AC.
The compression rate estimation unit (compression rate estimation means) 503 has a function of estimating (estimating) the compression rate when new dictionary data is generated. That is, when the compression rate estimation unit 503 receives an instruction from the switching unit 501, the compression rate estimation unit 503 starts an operation. The compression rate estimation unit 503 calculates a compression rate. Specifically, for example, similarly to the compression cost estimation unit 502, the compression rate estimation unit 503 generates pre-compression data by using a part of the compressed data stored in the first storage unit 103. . The compression rate estimation unit 503 then determines how much the data amount after compression is smaller than the data amount before compression for each of a plurality of preset compression algorithms with respect to the data before compression. The compression ratio, which is an index indicating the above, is calculated. Then, the compression rate estimation unit 503 scores the calculated compression rate using, for example, Expression (10) or Expression (11), and outputs the scored data to the switching unit 501.
SB1 = W13 × R1 (10)
SB2 = W14 × R2 (11)
In Expression (10), SB1 represents data (compression ratio score) obtained by scoring the compression ratio of data compressed using the compression algorithm 1. W13 indicates a weight constant. R1 indicates the compression rate corresponding to the compression algorithm 1. In Expression (11), SB2 represents data (compression ratio score) obtained by scoring the compression ratio of data compressed using the compression algorithm 2. W14 represents a weight constant. R2 indicates the compression rate corresponding to the compression algorithm 1.
The technique for obtaining scored data by using formula (10) or formula (11) is merely an example of a technique for scoring data. The method of scoring data by the compression rate estimation unit 503 is not particularly limited as long as it is a method of calculating using the compression rate.
The switching unit 501 has a function of determining whether or not to switch the compression algorithm based on the data received from the compression cost estimation unit 502 and the compression rate estimation beauty 503, respectively.
Specifically, when the compression rate is reduced due to a decrease in the use frequency of dictionary data, it is necessary to switch to an algorithm with a higher compression rate. In this case, the switching unit 501 selects a compression algorithm having a compression rate score higher than the current compression rate score and a compression cost score higher than the current compression cost score. When there are a plurality of switching candidate compression algorithms, the switching unit 501 calculates a total score using the following equation (12), for example.
SC1 = W15 × SA1 + W16 × SB1 (12)
In Expression (12), SC1 represents an overall score corresponding to the compression algorithm 1. W15 and W16 indicate weight constants. SA1 indicates a compression cost score corresponding to the compression algorithm 1. SB1 indicates a compression rate score corresponding to the compression algorithm 1. The values of the weight constants W15 and W16 are set in advance. The values of the weight constants W15 and W16 are appropriately adjusted depending on whether the compression cost (implementation cost) is important or the compression rate is important. That is, when the compression cost is more important than the compression rate, the weight constant W15 is set larger. On the other hand, when the compression rate is more important than the compression cost, the weight constant W16 is set larger.
On the other hand, when the data flow rate becomes lower than or equal to the threshold value T1, the data flow rate is reduced. Therefore, it is preferable to employ the following compression algorithm. The compression algorithm is a high-load compression algorithm that requires time for compression, but is a compression algorithm having a high compression rate.
In such a case, the switching unit 501 selects a compression algorithm having a high compression rate score even if the compression cost score is low. Specifically, the switching unit 501 calculates a total score for each compression algorithm using a mathematical expression similar to Expression (12). Note that the compression rate score is more important than the compression cost score by setting the weighting constant W16 in the mathematical formula to be large.
The switching unit 501 determines which compression algorithm to switch to based on the result of comparing these total scores.
In addition, when the data flow rate exceeds the threshold value T2, the data flow rate is large, so it is preferable to employ the following compression algorithm. The compression algorithm is a low-load compression algorithm even if the compression rate is low.
In such a case, the switching unit 501 selects a compression algorithm having a high compression cost score even if the compression rate score is low. Specifically, the switching unit 501 calculates a total score for each compression algorithm using a mathematical expression similar to Expression (12). Note that the compression cost score is more important than the compression rate score by setting the weighting constant W15 in the mathematical formula to be large.
The switching unit 501 sets the compression algorithm selected as described above as a new compression algorithm used by the compression unit 102. That is, the switching unit 501 switches the compression algorithm used when compressing data. In addition, the switching unit 501 issues an instruction to start an operation to the update unit 504 and outputs information indicating the compression algorithm after switching.
The update unit (update unit) 504 starts the operation when receiving an instruction from the switching unit 501. The update unit 504 updates information indicating from which part of the data stored in the first storage unit 103 to which data is compressed with which compression algorithm. Further, the update unit 504 updates information indicating from which part of the dictionary data stored in the second storage unit 104 to which dictionary data is used by which compression algorithm.
Specifically, the compression algorithm information received from the switching unit 501 by the update unit 504 includes at least information about the compression algorithm name. Then, the update unit 504 stores (adds) compression algorithm information in each of the first storage unit 103 and the second storage unit 104 so as to know which data is related to which compression algorithm. .
For example, the data stored in the first storage unit 103 is classified for each compression algorithm as follows.
[First storage unit 103]
Compression algorithm A
ABCABABC
DACEFDAS
AXMCALDA
Compression algorithm B
ANSKANX
CIKADLAL
Compression algorithm C
DKLABNKC
ALKDKAJD
SLDKJALL
The same applies to the second storage unit 104.
Note that a compression algorithm that is first employed by the compression unit 102 is set in advance. In the fifth embodiment, the compression unit 102 has a function of generating dictionary data and registering the generated dictionary data in the second storage unit 104 in addition to the function of compressing data. .
Hereinafter, an operation example (data compression method) of the data compression apparatus 500 of the fifth embodiment will be described with reference to the flowchart of FIG. In FIG. 22, the same reference numerals as those in FIG. 18 are given to portions showing the same operations as those shown in FIG. 18, and the duplicate description thereof is omitted.
The flowchart of FIG. 22 shows the processing procedure of the computer program executed by the control device (CPU) 101 in the data compression apparatus 500, similarly to the flowchart of FIG. The computer program is stored in a storage medium (for example, a non-volatile storage medium) of a storage device such as a memory or a hard disk included in the data compression apparatus 500 as described above. Further, for example, the computer program is stored in a portable storage medium such as a compact disk (CD) or a memory card, and then transferred from the portable storage medium to the storage medium of the data compression apparatus 500. May be stored.
When the switching unit 501 determines in step S040 shown in FIG. 22 that the usage frequency of the dictionary data has not decreased, the switching unit 501 calculates the data flow rate (step S410). That is, the switching unit 501 calculates the data flow rate using the equation (6) based on the total capacity data and the elapsed time data received from the compression unit 102. Then, the switching unit 501 determines whether the calculated data flow rate is lower than the threshold value T1 or whether the data flow rate is higher than the threshold value T2. As a result, when the switching unit 501 determines that the data flow rate is equal to or lower than the threshold value T1 or higher than the threshold value T2, the switching unit 501 issues an instruction to start the operation to the compression cost estimation unit 502 and the compression rate estimation unit 503.
Thereby, the compression cost estimation unit 502 and the compression rate estimation unit 503 start operation (step S420). That is, the compression cost estimation unit 502 calculates a compression cost score for each of a plurality of preset compression algorithms as described above. In addition, the compression rate estimation unit 503 calculates a compression rate score for each of a plurality of preset compression algorithms as described above.
The switching unit 501 determines whether to switch the compression algorithm based on the calculated compression cost score and compression ratio score (whether there is a compression algorithm more appropriate than the compression algorithm currently used) ( Step S430).
If it is determined that the compression algorithm is to be switched, the switching unit 501 selects the compressed compression algorithm as described above. Then, the switching unit 501 switches the compression algorithm used when compressing data to the selected compression algorithm (step S440). In addition, the switching unit 501 stores (adds) compression algorithm information in each of the first storage unit 103 and the second storage unit 104 so that it can be understood which data is related to which compression algorithm. To do.
As described above, according to the fifth embodiment, the switching unit 501 selects a compression algorithm with an emphasis on the compression rate when the data flow rate is equal to or less than the threshold T1. In addition, when the data flow rate is equal to or greater than the threshold value T2, the switching unit 501 selects a compression algorithm with an emphasis on compression cost. As described above, the switching unit 501 selects a compression algorithm based on the compression cost and the compression rate. Thereby, the switching unit 501 can select a highly effective compression algorithm from a plurality of compression algorithms including a compression algorithm other than the lexicographic compression algorithm. That is, the switching unit 501 can switch to a highly effective compression algorithm at an appropriate timing. For this reason, the data compression apparatus 500 of the fifth embodiment can maintain efficient data compression.
In addition, this invention is not limited to each above-mentioned embodiment, Various embodiment can be taken. For example, the data compression apparatus may have a configuration as shown in FIG. That is, the data compression apparatus 600 includes a compression unit (compression unit) 601, a monitoring unit (monitoring unit) 602, a timing control unit (timing control unit) 603, and a dictionary generation unit (dictionary generation unit) 604. doing. The compression unit 601 has a function of compressing data to be compressed based on dictionary data given in advance. The monitoring unit 602 has a function of calculating the use frequency of the dictionary data when the compression unit 601 compresses the data. The timing control unit 603 has a function of issuing an instruction to update the dictionary data when the calculated use frequency falls below a preset threshold value. In response to the instruction, the dictionary generation unit 604 has a function of newly generating the dictionary data using the data compressed by the compression unit 102.
In this data compression apparatus 600 as well, an efficient data compression rate can be maintained as in the above embodiments.
While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
This application claims priority based on Japanese Patent Application No. 2010-036806 filed on Feb. 23, 2010, the entire disclosure of which is incorporated herein.
Further, a part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating a usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
A dictionary update timing control step for instructing updating of the dictionary when the calculation result of the use frequency by the dictionary use frequency monitoring step falls below a set threshold;
A data expansion step of expanding the compressed data obtained in the compression step into data before compression;
Dictionary data stored in the dictionary data storage means by constructing new dictionary data from the uncompressed data obtained in the data expansion step in accordance with the dictionary update instruction in the dictionary update timing control step A dictionary construction step for replacing the new dictionary data with
Data compression method.
(Appendix 2)
The dictionary usage frequency monitoring step receives as input the dictionary data used when the compression target data is compressed in the compression step, the number of times each dictionary data is used, and the total amount of the compression target data. The data compression method according to appendix 1, wherein the number of uses is divided by the total amount of data to be compressed for each dictionary data to obtain a calculation result of the use frequency.
(Appendix 3)
The dictionary update timing control step instructs the updating of the dictionary when the total value of the calculation results of the usage frequencies of all the dictionary data obtained by the dictionary usage frequency monitoring step is smaller than the threshold value. Or the data compression method of 2.
(Appendix 4)
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating a usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
A dictionary update timing control step for instructing to update the dictionary when the calculation result of the use frequency by the dictionary use frequency monitoring step falls below a threshold;
A data expansion step of expanding the compressed data obtained in the compression step into data before compression;
A resource monitoring step that outputs the free capacity of the resource to be used;
A parameter adjustment step for variably setting the threshold value used in the dictionary update timing control step according to the free capacity of the resource output in the resource monitoring step;
According to the dictionary update instruction in the dictionary update timing control step, new dictionary data is constructed from the uncompressed data obtained in the data expansion step, and the dictionary data stored in the dictionary data storage means is stored. A dictionary construction step for replacing the constructed new dictionary data with
Data compression method.
(Appendix 5)
In the parameter adjustment step, the threshold used in the dictionary update timing control step is variably set to a smaller value as the free capacity of the resource from the resource monitoring step is larger, and the threshold is decreased as the free capacity of the resource is smaller. The data compression method according to appendix 4, wherein the data is variably set to a large value.
(Appendix 6)
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating a usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
When the calculation result of the use frequency by the dictionary use frequency monitoring step falls below a threshold value, a dictionary is constructed according to the data amount of the compressed data and the data amount of the dictionary data stored in the dictionary data storage means A dictionary construction cost estimation step for estimating the cost;
A compression rate estimation step for estimating a compression rate when the dictionary data is updated when a calculation result of the usage frequency by the dictionary usage frequency monitoring step falls below a threshold;
A dictionary update timing control step of comparing the dictionary cost with the compression ratio and determining whether or not to construct a new dictionary according to the comparison result;
A data expansion step of expanding the compressed data output from the compression step into data before compression;
When the decision to construct the new dictionary is made by the dictionary update timing control step, new dictionary data is constructed from the uncompressed data obtained in the data expansion step and stored in the dictionary data storage means A dictionary construction step of replacing the created dictionary data with the constructed new dictionary data;
Data compression method.
(Appendix 7)
The compression rate estimation step includes a new data constructed using uncompressed data obtained by decompressing a part of the compressed data using the dictionary data stored in the dictionary data storage means, and the pre-compressed data. The data compression method according to appendix 6, wherein a ratio with the compressed data compressed using a dictionary is estimated as the compression ratio.
(Appendix 8)
The dictionary update timing control step compares the first value obtained by scoring the dictionary cost with the second value obtained by scoring the compression rate, and the second value is the first value. 8. The data compression method according to appendix 6 or 7, wherein it is decided to construct the new dictionary only when it is larger than.
(Appendix 9)
The data expansion step includes:
A compressed data storage step for storing the compressed data output from the compression means;
In accordance with an instruction to update the dictionary in the dictionary update timing control step, the compressed data stored in the compressed data storage step is expanded and compressed with reference to the dictionary data stored in the dictionary data storage means. A compressed data decompression step for obtaining decompressed data which is the previous data;
A decompressed data storage step for storing the decompressed data obtained by the compressed data decompressing step;
The dictionary construction step starts when the decompression of all the compressed data stored in the compressed data storage step by the compressed data decompression step is completed, and the pre-compression stored in the decompressed data storage The data compression method according to any one of supplementary notes 1 to 8, wherein the new dictionary data is constructed based on the expanded data that is data.
(Appendix 10)
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating, for each dictionary data, the usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
A dictionary update timing control step for extracting and outputting dictionary data corresponding to a use frequency that is lower than a set threshold among the use frequencies for each dictionary data calculated by the dictionary use frequency monitoring step;
A partial data expansion step of expanding a portion of the compressed data obtained by the compression into data before compression using the dictionary data obtained by the dictionary update timing control step;
A dictionary data deletion step of deleting the dictionary data obtained by the dictionary update timing control step from a plurality of dictionary data stored in the dictionary data storage means;
A dictionary construction step of constructing new dictionary data from a part of the uncompressed data output from the data partial expansion step, and adding the new dictionary data to the dictionary data storage means;
Data compression method.
(Appendix 11)
The partial data expansion step includes:
A compressed data storage step for storing the compressed data obtained in the compression step;
Using the dictionary data obtained in the dictionary update timing control step, a part of the compressed data stored in the compressed data storage step is expanded to obtain a part of the compressed data. Steps,
A decompressed data storage step for storing the partial uncompressed data obtained by the compressed data partial decompression step;
With
The dictionary construction step starts operation when the expansion of a part of the compressed data stored in the compressed data storage step in the compressed data partial expansion step is completed, and the part stored in the expanded data storage step The data compression method according to appendix 10, wherein the new dictionary data is constructed based on the data before compression.
(Appendix 12)
A compression step of compressing sequentially inputted data to be compressed in accordance with a designated compression algorithm and outputting compressed data;
The data before compression in the compression step is compressed by each of a plurality of preset compression algorithms, and the compression cost, which is the ratio of the amount of data before compression and the time taken for compression, A compression cost estimation step for estimating each of the compression algorithms;
A compression rate estimation step of calculating a compression rate when the data before compression by the compression step is compressed by each of the plurality of compression algorithms set in advance;
When the use frequency of dictionary data to be referred to when compressing the data to be compressed in the compression step is lower than a threshold value, the compression cost obtained in the compression cost estimation step and the compression rate estimation step are obtained. A compression algorithm switching step for selecting, based on the compression rate, a compression algorithm having a compression rate and a compression cost higher than a compression algorithm currently used among the plurality of compression algorithms as a compression algorithm used in the compression step;
Data compression method.
(Appendix 13)
The compression algorithm switching step calculates a data flow rate of the compression target data compressed by the compression step, and the data flow rate is equal to or lower than a first threshold value or equal to or higher than a second threshold value greater than the first threshold value. When the data flow rate is equal to or less than the first threshold, based on the compression cost obtained in the compression cost estimation step and the compression rate obtained in the compression rate estimation step, A compression algorithm having at least a higher compression ratio than the currently used compression algorithm is selected as a compression algorithm to be used in the compression step, and the plurality of compressions when the data flow rate is equal to or greater than the second threshold. Among the algorithms, the compression cost is at least higher than the compression algorithm currently used. Data compression method according to Note 12, wherein the selecting the compression algorithm to use compression algorithms in the compression step.
(Appendix 14)
Dictionary data storage means for storing dictionary data for compression;
Compression means for compressing sequentially inputted data to be compressed with reference to the dictionary data stored in the dictionary data storage means, and outputting compressed data;
Data expansion means for expanding the compressed data output from the compression means into data before compression;
Dictionary use frequency monitoring means for calculating the use frequency of the dictionary data referred to when the compression means compresses the data to be compressed;
Dictionary update timing control means for instructing updating of the dictionary when the calculation result of the use frequency by the dictionary use frequency monitoring means falls below a set threshold;
In accordance with the dictionary update instruction output from the dictionary update timing control means, new dictionary data is constructed from the uncompressed data output from the data expansion means and stored in the dictionary data storage means. Dictionary construction means for replacing the newly created dictionary data with the constructed new dictionary data;
A data compression system.
(Appendix 15)
The dictionary usage frequency monitoring means includes the compression means that stores the dictionary data used when the compression means compresses the compression target data, the number of times each dictionary data is used, and the total amount of the compression target data. 15. The data compression system according to appendix 14, wherein the number of uses is divided by the total amount of data to be compressed for each dictionary data to obtain a calculation result of the use frequency.
(Appendix 16)
The dictionary update timing control means instructs update of the dictionary when the total value of the calculation results of the usage frequencies of all the dictionary data input from the dictionary usage frequency monitoring means is smaller than the threshold value. Or 15. The data compression system according to 15.
(Appendix 17)
Dictionary data storage means for storing dictionary data for compression;
Compression means for compressing sequentially inputted data to be compressed with reference to the dictionary data stored in the dictionary data storage means, and outputting compressed data;
Data expansion means for expanding the compressed data output from the compression means into data before compression;
Dictionary use frequency monitoring means for calculating the use frequency of the dictionary data referred to when the compression means compresses the data to be compressed;
Dictionary update timing control means for instructing dictionary update when the use frequency calculation result by the dictionary use frequency monitoring means falls below a threshold;
Resource monitoring means for outputting the free capacity of the resource to be used;
Parameter adjustment means for variably setting the threshold value used in the dictionary update timing control means according to the free space of the resource from the resource monitoring means;
In accordance with the dictionary update instruction output from the dictionary update timing control means, new dictionary data is constructed from the uncompressed data output from the data expansion means and stored in the dictionary data storage means. Dictionary construction means for replacing the newly created dictionary data with the constructed new dictionary data;
A data compression system.
(Appendix 18)
The parameter adjustment means variably sets the threshold used by the dictionary update timing control means to a smaller value as the free capacity of the resource from the resource monitoring means is larger, and the threshold is set as the free capacity of the resource is smaller. The data compression system according to appendix 17, wherein the data compression system is variably set to a large value.
(Appendix 19)
Dictionary data storage means for storing dictionary data for compression;
Compression means for compressing sequentially inputted data to be compressed with reference to the dictionary data stored in the dictionary data storage means, and outputting compressed data;
Data expansion means for expanding the compressed data output from the compression means into data before compression;
Dictionary use frequency monitoring means for calculating the use frequency of the dictionary data referred to when the compression means compresses the data to be compressed;
Dictionary construction cost estimating means for estimating a dictionary construction cost according to the data amount of the compressed data and the data amount of the dictionary data stored in the dictionary data storage means;
Compression rate estimating means for estimating a compression rate when the dictionary data is updated;
The dictionary construction cost obtained by instructing each of the dictionary construction cost estimation unit and the compression rate estimation unit when the calculation result of the usage frequency by the dictionary usage frequency monitoring unit falls below a threshold value. A dictionary update timing control unit that compares the dictionary cost from the construction unit with the compression rate from the compression rate estimation unit and determines whether or not to construct a new dictionary according to the comparison result;
When the decision to construct the new dictionary is made by the dictionary update timing control means, new dictionary data is constructed from the uncompressed data output from the data expansion means, and stored in the dictionary data storage means. Dictionary construction means for replacing stored dictionary data with the constructed new dictionary data;
A data compression system.
(Appendix 20)
The compression rate estimation means includes a new data constructed using uncompressed data obtained by decompressing a part of the compressed data using the dictionary data stored in the dictionary data storage means, and the data before the compression. Item 20. The data compression system according to appendix 19, wherein a ratio with the compressed data compressed using a dictionary is estimated as the compression rate.
(Appendix 21)
The dictionary update timing control means compares the first value obtained by scoring the dictionary cost from the dictionary construction cost construction means and the second value obtained by scoring the compression ratio from the compression ratio estimation means. The data compression system according to appendix 19 or 20, wherein the new dictionary is determined to be constructed only when the second value is larger than the first value.
(Appendix 22)
The data expansion means is
Compressed data storage means for storing the compressed data output from the compression means;
In accordance with the dictionary update instruction output from the dictionary update timing control means, the compressed data stored in the compressed data storage means is referred to the dictionary data stored in the dictionary data storage means. Compressed data decompression means for decompressing and obtaining decompressed data that is data before compression;
Decompressed data storage means for storing the decompressed data obtained by the compressed data decompressing means;
And the dictionary construction means starts operation when the compressed data expansion means ends expansion of all the compressed data stored in the compressed data storage means, and is stored in the expanded data storage means. The data compression system according to any one of supplementary notes 14 to 21, wherein the new dictionary data is constructed based on the decompressed data that is data before compression.
(Appendix 23)
Dictionary data storage means for storing a plurality of dictionary data for compression;
Compression means for compressing sequentially inputted data to be compressed with reference to the dictionary data stored in the dictionary data storage means, and outputting compressed data;
Dictionary use frequency monitoring means for calculating the use frequency of the dictionary data referred to when the compression means compresses the data to be compressed for each dictionary data;
A dictionary update timing control means for extracting and outputting dictionary data corresponding to a use frequency that is lower than a set threshold value among the use frequencies for each dictionary data by the dictionary use frequency monitoring means;
Data partial expansion means for expanding a part of the compressed data output from the compression means into data before compression using the dictionary data output from the dictionary update timing control means;
Dictionary data deletion means for deleting the dictionary data output from the dictionary update timing control means from a plurality of dictionary data stored in the dictionary data storage means;
Dictionary construction means for constructing new dictionary data from a part of the uncompressed data output from the data partial expansion means, and adding the new dictionary data to the dictionary data storage means;
A data compression system.
(Appendix 24)
The data partial expansion means includes
Compressed data storage means for storing the compressed data output from the compression means;
Using the dictionary data output from the dictionary update timing control means, a part of the compressed data that expands a part of the compressed data stored in the compressed data storage means to obtain a part of the compressed data Deployment means;
Decompressed data storage means for storing the partial uncompressed data obtained by the compressed data partial decompression means;
With
The dictionary construction means starts its operation upon completion of expansion of a part of the compressed data stored in the compressed data storage means by the compressed data partial expansion means, and is stored in the expanded data storage means 24. The data compression system according to appendix 23, wherein the new dictionary data is constructed based on a part of the data before compression.
(Appendix 25)
Dictionary data storage means for storing dictionary data used when the compression algorithm is lexicographic;
Compression means for sequentially compressing data to be compressed according to a designated compression algorithm and outputting compressed data;
The data before compression of the compression means is compressed by each of a plurality of preset compression algorithms, and the compression cost, which is the ratio of the amount of data before compression and the time taken for compression, A compression cost estimation means for estimating each of the compression algorithms;
Compression ratio estimating means for calculating a compression ratio when the data before compression by the compression means is compressed by each of the plurality of compression algorithms set in advance;
Obtained by instructing the compression cost estimating means and the compression rate estimating means to operate when the use frequency of the dictionary data referred to when the compressing means compresses the data to be compressed falls below a threshold value. Further, based on the compression cost from the compression cost estimation unit and the compression rate from the compression rate estimation unit, the compression rate and the compression cost are higher than the compression algorithm currently used among the plurality of compression algorithms. Compression algorithm switching means for selecting a high compression algorithm as the compression algorithm used in the compression means;
A data compression system.
(Appendix 26)
The compression algorithm switching unit calculates a data flow rate of the compression target data input to the compression unit, and the data flow rate is equal to or lower than a first threshold value or higher than a second threshold value greater than the first threshold value. And the compression cost from the compression cost estimation unit and the compression rate from the compression rate estimation unit obtained by instructing the compression cost estimation unit and the compression rate estimation unit to operate. Based on the above, when the data flow rate is less than or equal to the first threshold, a compression algorithm having at least a higher compression rate than the currently used compression algorithm is selected as the compression algorithm to be used by the compression means, When the data flow rate is equal to or higher than the second threshold, the data compression flow is currently used among the plurality of compression algorithms. Data compression system according to Note 25, wherein selecting a compression algorithm used in the compression means at least higher compression algorithm of said compression cost than contraction algorithm.
(Appendix 27)
To the computer that performs data compression,
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating a usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
A dictionary update timing control step for instructing updating of the dictionary when the calculation result of the use frequency by the dictionary use frequency monitoring step falls below a set threshold;
A data expansion step of expanding the compressed data obtained in the compression step into data before compression;
Dictionary data stored in the dictionary data storage means by constructing new dictionary data from the uncompressed data obtained in the data expansion step in accordance with the dictionary update instruction in the dictionary update timing control step A dictionary construction step for replacing the new dictionary data with
A computer program storage medium for storing a data compression program for executing the program.
(Appendix 28)
The dictionary usage frequency monitoring step receives as input the dictionary data used when the compression target data is compressed in the compression step, the number of times each dictionary data is used, and the total amount of the compression target data. 28. A computer program storage medium for storing the data compression program according to appendix 27, wherein the number of uses is divided by the total amount of data to be compressed for each dictionary data to obtain the calculation result of the use frequency.
(Appendix 29)
The dictionary update timing control step instructs the updating of the dictionary when the total value of the calculation results of the usage frequencies of all the dictionary data obtained by the dictionary usage frequency monitoring step is smaller than the threshold value. Or a computer program storage medium for storing the data compression program according to 28.
(Appendix 30)
To the computer that performs data compression,
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating a usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
A dictionary update timing control step for instructing to update the dictionary when the calculation result of the use frequency by the dictionary use frequency monitoring step falls below a threshold;
A data expansion step of expanding the compressed data obtained in the compression step into data before compression;
A resource monitoring step that outputs the free capacity of the resource to be used;
A parameter adjustment step for variably setting the threshold value used in the dictionary update timing control step according to the free capacity of the resource output in the resource monitoring step;
According to the dictionary update instruction in the dictionary update timing control step, new dictionary data is constructed from the uncompressed data obtained in the data expansion step, and the dictionary data stored in the dictionary data storage means is stored. A dictionary construction step for replacing the constructed new dictionary data with
A computer program storage medium for storing a data compression program for executing the program.
(Appendix 31)
In the parameter adjustment step, the threshold used in the dictionary update timing control step is variably set to a smaller value as the free capacity of the resource from the resource monitoring step is larger, and the threshold is decreased as the free capacity of the resource is smaller. A computer program storage medium for storing the data compression program according to attachment 30 that is variably set to a large value.
(Appendix 32)
To the computer that performs data compression,
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating a usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
When the calculation result of the use frequency by the dictionary use frequency monitoring step falls below a threshold value, a dictionary is constructed according to the data amount of the compressed data and the data amount of the dictionary data stored in the dictionary data storage means A dictionary construction cost estimation step for estimating the cost;
A compression rate estimation step for estimating a compression rate when the dictionary data is updated when a calculation result of the usage frequency by the dictionary usage frequency monitoring step falls below a threshold;
A dictionary update timing control step of comparing the dictionary cost with the compression ratio and determining whether or not to construct a new dictionary according to the comparison result;
A data expansion step of expanding the compressed data output from the compression step into data before compression;
When the decision to construct the new dictionary is made by the dictionary update timing control step, new dictionary data is constructed from the uncompressed data obtained in the data expansion step and stored in the dictionary data storage means A dictionary construction step of replacing the created dictionary data with the constructed new dictionary data;
A computer program storage medium for storing a data compression program for executing the program.
(Appendix 33)
The compression rate estimation step includes a new data constructed using uncompressed data obtained by decompressing a part of the compressed data using the dictionary data stored in the dictionary data storage means, and the pre-compressed data. 33. A computer program storage medium for storing a data compression program according to supplementary note 32 for estimating a ratio of the compressed data compressed using a dictionary as the compression ratio.
(Appendix 34)
The dictionary update timing control step compares the first value obtained by scoring the dictionary cost with the second value obtained by scoring the compression rate, and the second value is the first value. 34. A computer program storage medium for storing the data compression program according to appendix 32 or 33, which determines that the new dictionary is to be constructed only when it is larger than.
(Appendix 35)
The data expansion step includes:
A compressed data storage step for storing the compressed data output from the compression means;
In accordance with an instruction to update the dictionary in the dictionary update timing control step, the compressed data stored in the compressed data storage step is expanded and compressed with reference to the dictionary data stored in the dictionary data storage means. A compressed data decompression step for obtaining decompressed data which is the previous data;
A decompressed data storage step for storing the decompressed data obtained by the compressed data decompressing step;
The dictionary construction step starts when the decompression of all the compressed data stored in the compressed data storage step by the compressed data decompression step is completed, and the pre-compression stored in the decompressed data storage 35. A computer program storage medium for storing the data compression program according to any one of supplementary notes 27 to 34 for constructing the new dictionary data based on the expanded data that is data.
(Appendix 36)
To the computer that performs data compression,
A compression step of compressing sequentially input data to be compressed while referring to dictionary data stored in the dictionary data storage means, and outputting compressed data;
A dictionary usage frequency monitoring step of calculating, for each dictionary data, the usage frequency of the dictionary data referred to when compressing the data to be compressed in the compression step;
A dictionary update timing control step for extracting and outputting dictionary data corresponding to a use frequency that is lower than a set threshold among the use frequencies for each dictionary data calculated by the dictionary use frequency monitoring step;
A partial data expansion step of expanding a portion of the compressed data obtained by the compression into data before compression using the dictionary data obtained by the dictionary update timing control step;
A dictionary data deletion step of deleting the dictionary data obtained by the dictionary update timing control step from a plurality of dictionary data stored in the dictionary data storage means;
A dictionary construction step of constructing new dictionary data from a part of the uncompressed data output from the data partial expansion step, and adding the new dictionary data to the dictionary data storage means;
A computer program storage medium for storing a data compression program for executing the program.
(Appendix 37)
The partial data expansion step includes:
A compressed data storage step for storing the compressed data obtained in the compression step;
Using the dictionary data obtained in the dictionary update timing control step, a part of the compressed data stored in the compressed data storage step is expanded to obtain a part of the compressed data. Steps,
A decompressed data storage step for storing the partial uncompressed data obtained by the compressed data partial decompression step;
With
The dictionary construction step starts operation when the expansion of a part of the compressed data stored in the compressed data storage step in the compressed data partial expansion step is completed, and the part stored in the expanded data storage step 37. A computer program storage medium for storing the data compression program according to appendix 36 for constructing the new dictionary data based on the data before compression.
(Appendix 38)
To the computer that performs data compression,
A compression step of compressing sequentially inputted data to be compressed in accordance with a designated compression algorithm and outputting compressed data;
The data before compression in the compression step is compressed by each of a plurality of preset compression algorithms, and the compression cost, which is the ratio of the amount of data before compression and the time taken for compression, A compression cost estimation step for estimating each of the compression algorithms;
A compression rate estimation step of calculating a compression rate when the data before compression by the compression step is compressed by each of the plurality of compression algorithms set in advance;
When the use frequency of dictionary data to be referred to when compressing the data to be compressed in the compression step is lower than a threshold value, the compression cost obtained in the compression cost estimation step and the compression rate estimation step are obtained. A compression algorithm switching step for selecting, based on the compression rate, a compression algorithm having a compression rate and a compression cost higher than a compression algorithm currently used among the plurality of compression algorithms as a compression algorithm used in the compression step;
A computer program storage medium for storing a data compression program for executing the program.
(Appendix 39)
The compression algorithm switching step calculates a data flow rate of the compression target data compressed by the compression step, and the data flow rate is equal to or lower than a first threshold value or equal to or higher than a second threshold value greater than the first threshold value. When the data flow rate is equal to or less than the first threshold, based on the compression cost obtained in the compression cost estimation step and the compression rate obtained in the compression rate estimation step, A compression algorithm having at least a higher compression ratio than the currently used compression algorithm is selected as a compression algorithm to be used in the compression step, and the plurality of compressions when the data flow rate is equal to or greater than the second threshold. Among the algorithms, the compression cost is at least higher than the compression algorithm currently used. Computer program storage medium for storing data compression program according to Note 38, wherein the selecting the compression algorithm to use compression algorithms in the compression step.

  The present invention is effective for an apparatus (system) that compresses sequentially generated data such as log data output from a server or data output from a sensor by a lexicographic compression method.

100, 200, 300, 400, 500 Data compression device 102 Compression unit 105 Monitoring unit 106, 203, 303, 401 Timing control unit 107 Expansion unit 109 Dictionary generation unit 201 Resource monitoring unit 202 Adjustment unit 301 Cost estimation unit 302, 503 Compression Rate estimation unit 402 Partial expansion unit 403 Deletion unit 501 Switching unit 502 Compression cost estimation unit 504 Update unit

Claims (9)

Compression means for compressing data to be compressed based on dictionary data given in advance;
Monitoring means for calculating the frequency of use of the dictionary data when the compression means compresses the data;
Timing control means for issuing an instruction to update the dictionary data when the calculated use frequency falls below a preset threshold;
A data compression apparatus comprising: a dictionary generation unit that receives the instruction and newly generates the dictionary data based on the data compressed by the compression unit. Expansion means for expanding the data compressed by the compression means using the dictionary data;
The data compression apparatus according to claim 1, wherein the dictionary generation unit newly generates the dictionary data using data expanded by the expansion unit.   The data compression apparatus according to claim 1 or 2, wherein the monitoring unit calculates the use frequency by dividing the number of times the dictionary data is used by the total capacity of the data before compression. Resource monitoring means for detecting a free capacity of a resource for storing the data;
4. The data compression apparatus according to claim 1, further comprising an adjustment unit that variably sets the threshold value used by the timing control unit in accordance with an available capacity of the resource. Estimating means for estimating a cost required for newly generating the dictionary data and a compression ratio of the data compressed by the compression means based on the new dictionary data when the dictionary data is newly generated In addition, and
2. The timing control unit determines whether to issue the instruction to update the dictionary data by using the cost and the compression rate estimated by the estimation unit in addition to the threshold value. The data compression device according to any one of claims 4 to 4. For each of a plurality of preset compression algorithms, an estimation means for estimating a compression cost that is a ratio between the amount of data before compression and a time required for compression, and estimating a compression rate of the compressed data;
The switching unit according to any one of claims 1 to 4, further comprising switching means for selecting a compressed compression algorithm from among the plurality of compression algorithms based on the compression cost and the compression ratio. Data compression device. A deletion unit that deletes the dictionary data whose usage frequency is lower than a predetermined value from a storage unit that stores a plurality of the dictionary data;
7. The data compression apparatus according to claim 1, wherein the dictionary generation unit newly generates the dictionary data using a part of the data compressed by the compression unit. Compress data to be compressed based on dictionary data given in advance,
Calculating the frequency of use of the dictionary data when compressing the data;
When the calculated usage frequency falls below a preset threshold value, an instruction to update the dictionary data is issued,
A data compression method for receiving the instruction and newly generating the dictionary data based on the compressed data. A process of compressing data to be compressed based on dictionary data given in advance;
A process of calculating the use frequency of the dictionary data when compressing the data;
A process of issuing an instruction to update the dictionary data when the calculated use frequency falls below a preset threshold;
A program storage medium that stores a computer program that causes the data compression apparatus to execute a process of generating the dictionary data based on the compressed data in response to the instruction.

Images