JPH07334344A - Data coding device - Google Patents

Abstract

PURPOSE:To simplify the procedure of processing and to deal with the change of appearance frequency by exchanging the order of a dictionary item, to which a code word shorter than a dictionary item corresponding to a source data string is assigned, each time a data string is coded. CONSTITUTION:The data string to be inputted is first transmitted through a control means 6 to an instruction means 4, and a prescribed data string is instructed to a dictionary means 3. At the dictionary means 3, the data strings stored in the dictionary items are retrieved, and a deciding means 5 decides whether or not the relevant data string exists. When the input data string exists at the dictionary item, that input data string is coded as it is while using the code word corresponding to it, and the dictionary is updated by exchanging the order between the dictionary item in the dictionary means 3 corresponding to that data string input and one of dictionary items in the means 3 to which the code word shorter than the former dictionary item is assigned. Thus, coding corresponding to the local data string appearance frequency is enabled.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data compression algorithm, and uses the size of various files used in a file archiver, distributed file system, data communication, voice / image communication, computer network, etc. The present invention relates to a data compression algorithm for compressing and compressing.

[0002]

2. Description of the Related Art Conventionally, data compression algorithms of various principles have been proposed for the purpose of reducing the memory size and the time for file transmission / reception. Recently, with the advent of LZ codes, the development of codes with good performance that requires only one-pass processing of character strings has become active in the field of computer science.

Among them, the dynamic Huffman coding which carries out the Huffman coding by the one-pass processing has a great advantage in terms of processing time and compression rate and is widely used. The dynamic Huffman coding will be described below.

22 and 23 are flowcharts of the operation in the conventional dynamic Huffman coding. Initially, the code tree is a tree that has only leaves corresponding to the non-appearing data strings (data strings that have not appeared until the current encoding stage) (step S501). The data string is searched to see if there is a leaf corresponding to the input data string in the code tree (step S502). In the encoding of the first data string, the encoding is performed using a code tree having only leaves corresponding to the non-appearing data string, and 2
In the encoding of the data string after the th, the encoding is performed using the previously obtained code tree. When the leaf corresponding to the input data string exists in the code tree, it is encoded as it is using the code word corresponding to the input data string (step S503). When the leaf corresponding to the input data string does not exist in the code tree, it is new. In order to clearly indicate the appearance of the data string, the code word of the non-appearing data is sent (step S504), and an appropriate expression of the data string is set as the send code word (step S505). Then, the occurrence probability of each character (the appearance frequency of the non-appearing character is always 1) is recalculated (in the first data string, the appearance frequency of the character is 1, the appearance frequency of the non-appearing character is 1, and the total number of characters is 2). As
The occurrence probability is calculated) (step S506), and a new code tree is constructed by the Huffman code construction algorithm (step S507). After that, if there is an unprocessed data string in the file, the process returns to step S502, and if not, the process ends (step S5).
08).

[0005]

However, as in the conventional method described above, the occurrence probability is recalculated for each data string input,
Recreating the code tree complicates the processing procedure and increases the amount of calculation. In addition, since the code tree is based on the occurrence probabilities of all the data processed previously, there is a problem that it is difficult to reflect the local data string appearance frequency.

The present invention has been made to solve the above-mentioned conventional problems, and an object thereof is to provide a data encoding apparatus which simplifies the processing procedure and copes with a local change in the appearance frequency of a data string. There is.

Further, it is possible to provide a data encoding device which further simplifies the processing procedure by eliminating the need to newly reconstruct the code tree each time the data string is encoded and eliminating the occurrence of non-appearing data. To aim.

It is another object of the present invention to provide a data encoding device that facilitates retrieval of data strings stored in a dictionary. It is another object of the present invention to provide a data encoding device that performs optimal encoding for a case where a small number of data strings appear with a high occurrence probability and a case where it does not occur.

It is another object of the present invention to provide a data coding device which prevents extreme redundancy of codewords. Further, it is another object of the present invention to provide a data encoding device that implements updating of a data string stored in a dictionary by hardware.

[0010]

In order to achieve the above object, the data coding apparatus of the present invention comprises a dictionary means for forming a dictionary using sequentially input data strings, and a code for the data strings. Encoding means for performing encoding processing, instructing means for instructing a predetermined data string to the dictionary means, and determining whether or not the predetermined data string instructed by the instructing means exists in the dictionary means Determining means, exchanging means for exchanging the order of the dictionary term for the data string input and one of the dictionary terms to which a shorter codeword is assigned in the dictionary means, the dictionary means, the encoding means, and the instruction. Means, the determining means, and a control means for controlling the exchanging means, and each time one sequentially input data string is encoded, the original data string is encoded. The dictionary is updated by exchanging the order of the dictionary terms in the dictionary means for the data string and one of the dictionary terms in the dictionary means to which a shorter codeword is assigned, and the dictionary is updated. By not calculating the occurrence probability, the processing procedure is simplified and the amount of calculation is reduced.

Further, the data coding apparatus of the present invention includes a dictionary means having a dictionary corresponding to sequentially input data strings, a coding means for coding the data strings, and the dictionary means. In contrast, an instructing means for instructing a predetermined data string, an exchanging means for exchanging the order of the dictionary term for the data string input and one of the dictionary terms to which a shorter codeword is assigned in the dictionary means, and the dictionary means. , The encoding means,
The instruction means and the control means for controlling the exchange means are provided, and a dictionary term in the dictionary means for the original data string and a code word shorter than that are assigned each time one data string input sequentially is encoded. The dictionary is updated by exchanging the order with one of the dictionary terms in the existing dictionary means, and the processing procedure is performed by not calculating the occurrence probability of each data string and reconstructing the code tree. Is simplified and the amount of calculation is reduced.

Further, the dictionary means comprises a dictionary term means and a dictionary search means, and when the instructing means searches the dictionary means, the dictionary means is referred to search the dictionary term means. Is.

Further, the encoding means is provided with a plurality of code trees, and the code tree to be used is changed according to the input data file. Further, the encoding means includes a code generation means and a redundancy prevention means, and the redundancy prevention means, when the code generation means generates a code word of a predetermined number of bits or more, outputs an original data string to which a specific code is added. The output prevents the codeword from becoming redundant.

Furthermore, by performing encoding with 4 bits, a dictionary means, a dictionary search means, and an exchange means are configured by a latch, and table updating is performed using hardware.

[0015]

With the above structure, in the data compression algorithm of the present invention, a dictionary term in the dictionary means for the original data string and a code word shorter than that are assigned each time one character string of the sequentially input data string is encoded. The dictionary is updated by replacing the order with one of the dictionary terms in the existing dictionary means, and the occurrence probability of each data string is not calculated, so that the processing procedure can be simplified and the calculation amount can be reduced. It is possible to perform encoding corresponding to the frequency of occurrence of a typical data string.

Further, the dictionary means is provided with a dictionary corresponding to all the input data strings, and the code tree in the encoding means is a tree having leaves corresponding to the initial value of the dictionary in the dictionary means. Since it does not change during the encoding process, there is no need to reconstruct the code tree each time the data string is encoded, and the occurrence of non-appearing data can be eliminated. The processing procedure can be further simplified.

Further, when the dictionary means comprises the dictionary term means and the dictionary search means, it is not necessary for the instruction means to search from the head of the dictionary term means when searching the dictionary term means. Therefore, the dictionary means can be easily searched.

Furthermore, if the encoding means has a plurality of code trees and changes the code tree to be used according to the input data file, the case where a small number of data strings appear with a high occurrence probability is not Since an optimal code tree can be used for each case, optimal encoding can be performed according to the type of input file.

Further, the encoding means includes a code generation means and a redundancy prevention means, and the redundancy prevention means adds an original code to which a specific code is added when the code generation means generates a code word of a predetermined number of bits or more. By outputting the data string, it is possible to prevent the redundancy of the code word, which would generate output data having more bits than the original data string.

Further, if the encoding is performed with 4 bits and the dictionary term means, the dictionary search means, and the exchanging means are configured by latches and the dictionary is updated by using hardware, dynamic encoding is performed. Dictionary term means needed to do,
Rewriting (reading, writing) occupancy time of the dictionary search means and the exchange means can be shortened.

[0021]

【Example】

(Embodiment 1) Hereinafter, a first embodiment of the present invention will be described with reference to FIG.
~ It demonstrates with reference to FIG.

As shown in the schematic configuration of FIG. 1, the data encoding device 1 of the present embodiment has a dictionary means 3 that forms a dictionary using sequentially input data strings, and a code for the data strings. Encoding means 2 for performing encoding processing
And an instructing means 4 for instructing the dictionary means 3 of a predetermined data string, and a judging means for judging whether or not the predetermined data string instructed by the instructing means 4 exists in the dictionary means 3. 5, exchange means 7 for exchanging the order of the dictionary term for the data string input in the dictionary means 3 and one of the dictionary terms to which a shorter codeword is assigned, the dictionary means 3, the encoding means 2, The control means 6 controls the instruction means 4, the determination means 5, and the exchange means 7. Dictionary means 3
Is a dictionary term means 8 which constitutes a dictionary using sequentially input data strings as shown in the schematic structure of FIG.
And a dictionary search means 9 for searching a dictionary term.

Flow charts showing the operation of the data encoding apparatus in this embodiment are shown in FIGS. In the present embodiment, the code tree in the first encoding means 2 is a tree having only leaves corresponding to the non-appearing data string (characters not appearing up to the current encoding step) (step S10).
1). Initially, the dictionary term means 8 in the dictionary means 3 is provided with only the dictionary term corresponding to the non-appearing data string (step S102). The data string to be input is first sent to the instructing means 4 through the control means 6, and the dictionary means 3 is instructed to the predetermined data string (step S).
103). In the dictionary means 3, the data strings stored in the dictionary entry means 8 correspond to the data stored in the dictionary search means 9. In the dictionary means 3, the dictionary searching means 9 searches the data string stored in the dictionary term means 8 (step S104), and the judging means 5 judges whether or not the corresponding data string exists (step S104). S105). The first data string is encoded using a code tree that has only leaves corresponding to the non-occurring data strings, and the second and subsequent data strings are encoded using the previously obtained code tree. To convert.

When the input data string exists in the dictionary term, the code word corresponding to the input data string is used for encoding as it is (step S106), and the dictionary term in the dictionary means 3 for the data string input and the shorter code. The dictionary is updated by exchanging the order with one of the dictionary terms in the dictionary means 3 to which the word is assigned (step S107). Then, the dictionary search means 9 is updated by replacing the data of the dictionary search means 9 corresponding to the two dictionary terms (step S108).

When the input data string does not exist in the dictionary term, the code word is the code word of the non-appearing data (step S109) followed by an appropriate expression of the data string (step S110). After that, the data string is sent to the dictionary means 3 and the encoding means 2 through the control means 6, stored in the dictionary term in the dictionary means 3 (step S111), and the data corresponding to the dictionary search means 9 is recorded (step S112). ).

Then, the coding means 2 constructs a new code tree (step S113). The algorithm for constructing the code tree is the same as the algorithm for constructing the Huffman code. After that, if there is an unprocessed data string in the file, the process returns to step S103, and if not, the process ends (step S114).

This eliminates the need to recalculate the occurrence probability each time the data string is encoded, so that the processing procedure can be simplified and the amount of calculation can be reduced.
It is possible to perform encoding corresponding to the local data string appearance frequency. Further, the dictionary means 3 comprises a dictionary term means 8 and a dictionary search means 9, and since the dictionary term means 8 is searched by referring to the dictionary search means 9, it is necessary to search from the beginning of the dictionary term at the time of encoding. In addition, the processing procedure can be simplified and the calculation amount can be reduced.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to FIGS. As shown in the schematic configuration of FIG. 6, the data encoding apparatus 1 of the present embodiment performs dictionary processing on a dictionary means 3 having a dictionary corresponding to sequentially input data strings and an encoding process on the data strings. The encoding means 2, the instructing means 4 for instructing the dictionary means 3 of a predetermined data string, the dictionary term for the data string input in the instructing means 3 and the dictionary terms to which shorter code words are assigned. An exchanging means 7 for exchanging the order with one, a dictionary means 3, the encoding means 2, the instructing means 4, and a control means 6 for controlling the exchanging means 7 are provided. The dictionary means 3 has a dictionary term means 8 which forms a dictionary corresponding to the data strings sequentially input as shown in the schematic structure of FIG.
And a dictionary search means 9 for searching a dictionary term.

Flowcharts showing the operation of the data encoding apparatus in this embodiment are shown in FIGS. In the present embodiment, the code tree in the encoding means 2 is a tree having leaves corresponding to the initial value of the dictionary term means 8 in the dictionary means 3 (step S201), and it changes during the encoding process. There is no. The dictionary entry means 8 is provided with a dictionary corresponding to all the input data strings from the beginning (step S202). The input data string is first sent to the instructing means 4 through the control means 6, and the dictionary means 3 is instructed to the predetermined data string (step S203).
In the dictionary means 3, the data strings stored in the dictionary entry means 8 correspond to the data stored in the dictionary search means 9. In the dictionary means 3, the dictionary retrieval means 9 retrieves the data string stored in the dictionary term means 8 (step S204).

The input data string is encoded using the codeword corresponding to the input data string (step S
205). The dictionary is updated by exchanging the order of the data in the dictionary entry means 8 with respect to the data string input and one of the data in the dictionary entry means 8 to which a shorter codeword is assigned (step S206). Then, the dictionary search means 9 is updated by replacing the data of the dictionary search means 9 corresponding to the data in the two dictionary term means 8 (step S207). After that, if there is an unprocessed data string in the file, the process returns to step S203, and if not, the process ends (step S208).

This eliminates the need to recalculate the occurrence probability each time the data string is encoded, so that the processing procedure can be simplified and the amount of calculation can be reduced.
It is possible to perform encoding corresponding to the local data string appearance frequency. Further, the dictionary means 3 comprises a dictionary term means 8 and a dictionary search means 9, and since the dictionary term means 8 is searched by referring to the dictionary search means 9, it is necessary to search from the beginning of the dictionary term at the time of encoding. In addition, the processing procedure can be simplified and the calculation amount can be reduced. Further, it is not necessary to reconstruct the code tree every time the data string is encoded, and the occurrence of non-appearing data can be eliminated, so that the processing procedure can be further simplified.

(Embodiment 3) A third embodiment of the present invention will be described with reference to FIGS. Since only the encoding means 2 is different from the first and second embodiments, this point will be described in detail, and the explanation with emphasis will be omitted.

The coding apparatus 2 of this embodiment has a data string occurrence probability dense code tree 10 and a data string occurrence probability discrete code tree 1 as shown in the schematic configuration of FIG.
It comprises a plurality of one code trees.

11 to 15 are flow charts showing the operation of the data encoding apparatus of this embodiment. The coding means 2 uses the data string occurrence probability dense type coding tree 10 until the coding of a predetermined amount of data strings (step S301) at first (step S302). If the encoded data string has reached a predetermined amount, the sum of the code lengths of the code words generated up to that point is obtained (step S3).
03). If the total code length of the generated code words is less than or equal to a predetermined threshold value, the data string occurrence probability dense type code tree 10 is used for the subsequent coding of the data string (step S304), and the code length of the generated code word is used. Is greater than or equal to a predetermined threshold value, the data string occurrence probability discrete code tree 11 is used for subsequent encoding of the data string (step S305).

Here, the data string occurrence probability dense type code tree 10 can be constructed by a Huffman code construction algorithm or a code tree type can be used. A short code is given to a data string having a high occurrence probability, By giving a long code to a data string having a low probability, the compression rate of a data string having a high frequency of occurrence can be increased and effective coding can be performed. The data string occurrence probability discrete type code tree 11 can be configured by an algorithm or a code tree type, and a code tree that reduces the change in code length even if the occurrence probability of the data string changes is configured. With this, it is possible to reduce the appearance of a code having a long code length and perform effective coding. This makes it possible to use the optimum code tree for each of the case where a small number of data strings appear with a high occurrence probability and the case where such a case does not occur, so that the optimum encoding according to the type of the input file can be performed.

(Embodiment 4) A fourth embodiment of the present invention will be described below with reference to FIGS. Since only the encoding means 2 is different from the first to third embodiments, this point will be described in detail and redundant description will be omitted.

The coding means 2 in this embodiment comprises a code generation means 12 and a redundancy prevention means 13 as shown in the schematic configuration of FIG.
Generates a code word with a predetermined number of bits or more, the original data string with a specific code added is output.

17 to 20 are flow charts showing the operation of the data encoding apparatus in this embodiment. For example, when an 8-bit data string is encoded, the threshold value of the code length activated by the redundancy prevention unit 13 is set to 8 bits. The coding means 2 first performs coding by the code generation means 12 (step S401), and judges whether the code length exceeds a predetermined threshold value (step S40).
2). When a codeword having a code length of 8 hits or more occurs, a codeword of a redundant codeword is sent (the occurrence of a codeword having a code length of a threshold value or more is clearly indicated) (step S403), and the input data string Is output without any processing (step S404). When a code word having a code length of less than 8 bits is generated, the redundancy prevention means 13 is not activated and normal coding is performed (step S405). As a result, it is possible to prevent the redundancy of the code word, which would generate output data having more bits than the original data string.

(Fifth Embodiment) A fifth embodiment of the present invention will be described below with reference to FIG. The encoding algorithm in this embodiment is the same as in the first to fourth embodiments.

In this embodiment, the dictionary means 16 and the exchanging means 20 are constituted by latches by encoding a 4-bit data string, and the dictionary is updated by hardware. The exchanging means 20 comprises a dictionary term exchanging means exchanging means 23 (constituted by an address latch) and a dictionary retrieval means exchanging means 24 (constituted by an address latch). Dictionary term means 21 and dictionary search means 2 in the dictionary means 16
2 is constituted by a latch. Dictionary term exchange means 23
The dictionary term means 21 is rewritten by the, and the dictionary search means 22 is replaced by the dictionary search means exchanging means 24. As a result, the rewriting (reading, writing) occupancy time required for performing dynamic encoding can be shortened. Also, the encoding means 15 is provided by hardware.
By configuring the instruction means 17, the exchange means 18, and the control means 19, respectively, the data encoding device 14 can be implemented as hardware.

[0041]

As described above, in the data compression algorithm of the present invention, a dictionary term for an original data string and a code word shorter than that are assigned every time one character string of a sequentially input data string is encoded. The dictionary is updated by exchanging the order with one of the dictionary terms, and since the occurrence probability of each data string is not calculated, the processing procedure can be simplified, the amount of calculation can be reduced, and the frequency of occurrence of local data strings Can be encoded.

The dictionary has dictionary terms corresponding to all the input data strings, and the code tree has leaves corresponding to the initial values of the dictionary, and changes through the encoding process. If there is nothing, it is not necessary to reconstruct the code tree each time the data string is encoded, and it is possible to eliminate the occurrence of non-appearing data, so that the processing procedure can be further simplified.

Further, when the dictionary search is performed using the dictionary search table when searching the dictionary, it is not necessary to search from the beginning of the dictionary for each dictionary search, so that the dictionary can be easily searched. You can search.

Furthermore, if there are a plurality of code trees and the code tree to be used is changed according to the input data string, the case where a small number of data strings appear with a high occurrence probability and the case where they do not occur occur respectively. Since the optimal code tree can be used, optimal encoding can be performed according to the type of input file.

Further, when a code word having a predetermined number of bits or more is generated, if an original data string to which a specific code is added is output, output data having more bits than the original data string is generated. Codeword redundancy can be prevented.

Further, the encoding is limited to 4 bits, the dictionary and the dictionary search table are constructed by the latch, and the dictionary update,
If the dictionary search table is updated using hardware, it is possible to reduce the time required to rewrite (read, write) the dictionary and the dictionary search table required for performing dynamic encoding.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram showing a data encoding device according to a first embodiment of the present invention.

FIG. 2 is a schematic configuration diagram of a dictionary means in the data encoding device according to the first embodiment of the present invention.

FIG. 3 is a flowchart showing an operation of the data encoding device according to the first embodiment of the present invention.

FIG. 4 is a flowchart showing an operation of the data encoding device according to the first embodiment of the present invention.

FIG. 5 is a flowchart showing the operation of the data encoding device according to the first embodiment of the present invention.

FIG. 6 is a schematic configuration diagram showing a data encoding device according to a second embodiment of the present invention.

FIG. 7 is a schematic configuration diagram of dictionary means in the data encoding device according to the second embodiment of the present invention.

FIG. 8 is a flowchart showing an operation of the data encoding device according to the second embodiment of the present invention.

FIG. 9 is a flowchart showing the operation of the data encoding device according to the second embodiment of the present invention.

FIG. 10 is a schematic configuration diagram of a coding means in a data coding apparatus according to a third embodiment of the present invention.

FIG. 11 is a flowchart showing an operation of the data encoding device according to the third embodiment of the present invention.

FIG. 12 is a flowchart showing an operation of the data encoding device according to the third embodiment of the present invention.

FIG. 13 is a flowchart showing the operation of the data encoding device according to the third embodiment of the present invention.

FIG. 14 is a flowchart showing an operation of the data encoding device according to the third embodiment of the present invention.

FIG. 15 is a flowchart showing an operation of the data encoding device according to the third embodiment of the present invention.

FIG. 16 is a schematic configuration diagram of a coding means in a data coding apparatus according to a fourth embodiment of the present invention.

FIG. 17 is a flowchart showing an operation of the data encoding device according to the fourth embodiment of the present invention.

FIG. 18 is a flowchart showing an operation of the data encoding device according to the fourth embodiment of the present invention.

FIG. 19 is a flowchart showing an operation of the data encoding device according to the fourth embodiment of the present invention.

FIG. 20 is a flowchart showing an operation of the data encoding device according to the fourth embodiment of the present invention.

FIG. 21 is a schematic configuration diagram of a data encoding device according to a fifth embodiment of the present invention.

FIG. 22 is a flowchart showing the operation of a conventional data encoding device using the dynamic Huffman method.

FIG. 23 is a flowchart showing the operation of a conventional data encoding device using the dynamic Huffman method.

[Explanation of symbols]

 1, 14 Data Encoding Device 2, 15 Encoding Means 3, 16 Dictionary Means 4, 17 Instructing Means 5, 18 Judging Means 6, 19 Control Means 7, 20 Exchange Means 8, 21 Dictionary Term Means 9, 22 Dictionary Search Means 10 Data String Occurrence Probability Dense Code Tree 11 Data String Occurrence Probability Discrete Code Tree 12 Code Generation Means 13 Redundancy Prevention Means 23 Dictionary Term Means Exchange Means 24 Dictionary Search Means Exchange Means

Claims (6)

[Claims] 1. A dictionary means for forming a dictionary using sequentially input data strings, an encoding means for performing an encoding process on the data strings, and a predetermined data string for the dictionary means. Instruction means to
Determination means for determining whether or not the predetermined data string instructed by the instructing means exists in the dictionary means, and a dictionary term for a data string input and a code word shorter than that are assigned in the dictionary means. An exchange means for changing the order of one of the dictionary terms, a dictionary means, the encoding means, the instructing means, the judging means, and a control means for controlling the exchange means are provided, and a data string to be sequentially input is provided. Each time one encoding is performed, the dictionary is updated by exchanging the order of the dictionary terms in the dictionary means for the original data string and one of the dictionary terms in the code means to which a shorter codeword is assigned. A data encoding device characterized by the above. 2. A dictionary means having a dictionary corresponding to sequentially input data strings, an encoding means for performing an encoding process on the data strings, and an instruction of a predetermined data string to the dictionary means. Instruction means,
Exchange means for exchanging the order of the dictionary term for the data string input and one of the dictionary terms to which a shorter codeword is assigned in the dictionary means, the dictionary means, the encoding means, the instructing means, and the exchanging means. Of the dictionary means for the original data string and the dictionary means to which a shorter code word is assigned each time one data string that is sequentially input is encoded. A data encoding device characterized in that the dictionary is updated by changing the order of one of the dictionary terms. 3. The dictionary means comprises a dictionary search means, and when the instruction means searches the dictionary means, the dictionary means is referred to to search the dictionary means. The described data encoding device. 4. The data encoding device according to claim 1, wherein the encoding means comprises a plurality of code trees, and the code tree to be used is changed according to an input data file. . 5. The encoding means comprises a redundancy prevention means, and the redundancy prevention means outputs an original data string to which a specific code is added when a code word having a predetermined number of bits or more is generated. The data encoding device according to any one of claims 1 to 4. 6. A control means, a dictionary means, a dictionary search means, an instruction means, and an exchange means are constituted by a latch by performing encoding with 4 bits, and table updating is performed using hardware. The data encoding device according to any one of 1 to 5.