JP3350118B2 - Data encoding method and data restoration method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data encoding system and a data restoration system.

[0002]

2. Description of the Related Art In recent years, with the development of office automation (OA), various media such as characters, figures, images, and the like can be mixed in one document. Mixed information such as character codes and black-and-white binary images is now handled as document data in G4 facsimile and optical disk file systems, etc., along with their layout information. The amount of such information is also rapidly increasing. It is increasing. When document information composed of such multimedia is used as digital data, the data amount of image information is generally ten times to several tens times larger than the data amount of character codes. For this reason, when image information is handled in data storage or data transmission, etc., in order to perform such processing efficiently, the amount of data is compressed by omitting redundant portions in the data, thereby reducing the storage capacity. And transmission efficiency.

However, in a large-capacity file system or a document database, the character code information in the document data becomes large as a whole, so that not only image information but also character code information must be compressed.

A universal encoding method is known as a method capable of compressing various data such as character codes and image data by one method. As a representative method , for example, a Jib Lempel code (Seiji Munakata) , "Ziv-
Lempel's Data Compression Method, '' Information Processing, Vol. 26, No. 1, Jan.
1985).

There are two algorithms for the Jib-Lempel code, a universal type and an incremental paring type.

Further, as an improvement of the universal algorithm, there is an LZSS code (TCBell, "Better OPM / L
Text Compression ", IEEE Trans. On Commun., Vol.COM-
34, No. 12, Dec. 1986).

[0007] An LZW (Lempel-Ziv-Welch) code is also an improved version of the incremental decomposition type algorithm.
(TA Welch, "A Technique for High-Performance Data
Compression ", Computer, June 1984).

[0008] Among these encoding methods, LZW codes have recently been used for compression of files stored in storage devices because of their high speed processing and simple algorithms. I have.

[0009] Here will be described the two algorithms universal type and incremental decomposition type of Ziv Lempel coding is a typical method of the universal coding. 1. Universal type algorithm This algorithm requires a large amount of computation, but can provide a high compression rate. The maximum length sequence (subsequence) that matches data to be encoded from an arbitrary position in the past data sequence
And encodes it as a copy of a past series.

FIG. 11A shows the basic concept of the encoding of such a universal jib Lempel code. Figure (a)
.. Abc... Stored in the P buffer shown in FIG. On the other hand, data (character string) “abcdef” to be encoded is input and stored in the Q buffer.

In such a state, when encoding the data in the Q buffer, the data sequence in the P buffer is scanned using the data sequence in the Q buffer as a key, and the data sequence in the Q buffer is scanned in the P buffer. Is obtained ("abc" in the example of FIG. 3A). Then, in order to specify the substring having the maximum length in the P buffer, a set of information in the format shown in FIG. This set of information includes "the start position of the maximum matching sequence in the P buffer" (the address of "a" in the example of FIG. 3A), and "length of matching" (in the example of FIG. 3 "), and" next symbol "(" d "in the example of FIG. 3A).

Subsequently, the encoded sequence (in this case, "abc") in the Q buffer is moved and stored in the P buffer to obtain a new past data sequence. Hereinafter, the same operation is repeated for the remaining data series “def” in the Q buffer, and the remaining data series in the Q buffer is
The data sequence is decomposed into partial strings already stored in the buffer, encoded as described above, and the data series in the P buffer is updated.

2. Incremental decomposition type algorithm Although this algorithm is inferior in compression ratio to the universal type, the algorithm is simple and the calculation is easy, so that high-speed processing can be performed.

L which is a typical method of this algorithm
The ZW encoding method will be described with reference to a flowchart shown in FIG. 12, a dictionary (learning dictionary) shown in FIG. 13, and a schematic diagram of data conversion shown in FIG.

The LZW encoding has one rewritable dictionary (learning dictionary), divides an input character string into different character strings (substrings), and assigns reference numbers in the order in which these character strings appear. And registering the character string currently input in the dictionary with a reference number assigned to a matching character string of the maximum length already registered in the dictionary. In the following description, one word unit of data is referred to as a character, and data connected with an arbitrary word is referred to as a character string, following the name used in information theory.

In the LZW encoding process, first, at step S
In step 1, initialization is performed to register in advance a character string consisting of one character for every character in the dictionary DC. That is, for example, when one character is encoded by an 8-bit code, a character string consisting of one character for every 256 characters at maximum is initially registered at addresses 0 to 255 of the dictionary DC. Thus, for example, as shown in FIG. 13 , addresses 0, 1,.
, 255, the alphabet "a",
“B”, “c”,..., Hiragana, katakana, numerals, and the like are registered. It should be noted that the character string table B1 shown on the left side of FIG. 13 is supplementarily shown for ease of explanation.

In the following description, an example in which an input character string as shown in FIG. 14 is input will be described for easy understanding. First, at step S1, the leading address n for writing dictionary D _C, which is the next address of the storage address of the last character string the initial registration "2
56 "is set as the storage address to the dictionary D _C of the character string to be newly registered.

[0018] Then, similarly in step S1, searches the dictionary D _c the first character K which is input as the key data (index), reference numeral omega (reference number of a character K which is registered in the dictionary D _C) And assign this to the initial string (prefi
x string). As a result, the input character string is changed to, for example, "ababcbababaaaa" as shown in FIG.
If aaaa ", dictionary D _C is searched for, which is the first letter K" a "as an index," obtained as a reference number "0" of a "reference number ω, the reference number" 0 "is the word This is the initial character string (see the output code column in FIG. 14).

Next, in step S2, the next character K of the input character string is read. Thus, the character “b” next to the first input character “a” is read. Subsequently, in a step S3, it is determined whether or not the character K is present. this is,
This is a process for determining whether or not the input character string has not been completed yet.

In the case of the input character string shown in FIG. 14, the character string "b" next to "a" is read in step S2, and the character string has not been finished yet. determination, and then in step S4, a character string "ωK" searches whether are registered in the dictionary D _C.

Thus, the initial character string ω (here, reference number “0”) obtained in step S1 is added to step S1.
(In this case "b") read in two characters K is the string "0b" plus, whether or not it is registered in the dictionary D _C is examined.

If the result of this search is No, the process proceeds to step S6, where the code "code (ω)" of the reference number ω of the character K obtained in step S1 is output, and the character string "ωK" It is registered in the address n of the dictionary D _C to impart a new reference number n in.

Thus, in the case of the input character string shown in FIG. 14, first, the code of "0" which is the reference number ω of "a" is output, and the character string "0b" which is not detected is replaced with the reference number. been granted the "256", the address of the dictionary D _C 2
56 is registered.

[0024] Then, similarly in step S6, along with replacing the input character K read in step S2 to the reference numbers omega, the address n of the dictionary D _C is incremented "1", the process returns to the step S2 next character K Read.

[0025] Thus, in the example of the input string in FIG. 14, reference numeral ω is replaced by "b" is a reference number "1", the dictionary D in _C of a character string next newly registered Registration address n is incremented by "257"
Changes to

On the other hand, if the character string "ωK" is registered in the dictionary D _C in step S4, in this case, step S5
The process proceeds to step S2 to the replace the string "ωK" at reference numeral omega, not find in step S4 string "ωK" from the dictionary D _C returns to step S2 again
Steps S5 to S5 are repeated to continue searching for the character string having the maximum matching length.

The processing of LZW encoding performed by such a method is described by an input character string "ababcbaba" shown in FIG.
babaaaaaaa ",
First, when inputting the first letter "a", the dictionary D there is no character string that matches the other "a" _{and C,} reference numeral code reference number "0" assigned to "a" (0) Is output. Then, registered in the dictionary D _C by giving the reference number "256" to the extended character string "ab". Figure 13 shows the actual dictionary registration.
Is registered in the form of a character string “0b” as shown on the right side of “.

Subsequently, the second character "b" becomes the head of a new search character string. In this case, the dictionary D _C outputs the sign of the reference numbers "1" since there is no character that matches the other characters "b" which are assigned to the letter "b" code (1),
Is not registered in the extended character string "ba" is also still Dictionary D _C simultaneously represents a character string "ba" in "1a",
Given the reference number "257" is registered in the dictionary D _C is. Then, next, the third character “a” is replaced with the next search character string “ω
K ”. Similarly, by continuing such processing, the input character string “abab” shown in FIG.
“cbababaaaaaaa” is “0, 1, 256, 2, 257, 260, 0, 2” shown in the output code column of FIG.
62, 263 ", and as a result, the input character string is compressed.

Next, an algorithm for restoring LZW encoded code data as described above will be described with reference to the flowchart of FIG. Further, as a specific example of the restoration, the output code string “0,
1, 256, 2, 257, 260, 0, 262, 26
"3" as an input code string is shown again in FIG.

First, in step S11, as in the case of the LZW encoding, a character string consisting of one character for every character is initially registered in the dictionary Dd. In the specific example described below, each of the characters “a”, “b”,
"C",... Are represented by reference numbers "0", "1",
Are registered in the dictionary Dd by adding "2",..., And "2" which is the next address of the storage address of the last character string initially registered is added to the writing start address n of the dictionary Dd.
56 "is set as the storage address n of the newly registered character string in the dictionary Dd.

Next, in step S11, the first code CODE is read, and a reference number corresponding to the code CODE is set to OLDω. As a result, FIG.
(a) In the example of the input code string shown, the code code (0) of the reference number “0”, which is the first input code, is read, converted to the reference number “0”, and set to OLDω.

Subsequently, in step S11, the character K corresponding to the reference number "OLDω" is restored. In this process, since the first input code CODE corresponds to one of the one-character reference numbers initially registered in the dictionary Dd as described above, the code that matches the input code CODE is used.
The code (K) is searched from the dictionary Dd, and the corresponding character "K" is output. Note that the output character "K" is also set in FINchar in preparation for an exception process performed as necessary later.

Thus, in the example of the input code string shown in FIG. 16A, first, the character "a" corresponding to the reference number "0"
Is restored and output, and is also set in FINchar.

Subsequently, in step S12, the next input code CODE is read. That is, in the example of the input code string shown in FIG. 16A , the code code (1) of “1” is read. Then, in a step S13, it is determined whether or not there is a code CODE newly read, that is, whether or not the code input is completed. In the example of the input code string shown in FIG.
In step S12, the code code (1) having the reference number "1" is read as a new input code CODE.

As described above, if there is a new input code CODE, the process proceeds to step S14, where the input code CODE is set.
Is set to INω. Thus, in the example of the input code shown in FIG. 16A, the reference number “1” is set to INω.

Next, in step S15, it is determined whether or not the reference number "ω" has already been registered in the dictionary Dd (ω ≧
n) is determined. In this process, normally, the read code CODE is already registered in the dictionary Dd in the previous process, so that ω <n, and the process proceeds to step S16.
The dictionary Dd is searched, and the character string ω′K corresponding to the reference number “ω” is read from the dictionary Dd, and the character string corresponding to the reference number “ω” is a two-character character string “ω′K”. Is determined. If it is a two-character string "ω'K", the character "K" is temporarily stacked in step S17, and the reference number "ω '" is set as a new reference number ω, and the process returns to step S16 again. This step S16, S17
Is repeated recursively until the character string corresponding to the reference number ω becomes one character "K", and the process proceeds to the last step S18, where the last restored character K is output, and then all of the characters stacked in the step S17 are output. LIFO (Last
(In First Out) pop-up and output (restoration and output of code CODE read in step S12). Further, in step S18, after setting the first character K of the restored character string to FINchar, the first character K of the restored character string is represented by a set (OLDω, K) from the reference number OLDω of the previously restored character string and the first character K of the character string restored this time. The character string is registered at the address n of the dictionary Dd with a new reference number "n". Subsequently, the address n is incremented by "1", and "n + 1" is set as a registration address n of a character string to be registered next in the dictionary Dd.
The reference number “ω” corresponding to the code CODE restored this time, which has been set to
Return to 12.

Thus, in the case of the input code shown in FIG. 16 (a), as shown in FIG. 16 (b), the code CODE (= code (1)) of the reference number "1" read second. The character "b" is restored and output from
ar and the code COD that was previously restored
A character string “0b” consisting of a series of a reference number “0” corresponding to E (= code (0)) and one character “b” restored this time
Is given a new reference number "256" and registered in the dictionary Dd.

The registered address n of the dictionary Dd is "2
After being updated to “57”, the reference number “1” corresponding to the code CODE (= code (1)) restored this time is set in OLDω, and the third code co is set in step S12.
de ( 256 ) is read.

Then, the character string "0b" obtained by searching the dictionary Dd is replaced with the character string "ab", and the character string "ab" is output. At the same time, a character string “1a” (= “ba”) obtained by combining the reference number “1” corresponding to the code code (1) restored last time and the first character “a” of the character string restored this time is new. Reference number "25
"7" is added and registered in the address "257" of the dictionary Dd.

On the other hand, if it is determined in step S15 that the read code (ω) is not registered in the dictionary Dd in the previous process (ω ≧ n), the process proceeds to step S19 to perform an exception process. . In this exception processing, first, after outputting the first character "FINchar" of the previously restored character string, the reference number "OLDω" corresponding to the code CODE restored last time is set as the reference number ω, and then the previously restored character string is restored. A character string “OLDω, FINchar” obtained by adding the first character “FINchar” of the character string is obtained, a reference number corresponding to the new character string is set to INω, and the process proceeds to step S16.

Thus, for example, in the case of the input code string shown in FIG.
Reference number "260" corresponding to code (260) of
Are not defined in the dictionary Dd at this time. In this case, first, in step S19, the previously restored code co
After the first character (FINchar) of the character string “bab” corresponding to de (257) is output, the previously restored character string “ba” is added to the reference number “257” corresponding to the code code (257) previously restored. Is obtained by adding the first character "b" of ".", A reference number "260" is given to this character string, and this reference number is set to INω. Then, by repeating the processing of steps S16 → S17, the characters are stacked one by one in the order of “a” and “b”. In step S18, the character string "ab" is output by a pop-up operation, and finally the code
The de ( 260 ) is restored and output as a character string of "bab", and the character string "257b" is added to the reference number "260" and registered in the dictionary Dd (see FIGS. 3B to 3E). ).

Thereafter, by repeating the same processing sequentially, the input code string shown in FIG. 16A is restored to the character string shown in FIG.

The data compression by the LZW coding described above is not a method of performing compression by assuming the statistical property and continuity of target data in advance as in other methods, but completes the original information by coding. Because it is an information storage type data compression method that is restored to
It is suitable for compression of data that requires complete restoration, such as program source code or object code .

Since the LZW code can be directly applied to an arbitrary symbol string, if image data is divided into a certain amount of data and the data is handled in the same manner as a character code, the image data is also compressed by LZW coding. can do. Therefore, it is possible to compress information in which a plurality of types of data having different properties, such as a character code and image data, are mixed by LZW encoding.

However, the conventional Jib Lempel (Ziv-Lemp
el) Encoding is performed using only one rewritable dictionary, and this dictionary is updated with input data, and when the dictionary capacity is full (when the empty capacity is exhausted)
The data is coded by clearing immediately or clearing the compression ratio when the capacity is full and then clearing the compression rate and restarting the dictionary registration from the beginning.
For this reason, at the time when the number of registered data in the dictionary after initial or clearing is small, the properties of the input data cannot be sufficiently learned, and it has been difficult to obtain a high compression rate.

Even if the number of registered dictionary data increases,
When the change in the properties of the input data is large, only the contents that reflect the average properties are registered in the dictionary, so that the dictionary cannot be used efficiently, that is, the data compression rate is low. .

The present invention has been made in view of the above circumstances, and when the properties of input data are known in advance, an initial dictionary suitable for the properties of the input data is loaded into the learning dictionary and compressed. After the property of the input data changes, the content learned in the learning dictionary is reflected in the initial dictionary, and then the initial dictionary that matches the input data with the changed property is loaded into the learning dictionary. Accordingly, an object of the present invention is to realize a data encoding method and a data decompression method that can always obtain highly efficient compressed data by creating an initial dictionary that always incorporates new characteristics.

[0048]

SUMMARY OF THE INVENTION The present invention relates to a data encoding system.
This method is applied to a method and a data restoration method for restoring the code.

The data encoding method according to the first aspect of the present invention (see FIG. 1) comprises a rewritable learning dictionary 1 and a plurality of initial dictionaries 2-1 and 2 corresponding to input data having different properties.
,..., 2-n, and encoding means 3 for encoding the input data at fixed intervals based on the learning dictionary 1 to perform data compression, and an initial dictionary at fixed intervals of the input data. The initial dictionary 2-i (i = 1, 2,...) Corresponding to the properties of the input data in 2-1, 2-2,.
., N) are loaded into the learning dictionary 1 and the encoding means 3
When the data compression for a certain section is completed, the dictionary changing means 4 changes the contents of the initial dictionary 2-i based on the contents of the learning dictionary 1.

The dictionary changing means 4 counts the number of times the character string of the dictionary of the learning dictionary 1 loaded from the initial dictionary 2-i is referred to within the certain section, for example. Based on the reference frequency obtained by counting,
The content of the learning dictionary 1 is reduced, and the content after the reduction is set as a new initial dictionary 2-i. Also, for example, in the above-mentioned certain section, the reference count of the character string registered in the initial dictionary 2-i loaded in the learning dictionary 1 and the reference of the character string not registered are set within the fixed section. The number of times was counted, respectively, and based on those count values and the threshold values respectively corresponding to those count values, the character string group consisting of the referenced registered character strings and the referenced unregistered character string were found. After each character string group composed of character strings is reduced, the reduced character string groups are merged to form a new initial dictionary 2-i.

The data restoration method according to the fourth aspect of the present invention (see FIG. 2) includes a rewritable learning dictionary 11 and a plurality of initial dictionaries 12-1 corresponding to input codes having different properties.
12-2,..., 12-n and a learning means 11 for restoring an input code for each fixed section, and an initial dictionary 12-
, 12-n, the initial dictionary 12-i corresponding to the properties of the input code is loaded into the learning dictionary 11, and the input code When the restoration is completed, the initial dictionary 12-i is replaced with the learning dictionary 1
1 and a dictionary changing means 14 for changing the contents to 1.

The dictionary changing means 14 may be, for example,
As described above, the number of times the character string of the dictionary of the learning dictionary 11 loaded from the initial dictionary 12-i is referred to in the above-described fixed section is counted, and the learning frequency is determined based on the reference frequency obtained by the counting. The content of the dictionary 11 is reduced, and the content after the reduction is set as a new initial dictionary 12-i. Further, for example, within the certain section, the reference count of the character string registered in the initial dictionary 12-i loaded in the learning dictionary 11 and the reference of the character string not registered are set in the fixed section. The number of times was counted, respectively, and based on those count values and the threshold values respectively corresponding to those count values, the character string group consisting of the referenced registered character strings and the referenced unregistered character string were found. After each character string group composed of character strings is reduced, the reduced character string groups are merged to form a new initial dictionary 2-i.

[0053]

First, in the data encoding method, a plurality of initial dictionaries corresponding to input data having different properties are used in addition to a normal rewritable learning dictionary. Then, for each fixed section of the input data, an initial dictionary corresponding to the property of the input data is selected and loaded into the learning dictionary. Is encoded. Thereafter, the original initial dictionary is rewritten with the contents of the learning dictionary that has been learned and updated in the encoding process.

At this time, after the contents of the learning dictionary updated / learned based on the reference frequency of the character string at the time of encoding are reduced and the dictionary size is reduced, the original initial dictionary is rewritten.

Alternatively, reference is made to each of the character strings registered and unregistered in the initial dictionary, which has been made into a learning dictionary, based on the reference frequency and the threshold value corresponding to each reference frequency. After the character string group composed of the extracted character strings is reduced, the reduced character string group is merged to create a new dictionary, and the original initial dictionary is rewritten.

As a result, encoding with a high compression ratio can be realized even for input data in which data having different properties are mixed. Next, in the data restoration method for the encoded data, similarly, a plurality of initial dictionaries corresponding to input codes having different properties from those of a normal rewritable learning dictionary are used. Then, for each fixed section of the input code, an initial dictionary corresponding to the property of the input code is selected and loaded into the learning dictionary, and the input code is restored by using the initial dictionary converted into the learning dictionary.

At this time, after the contents of the learning dictionary updated / learned based on the reference frequency at the time of restoration are reduced and the dictionary size is reduced, the original initial dictionary is rewritten. Alternatively, based on the reference frequency of each of the registered and unregistered character strings in the initial dictionary that has been converted into the learning dictionary, After the character string group composed of columns is reduced, the reduced character string group is merged to create a new dictionary, and the original initial dictionary is rewritten.

As a result, the data encoded by the above-mentioned encoding method is restored.

[0059]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 3 is a diagram illustrating the basic concept of the data encoding method according to the present embodiment.

In this data encoding method, in addition to the normal rewritable learning dictionary 33-2 used in the LZW encoding, an initial dictionary 32-1 for the object code used for encoding the object code, and the code of the source code If three types of dedicated initial dictionaries are prepared, an initial dictionary for source code 32-2 used for image conversion and an initial dictionary for image 32-3 used for encoding image data, and if the properties of the original data 30 are known in advance, , The learning dictionary 33
-2, the data compression by the LZW encoding of the original data 30 is performed. In this embodiment, the encoding method and
LZW coding method is used, but is not limited to this.
Never.

The original data 30 includes data type information indicating the type of data such as an object code, a source code or image data, and size information indicating the length (size) of the data indicated by the data type information. The data is input to the control unit 31 together with the data information. The control unit 31
Before starting data compression by encoding, the data type information of the input data information input together with the original data 30 is referred to, and it is determined whether the input data is object code, source code, or image data. If the property is known, an initial dictionary suitable for the property is selected from the object code initial dictionary 32-1, the source code initial dictionary 32-2, and the image initial dictionary 32-3. The data is loaded into the learning dictionary 33-2 of the encoding unit 33.

If the type (property) of the input data does not correspond to any of the initial dictionaries 32-1, 32-2 or 32-3, or if the property of the input data is unknown, the learning is performed. No initial dictionary is loaded in the learning dictionary 33-2, and the learning dictionary 33-2 is used as a normal learning dictionary.

The encoding unit 33 transmits the selected initial dictionary 32-1 (or 32) loaded into the learning dictionary 33-2.
-2, 32-3), the LZW encoder 33-1 encodes and compresses the input data with the data length indicated by the size information of the input data information as one block, and compresses the data with the configuration shown in FIG. Output compressed data sequence.

After that, the contents of the learning dictionary 33-2 learned / updated by the above-mentioned encoding process are stored in the initial dictionary 32-1 (or the initial dictionary 32-1 selected before the above-mentioned encoding process and loaded into the learning dictionary 33-2. 32-2, 32-3), and the contents of the initial dictionary are updated.

As described above, the input data is encoded from the initial stage of the encoding by the initial dictionary suitable for the type (property) of the input data. Thus, the data compression is performed at a high rate without the compression ratio decreasing as in the case of (1). Furthermore, the initial dictionary itself is also updated based on the contents of the learning dictionary that has been learned and updated, and an initial dictionary that is more suitable for the characteristics of the input data can be obtained. As a result, the overall compression ratio can be increased. .

As shown in FIG. 4, the compressed data sequence is added to the LZW encoded compressed data 42 and the head of the compressed data 42, and the compressed data 42 is subjected to LZW encoding using any dictionary. A plurality of input data information 41 including a dictionary flag indicating whether the compressed data has been encoded and size information indicating the length of the original data (the size of the encoded block) encoded in the compressed data. It is a data string composed of pairs. The input data information 41 is for informing the data decompressor of which dictionary should be used when decompressing the compressed data 42 and the size of the decompressed data. As described above, the compressed data sequence is composed of a plurality of consecutive sets of [input data information 41 (dictionary flag and original data size information) and compressed data 42] appearing in units of encoded blocks.

FIG. 5 shows an example of a method for creating the object code initial dictionary. As shown in the figure, the initial dictionary for object code 32-1 is prepared by first preparing a normal learning dictionary 52, and converting an object code 51 into an LZW code by the LZW encoding unit 53.
This is performed in the process of creating the compressed data 54 by encoding. That is, the LZW encoding unit 53 sequentially encodes the object code 51 with the new LZW
The codes and the corresponding object code strings are registered in the learning dictionary 52. When the LZW encoding is completed for all the object codes 51, the learning dictionary 52 is a dictionary reflecting the properties of the object codes 51. Therefore, the learning dictionary 52 is used as the object code initial dictionary 32-1.

In this embodiment, initial dictionaries are sequentially created for other input data having different properties as needed. FIG. 6 shows a method of creating the image data initial dictionary 32-3 used for LZW encoding of image data. The creation of the initial dictionary for image data 32-3 is performed in the same manner as the creation of the initial dictionary for object code 32-1.
The W encoding unit 63 refers to and registers the learning dictionary 62 in the process of LZW-encoding the predetermined image data 61, and when the LZW encoding of all the image data 61 is completed, the properties of the image data 61 are changed. The learning dictionary 62 that is the reflected dictionary is used as the image data initial dictionary 32-3.

Similarly, a source code initial dictionary 32-2 for LZW encoding of the source code is created. In the present embodiment, using the initial dictionary created in this way,
Data compression by LZW encoding shown in FIG. 3 is performed. In this case, when the content of the learning dictionary 33-2 updated by learning is loaded into the original initial dictionary 32-1, 32-2, or 32-3, the dictionary content whose size has been increased by learning / updating is directly used. Instead of loading, moderate size,
For example, the size is reduced to the size of the original initial dictionary and then loaded.

FIG. 7 is a schematic diagram showing a method of loading the original dictionary after reducing it to the size of the original initial dictionary. Learning dictionary 3
In 3-2, since it is necessary to register new data while learning, a large size is set in advance ((a) in FIG.
). Initial dictionary 32 corresponding to this learning dictionary 33-2
-1 (or 32-2, 32-3) is loaded to create a learning dictionary 33-2 (FIG. 9B). As the compression of the input data by the LZW encoding progresses, the learning dictionary 33-2 is sequentially updated, and the dictionary data of the learning dictionary 33-2 increases (FIG. 3 (c)). When the LZW encoding is completed, the reference frequency is counted for each character string in the dictionary, and the initial dictionary 32-i (i =
Learning dictionary 33 obtained by reducing 1, 2, 3) to the same size
-2 is created (FIG. 4 (d)). Then, the learning dictionary 33-2 is used as the initial dictionary 32-1 (or 32-1) used this time.
-2, 32-3). As a result, the contents corresponding to the latest input data are always stored in the initial dictionary 32-1 (or 32-2, 32-3).

The dictionary data of the learning dictionary 33-2 forms a decomposition component tree which is successively connected to the already-existing component of the character string and the component copy, and is branched in some places. (For more information, see Seiji Munakata, "Ziv-Lempel Data Compression Method," Information Processing, Vol. 26, No. 1, Jan. 1985.) In order to reduce the size of the learning dictionary 33-2, instead of the method based on the reference frequency described above, pruning for deleting a tree segment of a decomposition component tree of a character string formed as dictionary data is also performed. The result of a simple dictionary reduction is obtained.

In the method shown in FIG. 7, the contents of the learning dictionary 33-2 obtained as a result of learning are stored in the original initial dictionary 32-2.
1 (or 32-2, 32-3), but the original initial dictionary 32-1 (or 32-2, 32-
A character string corresponding to the property of the input data corresponding to 3) is also registered. Therefore, it may not be appropriate to completely replace the original initial dictionary 32-1 (or 32-2, 32-3).

Based on such considerations, the initial dictionary 32-
FIG. 8 shows a method of partially replacing 1 (or 32-2, 32-3). In the figure, before performing data compression by LZW coding, an empty learning dictionary 33-2 ((a) in the figure) and an initial dictionary 32-1 (or 32-2,
2-3) is loaded to create a learning dictionary 33-2 (FIG. 13B), and LZW encoding of the input data is performed using the learning dictionary 33-2, and the method shown in FIG. The learning dictionary 33-2 is created in the same manner as (2) in FIG.

After the end of the LZW encoding, the size of the learning dictionary 33-2 is reduced by the reference frequency or the pruning to create a learning dictionary 33-2 half the size of the initial dictionary. Figure (d)). On the other hand, the size of the initial dictionary 32-1 (or 32-2, 32-3) is similarly reduced to 1/2 by the reference frequency or pruning (FIG. 11 (f)). Next, this initial dictionary 32-1
(Or 32-2, 32-3) and the reduced learning dictionary 33-2 are merged into a new initial dictionary 32-
1 is created (FIG. 9 (g)).

FIG. 9 is a flowchart showing an algorithm of the above-mentioned encoding method. In this process,
An input counter CT for counting input original data for each word is used. In addition, the initial dictionaries 32-1 and 32-
2 and 32-3 are prepared.

In the figure, first, an input count initial value corresponding to input data (original data) of one block size is set in an input counter CT (step S1). The block size of the input data set here is, for example, "object code of 100 kbytes".
The determination is made based on information indicating data (object code) having the same property and a size (100 kbytes) that is a continuous length of the data.

Next, an initial dictionary corresponding to the property of the input data is loaded (step S2). Thus, for example, if the input data is the object code, the initial dictionary for object code 32-1 is loaded into the learning dictionary 33-2. If the nature of the input data is unknown, a normal (empty) learning dictionary 33-2 is used.

Subsequently, the code (dictionary flag) representing the initial dictionary to be used and the size of the input data to be encoded are output (step S3). As a result, the input data information 41 of the compressed data sequence shown in FIG. 4 is output.
For example, in the case of an object code of 100 kbytes, a dictionary flag indicating the initial dictionary 32-1 for the object code and a code indicating 100 kbytes are output.

Next, data for one character is input (step S4). And the input data is "EOF" (end)
Is determined (step S5). If "EOF", it is determined that the file of the input data has been completed, and the processing is immediately terminated. If the input data is not "EOF", the input counter is determined. CT is decremented (step S6), and the value of the decremented input counter CT indicates the end of one block of the input data (CT = 0).
It is determined whether or not this is the case (step S7).

If the data processing of one block has not been completed yet (CT ≠ 0 in step S7), the input data is encoded using the dictionary selected and loaded in step S2, and the dictionary learning / An update process is performed (step S8), and the number of times a character string that matches at the time of reference is referenced is counted (step S9).

Further, only the original initial dictionary of the selected / loaded dictionary is referred (again, step S8), and the number of times of reference to the matched character string is counted (again, step S9).
Then, the process returns to step S4.

The above steps S4 to S9 are repeated, and the encoding of one block of input data is advanced to the end. In step S7, it is confirmed that the input counter CT has become 0, and the flow proceeds to step S10.

In step S10, the initial dictionary is compressed (reduced) based on the reference frequency.
By compressing (reducing) the learning dictionary based on the reference frequency, in step S12, the reduced initial dictionary and learning dictionary are merged, and the dictionary obtained by the merge is replaced with the original initial dictionary. The original initial dictionary is updated, and the process returns to step S1.

In this way, the input data is coded by the same initial dictionary in block units to form a coded block, and a dictionary flag indicating the dictionary used for coding and data indicating the size of the original data are provided at the head of the coded block. It is output as compressed data (see FIG. 4) in the added block unit.

At this time, the input data such as the object code, the source code, and the image data are efficiently encoded from the start of the processing by the corresponding initial dictionary, and the data is compressed at a high rate. Then, the used initial dictionary becomes an initial dictionary corresponding to the property of the input data by the replacement after the merge. This further improves the compression ratio for the next input data of the same type.

When the learning dictionary and the initial dictionary are compressed, a threshold is set for each reference frequency and compression is performed. In that case, the threshold of the reference frequency of the initial dictionary is set to IL,
When the threshold of the reference frequency of the learning dictionary is LL, IL <
If <LL, updating of the initial dictionary with a high response speed to the input data can be realized. On the other hand, if IL >> LL, a configuration in which the contents of the initial initial dictionary are not easily changed can be achieved.

Next, the decompression of the coded compressed data will be described with reference to the flowchart of FIG.
Also in this restoration processing, the input counter CT and the initial dictionaries 32-1, 32-2, and 32-3 are used. The input data is a compressed data sequence shown in FIG. 4. As described above, one block is composed of input data information 41 including a dictionary flag and a data size, and a subsequent compressed data block 4.
Consists of two.

First, a dictionary flag, which is the first input data of one block of compressed data to be input, is read, and the type of dictionary used for encoding the compressed data is set (step S21).

Subsequently, the size of one block, which is the second input data of the compressed data, is read and set as an initial value in the input counter CT (step S22).
As a result, the number of original data restored from the input data (compressed data) is set.

Next, the initial dictionary set in step S21 is loaded as a learning dictionary (step S2).
3). Thus, the learning dictionary used when the input data (compressed data) is encoded is set.

Subsequently, compressed data is input (step S
24) If the input data is not “EOF” indicating the end of the file (step S25), the compressed data is decoded (decompressed) by the learning dictionary and output (step S26). In this process, the learning dictionary is sequentially learned and dictionary data is registered.

Subsequently, at the time of restoration, the number of times of reference to the matched character string in the learning dictionary is counted (step S2).
7). In this case, as in the case of the encoding, the loaded original dictionary is also referred to (Step S26 again), and the number of times of reference to the matched character string is counted (Step S2 again).
7).

Then, the input counter CT is decremented by the number of the restored words (step S28).
With reference to the decremented value of the input counter CT, the restoration of the compressed data for one block is completed (CT
= 0) (step S29).

If all of the compressed data for one block has not been restored (CT ≠ 0), the flow returns to step S24 to input the next compressed data. The above steps S24 to S29 are repeated to complete the decompression of one block of compressed data, and in step S29, CT = 0
Then, the process proceeds to step S30.

In step S30, the initial dictionary is compressed (reduced) based on the reference frequency.
The learning dictionary is compressed (reduced) based on the reference frequency. In step S32, the reduced initial dictionary and the learning dictionary are merged, and the dictionary obtained by the merge is replaced with the original initial dictionary. Is updated, and the process returns to step S21.

As described above, as in the case of encoding, the restoration of the compressed data proceeds while the initial dictionary is updated for each block. If the input data is "EOF" in step S25, the process is immediately terminated.

[0097]

According to the present invention, a plurality of initial dictionaries in which the characteristics of different types of input data are taken in advance are prepared, and the input data is encoded using a dictionary corresponding to the characteristics of the input data. In addition, it is possible to prevent a decrease in the compression ratio at a time when the number of registered learning data is small at the beginning of the compression start, and it is possible to realize a high compression ratio from the beginning of the encoding. Also,
Since the contents of the initial dictionary are updated according to the actual input data, encoding is always performed using a dictionary that is well suited to the characteristics of the input data, and high-speed compression can be realized regardless of the type of input data.

Therefore, it is possible to obtain a high compression ratio even for input data in which different types of data are mixed.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of a data encoding method according to the present invention.

FIG. 2 is a principle diagram of a data restoration method according to the present invention.

FIG. 3 is a conceptual diagram of a basic configuration in an encoding method according to an embodiment.

FIG. 4 is a diagram illustrating the structure of a compressed data sequence according to one embodiment.

FIG. 5 is a diagram (part 1) illustrating an example of a method of creating a first initial dictionary;

FIG. 6 is a diagram (part 2) illustrating an example of a method of creating an initial initial dictionary;

FIG. 7 is a schematic diagram (part 1) illustrating a method of feeding back a learning dictionary to an initial dictionary.

FIG. 8 is a schematic diagram (part 2) illustrating a method of feeding back a learning dictionary to an initial dictionary.

FIG. 9 is a flowchart illustrating an algorithm of data compression according to an embodiment;

FIG. 10 is a flowchart illustrating an algorithm for restoring compressed data according to one embodiment.

FIGS. 11A and 11B are diagrams illustrating the basic concept of encoding of a universal type Jib-Lempel code.

FIG. 12 is a flowchart illustrating an algorithm of LZW encoding.

FIG. 13 is a diagram illustrating a configuration of a dictionary used for LZW encoding.

FIG. 14 is a schematic diagram illustrating an LZW encoding method.

FIG. A flowchart for explaining an LZW code restoration algorithm
It is a low chart.

FIG. 16 (a), (b), (c), (d), (e) are incremental decomposition type jib
FIG. 3 is a schematic diagram illustrating restoration of a impel code.

[Explanation of symbols]

 1, 11 learning dictionary 2-1, 2-2, ..., 2-n initial dictionary 12-1, 12-2, ..., 12-n initial dictionary 3 encoding means 4, 14 dictionary changing means 13 Restoration means

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Hirotaka Chiba 1015 Uedanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Inside Fujitsu Limited (72) Inventor Masahiro Mori 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Fujitsu Limited ( 56) References JP-A-4-265020 (JP, A) JP-A-3-247167 (JP, A) JP-A-63-151224 (JP, A) JP-A-3-247168 (JP, A) JP Hei 4-80813 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 5/00 G06T 9/00 H04N 1/41 H04N 1/413 H03M ⁷ /30-7/46

Claims (6)

(57) [Claims] In a data encoding method, a rewritable learning dictionary (1) and a plurality of initial dictionaries (2) corresponding to input data having different properties are provided.
-1), (2-2),..., (2-n), and the initial dictionary (2-
(1) The input in (2-2),..., (2-n)
Initial dictionary (2-i; i = 1,
2,..., N) into the learning dictionary (1).
Dictionary changing means (4), and the initial dictionary (2-i; i)
= 1, 2, ..., n), the learning dictionary (1)
Input data is coded at regular intervals based on
Encoding means (3) for performing compression , wherein the dictionary changing means (4) is provided by the encoding means (3).
When the data compression for a certain section is completed, the learning
Based on the contents of the book (1), the contents of the initial dictionary (2-i) are
A data encoding method characterized by changing . 2. The dictionary change unit (4) counts the number of times a reference is made to a character string of a dictionary of the learning dictionary (1) loaded from the initial dictionary (2-i) in the certain section, The content of the learning dictionary (1) is reduced based on the reference frequency obtained by the counting, and the reduced content is used as a new initial dictionary (2-i). 2. The data encoding method according to 1. 3. The dictionary rewriting means (4) determines the number of times of reference to a character string registered in the initial dictionary (2-i) loaded in the learning dictionary (1) in the certain section. A character string group consisting of the registered registered character strings is counted based on the counted values and the threshold values respectively corresponding to the counted values and the counted values. And reducing a character string group composed of referenced and unregistered character strings, and merging the reduced character string groups to form a new initial dictionary (2-i). Item 2. The data encoding method according to Item 1. 4. A data decompression method for decompressing compressed data LZW-encoded by the data encoding method according to claim 1, wherein the data decompression method supports a rewritable learning dictionary and input codes having different properties. Multiple initial dictionaries (12
-1), (12-2),..., (12-n); and a restoring means (13) for restoring an input code at regular intervals based on the learning dictionary (11); For each fixed section of the input code, the initial dictionary (12-
An initial dictionary (12-i) corresponding to the property of the input code in 1), (12-2),..., (12-n) is loaded into the learning dictionary (11). The restoration means (1
When the restoration of the input code in a certain section according to 3) is completed,
A dictionary changing means (14) for changing the initial dictionary (12-i) to the contents of the learning dictionary (11). 5. A data restoration method for restoring compressed data LZW-encoded by the data encoding method according to claim 2, wherein said dictionary changing means (14) is configured to:
i) counting the number of times a reference is made to the character string of the dictionary of the learning dictionary (11) loaded in the fixed section in the fixed section, and based on the reference frequency obtained by counting, the learning dictionary (11). 5. The contents after the reduction are used as a new initial dictionary (12-i).
Data recovery method described. 6. A data restoration method for restoring compressed data LZW-encoded by the data encoding method according to claim 3, wherein the dictionary changing means (14) is configured to perform the learning for the learning within the fixed section. The number of times of reference to a character string registered in the initial dictionary (12-i) loaded into the dictionary (11) and the number of times of reference to a character string not registered are counted. After reducing the character string group consisting of the referenced registered character strings and the character string group consisting of the referenced unregistered character strings based on the threshold values respectively corresponding to the count values, By combining the character strings after the reduction, a new initial dictionary (2-
5. The data restoration method according to claim 4, wherein i) is set.